Lesson 2 · The substrate

Values and expressions

Why your loader has two front ends, and why that is not the accident you think it is.

You said the dual front end — runtime reflection and source parsing — was accreted. This lesson argues it was forced, that you can't remove it, and that the useful move is to write down the rule it currently follows only in your head.

It rests on one distinction, which is the most important idea in this whole path:

A running program holds values. A generator must emit expressions. There is no general way back from one to the other.

Why repr is not the way back

It looks like it should be. repr's documented aspiration is to produce a string that eval turns back into an equal object. So why can't you read User.model_fields["id"].default off the runtime class and write repr of it into the file?

It fails in three tiers, and they matter in ascending order because each is quieter than the last.

Tier 1 — not source at all

<object object at 0x1028c4b20>
<function <lambda> at 0x1039419b0>
<_io.StringIO object at 0x103a81ab0>
<generator object <genexpr> at 0x103ae0450>

Angle brackets aren't Python. You'd notice within a minute. Harmless, because loud.

Tier 2 — valid syntax, unbound names

repr(datetime(2020, 1, 1))  →  'datetime.datetime(2020, 1, 1, 0, 0)'
repr(Tag(label="x"))        →  "Tag(label='x')"

Both are legal Python. Neither tells you that the file now needs import datetime or from tagging import Tag. A value carries no import list. This is precisely the job bindings.py does — and it can only do it because it is handed an expression to walk for free names. Hand it a value and there is nothing to walk. Still fairly loud: the generated file fails at import.

Tier 3 — valid syntax, bound names, wrong meaning

This is the answer to the question you passed on, and there's an instance of it sitting in your own test corpus. Consider:

class Tagged(BaseModel):
    tags: list[Tag] = Field(default_factory=list)

Ask the runtime for the default and you get []. Emit repr of it:

tags: list[Tag] = []

Valid syntax. Bound names. Imports fine. Passes a golden test if the golden file was recorded from the same wrong code. And it is a different program: the source says build a fresh list per instance, and the output says here is one list. You have silently converted a factory into a shared default — Lesson 5's Hazard 2, introduced by the generator rather than by the user, in a place where nothing raises.

Worse, the tell is invisible in the value. [] from a default_factory and [] written literally are the same object shape. The distinction exists only in the source text. Which is the general statement of tier 3:

Evaluation is lossy. It discards exactly the information a generator needs: how the value was going to be produced.

The same loss elsewhere: Field(default=3, gt=0) evaluates to 3 and the constraint goes somewhere else entirely; an enum member reprs as <Color.RED: 1>; a float computed as 1/3 reprs as a decimal literal that is a different expression with the same value. Every one of these is a place where “ask the object” and “read the source” disagree, and the source is right.

Exercise A — Pick the oracle

Six questions the loader has to answer. Which source of truth can answer it? Decide before clicking.

So: the rule your loader already follows

Work through Exercise A and the split falls out. It is not arbitrary:

And now the answer to the half of question 1 you skipped: where does the loader need both? Look at ModuleSource. It calls inspect.getsource to get the text and ast.parse to get the tree — then ast.get_source_segment(text, node) uses the node's line and column span to slice the original text. The tree is used only to locate; the text is what's kept.

That is deliberate and it is the third thing in this lesson worth remembering. ast.unparse(node) would give you an expression back, but a normalised one — reformatted, requoted, comments gone. By slicing the original text instead, the generated file reproduces what the programmer wrote, character for character. A parse tree used as an index into source, rather than as a replacement for it, is called keeping the source map. It's the same reason compilers carry spans around long after they've stopped needing to re-read the file.

Exercise B — The quiet one

class Preset(BaseModel):
    tags: list[Tag] = Field(default_factory=lambda: [Tag("new")])

The lever you didn't have until 3.14

One column of that table just got a second oracle, and it's worth knowing before you decide what to do with the accretion.

PEP 649 (shipped in 3.14) makes annotations lazily computed via an __annotate__ method, and PEP 749 adds annotationlib, which can ask a class for its annotations as strings — from the runtime object, with no file access:

annotationlib.get_annotations(User, format=Format.STRING)
# {'id': 'UUID', 'tags': 'list[Tag]', 'at': 'datetime | None', 'ref': 'Tag | None'}

That is the annotation column of your source front end, for free. But measure before you believe it — I ran it, and it is normalised, not source-faithful:

Read those last two rows again. The quotes are gone. A string forward reference comes back unquoted — which is the canonicalisation Lesson 3 wants, and it would have made the partial_none bug in Lesson 3 unreachable, for free, by construction.

So the honest assessment of the accretion: annotations could plausibly move to the runtime oracle and shrink module_source.py. Defaults and bases cannot — nothing normalises those back into expressions, because of everything above. The dual front end stays. What changes is that it stops being accidental.

Exercise C — Recall

status · resources · mission