Lesson 8 · Hygiene

Hygiene: two directions, three fixes

You said names break and you didn't know the families of fix. There are exactly two ways they break, and the fix you pick follows from which one — plus one rule about who owns the name.

The two directions

Vocabulary first, because the whole subject collapses into two words. An identifier is bound in an expression if that expression contains its binder — the def, the parameter, the assignment, the import. Otherwise it is free, and its meaning comes from outside. Capture is when a name that was free gets accidentally bound by someone else's binder. Two directions, and both have proper names:

Clinger and Rees's Macros That Work is where these are pinned down as separate obligations. A macro system that satisfies both is called hygienic; most people use “hygiene” loosely for the pair, but knowing they are two properties is what lets you notice you have solved one and not the other.

You have both problems already, without user code: ImportCollisionError is direction 2 detected by hand, and ShadowedImportError is direction 1 detected by hand. gates.py is a hygiene checker written before anyone said the word.

Exercise A — Find the capture

Here is the output your fake() generator will emit. Two free-text answers; the fields check themselves.

from app.models import Invoice
from pydantic.experimental.missing_sentinel import MISSING

def fake_invoice(*, total: int = 1, note: str = "note", **overrides) -> Invoice:
    return Invoice(**({"total": total, "note": note} | overrides))

Both answers are names your template put in the file. That is the general shape: a capture is always a collision between a name you introduced and a name the user owns — which is why the fix is always about deciding who has the right to be renamed.

The three fix families

1. Rename on introduction — gensym

Every binder you introduce gets a name nothing else can have: _pcg_overrides rather than overrides. The name comes from generate-symbol, the Lisp function that hands out identifiers guaranteed fresh. Fixes direction 1, completely and cheaply, and it is the only family that scales to arbitrary spliced code — because you cannot enumerate the names a user might use.

One constraint specific to you, and it is a real one: your output is committed to a repository and read by humans. A counter-based gensym (overrides_1, overrides_2) makes the emitted text depend on generation order, which means an unrelated change reorders names and your Lesson 4 golden tests all go red. So: a reserved prefix and a deterministic suffix derived from the thing being generated, never from a global counter. Ugly names, stable diffs.

2. Resolve at the definition site

Fixes direction 2. Never emit a bare free name and hope the output file resolves it the way you meant. Instead emit, alongside it, the import that makes it mean that — and alias it into a namespace you own:

from pydantic.experimental.missing_sentinel import MISSING as _pcg_MISSING

This is what Rust's quote! users mean by always writing ::core::option::Option with the leading ::, and it is what syntax-case does for you automatically by remembering, per identifier, the scope it was written in. Your Import node already has an alias field. The diff is: the renderer aliases every import it introduces, and then ImportCollisionError becomes unreachable — two different modules binding the same name is no longer a conflict, because neither one is bound under that name.

A gate you can make unreachable by construction is a gate you should delete.

3. Detect and reject

What you do today, and it stays the right answer in exactly one situation: when both colliding names belong to the user. You cannot rename a field — it is the payload. You cannot rename a generated class — other modules import it, so it is API. So a model named Invoice generated into a file that must import Invoice is not fixable by renaming; it is a recipe the user has to change, and ShadowedImportError is correct.

The rule that ties the three together:

Rename what you introduced. Resolve what you referenced. Reject only what the user owns on both sides.

Exercise B — Pick the family

One of the three for each name. Answer before revealing.

The principled version, and why you won't build it

The state of the art is not renaming at all. In Racket, an identifier is not a string: it is a syntax object carrying the set of scopes it was written in, and two identifiers are the same variable only if name and scopes agree. Matthew Flatt's Binding as Sets of Scopes is the current formulation; Dybvig, Hieb and Bruggeman's Syntactic Abstraction in Scheme is the syntax-case design it descends from. Under that model both directions of capture are impossible rather than avoided.

You should not build it, for one decisive reason: your output is read. Scope sets work because the renaming they imply is invisible — the expander consumes its own output. Yours goes into a file a colleague opens in an editor, so every rename you make is a permanent cost to that reader, and the budget for ugly names is small. That is a genuine engineering difference, not a shortcut.

Steal one idea from it, though: an identifier only means something together with where it came from. You already compute that — bindings.py maps each free name in an annotation to the import that binds it in the defining module. That table is your syntax object, and when user-supplied bodies arrive it is the thing that answers “what did this name mean where it was written?”. Extend it, don't invent something new. Note the rejection you'll need already exists too: UnresolvableNameError — a body referring to a name its own module neither imports nor defines is a defect you can report precisely, and one Lesson 6 says to report alongside all the others.

status · resources · mission