Learning path

Compiler & metaprogramming fundamentals

Grounded in pydantic-codegen. See MISSION.md · RESOURCES.md

Assessment basis. Everything below comes from a ten-question diagnostic, not from reading the repository. The repo is largely LLM-written and is treated here as context for what the lessons should be about, never as evidence of capability. An earlier version of this page got that wrong and has been discarded.

Knowledge graph

confirmed inner fringe — lesson written completed outer fringe — unlocked next locked

Green nodes are clickable. Press F or the corner button to maximise; Esc to exit.

Lessons

#	Lesson	Written because
1	The Phase Map	The vocabulary gap is the stated blocker on entering the literature. Now anchored on the narrow waist, since that turned out to be already-held.
2	Values and expressions new	Q3 blank and Q1 half-blank. Written to close both: why `repr` can't produce source, and why that forces the dual front end rather than excusing it.
3	How opaque should an IR node be?	The original question. Unchanged in content — the diagnostic suggests it will land.
4	Testing a code generator new	Q5 blank + explicitly requested. Absorbs Q4 (pass algebra) and Q10 (round-trip), since the properties you'd assert are the algebra.
5	Deriving a `fake()`	The original question. Moved last: it depends on 2 (values vs expressions) and 3 (structured types), which weren't in place.
6	Errors that accumulate new	Q6 partial, and explicitly wanted. Leads with the mechanism you were reaching for — the bind/ap split — then the half that collecting doesn't buy you: recovery, error nodes, cascade suppression.
7	Emitting statements new	Q8's real answer, and the deadline that arbitrary user methods put on the f-string renderer. Quasiquotation, unquote-splicing, and the fifteen lines that give you both in Python.
8	Hygiene: two directions, three fixes new	Q7 blank. Now unblocked, since the fix families only make sense once statement emission exists. Reframes both existing gates as hygiene enforced by detection.
9	Metamorphic relations, and where the inputs come from new	Requested from Lesson 4 feedback (“quite interested in metamorphic properties / property-based testing”). Extends axis 3: MRs as (T, R) pairs, invariance vs equivariance, generating specs rather than IR or source, and how a property goes vacuous.
10	Recovery, poison, and the sink new	Requested directly: lesson 6 gave recovery, error nodes and the sink as a four-row table with no code. This is the code, plus a five-snippet bug hunt on the shapes they fail in — silent poison, the shared-sink delta, cardinality-sensitive recovery — and `ErrorGuaranteed` as the fix that makes one of them unwritable.
11	Spans and provenance new	The debt lessons 6 and 10 created twice. Leads with the finding that matters for a generator: provenance is a sum type, not a byte range, because synthesized nodes have a derivation and no location. Plus the five Python position facts, four of which are off-by-one.
12	Canonicalisation new	Closes the Q4 gap with the two missing terms — commutativity, confluence — and the surprise: order stops mattering because the input space shrank, not because the passes commute. Ends on the conflict with lesson 11, where normalising erases what a good message needs.
13	User methods, end to end new	The capstone that consumes 5, 7, 8, 11 and 12. Leads with the decision that removes most of the work — inherit rather than move — then the four Python hazards that only bite when code moves, and the differential oracle that makes the whole thing testable.
14	Edges and reverse names new	Session-2 diagnostic: Q7 blank and Q8 partial, both consequences of one broken invariant — the IR is a graph now. Act I is the missing mechanism behind the correct “one FK on the many side” answer; Act II names what collides, which is a reverse query name in a namespace Django owns. Corrected from feedback: the claim that the FastAPI backend wants both directions was wrong. Payload models show one direction too — which makes the placement argument stronger, since all backends want the same fact.
15	Cycles new	Q11, blank. “Cycles” is four problems with four fixes, and the lead finding is that the topological sort everyone reaches for is dead code — Django's lazy reference kills problems 1 and 2 outright, and the only one that matters is a cycle in the required subgraph, which is unsatisfiable and therefore a diagnostic rather than a codegen problem.
16	Lowering vs elaboration new	Written from the override answer — `max_length` inferrable, `on_delete` and `db_index` defaulted — which is exactly the lowering/elaboration line, plus a third category (derived) for `related_name`. Five of eight attributes turn out to be lowered. The trap is that effort is not the discriminator, and the mechanical bite is that pydantic v2 keeps constraints in `FieldInfo.metadata`, so the obvious attribute access silently drops them.

Ticking a lesson recolours its node in the graph above and is saved locally.

Diagnostic, session 1

#	Topic	Result	Read
2	Why the IR, not the class object	strong	“Waist problem: N inputs and M outputs.” Unprompted, correctly named, and the reason rather than a rationalisation. This is the single best answer of the ten.
8	Templates vs AST construction	strong	“See 2” — transferred the waist argument to a new question, plus static type-safety. Slightly off-target for what templates specifically buy, but the transfer matters more than the hit.
9	IR extension for user methods	strong	Named methods, bodies, and binding kind. Then made a clean ownership call — user's problem, error immediately — without hedging.
10	Generator as fixpoint	strong	“Round-trip and inverses, worth a test.” Correct, and the second unprompted reach for the idea.
4	When pass order doesn't matter	partial	“Idempotency is not enough” — knows the property, knows it's insufficient, lacks commutativity / confluence and hasn't tested either. One term from complete.
6	Accumulating diagnostics	partial	“Functions should return errors” — reaching for a result type. No model of which errors are recoverable, or what the return type becomes.
1	`getsource` vs `ast.parse`	partial	Distinguished them correctly; couldn't say where both are needed. Consistent with the front end being accreted — the rule is followed but not held.
5	Testing a generator	blank	“Not sure. Probably something about round-trip. Perhaps property-based? Would love to learn more.” Correct instincts, no framework. Highest-leverage gap and self-identified.
3	`repr` as source	blank	Load-bearing: it is the reason the dual front end exists and the reason `fake()` is hard. Straight to Lesson 2.
7	Hygiene	blank	Sensed that names break; no families of fix. Fine to defer — the fix families only make sense once statement emission exists. Becomes urgent the moment user methods ship.

Overall read

Strong architectural instinct, thin mechanism, no literature. Reasons from first principles and transfers ideas across problems unprompted (the waist argument appears twice, round-trip twice). Decisive about ownership and boundaries rather than hedging. What's missing is consistently the named technique and the specific Python-level fact — not the judgement to use them.

Practical consequence for teaching: skip motivation, lead with the mechanism, name things aggressively. The failure mode to avoid is explaining why a technique is worth having — that part is already there, and it's what makes the blanks feel like blanks rather than disinterest.

Diagnostic, session 2 — the Django backend

Eleven questions across two rounds, on going from a pydantic domain model to a Django model. Question 5 (rank your regrets) was rejected, correctly — the items are not independent, so ranking them invites a hedge rather than an answer. Question 6 (import the package or ast.parse it) was called bad, and it was: the front end is already dual by lesson 2's argument, so the binary is false. What whole-program resolution changes is when declarations must all be in hand, not which reader supplies them.

#	Topic	Result	Read
9	Value object with no table	strong	“Flattened fields.” Immediate and correct. This is Fowler's Embedded Value; the only thing missing was the prefix, and that the prefix must derive from the owning field rather than the value type or two `Money` fields collide.
10	Identity ownership	strong	“Django wins; a domain `id` field should resolve to a primary key.” Decisive, and the reason given was the right one. Fowler's Identity Field.
1	Compilation unit	strong	“Whole domain package, otherwise we can't resolve references.” Correct and correctly justified. Did not name the cost — the unit of recompilation grew with it.
2	Cardinality and owning side	partial	“One FK, on the many side, so on User.” The right output. The second half — what in the pipeline guarantees it — was not attempted, and a per-field backend cannot: the fact it needs is not in its argument. Same shape as session 1: strong call, thin machinery.
3	Elaborating `on_delete`	partial	Chose `DO_NOTHING` because soft deletes are handled in the domain — a real decision, and one that needs correcting: `DO_NOTHING` leaves the constraint in place, so a hard delete raises a driver-level `IntegrityError`. `PROTECT` states the same prohibition as a diagnostic. The override half of the question was skipped, twice.
8	Reverse-name collisions	partial	“Bidirectional reference, collides in Django's inferred methods.” The right instinct, one term short — it is a reverse query name, and the namespace is the target model's. No model of the four shapes (`fields.E302`–`E305`) or of the fix families.
7	Where the dedup pass lives	blank	The mechanism gap made explicit. Highest-leverage of the round, and the entry point for lesson 14.
11	Mutually non-nullable FKs	blank	Cycles. Untouched, and it is the one problem in this cluster where no amount of ordering helps — the fix has to be a decision about the domain. Lesson 15.

Overall read

The resolution half of this cluster is largely already held: compilation unit, owning side, identity ownership and embedded values all came back clean and unprompted, and three of them are named patterns in Fowler's catalogue. So the symbol-table lesson that looked like the obvious entry point would have been mostly review — the expertise-reversal trap — and two-phase resolution is folded into lesson 14 as mechanism rather than taught as its own thing.

What is genuinely absent is narrower and more specific than expected: the consequences of the IR no longer being a tree. Both blanks and one partial are the same fact seen three times — an edge is not a field, so nothing that maps over fields can be correct about edges. The second gap is Django's own naming semantics, which is not compiler knowledge at all but is where hygiene (lesson 8) now actually bites.

Also worth noting: two of eleven questions were rejected as badly posed, and both rejections were right. Take those at face value the way the blanks are taken.

Outer fringe — next session candidates

The Django cluster is written out. All four load-bearing decisions in it now have a lesson and a diff: resolution and edge identity (14), satisfiability and build order (15), the lowering/elaboration line (16). What remains on the fringe is older than this cluster and none of it is about Django.

Topic	Why now
Reading the literature	Overdue three times, and the only mission criterion never tested. Session 2 supplied a much easier on-ramp than the nanopass paper: five of the eight rows in lesson 16's table are named patterns, and three of your own session-2 answers turned out to be Fowler's Foreign Key Mapping, Identity Field and Embedded Value. Run the exercise in the easy direction first — map your answers onto published names — then in the hard direction, mapping a paper onto the code. Shape it as an exercise, not an explainer; what cannot be placed picks the next lesson by evidence.
Nanopass discipline	Held over twice, and the Django cluster raised the stakes rather than the interest: 14's resolve pass produces an IR containing `Relation` nodes the input did not have, and 15 adds a pass whose only output is a diagnostic. That is the first place per-pass IRs would be materially different rather than tidier — an IR where `Relation` is non-optional after resolution and absent before it. The open question is unchanged and is a types question, not a macros one.
Extension registry	Now partly answered by 16 rather than still open: the trigger condition is written down (the first per-field `on_delete` override request) and so is the shape (a sidecar beside the domain package, keyed by model and field, drift reported as a diagnostic). What is left is only the `fake()` half — user-supplied factories for types the deriver cannot handle, where Hypothesis's `register_type_strategy` is the prior art. Smaller than it was.

Recommendation: the literature session, and shaped as five mapping exercises rather than prose. It is the last untested mission criterion, it has never been attempted, and after three lessons of naming things it is the cheapest it will ever be. Nanopass is the natural follow-on because the paper is the one to read, so the two compose into a single session if the mapping goes quickly.

Two things deliberately not taught in this cluster. Symbol tables and two-phase resolution appear inside 14 as mechanism rather than as a topic, because questions 1, 2 and 10 showed that part is already held. Sum types across the waist — one table with nullable columns versus table-per-variant, which is Fowler's Single versus Class Table Inheritance and Django's abstract/multi-table/proxy — is genuinely unaddressed and stays locked, since it needs a domain model that actually has a sum type in it.

Confirmed context

Dual front end	accreted, not designed. Lesson 2 argues it's nonetheless forced; the work is to write the rule down.
Generated files re-read?	No. Rules out staleness and incremental-generation concerns entirely. Round-trip stays useful as a property, not as a workflow.
Error reporting	All errors, not the first. Currently fail-fast; this is a real structural change and a whole lesson.
Django output	Python source. One target language throughout — the renderer never needs to be polyglot.
Compilation unit	The whole domain package. Confirmed session 2. Nothing can be generated from one model in isolation, because whether an id type resolves is a fact about another file.
Identity	Django owns the primary key. A domain `id: XId` field lowers to `primary_key=True` rather than coexisting with an implicit `AutoField`.
Value objects	Flattened into the owning table with a prefix taken from the owning field name. Fowler's Embedded Value. No separate table, no `JSONField`.
`on_delete`	Elaborated, defaulted. Was `DO_NOTHING`; lesson 16 argues for `PROTECT`, since the domain prohibits hard deletes and `DO_NOTHING` converts that prohibition into a driver-level `IntegrityError`. Open for your call.
`db_index`	Elaborated, defaulted. Confirmed. Safe to default because being wrong costs an unused index.
`max_length`	Lowered from the pydantic constraint. Confirmed — and note it is not optional to state something, since `CharField` requires it (`fields.E120`); the real choice is `CharField(max_length=n)` against `TextField()`, i.e. whether the constraint lives in the schema or only in validation.
Django override surface	Not built, by decision. Defaults suffice for two elaborated attributes. The trigger for building a sidecar is written down in lesson 16: the first per-field `on_delete` override request. When it comes it goes beside the domain package, not on the model, so the domain never imports Django.
`fake()` scope	Tests only, for now. But the general requirement is arbitrary user-specified methods on generated output, which is the larger feature `fake()` is one instance of. This reshaped the graph more than anything else.

Teaching preferences

The repo is not evidence. It is LLM-written. Critique design decisions and use it for worked examples; never infer capability from it, and never praise it.
Answers “don't know” cleanly and without padding. Take blanks at face value — they're accurate.
Wants fundamentals over recipes; asked to be pushed deep. Lead with mechanism, not motivation.
No formal CS background: name every term on first use, assume no familiarity with the literature.
Every lesson must cash out as a decision or a diff in pydantic-codegen.

Updated after lessons 14, 15 and 16. Lesson 14 is complete and carries one correction from feedback — the claim that payload models want both directions was wrong, and the corrected argument is stronger. Lessons 15 and 16 were generated from the two remaining gaps in the session-2 diagnostic (Q11 blank, Q3's skipped override half) plus the answer on max_length/defaults, which turned out to be the lowering-versus-elaboration line and shaped 16 directly. The Django cluster is now written out; the fringe holds nothing Django-specific, and the literature-mapping session is overdue three times with the easiest on-ramp it has had.