Lesson 10 · Diagnostics, part 2

Recovery, poison, and the sink

Lesson 6 gave you three words in a table. This is the code, and it is almost entirely about one hazard: recovery that loses the diagnostic. Every bug below is a real compiler bug shape.

Where lesson 6 left off

Three decisions, restated so this lesson has something to hang on. Accumulate siblings — checks that consume nothing from each other. Recover to reach a later phase, by substituting an error node for the value you could not build. Suppress the diagnostics that recovery itself caused.

The thing all three write into is the diagnostic sink — rustc's DiagCtxt. Here it is, whole:

class Severity(Enum):
    FATAL = auto()
    ERROR = auto()
    WARNING = auto()

@dataclass(frozen=True)
class Diagnostic:
    code: DiagnosticCode
    severity: Severity
    message: str

class Sink:
    def __init__(self) -> None:
        self._diagnostics: list[Diagnostic] = []

    def report(self, diagnostic: Diagnostic) -> None:
        self._diagnostics.append(diagnostic)

    def errors(self) -> int:
        return len([d for d in self._diagnostics if d.severity is not Severity.WARNING])

Decision 1 — a mutable sink, or diagnostics in the return type?

Lesson 6 argued that list[Diagnostic] plus concatenation is the applicative, and that is true. But once you add recovery, a pass has to return two things: the value it managed to build, and what went wrong. Written purely, every signature in the codebase becomes a pair:

def load_one(target: Target) -> tuple[Model | Poisoned, list[Diagnostic]]

And every call site becomes plumbing. Real compilers take the other branch: thread one mutable sink through, and let the return type say only what was produced. Two things that buys you, and one it costs you, worth being able to state:

Take the mutable sink. Then spend the rest of the lesson defending against the cost.

Error nodes: three encodings, and the two questions that pick one

An error node is whatever you put where the real value should have gone. There are only three shapes:

# A — absence. The failed model is simply not in the list.
def load_all(targets: list[Target], sink: Sink) -> list[Model]

# B — an explicit poison node. A sum type; the failure is a member of the IR.
@dataclass(frozen=True)
class Poisoned:
    name: ModelName
Loaded = Model | Poisoned

# C — a flag on the real node. The model exists but is marked untrustworthy.
@dataclass(frozen=True)
class Model:
    name: ModelName
    fields: list[Field]
    poisoned: bool

Absence is free, so the interesting work is knowing when it is a lie. Two questions about every downstream consumer:

Recovery is allowed to lose information. It is never allowed to make a failing check pass.

That is the safety property, and it has a name worth using: conservative recovery. Under absence, rename_model violates it — which decides the encoding. Take B: poison is a member of the IR, carries the name, and is visible to anything that counts. C is what you reach for when the node has already been handed out and you cannot change its type; it costs you the compiler's help, because if model.poisoned is a check every pass can forget and none is forced to make.

The best idea in this lesson: make silent poison unrepresentable

The most common recovery bug, by a wide margin, is producing an error node and forgetting to report anything — except SomeError: continue. The run then succeeds, writes files, and one model has quietly vanished. That is strictly worse than the fail-fast you started with.

rustc fixes this in the type system. ErrorGuaranteed is a zero-sized token that cannot be constructed outside the diagnostics crate; you get one only by actually emitting an error. Its error nodes carry one — so holding a value of that type is a static guarantee that compilation will fail. The dev guide is precise about the direction: it means an error has already been emitted, not that one will be.

Python has no privacy, but it has enough. The token constructor takes a marker only the sink can supply:

class _SinkKey:
    """Instantiated once, in Sink.__init__, and never exposed."""

@dataclass(frozen=True)
class Reported:
    _key: _SinkKey

class Sink:
    def __init__(self) -> None:
        self._diagnostics: list[Diagnostic] = []
        self._key = _SinkKey()

    def report(self, diagnostic: Diagnostic) -> Reported:
        self._diagnostics.append(diagnostic)
        return Reported(self._key)

@dataclass(frozen=True)
class Poisoned:
    name: ModelName
    reported: Reported

Now Poisoned cannot be built without a Reported, and a Reported cannot be built without a diagnostic in the sink. Snippet 2 of Exercise A stops being writable.

Cascade suppression, as code

Suppression means: a diagnostic derived from a poisoned value is noise, so don't report it. The tempting implementation is a check in every pass —

def omit(models: list[Loaded], field: FieldName, sink: Sink) -> list[Loaded]:
    for model in models:
        if isinstance(model, Poisoned):
            continue
        ...

— and it rots on the first pass someone adds. Put the check in the driver instead, and give the pass a signature that cannot see poison:

type Pass = Callable[[Model, Sink], Model]

def each(pass_: Pass, models: list[Loaded], sink: Sink) -> list[Loaded]:
    return [m if isinstance(m, Poisoned) else pass_(m, sink) for m in models]

Two things happened there. Poison is propagated rather than dropped — the list keeps its length and its names, which is what conservative recovery needed. And forgetting to suppress is now a type error rather than a missing if: a pass declares Model, so it never receives a Poisoned to mishandle. This is the same move as lesson 5's sentinel and lesson 8's hygiene-by-construction — push the obligation into a place where the checker enforces it.

The general term for this shape is taint propagation: poison spreads along data dependencies, and a diagnostic is suppressed exactly when one of its inputs is tainted. The alternative you will see in older compilers is error-count gating — “stop the phase if anything failed” — which is lesson 6's policy P2, and it suppresses far more than the cascade.

Exercise A — Bug hunt

Five snippets. Click the line that carries the defect, or say no bug. Not all of them are broken; a wrong click reveals the answer, so commit before you click.

Exercise B — Decisions

Answer in the box from memory, then reveal and compare. These are the four I would ask in a design review.

What changes in the repo

Five diffs, in dependency order. The first three are mechanical; the last two are the ones that keep this from rotting.

And two tests, which are the ones that matter more than the diffs:

def test_reports_every_independent_defect() -> None:
    # n seeded defects that do not consume each other -> exactly n diagnostics.
    # Fails if suppression is too eager.

def test_no_poison_without_a_diagnostic() -> None:
    # every Poisoned in the output has a diagnostic whose code names it.
    # Fails if recovery went silent.

Those two pull in opposite directions, which is exactly why you want both: the first fails when you suppress too much, the second when you suppress too little. A single test in either direction is one you can satisfy by making the other worse.

Exercise C — Free recall

Lesson 6 and lesson 10 interleaved, since the terms are easy to confuse. Type the term; the field checks itself.

The debt this creates

Every diagnostic in this lesson is a code and a message with nowhere to point. Poisoned carries a ModelName, which is the best location available to you — and it is not good enough the moment a defect is about a field, or about the recipe line rather than the model. The outer fringe was already spans and provenance; this lesson is the second time it has come up, and the second time is usually when it stops being optional.

status · resources · mission