Frontend testing · Lesson C3

Size, Scope, and What Belongs in e2e/

You said "no idea" to what would keep a behaviour out of the E2E suite. That blank is the most expensive one in the assessment, because it is the only lever that moves flake rate and CI time at the same time.

~18 min · after C2 · unlocks the team-standards work

1. Two words that are always confused

Almost every argument about "unit vs integration tests" is two people using one word for two independent things. Google's testing chapters split them, and the split is the whole lesson.

Axis 1

Size

What resources the test is allowed to touch. Nothing to do with how much code runs.

  • Small — one process, one thread. No network, no real filesystem, no sleeping.
  • Medium — one machine, multiple processes, localhost only. A real database is allowed.
  • Large — multiple machines, real network, real services.
Axis 2

Scope

How much code the test exercises and holds responsible.

  • Narrow — one class or function.
  • Medium — a few units interacting.
  • Large — an entire user journey.

A narrow-scope test can still be large-size. That combination is almost always a mistake — and almost always accidental.

The third term completes the vocabulary. Hermetic: a test carries everything needed to set up, execute and tear down its own environment, and assumes nothing about the outside world — including the order tests run in.

2. Where your suite sits

Plot it honestly:

graph TD
  subgraph L["LARGE size · real backend, real network, real time"]
    F["forms/ — 20 field specs
narrow scope"]
    T["tables/ — 11 specs
medium scope"]
    E["editor/ — transform, animation
medium scope"]
    S["settings/, products/, shop/
large scope"]
  end
  subgraph M["MEDIUM size"]
    X["(empty)"]
  end
  subgraph SM["SMALL size"]
    U["src/lib/*.test.ts · eslint-plugins/*.test.ts
narrow scope"]
  end
  style F fill:#fee2e2,stroke:#dc2626,stroke-width:2px
  style X fill:#f5f5f4,stroke:#a8a29e,stroke-dasharray: 4 4
  style U fill:#dcfce7,stroke:#16a34a

Two things stand out. The medium band is empty — there is no level that runs real components with real browser events but no backend. And the red box is the anomaly: twenty specs of narrow scope at large size. number-field.spec.ts holds one component responsible while paying for a database reset, an auth setup, a Vite build, a browser, and a real HTTP round trip.

The finding that makes this urgent. Google measured flakiness against test properties across their whole corpus and found it correlates with size and with number of dependencies — not with what the test is testing. A narrow-scope behaviour tested at large size inherits every flake source of the large tier while covering one component's worth of risk. That is the worst available trade, and your suite makes it twenty times.

3. The criterion

Here is the answer to the question you couldn't answer, as a rule you can put in review comments.

A behaviour earns an e2e/ spec only if the thing that could break lives in the integration — that is, only if no smaller test could observe the failure.

Concretely, at least one of these must be true:

  1. Contract. The failure is a disagreement between frontend and backend about a payload, status or shape.
  2. Routing, auth or persistence. The behaviour spans a navigation, a session, or a reload.
  3. Cross-component state. Two independently-owned parts of the app must agree — a mutation in one panel invalidating a query read by another.
  4. Browser-native behaviour. Real pointer capture, real drag, real clipboard, real file input, real focus order. The editor's gestures live here, permanently.
  5. It is a designated smoke path. A small, explicitly-listed set of journeys that must work, tested end to end because they are the deploy gate.

If none applies, the behaviour is component-level, and putting it in e2e/ is duplicated coverage bought at roughly a hundred times the price.

Run number-field.spec.ts through it. Does a bad thousands separator involve a contract? No. Routing or persistence? No. Cross-component agreement? No. Browser-native input — maybe, if the spec covers real keyboard behaviour, caret position, or paste. Smoke path? No.

So the honest answer for most field specs is: only clause 4, and only for the parts that genuinely exercise browser input. That is a real reason to keep some of them at this level — and it is a much smaller set than twenty files.

4. Khorikov's four pillars — why you can't just move everything down

The counter-argument deserves its strongest form. Khorikov's model says a test's value is four properties, and that two of them cannot both be maximised:

Protection against bugsHow much real risk the test covers. Rises with how much of the real system runs.
Resistance to refactoringWhether the test survives a change that keeps behaviour identical. Falls as the test learns implementation detail.
Fast feedbackWall-clock to a verdict. Falls off a cliff at large size.
MaintainabilityCost to read and to keep working.

Your suite's conventions are a deliberate, coherent bet on the first two. Black-box (no src/ imports), generated types from the real API, real backend — that is maximum protection and maximum refactoring resistance, paid for in feedback speed. It is a defensible position, and it is why the suite is good.

The problem is not the bet. It is that the bet is being placed on every test identically, including tests whose risk does not justify the premium. Level allocation is not an argument against your conventions — it is the claim that one policy for 200 heterogeneous specs is the thing costing you.

Fowler's version, from On the Diverse and Fantastical Shapes of Testing: the proportions in the pyramid/trophy/honeycomb arguments matter far less than whether tests are fast, reliable, expressive, and fail only for useful reasons. That last clause is the one to keep. A field spec that fails because a parallel worker changed the shared company failed for a useless reason.

5. The missing middle, priced

If the forms specs should move down, they need somewhere to land. Three candidates, in honest order:

Playwright component testing

experimental

Real Chromium, real events, no backend. Reuses the locator API and the test-id convention you already have, so the migration is mechanical. The cost is that Playwright still labels it experimental — a real risk to weigh, not a reason to dismiss.

Vitest + Testing Library

mature

Fast, boring, already in the repo for src/lib. The cost is a simulated DOM: no real layout, no real pointer capture, and jsdom lies about enough that editor and drag behaviour is out of reach. Also pulls against the black-box rule, since it imports src/.

Playwright + route.fulfill against the real app

available today

Same browser, same specs, same conventions — but the backend is stubbed, so no database reset, no seed project, no cross-worker collisions. Still large-size by Google's definition, but the dependency count drops sharply, and that is the thing flakiness correlates with. The cheapest experiment available, and it needs no new tooling.

6. Execution: classify these

Eight specs from your repo. Click a spec, then click the level it belongs at under the criterion in §3. Feedback is immediate, and the reasoning appears as you go.

Spec
Belongs at

7. Free recall

Without scrolling: state the criterion for keeping a behaviour in e2e/, in your own words.

8. What this buys you this week

  1. Run the §3 criterion over e2e/tests/forms/, one file at a time, and write the verdict next to each. Do not move anything yet — the list itself is the artefact, and it is the first draft of the team standard.
  2. Pick the single clearest narrow-scope-at-large-size spec and port it to route.fulfill, keeping everything else identical. Measure both: wall-clock, and whether it still fails when you break the component.
  3. Add the criterion to e2e/CLAUDE.md as a question a reviewer asks, not a rule: "which of the five clauses justifies this being an e2e spec?" A question survives disagreement better than a rule.
  4. Note in STATUS which clause the editor specs rely on. It is clause 4, permanently, and that is worth being explicit about — the editor is not a candidate for moving down, and the team should know why.