The Collision Surface Audit
A2 gave you a price list. This is the survey you run before you buy anything: a repeatable, four-question procedure for enumerating everything two workers can touch — including the surfaces that are not in the database, which is where the failures nobody can reproduce come from.
~18 min · after A2: The Isolation Ladder · pairs with A4
1. Why an audit and not a hunt
When you assessed your suite's collision surface in the first session, you reasoned about the system under test and got it right. What you missed were the surfaces that are not the application at all: the storageState file on disk, seed artefacts, ports, output directories.
That is the normal failure of this task, and it has a structural cause. Debugging a flake is retrospective — you start from a symptom and reason backwards, so you only ever consider surfaces the symptom pointed at. An audit is prospective: you walk a fixed checklist and ask what is on each layer, whether or not anything has failed there yet. The checklist is doing the remembering so you do not have to.
A collision surface is any mutable thing addressable by more than one concurrently-running actor. Three conditions, all required: mutable, addressable by more than one, concurrent. Break any one and it is not a surface.
Those three conditions are also your escape hatches, and they map onto the ladder: make it immutable (rung 0-read-only), make it addressable by exactly one (rungs 1, 3, 4, 5), or remove the concurrency (serialisation). There is no fourth option.
2. The four questions
Per resource, in order. Question 3 is the one that does the work, and the one people skip.
What does this test write?
Not what it reads. Reads never collide. List every mutation: records created, updated, deleted; settings changed; files written; external calls with side effects. If the list is empty, the test cannot cause a concurrency flake — though it can still suffer one.
Who else can address that write?
Other tests in the same file, other files in the same project, other projects, setup projects, the application's own background work. Under fullyParallel the answer widens; the audit is only valid for a stated parallelism configuration.
How is the thing addressed?
There are exactly four ways, and three of them are collisions waiting to happen:
- By unique handle — an id or UUID this test owns. Safe.
- By fixed name — a literal string. Collides with every other actor using that literal, and with leftovers from previous runs.
- By position or count — "the first row",
toHaveCount(25),items[i]. Collides with anything that changes the collection, including inserts by unrelated tests. - By singularity — "the company", "the logged-in user", "the flag". Collides with everything, always. There is only one.
What survives failure?
Run the test and imagine it throws halfway. What is left behind? Whatever survives is poison state, and with retries: 3 the very next thing to encounter it is that test's own retry — which is why a leaky test tends to fail all four attempts and look like a hard bug.
Q3 is where the audit earns its keep. Q1 and Q2 are obvious enough that people do them instinctively. Q3 converts a vague worry ("this touches products, and so do other tests") into a decision, because each addressing mode maps directly onto a rung: fixed name → rung 1, position/count → stop asserting on the container, singularity → rung 2 or 5.
3. The seven layers
Q1 asks what the test writes. This is the checklist that stops you answering only "database". Click each to expand.
4. Free recall
Cover section 3. Name as many layers as you can, and for each, one concrete thing on it in a Playwright suite. Then reveal.
Application data · Identity & auth · Filesystem · Process & network · External services · Time · Application background work
If you produced application data and nothing else, that is the same gap as the first assessment — and it is the one that makes flakes irreproducible, because layers 3 through 7 do not show up in a database diff.
One layer is not on the list and is worth knowing why: browser storage — cookies, localStorage, IndexedDB. Playwright gives every test a fresh BrowserContext, so this is isolated by construction. Knowing which layers the framework already handles is as valuable as knowing which it doesn't.
5. Classify the surface
For each item: is it a genuine collision surface, shared-but-immutable, or isolated by the framework?
Interleaved deliberately — these three are easy to tell apart in isolation and hard to tell apart in a list.
6. Audit a spec
Click every line that writes to a shared surface. There are three. The verdict updates as you find them.
7. Your suite, audited
The seven layers walked over apps/frontend/e2e, at the current configuration: fullyParallel unset (parallel across files, serial within), eleven projects, three setup projects, retries: 3.
| Layer | What is on it here | Verdict |
|---|---|---|
| Application data | One company. Products, departments, folders, schemas, frame types, settings — all in it, all writable by all eleven projects. | hot |
| Identity & auth | One DEFAULT_USER, one e2e/.auth/user.json, read by every project and by the worker-scoped authedRequest. Any spec that changes this user's password, role, permissions or language mutates the credential every other worker is holding. | hot |
| Filesystem | .auth/ is deleted and rewritten by globalSetup + auth.setup.ts, once per run, before workers start. Traces and reports are per-test paths. Fixture media (test-image.jpg, test-video.mp4) is read-only. | cold |
| Process & network | PREVIEW_PORT with reuseExistingServer: false, and a globalTeardown that explicitly reaps the vite child because pnpm may not forward SIGTERM. One backend container set per run. Singular, but singular by design and serialised into global setup/teardown. | warm |
| External services | Invitations and password reset imply mail. If two specs address the same inbox and select "the latest message", that is addressing-by-position on a shared collection — the classic form. | warm |
| Time | No clock manipulation in evidence. Not a surface here. | cold |
| Background work | Renders. globalSetup points the backend's FRONTEND_URL_INTERNAL at the preview server so the company-robot render hits the same build. That is a server-side headless browser doing work concurrently with your tests, against shared data. | warm |
Two things this audit says that a database-only view would not:
- Identity is a separate hot surface from data, and it is worse. Corrupt a product and one project fails. Corrupt
DEFAULT_USER— a password change, a permission change, a language toggle — and every worker holding that JWT starts failing at once, in a way that looks like a suite-wide outage rather than a test bug. Theauthproject is the highest-risk directory in the repo for exactly this reason. - Renders are a concurrent writer you did not write. A render running against the shared company sees whatever state the tests have put it in at that instant. That direction is a source of flakes in the render specs; the reverse direction — the render writing output back — is worth checking, because a background writer breaks the mental model where "the actors" means "the workers".
Filesystem is cold, and that is a fact about the current configuration, not a property of the suite. It is cold because .auth is written once by a setup project before any worker starts. Move authentication into a per-worker fixture — the natural first step toward rung 5 — and this row becomes hot immediately: N workers writing N state files, and any shared path between them is a race. Re-run the audit when you change parallelism or scope, not once.
Retain this
- Audit one spec this week. Pick a spec that has failed recently. Four questions, seven layers, ten minutes. The output is a rung per resource — which is directly actionable, unlike "this test is flaky".
- Check the identity surface first. Grep
e2e/tests/authandtests/settingsfor anything that changesDEFAULT_USER's password, language or permissions. If such a spec exists and is not serialised or guarded, it is a suite-wide poison generator and it outranks everything else on your list. - Retrieval, one week out. The four questions in order, the four addressing modes, and the seven layers. If the layers are the part you lose, that is expected — write them on the wall next to the suite.
Fixture Scope Is a Sharing Decision
This audit repeatedly landed on "shared across a worker". A4 is about the declaration that decides that — and the several ways a fixture leaks state it appears to own.
Sources: Playwright · Parallelism · Flaky Tests in JavaScript · RESOURCES.md · STATUS.html · A1 · A2