The Determinism Inventory

A test is deterministic when nothing outside it can change its result. Your suite currently leaves four inputs to the environment. Three of them have a one-line fix; the fourth is the interesting one.

~18 min · after C1 · closes the Q4 blank

1. The term

Determinism, in Kent Beck's Test Desiderata framing: if nothing changes, the test result shouldn't change. It sits alongside isolation — same result regardless of run order — as one of twelve properties you trade against each other.

That framing matters because it names what retries: 3 is. Retries do not add determinism. They buy an appearance of it by paying in wall-clock time and in lost signal: a test that passes on attempt three is recorded as passing, and the nondeterminism it proved is discarded. Beck's rule is that you give up a property only to gain a more valuable one. Here you gave up determinism to gain… nothing. It was never a decision.

A test's result is a function of its inputs. Enumerate the inputs and you have the audit:

graph LR
  T[Test result] --- A[Test code]
  T --- B[App code]
  T --- C["⏱ Time"]
  T --- D["🌐 Network"]
  T --- E["🌍 Locale & timezone"]
  T --- F["🗄 Shared backend state"]
  T --- G["⚙️ Machine resources"]
  style C fill:#fee2e2,stroke:#dc2626
  style D fill:#fee2e2,stroke:#dc2626
  style E fill:#fee2e2,stroke:#dc2626
  style F fill:#fef3c7,stroke:#d97706
  style G fill:#fef3c7,stroke:#d97706
  style A fill:#dcfce7,stroke:#16a34a
  style B fill:#dcfce7,stroke:#16a34a

Red is uncontrolled in your suite today. Amber is partially controlled — shared state by convention and guard fixtures (that's A2/A3), resources not at all. Green is the only part your existing conventions actually govern.

2. Locale and timezone: the cheapest fix in the repo

Your config sets reducedMotion and nothing else from Playwright's emulation surface. timezoneId and locale are unset, which means they come from the machine — your laptop in one place, the CI runner in another.

Now consider what the suite asserts. money-field.spec.ts, date-fields.spec.ts, multi-date-time-field.spec.ts, i18n/live-reformat.spec.ts. Every one of those asserts a string produced by Intl, and Intl reads its defaults from the environment.

use: {
  baseURL: PREVIEW_URL,
  locale: 'en-GB',        // pin it — whatever the app should be tested under
  timezoneId: 'Europe/Copenhagen',
  contextOptions: { reducedMotion: 'reduce' },
}

This is a two-line change that converts a class of latent CI-vs-local disagreement into a config fact. It is worth doing even if it has never yet bitten you, because the failure it prevents is one that looks like a product bug.

3. TanStack Query: the defaults are a nondeterminism list

You said "don't know" on this one, so here is the mechanism rather than the answer. TanStack's documented defaults, read as an audit:

Compose the first two and you get the concrete answer to the question you couldn't answer:

Note what this is in the taxonomy from A1: it presents as async-wait, but the cause is concurrency over shared backend state, with the cache as the delay line that makes the two events overlap. Misfiling it costs you the fix.

4. Why app-idle absorbs this rather than fixing it

Your fixtures/app-idle.ts holds the test open at teardown until the app stops fetching, mutating and navigating. That is genuinely good engineering, and it solves a real problem: a request outliving its test and landing during the next one.

But look at when it acts. It is a teardown guard. It protects the next test from this test's in-flight work. It does nothing about a refetch that starts and lands in the middle of the current test, between two assertions — which is exactly the race above.

The word for the distinction is worth having: app-idle gives you quiescence at the boundary, not quiescence during. The fix for intra-test races is not a better guard, it is removing the reason a refetch happens at all — and that is a product config question (a non-zero staleTime for read-heavy lists), not a test one.

5. Time: the lever with a trap in it

page.clock installs a fake clock that intercepts Date, Date.now, setTimeout, setInterval, performance.now — and requestAnimationFrame. Zero uses in your suite.

Read that list against your suite and the trap is immediate. Your canvas gestures depend on real animation frames. dragBetween steps pointer moves across rAFs precisely so Gesto and Moveable register them; the actionability stable check from C1 needs two consecutive frames. Install a fake clock and both stop advancing unless you drive them.

One more trap, from the fake-timers engine underneath: advancing a clock does not flush promises. Fast-forwarding past a debounce fires the timer, but the await chain it kicks off still needs microtask turns. In a Playwright test your web-first assertion polls, so this mostly resolves itself — which is exactly why the bug, when it appears, is baffling.

6. Network: what you cannot currently test

Four route calls across the whole suite. The consequence isn't flakiness — it is a coverage hole shaped exactly like the real world:

• A 500 from an endpoint the user is mid-form on. A healthy backend will not produce one on request.
• A request that never resolves — the loading state that stays. tables/loading-state.spec.ts exists; how does it produce a slow response without a hard wait?
• route.abort() for offline. Note your own convention: an aborted request fires requestfailed, not response, so backendErrorGuard needs no declaration — which means the guard cannot see it.
• An optimistic update whose mutation then fails. This is the bug a manually-handled cache actually ships, and it is unreachable without failing a request on demand.

7. Bug hunt

Click the line that makes each test nondeterministic — or "no bug". One of the four is clean.

8. Free recall

Without scrolling: name the environmental inputs a browser test's result depends on, beyond the test code and the app code.

9. What this buys you this week

1. Pin locale and timezoneId in playwright.config.ts. Two lines, removes a whole input.
2. Find out what staleTime the app sets for the products and asset-library lists. If it is 0, you have located the intra-test refetch race — and the fix lives in the app, not the test.
3. Write one spec that could not exist before: route.fulfill a 500 on a product save and assert the optimistic update rolls back. Declare it through backendErrorGuard.
4. Add to e2e/CLAUDE.md: page.clock is forbidden in any spec that uses lib/editor gestures, because it fakes requestAnimationFrame. Write the reason down before someone discovers it the hard way.