# Property-Based Testing Resources

⭐ marks the core set. Batch 1 added 2026-10-06.

## Which property? — catalogues of property shapes

- ⭐ [Article: "Choosing properties for property-based testing" — Scott Wlaschin (F# for Fun and Profit, 2014)](https://fsharpforfunandprofit.com/posts/property-based-testing-2/)
  The seven patterns: **different paths, same destination** (commutativity), **there and back
  again** (inverse / round-trip), **some things never change** (invariant), **the more things
  change, the more they stay the same** (idempotence), **solve a smaller problem first**
  (structural induction), **hard to prove, easy to verify**, and **the test oracle**.
  Use for: the vocabulary of the whole path. Every exercise names one of these.
- ⭐ [Paper: "How to Specify It! A Guide to Writing Properties of Pure Functions" — John Hughes (TFP 2019, LNCS 2020)](https://research.chalmers.se/en/publication/517894)
  ([PDF](https://research.chalmers.se/publication/517894/file/517894_Fulltext.pdf)). Five
  approaches — **validity**, **postcondition**, **metamorphic**, **inductive**, **model-based** —
  run against eight deliberately buggy implementations. Result: validity properties miss five of
  eight bugs; model-based and metamorphic properties catch nearly all of them.
  Use for: which kind of property actually finds bugs; the "construct a case whose outcome is easy
  to predict" trick.
- [Article: "Choosing properties in practice, part 3: Properties for a dollar object" — Scott Wlaschin](https://fsharpforfunandprofit.com/posts/property-based-testing-5/)
  The closest thing in the series to a domain object: setter/getter inverse, idempotent set, and a
  `map`-based "different paths" property that ends up reshaping the API.
  Use for: applying the patterns to a value object instead of a list.
- [Article: "The Enterprise Developer from Hell" — Scott Wlaschin](https://fsharpforfunandprofit.com/posts/property-based-testing/)
  Why example-based tests can be satisfied by a wrong implementation, and how properties stop it.
  Use for: motivation; the `add` example.

## When is it worth it? — evidence from practice

- ⭐ [Paper: "Property-Based Testing in Practice" — Goldstein, Cutler, Dickstein, Pierce, Head (ICSE 2024)](https://www.cis.upenn.edu/~bcpierce/papers/icse24-pbt-in-practice.pdf)
  30 interviews at Jane Street. PBT is used *opportunistically* in **high-leverage** scenarios:
  **differential/model-based** (17/30, the most common), **round-trip** (11/30), **classical**
  (11/30), **catastrophic-failure** (7/30). Mantra from one participant: *"most useful when… you
  have a really good abstraction with a complicated implementation."* Hidden mutable state and
  interaction with the outside world make properties harder to find.
  Use for: the decision rule — is there a cheap abstraction to compare against?
- [Article: "Metamorphic Testing" — Hillel Wayne (2019)](https://www.hillelwayne.com/post/metamorphic-testing/)
  Testing without knowing the right answer: transform the input, predict how the output must
  change. Includes metamorphic bugs found in the Spotify and YouTube web APIs via pagination and
  ordering relations.
  Use for: list endpoints, search, filtering, pagination — the most CRUD-shaped material found.

- ⭐ [Paper: "Metamorphic Testing of RESTful Web APIs" — Segura, Parejo, Troya, Ruiz-Cortés (IEEE TSE 44(11), 2018)](https://idus.us.es/handle/11441/73547)
  Six **metamorphic relation output patterns (MROPs)** for any API with filtering, ordering and
  pagination: **equivalence, equality, subset, disjoint, complete, difference**. 60 relations on
  Spotify and YouTube found 11 issues, 10 confirmed (e.g. page size changing the result count;
  a filter that *added* rows). Assumes CRUD semantics explicitly.
  Use for: every list-endpoint property; the source/follow-up vocabulary.
- [Paper: "Metamorphic Testing: A Review of Challenges and Opportunities" — Chen et al. (ACM CSUR 51(1), 2018)](https://hub.hku.hk/handle/10722/254895)
  Survey by the technique's originators. Use for: definitions, MR identification, the wider literature.

## Stateful / model-based testing — PBT for CRUD resources

- ⭐ [Docs: "Model based testing" — fast-check](https://fast-check.dev/docs/advanced/model-based-testing/)
  TypeScript API. A **command** has `check(model)` (may it run now?) and `run(model, real)` (do it
  to both, assert they agree). `fc.commands` generates and shrinks sequences of commands.
  Use for: every code exercise involving a repository, service or API.
- [Article: "Rule Based Stateful Testing" — David R. MacIver (Hypothesis, 2016)](https://hypothesis.works/articles/rule-based-stateful-testing/)
  Generating whole programs of operations rather than single inputs; rules, bundles, invariants.
  Use for: the conceptual model behind fast-check's commands, from the Hypothesis author.
- [Paper: "Deriving Semantics-Aware Fuzzers from Web API Schemas" — Hatfield-Dodds & Dygalo (ICSE 2022)](https://arxiv.org/abs/2112.10328)
  Schemathesis: PBT generated from an OpenAPI schema, including stateful sequences via OpenAPI
  links. Finds 1.4–4.5× more unique defects than the next-best fuzzer.
  Use for: the catastrophic-failure property applied to a whole HTTP API for near-zero effort.

## Gaps

- No high-trust, worked write-up of PBT on a database-backed CRUD service (repository + real DB).
  The Elixir Forum thread "Property Testing a CRUD API" and Quviq's QuickCheck examples come
  closest; neither is authoritative.