# Data Synchronisation Resources

⭐ marks the core set. Batch 1 and batch 2 added 2026-09-11 after Martin chose directions A (command log)
and C (CRDTs + local-first). Replication theory (Kleppmann's notes) is kept as the shared foundation.

## The command log and the authoritative server — "sync engine" practice

- ⭐ [Docs: "How Replicache Works" — Rocicorp](https://doc.replicache.dev/concepts/how-it-works)
  The cleanest written statement of the model Martin sketched. **Mutators** are named functions;
  a **mutation** is a record of one invocation with its arguments; the client keeps a **pending**
  list; the server is authoritative; on **pull** the client *"rewinds the state of the Client View
  to the last version it got from the server, applies the patch … then replays any pending
  mutations on top"* — explicitly compared to `git rebase`. Also: *"it's possible and common for
  mutations to calculate a different effect when they run during rebase."*
  Use for: canonical names for dirty/committing/committed; the rebase loop; why the server runs the
  same mutator.
- ⭐ [Blog: "How Figma's multiplayer technology works" — Evan Wallace (2019)](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/)
  ([archived copy, 2023-02-23](https://web.archive.org/web/20230223081016/https://www.figma.com/blog/how-figmas-multiplayer-technology-works/))
  Rejected OT and full CRDTs; a central server is the single ordering authority; per-property
  **last-writer-wins** registers; the client keeps an **unacknowledged** flag so its own optimistic
  edit is not overwritten by a broadcast that arrives first. Concrete failure cases (tree
  reparenting cycles, undo across users).
  Use for: what "server decides" buys; the acknowledgement problem; where LWW is acceptable.
- ⭐ [Talk + post: "Scaling the Linear Sync Engine" — Tuomas Artman (2023)](https://linear.app/now/scaling-the-linear-sync-engine)
  ([YouTube](https://www.youtube.com/watch?v=Wo2m3jaJixU)). Normalised object graph in memory
  (MobX), a **transaction queue** to the backend, partial sync, and the operational problems that
  appear at scale (bootstrap size, delta sync, ordering).
  Use for: a production-scale instance of the command-log model; what changes at scale.
  Secondary: [reverse-engineering write-up — wzhudev](https://github.com/wzhudev/reverse-linear-sync-engine)
  (community, detailed, not official — cross-check against Artman).
- ⭐ [Docs: "Writing Data" — Zero (Rocicorp)](https://zero.rocicorp.dev/docs/writing-data)
  Replicache's successor; same vocabulary. **Mutator** (the function), **mutation** (*"a record of the
  mutator having run with certain arguments"*), **pending mutations**, **server-authoritative**. Adds
  the full lifecycle: client execution → optimistic display → push → server execution in a
  transaction → replication to `zero-cache` → client **reconciliation**, where each pending
  mutation's optimistic effect is *rolled back* and replaced by the authoritative rows.
  Use for: the most explicit written lifecycle of one mutation; the "rollback then replace" detail.
- [Docs: "Overview" — TanStack DB](https://tanstack.com/db/latest/docs/overview)
  Different words for the same model: **optimistic state** layered over immutable **synced data**;
  a **transaction** groups mutations; *"if the handler throws an error, the optimistic state is
  rolled back."* The two-loop framing — *"an instant inner loop of optimistic state, superseded in
  time by the slower outer loop of persisting to the server and syncing the updated server state
  back."*
  Use for: seeing the same architecture named without Replicache's git metaphor.
- [Guide: "Writes" — ElectricSQL](https://electric.ax/docs/guides/writes)
  A ladder of four **write patterns**: online writes → optimistic state → shared persistent
  optimistic state → **through-the-DB sync** (local **shadow tables** holding local changes beside
  synced tables). Uses **rebase** for reapplying local state over incoming synced changes.
  Use for: the pattern ladder — each rung is a design a team might stop at, and the guide says what
  each one costs.
- [Gist: "Building an offline realtime sync engine" — Paulus Esterhazy](https://gist.github.com/pesterhazy/3e039677f2e314cb77ffe3497ebca07b)
  A practitioner's checklist: queueing offline, retry, ordering, de-duplication, rebase. Medium
  trust (a gist), but a good inventory of the concerns.
  Use for: a checklist to audit a design against.
- [Article: "Event Sourcing" — Martin Fowler (2005)](https://martinfowler.com/eaaDev/EventSourcing.html)
  and [bliki: CQRS](https://martinfowler.com/bliki/CQRS.html). *"All changes to application state
  are stored as a sequence of events."* Source of the terms **command** vs. **event**, **projection**,
  **replay**, **snapshot**. Martin's "frontend state is a projection over state + commands" is this
  pattern applied at the client.
  Use for: the command/event distinction and the word *projection*, with its provenance.

## Operational Transformation — the brief sketch only

- [Paper: "Concurrency control in groupware systems" — Ellis & Gibbs (SIGMOD 1989)](https://www.lri.fr/~mbl/ENS/CSCW/2012/papers/Ellis-SIGMOD89.pdf)
  Origin of OT (the GROVE editor). Each site applies its own operation immediately; a remote
  operation is **transformed** against concurrent local ones before applying, so index-based edits
  land in the right place. Requires causality preservation and, in practice, a central server to
  fix the order (Google Docs). Read the abstract and §1 only — the mission caps OT at a sketch.
  Use for: what OT *is*, in one paragraph, and why Figma called it "unnecessarily complex."

## Replication theory — what has to be true for replicas to agree

- ⭐ [Lecture notes: "Distributed Systems" — Martin Kleppmann, Cambridge Part IB (2021–22)](https://www.cl.cam.ac.uk/teaching/2122/ConcDisSys/dist-sys-notes.pdf)
  ([videos](https://www.classcentral.com/course/youtube-distributed-systems-lecture-series-53062),
  [source](https://github.com/ept/dist-sys), CC BY-SA). Lectures 3–4: happens-before, Lamport and
  vector clocks, broadcast ordering (FIFO / causal / total). Lecture 5–8: replication, quorums,
  consensus, **state-based vs. operation-based CRDTs**, eventual consistency, collaboration software.
  Use for: the theory under every engine above; the ordering guarantees a command log actually needs.
- [Book: _Designing Data-Intensive Applications_ — Kleppmann, ch. 5 "Replication"](https://dataintensive.net/)
  Leader-based vs. multi-leader vs. leaderless; **replication lag** anomalies (read-your-writes,
  monotonic reads, consistent prefix); the multi-leader conflict-resolution catalogue.
  Use for: naming the anomalies an optimistic client can show its own user.

## CRDTs

- ⭐ [Report: "A comprehensive study of Convergent and Commutative Replicated Data Types" — Shapiro, Preguiça, Baquero, Zawirski (INRIA RR-7506, 2011)](https://hal.inria.fr/inria-00555588/)
  The founding paper. Defines **CvRDT** (state-based, merge is a join on a semilattice) and
  **CmRDT** (operation-based, concurrent ops commute); **strong eventual consistency**. Catalogue:
  counters, LWW-register, G-Set, 2P-Set, OR-Set, graphs, sequences.
  Use for: the definitions, and the register/set types you would actually use in an app.
- [Paper: "Conflict-free Replicated Data Types" — Shapiro, Preguiça, Baquero, Zawirski (SSS 2011)](https://www.lip6.fr/Marc.Shapiro/papers/2011/CRDTs_SSS-2011.pdf)
  The short conference version of the report above; defines **Strong Eventual Consistency** (SEC):
  replicas that have received the same updates are in the same state, with no conflict resolution
  and no roll-back. Read this one first, the report for the catalogue.
- [Tutorial series: "An introduction to Conflict-Free Replicated Data Types" — Lars Hupel](https://lars.hupel.info/topics/crdt/01-intro/)
  Eight interactive parts in the browser: lattices, state-based types, combinators. Rigorous,
  short, executable.
  Use for: building intuition for *join* and *monotonicity* by hand before reading Shapiro.
- [Blog series: state-based and operation-based CRDTs — Bartosz Sypytkowski](https://www.bartoszsypytkowski.com/tag/crdt/)
  Starts at [state-based](https://www.bartoszsypytkowski.com/the-state-of-a-state-based-crdts/),
  then [op-based protocol](https://www.bartoszsypytkowski.com/operation-based-crdts-protocol/) and
  [op-based registers and sets](https://www.bartoszsypytkowski.com/operation-based-crdts-registers-and-sets/).
  Practitioner-grade; makes the delivery requirements of op-based CRDTs (causal, exactly-once)
  concrete.
  Use for: the operational side Shapiro leaves abstract — what the transport must guarantee.
- [Docs: "Conflicts" — Automerge](https://automerge.org/docs/reference/documents/conflicts/)
  The only case Automerge cannot merge: concurrent writes to the same property; it picks a
  deterministic winner and keeps the losers retrievable. A production statement of what
  "conflict-free" does and does not mean.
  Use for: correcting the common misreading that CRDTs "resolve" conflicts.
- ⭐ [Talk: "CRDTs: The Hard Parts" — Martin Kleppmann (Hydra 2020)](https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html)
  Interleaving anomalies, moving elements, tree reparenting, undo — the cases where "it converges"
  is not the same as "it converges to what the user meant."
  Use for: the honest cost side of CRDTs; pairs directly with Figma's reasons for not using them.
- [Talk + code: "CRDTs for Mortals" — James Long (dotJS 2019)](https://www.youtube.com/watch?v=DEcwa68f-jY)
  ([crdt-example-app](https://github.com/jlongster/crdt-example-app),
  [annotated fork](https://github.com/clintharris/crdt-example-app_annotated)). A complete small
  system in JavaScript: **hybrid logical clocks** for timestamps, per-field LWW over a message log,
  Merkle tree to detect divergence. The design behind Actual Budget.
  Use for: the smallest end-to-end CRDT sync you can read in an afternoon; TypeScript-adjacent.

## Local-first

- ⭐ [Essay: "Local-first software: You own your data, in spite of the cloud" — Kleppmann, Wiggins, van Hardenberg, McGranaghan (Ink & Switch, Onward! 2019)](https://www.inkandswitch.com/essay/local-first/)
  The seven ideals (no spinners, multi-device, offline, collaboration, longevity, privacy, user
  control) and a scorecard of existing architectures against them. Argues CRDTs are the enabling
  technology; reports on the Automerge prototypes and what was hard.
  Use for: defining "local-first" precisely; deciding which ideals a given product needs.

- [Keynote: "The past, present, and future of local-first" — Kleppmann (Local-First Conf 2024)](https://martin.kleppmann.com/2024/05/30/local-first-conference.html)
  Five years on: the developer-side benefits, the reclassification of what counts as local-first,
  and the proposal that **sync servers become a commodity** alongside peer-to-peer sync. Where
  Linear-style engines sit relative to the 2019 ideals.
  Use for: placing server-authoritative engines and CRDT-based systems on one map.

## Gaps

- **No neutral source comparing "server-authoritative command log" (Replicache/Linear/Figma) with
  "peer CRDT" (Automerge/Yjs) as design choices.** Each vendor argues its own case. Lessons must
  present the trade-off as synthesis and cite both sides.
- **Terminology for the client-side command lifecycle is not standardised.** Replicache and Zero say
  *mutator / mutation / pending / server-authoritative*; Figma says *unacknowledged*; TanStack DB
  says *optimistic state over synced data*, grouped in a *transaction*; Electric says *optimistic
  state*, *shadow tables*, *rebase*. No paper fixes the words. Lessons adopt the Replicache/Zero set
  and show the mapping.
- **Rejection handling** — what to do with pending commands that depend on a rejected one — is
  under-documented everywhere. Needs synthesis.