Delta is Meta’s object store for the layer that has to come back first when everything else is down — build artifacts, distribution packages, bootstrap data. That constraint drives the whole design. For a system whose job is restoring the fleet, complexity is only justified when it buys reliability, and performance is explicitly secondary.

It implements chain replication: servers arranged in a linear chain, writes entering at the head and reads served only by the tail.

  • Writes hit the head, persist locally, forward down the chain, and are acknowledged only after the tail has durably stored the data.
  • Reads go to the tail, so only fully replicated data is ever visible. Strong consistency falls out of the topology rather than a consensus protocol.
  • The head is always the write leader. No election, no quorum.

The trade-off is stated plainly rather than hidden:

  • Storage efficiency loses. Every host in a chain holds the whole dataset; erasure coding would be cheaper.
  • Average write latency loses. A write is done only when every link in the chain has persisted it.
  • Fault tolerance holds up. A chain of n nodes tolerates n − 2 failures, comparable to quorum replication.
  • Consensus complexity disappears. Quorum systems need consensus and leader election to maintain quorum; here the scope narrows to a chain-host mapping.

The operational detail is where the real engineering lives. Sibling hosts detect failures via heartbeats and failed acknowledgements, and a host is expelled once two peers suspect it — a limit of one would let two hosts vote each other out simultaneously. Timeouts were tuned by performance testing, because too short amplifies transient network blips and too long burns client latency. They deliberately accepted more false positives because repair is automated.

Two later additions are worth stealing. Apportioned queries let every node serve reads after checking with the tail whether their local copy is clean (committed everywhere) or dirty, which scales read throughput linearly with chain length without weakening the guarantee. And rejoin is asymmetric: a recovering host accepts new writes while it catches up but defers reads upstream until it is fully synchronised.

The reusable idea is that a linear placement plus a rule about which end answers converts an ordering problem into a topology problem — no quorum, no election, no reconfiguration protocol. You pay in write latency and disk, and Meta is honest that this is the trade.

The caveat is that the constants — chain lengths of four or more, a vote limit of two, standby hosts for chains missing over half their nodes — are tuned to a specific fleet. Simplicity at the architectural level gets re-complicated at the operational level, which is true of every distributed system that survives contact with production.