GitHub’s Karthik Raman and Brian Celenza describe a storage redesign built around Git’s split between immutable objects and mutable references. Objects can be stored independently. Updating a branch pointer is where writers need agreement.
The design makes that boundary explicit:
- Store data before publishing pointers. A worker uploads a pack of Git objects to Azure Blob Storage. It then commits the pack manifest and reference changes together.
- Keep agreement small. Ref-CAS, a replicated reference log, locks only the references a transaction touches. Concurrent writes to separate branches can proceed in parallel.
- Make unchanged reads cheap. A worker checks its cached version against the log. If the repository hasn’t changed, the response carries no reference data.
- Separate maintenance from requests. Dedicated workers merge packs and verify that the replacement contains exactly the same objects. New packs enter the manifest before old ones leave; deletion happens later.
The failed prototypes are worth reading too. One shared storage blob serialized writes. Splitting references across blobs improved writes, but forced reads to check every segment and wait for the slowest. That trade hurt a workload where reads vastly outnumber writes.
GitHub reports 7,200 pushes per second in internal testing; the architecture is still in limited private preview. The enduring idea is to isolate the small state that needs coordination, then give request handling and background maintenance separate room to grow.