Cloudflare’s Rust DNS platform (Big Pineapple) holds over 250 billion cache entries at any given time — so wasting a single byte per entry costs 250 GB of RAM across the fleet. Five successive changes to how entries are stored cut the per-entry footprint by more than half and freed roughly 100 terabytes, equivalent to the RAM in 130 of their Gen 13 servers.

The optimizations, each small and obvious in retrospect:

  • Vec<T> and String carry an 8-byte capacity field that is dead weight for immutable cache entries — Box<[T]> and Box<str> drop it plus the heap over-allocation Vec reserves for growth (64 bytes per entry, over 15 TB fleet-wide)
  • Answer, authority, and additional sections merged into one list with u16 offsets instead of three separate lists (28 bytes per entry)
  • RecordData enum was sized to its largest variant (NAPTR, 144 bytes) while A/AAAA records — 80%+ of traffic — need 4–16 bytes; boxing the rare large variants recovered ~120 bytes per common record
  • Record owners identical to the queried domain are dropped and inferred from the cache key at read time
  • Record data stored as raw wire bytes in a single contiguous Box<[u8]> with 2-byte length prefixes — kills per-variant allocations, and most record types copy straight into the outgoing response without re-serialization

The measured results:

  • Per-entry footprint: 953 → 420 bytes (-56%); per-entry allocations 1.1 KB → 461 bytes
  • ~100 TB freed fleet-wide; p99 resident memory dropped from 9.3 GB to 5.3 GB per instance
  • Insert throughput up 43%, lookup latency down 19% — the cache got faster, not slower

The methodology matters as much as the changes: a custom allocator wrapper tracked per-entry allocations in benchmarks, and production resident memory was verified across percentiles during a staged two-month rollout.

A textbook death-by-a-thousand-cuts optimization — and a reminder that fewer allocations and better memory locality usually win on space and speed at the same time. The freed memory is being reinvested into cache capacity rather than smaller bills.