engineering
Building a CI cache that does not throttle
How we serve cache restores at 1 GB/s, and why bandwidth matters more than capacity.
Priya Raghavan · Infrastructure · 2026-04-23 · 9 min read
Cache discussions focus on capacity — the 10 GB cap, eviction policy, hit rate. In practice bandwidth is the constraint people feel and rarely name.
Why restore speed dominates
A cache entry only helps if restoring it is faster than recreating it. At a low enough transfer rate, every cache becomes a net loss, and it will report a perfect hit rate the whole time.
What we built
- Cache stored on NVMe in the same availability zone as the runner pool
- Content-addressed chunks so unchanged parts are not re-transferred
- zstd compression, chosen for decompression speed rather than ratio
- Parallel chunk fetch saturating the link rather than a single stream
The result is roughly 1 GB/s sustained restore. That changes which things are worth caching at all — at that rate, caching a 6 GB toolchain directory becomes obviously correct rather than marginal.
Your next build could be twice as fast, at half the price
Start free. Migrating away is the same one line, and we publish that diff too.