OBSERVABILITY Signal 433
Datadog's gitretriever sustained 20× CI traffic growth while holding median latency near 40 ms
↗
Datadog built gitretriever, a Git mirror serving code to CI workloads, which now handles over 100 million weekly requests with median latency around 40 ms despite 20× traffic growth since launch.
The core problem was a scaling mismatch: read traffic grows with CI jobs but write costs grow with replicas in a replicated architecture, so adding nodes made things worse. AI coding agents drove an order-of-magnitude increase in Git traffic, pushing traditional scaling approaches past their limits for organizations with large monorepos.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
gitretriever served over a billion Git requests and hundreds of terabytes of code in its first 4 months, now handling more than 100 million requests weekly.
Previous fixes, adding capacity, larger instances, dedicated backends, CDN proxies, failed because replication overhead scaled with node count rather than read demand.
AI coding agents hit Git far harder and more often than human contributors, driving an order-of-magnitude traffic increase on top of existing deployment, auditing, and security load.
THE CLUSTER