How we cut CDN metadata lookup latency by 91%

Vercel changed the way the CDN retrieves routing metadata from fetching one object per path to fetching a bounded shard that holds many paths. A Bloom filter check precedes the fetch, letting the CDN skip lookups for paths that are known to be absent. When the filter passes, the CDN retrieves the shard, warms the cache for all its paths, and then locates the needed entry locally. This shift turned many individual cache misses into a single warm-fill operation.

Adopting the shard approach adds build-time work: the system must create JSONL files, compute inline indexes, and encode pointers as six-bit Base64 characters. At runtime the CDN must perform a binary search on those pointers and decode only the matched line, which adds a small computational step compared with a direct object fetch. The extra code and memory for the indexes increase the deployment footprint slightly.

The technique works best when projects contain hundreds of thousands of paths and are deployed frequently, because the amortized cost of a shard fill is spread over many lookups. For very small projects with only a few paths the overhead of managing shards and indexes can outweigh the latency gain. If the bound on shard size is set too low, the grouping advantage diminishes, and if individual metadata entries are large, a shard may still exceed desirable fetch sizes.

As a result, Vercel measured a 91% reduction in P99 metadata lookup latency and observed faster deployments because each request now requires fewer round trips to origin storage. The CDN continues to process over 80 million routing instructions per second, but the average cost per instruction dropped due to the reduced metadata fetch overhead. These gains persist as long as the shard size remains tuned to the typical metadata volume per deployment.