ARCHITECTURE Signal 423
How Netflix Scaled Its Real-Time Service Map
↗
Netflix restructured the ingestion pipeline behind its Service Topology graph into three independent stages, swapping gRPC for server-sent events and pushing backpressure all the way back to Kafka instead of dropping records.
For teams running their own dependency-mapping or graph-from-telemetry systems, the interesting decisions are not Netflix's but the trade-offs they imply: trading freshness for completeness, replacing gRPC streaming with SSE under heavy backpressure, and using consistent hashing against a service registry to avoid a rebalance protocol. Only one feed carried the story, so the framing should be treated as Netflix's own engineering narrative rather than independent reporting.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
The pipeline was split into aggregation, intermediary resolution, and enrichment-plus-persistence so that hot intermediaries no longer concentrate I/O and CPU on a small set of instances.
Backpressure now propagates from the graph store through Apache Pekko Streams up to the Kafka consumer, pausing consumption rather than dropping flow records, which favors delayed freshness over an incomplete map.
gRPC was removed from internal stage-to-stage transport in favor of server-sent events, which Netflix describes as lighter at its scale, while the gRPC client API for Service Topology was left untouched.
THE CLUSTER