INFRA Signal 503
Kubernetes probes explained: startup, readiness, and liveness checks prevent dropped requests and restart loops
An in-depth walkthrough demonstrates how Kubernetes' three probe types work, how to configure them, and how misconfiguration leads to traffic loss and unnecessary restarts.
Without probes, Kubernetes marks containers Ready the moment they start, sending traffic to processes that are still initializing and causing request failures. Understanding probe behavior is essential for reliable rollouts and avoiding CrashLoopBackOff scenarios that can take minutes to recover from. The article also surfaces a bug found in Kubernetes itself through its simulation approach.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Kubernetes provides three probe types, startup, readiness, and liveness, each serving a distinct purpose in container lifecycle management.
Without probes, containers are considered Ready immediately upon start, causing requests to fail during initialization periods.
The author ported over 100,000 lines of Kubernetes Go code to TypeScript to create interactive browser-based demos, and verified behavior against k3s.
THE READ
What the cluster adds up to.
The article's core argument is that probes are not optional decoration but a necessary mechanism for correct traffic routing. A container without any probe is marked Ready as soon as it starts, even if it is still performing initialization work and not listening on its service port. This means any client sending requests during that window will experience failures, whether from an ingress controller, a load balancer, or inter-service calls. The interactive demos make this visible by showing requests failing in real time when a container restarts without probe configuration.
Kubernetes offers three probe types that address different lifecycle stages. Startup probes determine whether the application has initialized, readiness probes determine whether it can receive traffic, and liveness probes determine whether it needs to be restarted. Each probe can use one of four mechanisms: httpGet (status codes 200-399 count as success), tcpSocket, exec, or grpc. Probes are executed by the kubelet on each node, with configurable parameters like periodSeconds and failureThreshold controlling check frequency and tolerance for consecutive failures before the container is killed.
The CrashLoopBackOff behavior is worth noting for anyone operating Kubernetes in production. By default, the delay between restart attempts starts at 10 seconds and doubles with each crash, capped at a maximum of 5 minutes. This means a misconfigured liveness probe that kills a slow-starting container can create a recovery cycle that takes significant time to resolve. The article's demo shortens this to 3 seconds for illustration, but the production defaults mean probe misconfiguration has real and potentially severe operational consequences.
A subtle but important detail emerges around how readiness actually works. Even when a pod with a startup probe shows as NotReady, direct requests to the pod's IP address still succeed in reaching the container and can still fail. The readiness mechanism only takes effect when traffic is routed through Kubernetes' service abstraction, not when a client bypasses it by targeting the pod IP directly. This distinction matters for debugging: if inter-service communication uses pod IPs rather than services, readiness probes will not protect against sending traffic to unready containers.
The author's methodology is itself noteworthy. Rather than describing probe behavior from documentation alone, they ported over 100,000 lines of Kubernetes Go code to TypeScript to build a simulated cluster that runs in the browser. They verified the demos against k3s and in the process found a bug in Kubernetes. This approach of executable, verifiable explanation is more rigorous than typical tutorial content and gives the findings a degree of credibility that prose-only articles lack.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER
↗