The slowest 1% of your requests can ruin the experience for everyone, and you usually can't predict which request will be slow. Netflix's answer is almost cheeky: don't try to predict it. Just quietly ask a second time.
On stateful services, tail latency comes from unlucky moments — a replica mid-GC-pause, a node briefly overloaded. The request itself is fine; it just landed on a momentarily slow server. Waiting it out punishes the user for the server's bad luck.
Your data lives on several replicas. When you read, you ask one of them. Usually fast — but occasionally the replica you picked is having a bad moment (garbage collection, a load spike) and takes far longer than normal. You're now stuck waiting on a slow server, even though other replicas could have answered instantly.
Request hedging is the fix: if your first request is taking unusually long, send the same request to a second replica and use whichever answers first. You're hedging your bet against one server's bad luck.
The naive version — always send two requests — doubles your load. Netflix's version is disciplined: the server advertises a target and max latency per namespace (its SLO), and the client only fires a hedge request after the target elapses (say, the p95). So for the 95% of requests that are normal, nothing extra happens; the backup fires only for the slow tail. Hedges are themselves concurrency-limited so a wave of slowness can't trigger a self-inflicted load storm, and timeouts are GC-tolerant so a normal pause isn't mistaken for failure.
The non-negotiable precondition: idempotency. Hedging means the same operation may execute more than once, so it must be safe to repeat — exactly the property Stripe's idempotency keys guarantee. Hedging and idempotency are a matched pair; you cannot safely do the first without the second.
Worth knowing
Notice the threading-together of three earlier teardowns: hedging fights the GC pauses that pushed Discord off Go (#2), it's concurrency-limited using the same thinking as Netflix's adaptive limits (#6), and it's safe only because of idempotency (#12). These mechanisms aren't isolated tricks — they compose. Senior design is mostly knowing how the pieces fit, not memorising each in isolation.
The gap it reveals
Engineers obsess over average latency; the tail is what users feel. The realisation is that p99 is often caused by transient bad luck (GC, momentary load), not a slow request — so you can attack it by retrying against another replica, but only after a threshold and only if operations are idempotent. Knowing hedging without knowing its idempotency precondition is how you double-charge a customer.
In the interview room
"How would you reduce p99 latency?" rewards the candidate who says more than 'add a cache.' Hedging — send a backup request after the SLO target, use the first response — is a strong answer, provided you immediately add 'and this requires idempotent operations and a concurrency cap on hedges.' Stating the precondition unprompted is the senior signal.
The reframe
You can't eliminate the random bad moments inside a large fleet — some server is always mid-pause somewhere. So instead of trying to make every replica perfectly fast, Netflix designed around the certainty that some will be slow at any instant, and routed around it cheaply. Resilience is accepting the inevitable variance and building a graceful escape from it.
Don't predict which request will be slow. Make it cheap to stop waiting on the one that is.
Primary source →
infoq.com — How Netflix Ensures Highly-Reliable Online Stateful Systems