A huge chunk of the internet went down because a database query returned a few too many rows. Not a hack, not a traffic flood — a config file grew slightly too big, and a single line of defensive code that wasn't defensive enough did the rest.
On November 18, 2025, Cloudflare suffered a major outage. The trigger was almost insultingly mundane, and that's exactly why it's worth studying: the most catastrophic failures rarely come from exotic causes. They come from a small, internal, "trusted" thing quietly violating an assumption nobody wrote down.
Cloudflare's proxy — the program that sits in front of millions of websites — loads a "feature file" that configures its Bot Management. That file is generated internally by a query against a database. A routine database permissions change caused the query to start returning duplicate rows, which made the feature file bigger than expected — bigger than a hard-coded limit of 200 features the proxy was built to accept.
When the proxy hit that limit, the code didn't shrug and carry on. It crashed. And because every request flows through that proxy, the crash became errors for everyone.
The fatal detail is one line. The Rust proxy parsed the feature file and called .unwrap() on the result — a pattern that says "I'm certain this can't fail; if it does, panic." When the oversized file blew past the 200-feature limit, that assumption was false, the unwrap() panicked, and the proxy returned 5xx errors across the board.
It got eerier: the errors oscillated roughly every five minutes. Good and bad versions of the feature file were being regenerated and propagated in turn, so the system flickered between healthy and broken — the signature of a bad artifact moving through a distribution pipeline, which makes diagnosis maddening because the system keeps 'recovering.'
Cloudflare's own post-mortem actions read like a checklist every team should steal: treat internally-generated configuration like untrusted input (validate it before consuming), add global kill switches to disable a misbehaving subsystem fast, and prevent one component's failure (including diagnostic dumps) from exhausting resources and cascading.
Worth knowing
The phrase to internalise is "trust boundary." Engineers carefully validate external input but wave through internal data as inherently safe. This outage is the counterexample: the dangerous input was generated by Cloudflare's own pipeline. Anything your code consumes — even your own config — can violate an assumption, and the place you assumed it couldn't is exactly where the unwrap() hides.
The gap it reveals
Most engineers think outages come from traffic or attacks. The mature realisation is that internal data violating an unstated invariant — a config file exceeding a hard-coded limit — is a leading cause, and that defensive habits (validate internal inputs, fail soft not hard, kill switches) matter as much as scaling. Recognising that a lone unwrap()/assertion on 'impossible' data is a latent outage is hard-won operational wisdom.
In the interview room
In a design round, narrating failure modes earns trust: "this component loads a generated config — what if the config is malformed or oversized? I'd validate it, cap blast radius, and fail soft rather than crash the request path." Referencing the idea that internal inputs deserve the same suspicion as external ones signals you design for the outage you didn't see coming.
The reframe
Resilience isn't only about surviving the big, obvious threats — it's about not letting a tiny, internal surprise become a global one. The discipline is assuming every input can be wrong, every limit can be exceeded, and every 'this can't happen' eventually happens — then making sure that when it does, the failure is contained and soft, not total and hard.
The internet didn't fall to an attacker. It fell to an assumption nobody validated.
Primary source →
blog.cloudflare.com — Cloudflare Outage on November 18, 2025