Skip to navigation Skip to main content Skip to footer
Research

What the 13% Looks Like: A Case Study

By Steven van der Baan

24 July 2026

What the 13% Looks Like: A Case Study

This is the fifth piece in a series. The first, The Cartographer's Advantage, made the philosophical argument for why human judgment is structurally irreplaceable in security testing. The second, The Expedition Debrief, asked what that looks like in practice. The third, Teaching the Map-Makers, examined what AI's training corpus contains, and what it leaves out. The fourth, The Symmetry Problem, showed that both offensive and defensive AI navigate from the same map, and left a question standing: what happens on the terrain where both maps agree to end? This piece tries to answer it, with a name, a number, and a technique.

In 2025, James Kettle (Director of Research at PortSwigger, the company behind Burp Suite) presented work at Black Hat and DEF CON under the title "HTTP/1.1 Must Die." The paper argued, convincingly, that HTTP/1.1's handling of request boundaries is architecturally unfixable, that the ambiguity built into how the protocol was written creates desynchronisation vulnerabilities that cannot be patched individually across a complex enough stack, only removed at the root by moving upstream to HTTP/2.

The work produced more than $200,000 in bounties in roughly two weeks. One finding gave effective control over approximately 24 million websites through CDN cache poisoning. Several major platforms, T-Mobile, GitLab, Netlify, were affected by variants described in the research. Two CVEs followed: CVE-2025-32094 on an Akamai desync exploited through a specific combination of the OPTIONS method and obsolete header line folding; CVE-2025-49005 on a Next.js cache poisoning via middleware rewrite key separation. 

I want to use this work to answer the question I left standing at the end of the previous piece.

The Technique That Wasn't in the Corpus

HTTP request smuggling is not new. The basic vulnerability class, where a client and a server disagree about where one HTTP request ends and the next begins, was documented in Watchfire research in 2005 and revisited periodically over the following two decades. Anyone building a security AI in 2024 would have found it in every corpus worth training on.

What Kettle's 2025 work introduced were new variants that extended the technique into territory that was not in that corpus: the 0.CL desync (an inversion of the known CL.0 pattern), Expect-header-based desync as a vector that most defences weren't watching, browser-powered desync that uses browser behaviour to trigger server-side smuggling, and chunk-extension tricks that bypass current WAF defences. These variants were not refinements on known payloads. They were the product of understanding how five different systems, origin servers, CDNs, reverse proxies, caching layers, and browsers, each parse the same HTTP request, and then asking what happens in the gaps between those interpretations. 

That is a different cognitive operation than matching a known pattern against a target. 

To find a known request smuggling variant on a test target, you need a scanner with current signatures and a server running a vulnerable stack. The scanner has both. To find the Expect-header desync variant, you need a model in your head of how five different parser implementations interact across a multi-tier infrastructure that you've never seen before, and you need to look at a combination that nobody has published yet, because it hasn't been published yet.

 No training corpus contained that model. It couldn't have. The technique didn't exist until someone built it.

This is what the 13% looks like.

Reading the Numbers Correctly

The 87%/13% benchmark has appeared in various forms across this series, the rough proportion of documented versus novel or target-specific conditions where AI pentesting tools perform reliably versus where they don't. XBOW's performance on HackerOne offers a different angle on the same split: in June 2025, XBOW became the first non-human entity to top the leaderboard, with over 1,000 submissions in 90 days at speeds roughly 85 times faster than a human tester. Of those submissions, independent analysis found approximately 37.5% were valid confirmed findings. The remainder were false positives, duplicates, or informative notes.

I am not citing those figures to dismiss what XBOW built. Volume at speed on documented vulnerability classes is genuinely valuable, it's the fast, consistent surface sweep that frees human testers to do something else. The 37.5% validity rate on a real-world bug bounty programme is not a failure of the tool; it's a reflection of what the tool is optimised for. XBOW's co-founder has said directly that XBOW excels at volume, not business impact. That's honest, and it's the right framing. 

But put the two data points side by side: XBOW at scale producing volume, and Kettle producing a single technique that controlled 24 million websites. The contrast isn't about which is more impressive. It's about what kind of security value each represents, and whether your programme is structured to generate both. 

The $200,000 in bounties is not actually the interesting number. The interesting number is the 24 million websites, the scale of the finding that a single researcher produced by going into terrain that no automated tool was looking at. High-impact security findings, the ones that don't just win bug bounties but reshape how entire protocols are understood, concentrate at the boundary of the unmapped. That's where Kettle was working. It's where the AI tools weren’t and couldn't have been.

Why Faster Doesn't Reach Here

The response I hear most often when I raise this kind of argument is some version of: "yes, but AI is improving rapidly. Give it time."

I want to be precise about why that response misses the point, because it's not an argument against AI improvement, AI security tooling is improving rapidly, and that matters. The issue is structural. 

An AI tool improves by learning from examples. It gets better at HTTP request smuggling by training on more HTTP request smuggling variants, documented ones, the kind that make it into CVE databases and bug bounty corpora and conference papers. It gets faster, more comprehensive, better at generalising across the known variants. That's real improvement, and it's happening.

What it cannot do is learn the Expect-header desync before the Expect-header desync exists. The technique was novel because it wasn't derived from known examples, it was built by a researcher who understood the underlying systems well enough to ask a question that hadn't been asked before. The path to that question is not more training data. It's the kind of systems thinking that produces new training data for everyone else. 

This is the structural ceiling. On terrain that has been mapped, where examples exist, where patterns have been documented, AI tools improve continuously. On terrain that has not yet been mapped, where the finding doesn't exist yet, where the technique is waiting to be discovered by someone who understands the underlying systems, the improvement curve doesn't apply. There's nothing to train on, because the insight hasn't existed until a human creates it. 

Kettle's work will now train the next generation of tools. The Expect-header variant is in CVE databases, in conference proceedings, in every corpus that gets assembled next year. Future AI systems will find it reliably on targets that are vulnerable to the known signature. That's how the cycle works, human insight becomes documented terrain, documented terrain becomes training data, training data becomes automated capability. 

The frontier, by definition, is always where the training data isn't.

What This Means for How You Build a Security Programme

The practical implication is not "don't use AI security tools." That conclusion is as wrong as the opposite one.

It's that two different things need to be funded, and the industry has become better at articulating the value of one of them, AI-augmented coverage, while the value of the other has become harder to defend in a procurement conversation. The other thing is research: the investment in people who understand the underlying systems well enough to ask questions that don't have answers yet.

Kettle found the Expect-header desync because he spent years, as someone whose entire professional focus is the research frontier of web security, building a deep model of how HTTP stacks interact. That's not something you buy in a tool licence. It's something you develop in people over time, protect from the pressure to optimise solely for delivery throughput, and give the space to follow technical curiosity into terrain that might produce nothing for months before producing something that changes how the whole field understands a protocol. 

I'm not making a sentimental argument about human creativity. I'm making a structural one. The attack surface that nobody is looking at, the terrain where both the offensive AI and the defensive AI have agreed, without any intention, to stop, is also the terrain where the highest-impact vulnerabilities live. If your security programme is entirely oriented around what the AI tools can find, you're well-defended on the mapped terrain. The unmapped terrain is being tested by nobody. 

For buyers building or reviewing an AI-first security posture, a useful question to ask your provider is not whether their tooling can match what XBOW does on documented classes, most competent platforms can. The question is what the humans in their practice are doing that the tools can't. Who is working at the edge of the map? What are they finding there? How does that knowledge propagate back into the assessment methodology? 

If the answer to that question is "we let the tools handle it," that's useful information.

The Question Part Four Left Standing

I ended the previous piece in this series with a question I said I kept returning to: what happens on the terrain where both maps agree to end?

The Kettle research is one answer. Someone goes there. They go with the specific kind of expertise that takes years to build, that can't be compressed into a training run, that requires understanding the systems underneath the systems, and they come back with something that rewrites the map. 

The part worth sitting with is that this kind of work happens precisely because someone understood a system well enough to notice that the map was wrong. Kettle knew what the existing HTTP desync literature said the attack surface looked like. He also knew enough about how parsers behave to suspect the map was incomplete. The finding followed from the suspicion. 

Korzybski's observation has run as a thread through this series. The map is not the territory; the representation is not the thing. In the first piece, I used it to argue that AI-generated attack maps have structural gaps that can only be identified by someone who knows the territory. This piece is meant to show what that argument looks like when it's true, when a specific researcher, working in specific unmapped terrain, finds a specific technique that generates $200,000 in bounties and a finding that reaches 24 million websites. 

The map got redrawn. It took a human to do it.

References