XBOW topped HackerOne in 2025 and fixed nothing

XBOW, an autonomous offensive-security system, reached the top of HackerOne’s US leaderboard in 2025. Google’s Big Sleep agent reported an exploitable stack buffer underflow in SQLite before the code shipped. Both were announced as counts. Vulnerabilities found. Neither system publishes the second number - vulnerabilities fixed. That number does not exist because the fix is not the deliverable. The finding is.

This is the structural fact of vulnerability research. The incentive stack pays out at discovery and goes silent at remediation. Bug bounty pays on triage of a valid report. Pwn2Own scores the pop, not the patch. CVE assignment records that a flaw exists, not that it closed. Money, reputation, leaderboard rank - all settle at the moment something is found. Remediation is unpaid, unscored, and vendor-gated. So the metric every researcher and every automated finder optimizes is the count of things found. Frontier models made this asymmetry worse, not better. They scale the finding side. The fixing side stays human-paced, tied to patch cycles, regression suites, and enterprise deployment windows that move in weeks and quarters.

The deeper problem is not vanity metrics. It is what the metric reveals about how exploitation actually works. Attackers do not have oracle access to every latent bug in a codebase. They are not sitting on a comprehensive index of unfound flaws. Most of them watch a feed. The feed is CVE publication, the advisory text, the GitHub proof-of-concept, and the patch diff. Each of those is an intelligence product. Each one converts an unknown condition into a targetable one. The vulnerability was always there. What changed is that it became reported. Reporting is the trigger.

Consider the mechanism at the binary level. A vendor ships a patch. The patch is a delta. Bindiff the pre-patch and post-patch binaries, localize the changed function, and the modified bounds check or added validation call tells the analyst exactly where the flaw lived and what condition reaches it. The patch is a map to the bug it fixes. This is n-day development. The window opens the moment the fix is public and stays open until the fix is deployed everywhere. That gap - disclosure to deployment - is where the exploitation lives. The researcher’s find created the CVE. The vendor’s patch created the map. Neither created the exploitation. The report did.

Map this to real campaigns and the pattern holds. CVE-2021-44228, Log4Shell, CVSS 10.0 - JNDI lookup injection in Log4j, CWE-917 expression-language injection reaching remote class loading. Disclosed December 2021. Mass exploitation inside hours, not days. Scanning for the string in every reachable HTTP header and user-agent field started before most defenders had finished reading the advisory. The bug had shipped in production for years. It became weaponized the day it was named.

ProxyShell is cleaner still. CVE-2021-34473, CVSS 9.8, plus its two chain partners in Exchange. Orange Tsai presented the research at Black Hat USA in August 2021 without full exploitation detail. Within weeks, other researchers reconstructed the chain from the presentation and the patch, PoC hit public repos, and mass exploitation followed. The flaw was patched by Microsoft months earlier. The exploitation curve did not track the patch. It tracked the public reconstruction. The signal was the report, not the fix.

Citrix Bleed, CVE-2023-4966, CVSS 9.4 - sensitive information disclosure leaking session tokens from NetScaler ADC and Gateway. Quiet after the advisory. Then PoC went public, and a LockBit affiliate ran it against unpatched appliances at scale, replaying stolen session tokens to walk past MFA. ProxyLogon, CVE-2021-26855, CVSS 9.8 - the SSRF that anchored the Exchange chain. HAFNIUM used it as a zero-day. The volume event came after the March 2021 patch, when multiple unrelated groups reverse-engineered the fix and hit every Exchange server that had not yet applied it.

The counter-case proves the same rule from the other side. CVE-2023-34362, the MOVEit Transfer SQL injection, CVSS 9.8, CWE-89. Cl0p - TA505, FIN11 in overlapping attribution - exploited it as a true zero-day in May 2023, before any advisory existed, and ran mass exfiltration against hundreds of organizations. That is what a capable actor with an unreported bug looks like. They do not wait for the feed because they built their own. The distinction matters. A small number of well-resourced actors work off unreported flaws they discovered themselves. The overwhelming volume of exploitation - the opportunistic scanning that hits everyone - is disclosure-triggered. It is a reaction to reported vulnerabilities. Remove the report and that volume has nothing to react to.

This is why the finding-versus-fixing asymmetry is an operational problem, not a cultural complaint. Every published find is a starting gun. The research community, and now the automated systems piling onto bug-bounty leaderboards, are optimized to fire that gun as often as possible. MITRE frames the adversary side precisely. T1588.006, obtain capabilities: vulnerabilities. T1588.005, obtain capabilities: exploits. T1595.002, active scanning for vulnerabilities. T1190, exploitation of public-facing applications. The chain from a published CVE to a compromised host is a documented adversary workflow. The input to that workflow is the report. The finder produces the input and moves to the next target. The fix, if it happens, happens somewhere downstream, unmeasured, on someone else’s clock.

What this produces in telemetry is consistent and predictable. After a disclosure with public PoC, mass-scanning sensors light up. Shadowserver and GreyNoise-class collectors register a sharp rise in probes for the specific path, header, or parameter the PoC targets, sourced from bulletproof hosting and compromised residential ranges. WAF and reverse-proxy logs - Cloudflare-tier edge telemetry included - fill with the signature payload, often the literal PoC string copy-pasted with no modification. That is the tell. The early opportunistic wave is not customizing anything. It is running published code against the internet. On the host, the interesting events come after the probe lands. For a public-facing application exploit, that is the web service process spawning a shell interpreter - Sysmon Event ID 1, parent-child anomaly, w3wp.exe or a Java process spawning cmd.exe or bash. Then outbound connections the service never makes, Event ID 3, to infrastructure with no prior baseline.

The blind spot sits between the two. The scanning wave is loud and easy to detect and mostly useless to block after the fact, because by the time the correlation rule fires the mass exploitation is already in progress across every unpatched instance. The actual compromise - the payload that lands on a vulnerable host - is quiet relative to the noise around it. Detection engineering that alerts on scan volume is watching the starting gun. The round that hits is the process-lineage anomaly and the anomalous egress after the payload executes. Most environments tune for the former because it is high-volume and legible, and under-instrument the latter because it requires host telemetry and behavioral baselining the scan-counting dashboard does not need.

There is a second telemetry gap that the finding-not-fixing culture creates directly. Remediation has no equivalent signal. A find generates a CVE, an advisory, a scanner plugin, a news cycle. A fix generates a version bump in a manifest somewhere. There is no feed of what got patched, no leaderboard of closed exposure, no correlation rule that fires when a fleet finishes deploying. Defenders can see, in near real time, what the adversary now knows. They cannot see, with anything close to the same fidelity, what their own environment has actually closed. The asymmetry in the metric is also an asymmetry in visibility. The offense reads a rich, timely, structured feed of new capability. The defense reconstructs its own remediation state from patch-management exhaust that lags reality by days.

The technical reality after the patch ships does not resolve the way the CVE lifecycle implies. A CVE marked fixed means a fix exists. It does not mean the fix is applied. The patch boundary - Log4j 2.17.1, the relevant NetScaler build, the Exchange cumulative update - defines where the code is corrected. It says nothing about the population still running the vulnerable version, which for internet-facing appliances routinely stretches into months and for embedded and end-of-life systems never fully closes. Residual exposure is the difference between fixed-in-source and fixed-in-fleet, and that difference is exactly the exploitation window every one of the campaigns above lived inside.

The find is the easy half and the celebrated half. It is countable, announceable, and now automatable at scale by systems that will never touch a patch pipeline. The fix is the hard half, the unmeasured half, and the half that actually removes the exposure - and it happens after the report has already told the adversary where to aim. Attackers are not reacting to vulnerabilities. Vulnerabilities are silent. They are reacting to reports. The industry keeps score on the noise and pays no one for the silence. Until remediation is measured with the same fidelity as discovery, the feed will keep firing the starting gun faster than anyone deploys the fix. Found is a number. Fixed is the work no one counts.