Formal Verification's Hidden Assumptions
Formal verification has entered the mainstream through AI, bringing genuine economic value and widespread adoption. Yet the pervasive claim that systems are “formally verified” glosses over a fundamental problem: verification itself rests on unverified foundations—the definitions, mathematical models, and prover tools that enable the proofs.
Testing generates human-auditable evidence (bug reports that anyone can review), while formal proofs depend entirely on their own unexamined assumptions. Any flaw in those assumptions collapses the proof. AI has accelerated formal verification output roughly a thousandfold, far exceeding the capacity of traditional peer review. The author argues for mandatory specificity: each claim must name exactly what properties were proven and which components constitute the trusted base. Binary-level verification offers one path forward by reducing unverified assumptions through analysis at the machine-code layer rather than source code.
The field shows genuine promise for well-defined domains like single-threaded systems, but only if practitioners abandon vague “formally verified” language in favor of precise, caveated descriptions of what has actually been proven and what remains uncertain.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.