Anthropic’s Formal Verification Work Makes Fuzz Tests Part Of The Story

🔍 Read the full analysis: Anthropic’s Formal Verification Work Makes Fuzz Tests Part Of The Story on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and formal-verification company Theorem say they used Claude to prove no-crash properties for 239 fuzz-test entry points across 123 open-source libraries, checking the compiled programs at the RISC-V machine-code level. The effort also surfaced bugs the teams say earlier fuzzing had missed, but the proofs cover specific builds and conditions—not every version or use of the libraries.

The reported property is bounded and specific: for each of the 239 entry points, the compiled RISC-V program does not crash on inputs under 64 MB, including through an AddressSanitizer memory error or a failed assertion, subject to the project’s stated assumptions. The work starts with existing fuzz tests rather than a new, broad definition of what each library should do. Those tests become specifications for properties that can be checked against the compiled program.

In the pipeline described by Anthropic, the library and fuzz test are compiled into a RISC-V executable and instrumented with AddressSanitizer, which makes certain improper memory accesses trigger a crash. Theorem’s formal models of RISC-V instructions and Linux processes help produce a specification. Claude then writes a proof in Rocq, a proof assistant whose checker verifies whether the proof follows from that specification. When an attempt fails, the failure can point to a possible bug; Claude may patch the code and attempt a proof against a regenerated specification.

Anthropic says the project ran over several weeks and generated about 180 million lines of Rocq proof. A typical proof was about 600,000 lines, while the median executable contained 18,000 RISC-V instructions. The company says it is publishing proofs that did not require patching, some previously unpublished trusted components, and specifications for fuzz tests the system could not prove. The reported scale is notable, but the proof checker—not a human reading every proof line—is the mechanism used to verify the formal arguments.

What Machine-Checked Fuzz Tests Add

Fuzzing tests software by sending it many generated or modified inputs and watching for failures. It can find serious flaws, but testing a large number of inputs does not establish what happens for every possible input. A formal proof can establish a stated property across the modeled cases, which is why this approach could add a different kind of assurance to software testing.

The reported bug discoveries illustrate the gap. Anthropic says Claude found hundreds of bugs, including cases involving large inputs, unusual data layouts, or particular sequences of calls that may be difficult for a fuzzer to reach. The source describes, among other examples, a decompressor counter underflow associated with an out-of-bounds write on an input of about 20 MB, and an image-decoder read requiring a 64 MB input and more than 2 GB of memory. These are examples reported by the project, not evidence that every discovered issue has been independently reproduced or fixed.

The approach may also apply to properties beyond memory safety when a fuzz test checks them. Anthropic gives a compression test that asserts a decompressed result matches the original input: proving the test cannot crash can, under the formalized conditions, establish that round-trip behavior. That could make existing tests a practical starting point for verification. At the same time, proving code this way remains resource-intensive, and a result is only as broad as its specification, model, and assumptions.

From Fuzzing to Rocq Proofs

Anthropic frames the work against a security problem: software can contain bugs even after extensive testing, and vulnerabilities can be combined into attacks. The source cites recent Linux kernel vulnerability disclosures as an example of the scale of the problem. That observation is the company’s rationale for pursuing stronger guarantees; it does not mean the reported experiment verified the Linux kernel.

Formal verification has traditionally required substantial specialist effort. The source cites the seL4 microkernel as an example, saying its development took two person-years and its verification took 20 person-years. Anthropic and Theorem’s experiment instead uses a language model to produce proof scripts, with Rocq checking them mechanically. According to Anthropic, about 180 million lines of proof did not need human review line by line; confidence rests on the proof checker, formal models, theorem statements, and other trusted components.

The distinction between a test and its proof matters. A fuzz test generally treats a crash or failed assertion as a finding, while a fuzzer explores only the inputs it generates. Here, the property is attached to a particular compiled program and entry point. Anthropic says the work is, to its knowledge, the first large-scale application of AI formal verification to production software at the assembly-code level. That description is the company’s characterization of the project.

“discovered hundreds of bugs”

— Anthropic

Where the Proofs Stop

The results do not establish that the libraries are bug-free. They cover single-threaded, static RISC-V binaries linked against musl, with inputs below 64 MB and under the assumptions used by the formal models. The source says most users run x86 or ARM builds linked against glibc, so the reported proofs do not directly verify those commonly deployed configurations.

Other limits include the properties being checked and the behavior modeled. The method proves that the selected fuzz tests do not crash under the specified conditions; it does not automatically prove that every function behaves correctly, that all security vulnerabilities are absent, or that the software is safe in every deployment. The source also describes a logic flaw involving a file-loading restriction that was not always enforced, illustrating that relevant issues may not fit neatly into a no-crash guarantee.

Cost and scalability remain open questions. Anthropic says some proofs can consume hundreds of millions of tokens, while newer models and shared proofs may reduce the requirement. The source does not provide a complete cost breakdown, comparative results against other verification methods, or independent confirmation of the full bug count. It also does not specify, for each reported issue, its disclosure status or whether a fix has been released.

Publication and Broader Deployment

Anthropic says it is publishing the proofs that required no code patch, trusted components that were not already public, and specifications for tests it could not prove. Those materials should allow outside researchers to examine the theorem statements, assumptions, and proof artifacts. The source does not give a publication schedule for every component or say when the work will be repeated on other architectures.

The next practical test is whether the method can become cheaper and extend beyond the particular RISC-V and musl configuration. Wider use would require evidence about performance across other build targets, the reliability of model-generated proofs, and how teams handle defects discovered during the process. Until then, the report presents formal verification as a supplement to fuzzing and other security testing, rather than a replacement for them.

Readers should also look for follow-up disclosures from the library maintainers and for independent scrutiny of the released proofs. Those developments could clarify how many findings were confirmed, patched, or already known, and whether the proof pipeline can support routine verification at a lower cost.

Key Questions

What did Claude prove?

Anthropic and Theorem say Claude produced Rocq proofs that 239 selected fuzz-test entry points across 123 open-source libraries would not crash for inputs under 64 MB, in the specified compiled RISC-V setup and subject to the project’s assumptions.

Does this mean the libraries are bug-free?

No. The proofs cover particular properties, entry points, binaries, and input limits. They do not establish that every library version, architecture, or behavior is free of defects.

How is this different from fuzz testing?

Fuzzers try many generated inputs and can find crashes, but they do not necessarily reach every possible path. A formal proof can establish a specified property across the modeled inputs, although the result depends on the specification and assumptions.

What bugs did the project report finding?

Anthropic says the effort found hundreds, including issues involving large or unusual inputs and specific call sequences. The source gives examples such as an out-of-bounds write associated with a decompressor counter underflow and a read issue in an image decoder.

Can the proofs be applied to software people commonly run?

Not directly to every common build. The reported proofs cover single-threaded, static RISC-V binaries linked against musl; the source says typical deployments often use x86 or ARM with glibc. Applying the method there would require verification of those configurations.

Source: ThorstenMeyerAI.com