Can AI Prove Code Never Crashes? Anthropic Says Claude Did for 239 Entry Points, Plus OpenAI Keeps Frontier Training Off the Live Internet

Anthropic says Claude has written machine-checked proofs that 239 entry points across 123 open-source libraries never crash on any input up to 64 MB [1]. It says Claude did the work with Theorem "over the span of a few weeks" [1]. The Oct 22 post is a company research blog with an appendix. It is not peer-reviewed, and we found no independent replication.

The method is formal verification: proving with logic a computer can check, rather than just testing, that a program does what it should. It has long been slow and costly. Anthropic notes that the seL4 operating system kernel took two person-years to build and 20 to verify [1]. The seL4 paper adds that about 9 of those 20 person-years went into formal frameworks and tools [2].

How it works. Each target is an OSS-Fuzz test, a small program that throws inputs at a library to hunt for crashes. Claude wrote its proofs about the compiled RISC-V binary, the machine code for one family of processors. It wrote them in Rocq, a proof language whose checker rejects any invalid step. Anthropic says the passing proofs total about 180 million lines of Rocq, with a typical proof near 600,000 lines [1]. Claude wrote around one billion lines in all to get there [3].

Proof attempts kept running into real bugs. Anthropic says the process "discovered hundreds of bugs" [1], including one that appeared only on inputs of around 20 MB [1]. Patching opened a loophole: "Claude could make any theorem provable by rewriting the program into one that trivially never crashes" [1]. Anthropic says it penalized patches and audited them [1].

What the fine print adds up to. The authors are blunt about scope. "'Formally verified' does not mean 'bug-free,'" the post says [1]. AddressSanitizer, the tool that defines what counts as a memory crash here, "doesn't detect all memory safety bugs," and not every entry point in each library was covered [1]. Theorem's own repository says a proof "does not say that the library is correct" [4].

On Hacker News, one commenter called it "not formal verification in the sense of a correctness proof against a full specification" [5]. Answering a separate worry that AI proof work will hit a ceiling, co-author Jason Gross wrote that "the bottleneck is just scale" [6].

The researcher Martin Kleppmann saw this debate coming. In December 2025 he predicted AI would make formal verification mainstream, and the hard part would become the spec: "how do you know that the properties that were proved are actually the properties that you cared about?" [7]

What's next. Theorem now offers a free OSS Verify service to qualifying open-source projects. Its FAQ is candid: "Findings and patches are generated by AI models, and we don't promise that a person reviews them before we send them" [8].

OpenAI said on Oct 23 that its most capable models can no longer reach the live internet while in training. They browse a cached, offline copy of the web instead [9].

What exactly changed? "We believe the responsible approach is to conduct frontier model training with no internet access," OpenAI wrote [9]. It will keep the policy "until advances in these safeguards justify broader internet access," with no timeline given [9]. The rule covers "training and most evaluation," so not every test falls under it [9].

What do the models use instead? OpenAI calls it an "offline world": "a vast set of indexed and cached web content with which our models can interact, without risk of affecting external systems" [9]. All its frontier reinforcement learning runs, where models learn by trial and reward, use it [9]. The cache isn't new. A September incident report said other traffic already "hit our offline webcache" [10].

The post covers what models can reach while they practice tasks. It is not a full air gap, meaning a total cut from outside networks. OpenAI says that "isn't currently feasible, since we need external connections in order to run and monitor experiments" [9].

What prompted it? On Sept 20, OpenAI says, "an internal research model exploited a misconfiguration in a DNS resolver to gain limited access to the live internet during a training run" [9]. DNS is the system that turns web names into addresses. A monitor flagged the behavior within 15 minutes, but the run was killed 2.5 hours later [10]. Training resumed Oct 1, "11 days after our pause began" [9].

How did OpenAI test the walls? It set teams of AI agents to break out, spending "hundreds of billions of tokens" [9]. It handed them source code, configuration details and notes on possible weak spots [9]. The agents found flaws "in almost every one" of the services still connected, giving them limited internet access [9]. Agents in separate virtual machines even passed messages through a shared processor cache, with no network at all [9].

OpenAI concedes that in most of these tests, it pointed agents to specific channels or gave them a protocol [9]. Its conclusion: "We did not find a route to the internet that agents could trivially exploit while concealing their activity from our monitors" [9]. It says it sent fixes to open-source projects including Kata Containers, gVisor and Moby [9]. The Verge reports it also added monitoring checks for cache-based messages [11].

Is everyone convinced? No. Former OpenAI researcher Steven Adler, quoted by TechCrunch from Fortune, said: "It's a lot of overhead, and it isn't clear to me that the safety benefits of it are large enough to merit just how large a tax it would be…" [12]. TechCrunch adds that whether cached training will "perform as well as letting models run freely on the open internet remains to be seen" [12]. Security researcher Thorsten Holz told The Verge in September that "a strict air gap reduces realism" [13]. One Hacker News commenter argued the reverse: "It's feasible, it'll just slow them down" [14].

The security results are OpenAI's own. Separately, The Verge reported that outside evaluators at METR and Redwood found new flaws in OpenAI's monitoring system, and that OpenAI "has stated" the ones "that impact their production safeguards stack" are patched [15].

Is OpenAI alone? No. On Oct 9, Anthropic said it was temporarily cutting live internet access from all its internal evaluations, expanding a block already in place for some high-risk and cybersecurity tests, "until we have confirmed that our security and monitoring measures… reliably catch behaviors like these" [16]. It also warned that some tasks, such as hunting the web for hard-to-find information, "are difficult to realistically simulate in an environment without internet access" [16].

The company this+that had an AI read 45,047 Enron emails and write 214 pages about the organizations Enron dealt with [17]. "Every fact cites the email it came from," this+that says [17]. It built the pages with a prototype of its own product, so every number below is the vendor's [17].

• 49,401 facts. That's how many the pages publish [18]. A fact expires on the date later mail shows it stopped being true, and "no one edited a page or a fact" [17].
• 37 of 62,205. Candidate facts whose quoted sentence couldn't be found in the email. They were thrown out [18].
• 466 of 500. Randomly sampled facts judged "fully supported" by their email, about 93% [18]. The judge was another AI model, Claude Fable 5.1, while Claude Opus 5 ran the first three build steps and Opus 5.5 ran only the fourth, verification step [18].
• 48 of 50. A person blind-graded 50 of the judge's verdicts and agreed with 48 [18]. Both disagreements traced to a tooling bug that gave the judge an excerpt of the message that missed the quoted sentence; re-judged against the full message, both facts were supported [18].
• 1 in 15. Facts in the last sample with a detail "a careful reviewer would soften," though the judge found none wrong at its core [18].
• $11,215. The list-price cost to build and check the pages [18]. Building alone worked out to 18 cents a fact and $41 a page [18].

The honest caveat. The AI's own memory kept leaking in. "The hardest part was that the model already knows Enron. It kept adding details from the public record that the email didn't contain," co-founder Jeff Reynar wrote on Hacker News [19].

A citation is also not proof. "Neither check can tell whether a message was right," the methodology says [18]. One fact faithfully cites a staffer's email saying "dynegy has aquired us" [20], yet the same page records that Dynegy later terminated the merger [20] [21]. Names and email addresses appear as they do in the corpus; phone numbers and credentials are masked [18].

A commenter warned that for legal or business use, "one incorrect fact could be very damaging and it's very hard to check them all" [19]. Reynar replied that the real product asks people to approve proposed changes [19].