hn.today

“…and the proof is checked in Lean”

xenaproject.wordpress.com226 points94 comments
Screenshot of “…and the proof is checked in Lean”

Lean verification can provide strong evidence that a mathematical result is correct, but compilation alone is not enough - especially for AI-generated code. The key first check is whether the theorem stated in Lean actually matches the claim in the paper. A statement can compile while being wrong, and understanding formalized statements may require examining extensive code defining unfamiliar mathematical objects. Some OpenAI theorem-dump projects include Lean formalizations only of intermediate results, not the papers’ headline claims; roughly 38 of the 290 projects with Lean code did not even state the main theorem. The code must also compile, which can require substantial computing resources. AI-generated code should be run in a sandbox because it is not automatically safe.

Even after those checks, reviewers must establish that the proof really proves the intended statement and uses no illicit axioms or hidden changes to definitions. The tool lake comparator can compare the theorem in a trusted file with the one proved through untrusted imports and check which axioms the proof relies on. To guard against bugs in Lean’s official kernel, a paranoid mode can seek agreement from multiple independent kernels, including a formally verified one. These checks make a proof overwhelmingly credible, though they do not guarantee that a paper’s written argument is correct: the Navier-Stokes work illustrates that the Lean proof and its accompanying write-up can diverge. In some cases, translating the formal proof back into readable mathematics may be more reliable than trusting the paper’s exposition.

Read on xenaproject.wordpress.com94 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Programming

Aristotle Expands into Rust

Aristotle has expanded its formal verification platform to Rust, allowing developers to prove code meets specifications and contains no undefined behavior, including in unsafe blocks. It can also verify that Rust rewrites behave like their original C code, targeting critical software such as smart contracts and infrastructure. (harmonic.fun)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.