Lean verification can provide strong evidence that a mathematical result is correct, but compilation alone is not enough - especially for AI-generated code. The key first check is whether the theorem stated in Lean actually matches the claim in the paper. A statement can compile while being wrong, and understanding formalized statements may require examining extensive code defining unfamiliar mathematical objects. Some OpenAI theorem-dump projects include Lean formalizations only of intermediate results, not the papers’ headline claims; roughly 38 of the 290 projects with Lean code did not even state the main theorem. The code must also compile, which can require substantial computing resources. AI-generated code should be run in a sandbox because it is not automatically safe.
Even after those checks, reviewers must establish that the proof really proves the intended statement and uses no illicit axioms or hidden changes to definitions. The tool lake comparator can compare the theorem in a trusted file with the one proved through untrusted imports and check which axioms the proof relies on. To guard against bugs in Lean’s official kernel, a paranoid mode can seek agreement from multiple independent kernels, including a formally verified one. These checks make a proof overwhelmingly credible, though they do not guarantee that a paper’s written argument is correct: the Navier-Stokes work illustrates that the Lean proof and its accompanying write-up can diverge. In some cases, translating the formal proof back into readable mathematics may be more reliable than trusting the paper’s exposition.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.