Provable Inference

Provable inference aims to let users verify that an AI response came from a particular model, without access to its private weights or the computing resources needed to run it themselves. The proposed method uses zero-knowledge succinct proofs: a provider commits to specific model weights and supplies a compact certificate with each response. Users can check the certificate efficiently, while it reveals no information about the weights; the model’s architecture is not hidden. This could help customers detect substitutions by routing services, auditors confirm which model was evaluated, and security teams identify tampering or unauthorized model replacement. Regulators could also use proofs to check whether claimed inference workloads were served by the specified model, though such checks would not establish that a provider avoided training other models at the same time.

The research introduces FlashProve, an open-source prototype for generating these proofs. For the 400-billion-parameter Llama 4 Maverick model in 4-bit quantization, it reports proof generation above 100 tokens per second on one H100 GPU, with a roughly 19 MB proof for a 1,024-token request. Generating proofs takes about 70-95 times the GPU time required to run the model using vLLM, but is substantially faster than earlier open-source systems. The release includes a technical report, protocol specification, prover, two verifiers, and Lean formalizations of top-level security claims. The work is presented as an intermediate step intended to invite scrutiny and further development, not as a finished solution.