Proving a 32B language model's output, bit for bit, on one GPU

A new open-source system produces a cryptographic proof that a server ran a specific model and gave a specific reply. Proving takes about 50 times as long as answering, and the exact integer model it proves is somewhat less accurate than the original.

6 min readIIIOpen sourcePreprint, not peer reviewed
Save
Illustration for the article "Proving a 32B language model's output, bit for bit, on one GPU"AI illustration
Illustration: AI generated

What the study shows

A client that sends a prompt to a hosted language model gets text back and nothing else. It cannot tell which model wrote it, or whether the server used a cheaper version. Sable, from MoonMath.ai, attaches a cryptographic proof to the reply. This is a piece of evidence that anyone can check later with maths alone, without trusting the server or holding the model. The paper is a preprint and has not been peer reviewed.

A proof needs an exactly defined function. Normal model inference uses floating-point arithmetic, whose results shift slightly with the hardware and the batch. So the author defines an integer-only version of the model, called FO, in which every rounding step is fixed. A program converts a standard published checkpoint into FO. The server must then run FO, and the proof shows that FO produced the reply.

The author reports that one NVIDIA RTX PRO 6000 GPU proves all 64 layers of Qwen2.5-32B in about a minute and a half for a 4,096-token sequence. A 16,384-token sequence takes under seven minutes. The same prover handles two Qwen3 models, including a mixture of experts, a design where each token uses only a few of the model's internal sub-networks.

Proving takes 41 to 50 times as long as running the same request on the author's own engine for FO. Against a standard fast setup, 8-bit inference in the SGLang serving software, the factor is about 90. At public cloud prices, the author puts a 4,096-token proof at about 6 cents.

Exactness costs accuracy. Perplexity measures how well a model predicts text, and lower is better. FO's perplexity on a standard text set is 8.3% worse than the original model's. Common 8-bit formats lose under 2%. On the MMLU knowledge test, FO scores 1.4 points lower than the original. The author traces about a third of the text-prediction gap to the low precision used inside the attention step.

Proofs are large, from about 100 to almost 500 megabytes, and take under a minute to check on one processor core. They are not zero-knowledge, meaning they reveal parts of the computation. The author estimates that about 19,000 proofs would expose the model's weights. The system is therefore meant for openly published models and for verifiers such as auditors. The code is released under the MIT licence.

Our reading

In our reading, the main step is what the proof covers. Earlier checks of hosted models were approximate or statistical. Sable's statement is exact and covers the whole reply: every generated token, the sampling randomness and the point where generation stopped. A server that cut an answer short or swapped in a smaller model would fail the check.

The practical cost profile suits audits, not routine traffic. A proof taking minutes and hundreds of megabytes is too heavy for every request, but fine for spot checks. For example, an auditor could sample one request in a thousand from a provider that claims to serve Qwen2.5-32B. It could then check each sampled answer without needing the provider's hardware.

On our readiness scale this is level III, open source available. The prover, verifier and measurement scripts are public, and results are reported on public benchmarks. All measurements are the author's own on one machine, and no organisation is reported using it in production.

Free membership

Read the rest of this story

Membership is free. You get every article in full and the Issue every Sunday.

We send a sign-in link, no password needed. Already a member? Sign in

Readiness assessment

III
Open source
LevelStatusEvidence
I  Lab onlyMetThe system is a working research prover with a defined integer model, constraint system and soundness analysis.
II  BenchmarkedMetMeasured on real models (Qwen2.5-32B, Qwen3-32B, Qwen3-30B-A3B) with accuracy on WikiText-2, MMLU and GSM8K against BF16, FP8 and SmoothQuant. Proving time is compared per parameter and token with zkLLM and FlashProve. These are the authors' own runs. No third-party benchmark or independent replication is reported.
III  Open sourceMetProver, verifier, tests and measurement scripts released under the MIT licence at https://github.com/moonmath-ai/Sable. The paper does not report independent use or reproduction.
IV  In productionNot yetNo named organisation is reported using it in production. The proof is not zero-knowledge and is 106 to 477 MiB, which the authors say limits it to auditors of open-weight models.

What would move it up
Level 4 needs documented production use by a named organisation, such as an auditor verifying an open-weight model service. Practically, that likely needs zero-knowledge and recursive layers so proofs are small and private. Closing the accuracy gap to BF16 at small proving cost, and independent replication of the timings, would also help.

Checked automatically every week, last on 16 November.

The study

Tomer Solberg. Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models. arXiv, 10 November 2026. doi:10.48550/arxiv.2611.14083

Preprint, not peer reviewed · Licence CC BY

Notes

  1. Solberg, T. Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models. arXiv preprint, 10 November 2026, not peer reviewed. CC BY. Source
  2. Code: prover, verifier, tests and measurement scripts, MIT licence. Source
  3. Qwen2.5-32B proving times (Table 4, mean of three warm runs): 92.5 s for 1 x 4,096 tokens, 253.1 s for 4 x 4,096, 397.5 s for 1 x 16,384. Proof sizes 134.8, 239.0 and 345.7 MiB. Verification with compiled core on one CPU core: 18.8, 26.8 and 40.2 s.
  4. Qwen3 (Table 7): Qwen3-32B 121.8 s and Qwen3-30B-A3B 84.7 s for 4,096 tokens. Across all three models, proofs are 106 to 477 MiB and verify in 19 to 56 s.
  5. Overhead: 41 to 50 times the author's FO serving engine; about 90 times FP8 in SGLang. Cost: 6.4 cents per 1 x 4K proof at $2.50 per GPU-hour (CoreWeave price cited).
  6. Accuracy on Qwen2.5-32B (Table 12): WikiText-2 perplexity 5.580 for FO against 5.151 for BF16 (+8.3%); FP8 +0.7%, SmoothQuant INT8 +1.6%. Greedy-token agreement with BF16 86.3%. MMLU 81.8% against 83.2%; GSM8K 88.0% against 88.6%. 12-bit queries and keys would remove 37% of the gap (Table 13).
  7. Q/K split on Qwen3 removes 23% (Qwen3-32B) and 53% (Qwen3-30B-A3B) of the perplexity gap, at no proof cost. FO is within 1.6 points of BF16 on MMLU and 1.7 on GSM8K on both models.
  8. Soundness: round-by-round error 2^-103.4 in the random-oracle model, counting a 30-bit proof of work on the queries (Table 2). About 1.9 x 10^4 proofs would allow weight recovery (Section 4).
  9. Readiness: level III, open source available. Code and scripts released; public benchmark results are the author's own; no named production use.

Edited and signed by

Stefan PerdalEditor and publisher86 stories edited

Stefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.

How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

More on Safety & evaluation