# Proving a 32B language model's output, bit for bit, on one GPU

A new open-source system produces a cryptographic proof that a server ran a specific model and gave a specific reply. Proving takes about 50 times as long as answering, and the exact integer model it proves is somewhat less accurate than the original.

- Category: Safety & evaluation
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-12
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/proving-a-32b-language-model-s-output-bit-for-bit-on-one-gpu

## The study

Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models (preprint, not peer reviewed). DOI 10.48550/arxiv.2611.14083, arXiv 2611.14083. https://arxiv.org/abs/2611.14083

## At a glance

- What it shows: A single graphics card can produce a mathematical proof that a hosted AI model gave a particular answer, exactly as claimed. It works for models with 32 billion parameters, and each proof takes one to seven minutes.
- What it does not show: It does not keep the model or users' prompts private. The proofs are too large to send with every answer. It proves a slightly less accurate integer version of the model, not the original.
- Why it matters: Customers of hosted AI cannot currently check which model answered them, or whether an answer was cut short. This makes checkable proofs realistic for audits of openly published models.

## What the study shows

A client that sends a prompt to a hosted language model gets text back and nothing else. It cannot tell which model wrote it, or whether the server used a cheaper version. Sable, from MoonMath.ai, attaches a cryptographic proof to the reply. This is a piece of evidence that anyone can check later with maths alone, without trusting the server or holding the model. The paper is a preprint and has not been peer reviewed.

A proof needs an exactly defined function. Normal model inference uses floating-point arithmetic, whose results shift slightly with the hardware and the batch. So the author defines an integer-only version of the model, called FO, in which every rounding step is fixed. A program converts a standard published checkpoint into FO. The server must then run FO, and the proof shows that FO produced the reply.

The author reports that one NVIDIA RTX PRO 6000 GPU proves all 64 layers of Qwen2.5-32B in about a minute and a half for a 4,096-token sequence. A 16,384-token sequence takes under seven minutes. The same prover handles two Qwen3 models, including a mixture of experts, a design where each token uses only a few of the model's internal sub-networks.

Proving takes 41 to 50 times as long as running the same request on the author's own engine for FO. Against a standard fast setup, 8-bit inference in the SGLang serving software, the factor is about 90. At public cloud prices, the author puts a 4,096-token proof at about 6 cents.

Exactness costs accuracy. Perplexity measures how well a model predicts text, and lower is better. FO's perplexity on a standard text set is 8.3% worse than the original model's. Common 8-bit formats lose under 2%. On the MMLU knowledge test, FO scores 1.4 points lower than the original. The author traces about a third of the text-prediction gap to the low precision used inside the attention step.

Proofs are large, from about 100 to almost 500 megabytes, and take under a minute to check on one processor core. They are not zero-knowledge, meaning they reveal parts of the computation. The author estimates that about 19,000 proofs would expose the model's weights. The system is therefore meant for openly published models and for verifiers such as auditors. The code is released under the MIT licence.

## Our reading

In our reading, the main step is what the proof covers. Earlier checks of hosted models were approximate or statistical. Sable's statement is exact and covers the whole reply: every generated token, the sampling randomness and the point where generation stopped. A server that cut an answer short or swapped in a smaller model would fail the check.

The practical cost profile suits audits, not routine traffic. A proof taking minutes and hundreds of megabytes is too heavy for every request, but fine for spot checks. For example, an auditor could sample one request in a thousand from a provider that claims to serve Qwen2.5-32B. It could then check each sampled answer without needing the provider's hardware.

On our readiness scale this is level III, open source available. The prover, verifier and measurement scripts are public, and results are reported on public benchmarks. All measurements are the author's own on one machine, and no organisation is reported using it in production.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/proving-a-32b-language-model-s-output-bit-for-bit-on-one-gpu

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
