# Claude Code agent designs LLM quantisation recipes that beat public baselines

An MIT and NVIDIA preprint lets a coding agent search for the best way to compress each reasoning model. It gains about one to two accuracy points over the strongest public recipes for an 8-hour search on 8 GPUs, and slightly beats NVIDIA's own compressed release of a 320B model.

- Category: Efficiency
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-19
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/claude-code-agent-designs-llm-quantisation-recipes-that-beat-public-baselines

## The study

QDA: Closed-Loop LLM Quantization Design with Agents (preprint, not peer reviewed). arXiv 2611.23388. https://arxiv.org/abs/2611.23388

## At a glance

- What it shows: An AI coding agent can design, test and refine the settings used to shrink a language model so it is cheaper to run. The resulting models lose a little less accuracy than ones made with the best published methods, and they run on standard serving software.
- What it does not show: It does not show that the gains hold for other models, other agents or other hardware, and nobody outside the authors has repeated it. The code is promised but not yet released.
- Why it matters: New open models arrive every few weeks, and each currently needs an expert to hand-tune its compression. Automating that step could make cheaper, more accurate compressed models available sooner.

## What the study shows

Quantisation stores a model's numbers in fewer bits, for example 4 instead of 16, so it needs less memory and runs faster. The catch is lost accuracy. Many published methods reduce that loss, and they can be stacked. The exact mix of methods, settings and calibration data is called a recipe.

The authors, from MIT and NVIDIA, first show that no single recipe wins everywhere. On one Qwen model, the AWQ method beat GPTQ by about 5 points at 4-bit but lost by about 12 points at 3-bit. So each new model and precision needs its own search.

Their system, called QDA, hands that search to Claude Code running Claude Opus 5.5. The agent proposes a recipe, a shared library builds it, and an evaluator scores it. It then decides what to change next. Each search had 64 GPU-hours and cost $19 to $75 in agent fees.

The key trick is a fast score. Fully testing a compressed reasoning model takes 23 to 86 GPU-hours. Instead, the evaluator measures how far the compressed model's next-word predictions drift from the original's, a quantity called KL divergence. This takes minutes. The texts it uses are hidden from the agent, so the agent cannot game them. For a few promising candidates, the agent also asks for long answers to catch failures such as endless loops.

The authors report that across four reasoning models, QDA beat the strongest public recipe on average by 0.9 points at 4-bit weights, 1.4 points when activations are also cut to 8-bit, and 2.2 points at 3-bit weights. Tests covered maths, science, coding and instruction following.

On the 320B-parameter GLM-5.3-Flash, compressed to NVIDIA's 4-bit NVFP4 format, one 6.5-hour search kept 98.3% of the original's accuracy. NVIDIA's own release kept 96.9% and Red Hat AI's 96.3%. All QDA models run on existing serving kernels with no added slowdown.

This is a preprint and has not been peer reviewed. On our readiness scale it sits at level II, Benchmarked: the results use public benchmarks and public baselines, but the code is not yet available.

## Our reading

In our reading, the headline gains are modest but real where it counts. At 4-bit, one point of accuracy is close to the noise of a single search. At 3-bit, where compression usually hurts most, the gains are larger and more convincing. The clearest case is the DeepSeek-R1-Distill-Llama-8B model, where QDA beat plain GPTQ by about 6 points at 3-bit.

The more useful lesson may be about the setup rather than the agent. When the authors gave the same agent NVIDIA's ModelOpt toolkit instead of their library and evaluator, it did worse, even though it was allowed to select on the test benchmarks. When they swapped the agent for a simple automated search over the same library, it lost only up to about one point. The fast hidden score and the reusable library do much of the work.

A practical example: a team that wants to serve a newly released open model at 4-bit could run one overnight search on a single 8-GPU server. It would get a checkpoint that slots into its existing vLLM setup, instead of waiting weeks for a hand-tuned recipe.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/claude-code-agent-designs-llm-quantisation-recipes-that-beat-public-baselines

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
