A cheaper way to fine-tune small datasets without wiping out skills

A new preprint adds one extra term to standard fine-tuning: the model must stay loosely close to its own earlier self. On small datasets this kept more knowledge and often scored higher, for about 10% more training compute.

7 min readIIBenchmarkedPreprint, not peer reviewed
Save
Figure 1 from the paper "Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching"
Figure 1: the model being trained and a frozen copy of the original model both predict the next word; their predictions are flattened before comparison, and that comparison is added to the usual fine-tuning loss. Figure 1 from Giannis Daras et al., 2026, arXiv:2611.23229. CC BY 4.0.

What the study shows

Most language models are built in two stages. First comes pre-training on huge amounts of web text. Then comes fine-tuning on a much smaller, carefully chosen dataset, to teach a domain, a style or how to follow instructions. The common fine-tuning method is supervised fine-tuning (SFT), where the model learns to predict each next word in the new examples. With small datasets, SFT often overfits, meaning it learns the examples too closely, and the model loses skills it had before.

The authors, from MIT, Arizona State University and the University of Texas at Austin, propose Ambient SFT. A frozen copy of the original model acts as a teacher. The model being trained must roughly agree with that teacher, but only after both sets of predictions are blurred. The blurring uses temperature, a setting that flattens a model's word choices so it is less sure of any single word. At high temperature, the teacher only steers the broad shape of the answer, not the details. One setting controls how much the old model is trusted. In the main tests it was set to 2.5, and three in four training steps used plain SFT.

The authors report gains in three settings. In domain adaptation, a 1.5 billion parameter model was adapted to five kinds of text, such as code and news. Ambient SFT matched or beat the other methods on all five, measured by how well it predicted held-out text. The largest gain was on news, about 27% lower perplexity (a measure of prediction error). On two domains the gain was tiny, and on product reviews there was none.

In instruction tuning, the authors trained Qwen3 base models from 1.7 to 14 billion parameters on about 21,000 examples, 1% of a public dataset. Plain SFT improved instruction following but pushed most other scores, such as maths and general knowledge, below the untrained base model. Ambient SFT kept most of that knowledge. Its average score across eight public benchmarks was 8% to 20% higher than plain SFT, depending on model size. Gains also appeared on OLMo-3 and Llama-3.1 models. On the instruction-following test, IFEval, Ambient SFT scored lower than plain SFT at three of four sizes.

In reasoning, an OLMo-3 7 billion parameter model trained on about 22,000 worked-solution examples beat plain SFT on five of six benchmarks. The average gain was smaller, about 3.5%. The method was also compared with four other ways of limiting overfitting, and came out ahead in the authors' tests. The extra cost is one pass through the frozen teacher on a quarter of steps, which the authors put at about 1.1 times the training time of plain SFT.

This is a preprint and has not been peer reviewed. We rate it Level II, Benchmarked: it reports results on public benchmarks with a pinned evaluation setup, but the paper names no code or model release.

Our reading

In our reading, the useful idea is simple. Teams already know that fine-tuning on a small dataset can damage a model. The usual fixes are more data, earlier stopping, or penalties that keep the weights close to where they started. This paper offers a different anchor: keep the model's broad predictions close to its old self, while leaving it free to sharpen the details. The loss term is a few lines of code, and the pseudocode is in the paper.

The results are strongest exactly where the authors say they should be: little data, and a base model that already knows something about the target. Gains shrink as data grows. The authors report about 3% at 10% of the instruction data and about 9% at full data, against about 15% at 1% on the same model. On the domain-adaptation side, the base model's starting error on a domain ranked well with how much the method helped, though that rests on only five domains.

A practical example: a bank fine-tuning an open 8 billion parameter model on a few thousand internal question-and-answer pairs. With plain SFT, it might gain the house style but lose some arithmetic or general knowledge. The paper suggests Ambient SFT would keep more of that, at little extra cost. It might also follow formatting rules a little less tightly, which matters if outputs feed other software.

Free membership

Read the rest of this story

Membership is free. You get every article in full and the Issue every Sunday.

We send a sign-in link, no password needed. Already a member? Sign in

Readiness assessment

II
Benchmarked
LevelStatusEvidence
I  Lab onlyMetMethod, algorithm (Algorithm 1) and theory are described, with toy experiments on Tiny Shakespeare and GPT-2.
II  BenchmarkedMetResults on public benchmarks (MATH, GSM8K, HumanEval+, MMLU, PopQA, TriviaQA, IFEval, BBH, AIME, GPQA) using the public olmes harness at a pinned commit, against vanilla SFT and four tuned baselines, across Qwen3, OLMo-3 and Llama-3.1 models.
III  Open sourceNot yetThe text reads as if no code or weights link is given. The method is simple and pseudocode is in Algorithm 1, but no repository or checkpoints are named. Datasets used are public.
IV  In productionNot yetNo documented use by a named organisation in production.

What would move it up
To reach level 3 the authors would need to release training code and ideally checkpoints. To reach level 4 a named organisation would need to report using it in a real post-training pipeline. Independent replication at larger data scales, with seeds and error bars, would make the benchmark evidence stronger.

The study

Giannis Daras, Marios Papachristou, Adam Klivans et al.. Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching. arXiv, 17 November 2026. Read the paper

Preprint, not peer reviewed · Licence CC BY 4.0

Notes

  1. Daras, Papachristou, Klivans, Torralba and Daskalakis, 'Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching', arXiv preprint 2611.23229, November 2026. Not peer reviewed. CC BY 4.0. Source
  2. No code or weights link is given in the paper. Pseudocode appears as Algorithm 1 in Appendix B.
  3. Domain adaptation (Table 1): News validation loss 1.68 for Ambient SFT against 1.99 for the best baseline, reported as 26.7% lower perplexity. Reviews tied at 2.96. CS and BioMed margins were 0.01.
  4. Instruction tuning (Table 2): average relative gain over vanilla SFT of +20.3% (Qwen3-1.7B), +8.0% (4B), +10.2% (8B) and +8.7% (14B), on about 21,500 examples, 1% of Dolci-Instruct-SFT. Table 4: OLMo-3-7B +15.4%, Llama-3.1-8B +8.3%.
  5. Data scale (text, Section 4.3): +15.4% at 1%, +3.4% at 10% and +8.6% at 100% of Dolci-Instruct-SFT on OLMo-3-7B.
  6. Regulariser comparison (Table 5, Qwen3-1.7B): Ambient SFT +20.3%, KD at T=1 +17.0%, L2-SP +15.3%, NEFTune +0.8%, label smoothing -0.3%.
  7. Reasoning (Table 6, OLMo-3-7B, about 22K examples of Dolci-Think-SFT): +3.5% average, wins on 5 of 6 benchmarks, GSM8K 93.0 against 94.2, AIME 2024 44.5 against 39.8.
  8. Settings: T=2.5 and pc=0.75 (25% of steps use the matching loss); training about 1.1 times slower per step than plain SFT, per the authors.

Edited and signed by

Stefan PerdalEditor and publisher86 stories edited

Stefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.

How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

More on Language models