How large language models are trained, aligned and improved: new research on reasoning, training data and architectures, rated for how close it is to use.
Reflection's first open-weight model, Beam, runs only a small part of itself for each word yet scores close to much larger open models on coding and agent tests. The evidence comes almost entirely from the company's own preprint.
A new preprint adds one extra term to standard fine-tuning: the model must stay loosely close to its own earlier self. On small datasets this kept more knowledge and often scored higher, for about 10% more training compute.
A preprint tests 29 open-weight models and finds that, once randomness is turned up, the rule that picks each next word can matter as much as the model itself. Coding agents went from about half of issues fixed to none on that one setting.
A team at MBZUAI has published six language models for agent work, from phone-sized to server-sized, together with the data, code and checkpoints used to build them. The largest is competitive with open rivals on tool use, but trails closed models and has weak defences against hidden instructions.
NVIDIA has retrained Google's small Gemma 4 model to follow instructions and use tools better, while keeping replies short. It fits in about 4 GB of memory and comes with a helper that speeds up replies.
An Amazon team describes a set of checks that test each part of a language model training run against a trusted reference. Over two months, it found 61 problems across five new models, and about half raised no error at all.