# K2-Horizon releases six open models with their full training recipe

A team at MBZUAI has published six language models for agent work, from phone-sized to server-sized, together with the data, code and checkpoints used to build them. The largest is competitive with open rivals on tool use, but trails closed models and has weak defences against hidden instructions.

- Category: Language models
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-17
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/k2-horizon-releases-six-open-models-with-their-full-training-recipe

## The study

K2-Horizon: A Frontier Fleet of Open Source Language Models (preprint, not peer reviewed). arXiv 2611.21003. https://arxiv.org/abs/2611.21003

## At a glance

- What it shows: An Abu Dhabi research institute built six AI language models designed to carry out multi-step tasks, and released almost everything used to make them. On the team's own tests, the biggest model matches or beats most similar open models on tool-use and coding tasks.
- What it does not show: It does not show the models beat the best closed commercial systems overall, and no organisation is documented using them in a real product. The test results come from the authors, not from independent checks.
- Why it matters: Most open models share only the finished product. Here the data, code and training history are also public, so others can study, reproduce or adapt the process instead of starting from scratch.

## What the study shows

The K2 Team at the Institute of Foundation Models, part of Mohamed bin Zayed University of Artificial Intelligence, released a family of six models called K2-Horizon. This is a technical report on arXiv. It is a preprint and has not been peer reviewed.

The models range from 0.9 billion parameters, small enough for a phone, to 375 billion. The largest is a mixture-of-experts model, where only part of the network runs for each word. It uses about 23 billion parameters per word. Alongside the final models, the team says it released 380 intermediate checkpoints, about 21.6 TB of training data and 17 code repositories.

The authors compared the largest model with four open models of similar vintage and three closed ones. It led the open models on Toolathlon-Verified, a test of multi-step tool use, scoring 65.3. On Terminal-Bench 2.1, where an agent works in a live command-line session, it scored 70.2 and placed second among open models. Its clearest weakness was expert-level science, where it trailed the best open model by about 9 points on the HLE exam.

A mid-sized model tests a new design called Mixture-of-Value Attention, which applies the expert idea to part of the attention mechanism. It uses about one seventh of the active parameters of a dense 32B model trained on the same data. The authors report it stays within a few points of that model on every benchmark they ran.

The team also built Uno, small add-on modules that guess several words at once while the base model checks them. The authors say this keeps output identical to the base model. For the 7B model it gave about 1.5 times more throughput when serving 64 requests at once, and about 2.2 times for a single request.

About 10 trillion of the training words were synthetic, generated by other models and mostly not checked for correctness. The authors argue that wrong answers in long worked examples are ordinary noise, and that strict filtering would cut diversity. They also chose their data mix by predicting results at full scale. That reversed the choice that looked best in small test runs.

## Our reading

In our reading, the release itself is the main contribution. Benchmark wins are narrow and self-reported. But a full development history, from raw data to quantised builds, is rare at this scale. It gives researchers a way to study what each training step changes, which finished weights alone cannot do.

The report is also unusually frank about failures. One example: every coding trajectory placed the repository in the same folder. The model learned this habit so well that, when the folder moved, tasks took twice as many commands and the pass rate fell by 40%. Lessons like this are practical and transferable to any team building agent training data.

The performance picture is mixed. On agent tasks the largest model sits near the top of the open field with fewer active parameters than most rivals. On expert science and factual accuracy it is clearly behind. Against closed frontier models it trails on most rows of the authors' own table.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/k2-horizon-releases-six-open-models-with-their-full-training-recipe

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
