# Reflection's open-weight agent model nears bigger rivals on SWE-bench and Terminal-Bench

Reflection's first open-weight model, Beam, runs only a small part of itself for each word yet scores close to much larger open models on coding and agent tests. The evidence comes almost entirely from the company's own preprint.

- Category: Language models
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-20
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/reflection-s-open-weight-agent-model-nears-bigger-rivals-on-swe-bench-and-termin

## The study

Beam Model Technical Report (preprint, not peer reviewed). arXiv 2611.26358. https://arxiv.org/abs/2611.26358

## At a glance

- What it shows: Reflection has built a large open-weight AI model for coding and agent work. Its own tests put it close to bigger open models on software tasks, while it runs on fewer chips than many of them.
- What it does not show: It does not show the model beats the strongest open models, which still lead on many tests. Nor does it show the model working inside any real company.
- Why it matters: Companies that want an AI coding assistant they can run on their own hardware may get a cheaper option. The report also documents unusually detailed safety testing for an open release.

## What the study shows

Reflection, a US AI company, describes Beam, its first model with downloadable weights. Beam is a mixture-of-experts model, meaning only a few specialised parts of it run for each word it reads or writes. It has about 500 billion parameters in total but uses about 23 billion at a time. It reads up to 1 million tokens, roughly a few thousand pages, in one go.

The report is a preprint and has not been peer reviewed. Most scores come from the authors' own test runs.

On coding and agent benchmarks the authors report scores close to larger open models. Beam solves about 82% of tasks on SWE-bench Verified, a set of real software bugs to fix. It completes about 80% of tasks on Terminal-Bench 2.1, which tests work in a command line. GLM 5.2, a model with about 50% more parameters, scores at a similar level. Kimi K3 and DeepSeek V4.1 Flash score clearly higher on several agent and reasoning tests.

The model was mainly shaped by reinforcement learning, where it improves by trying tasks and being rewarded for correct results. This ran for about four weeks on roughly 10,500 NVIDIA GB300 chips and produced over 100 million practice attempts. The authors report test scores still rising at the end of the run.

For serving, Reflection released smaller compressed versions. The 4-bit version fits a full-length conversation on half as many chips and scores within about half a point of the full model on average. A small helper model that guesses upcoming words speeds up text generation by roughly two to three times.

On the authors' own safety suite, Beam refuses or redirects about 89% of harmful requests and answers about 82% of harmless ones. Reflection rates the model's risk as Limited in four areas: biological weapons, chemical weapons, loss of control and cyber attacks. That rating uses the company's own framework.

## Our reading

In our reading, the main news is not a new technique. It is a large, well-documented open-weight model aimed squarely at coding agents, released with weights the authors say anyone can use under the Apache 2.0 licence. That places it at level III on our Readiness Scale, open source available, although we could not confirm the download ourselves because the paper gives no link.

The headline claim of "frontier-level reasoning with three to four times less compute" needs care. The paper supports it mainly by pointing to the small number of active parameters. It gives no direct cost test against a named rival under the same conditions. The throughput gains it does measure compare Beam's own compressed versions with its own full version.

A concrete example: a bank or software firm that cannot send code to an outside service could run Beam's 4-bit version on four high-end GPUs and handle long coding sessions. On the authors' numbers it would fix most bugs of the SWE-bench kind. It would still trail the best open models on the hardest agent tasks.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/reflection-s-open-weight-agent-model-nears-bigger-rivals-on-swe-bench-and-termin

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
