Reflection's open-weight agent model nears bigger rivals on SWE-bench and Terminal-Bench
Reflection's first open-weight model, Beam, runs only a small part of itself for each word yet scores close to much larger open models on coding and agent tests. The evidence comes almost entirely from the company's own preprint.


What the study shows
Reflection, a US AI company, describes Beam, its first model with downloadable weights. Beam is a mixture-of-experts model, meaning only a few specialised parts of it run for each word it reads or writes. It has about 500 billion parameters in total but uses about 23 billion at a time. It reads up to 1 million tokens, roughly a few thousand pages, in one go.
The report is a preprint and has not been peer reviewed. Most scores come from the authors' own test runs.
On coding and agent benchmarks the authors report scores close to larger open models. Beam solves about 82% of tasks on SWE-bench Verified, a set of real software bugs to fix. It completes about 80% of tasks on Terminal-Bench 2.1, which tests work in a command line. GLM 5.2, a model with about 50% more parameters, scores at a similar level. Kimi K3 and DeepSeek V4.1 Flash score clearly higher on several agent and reasoning tests.
The model was mainly shaped by reinforcement learning, where it improves by trying tasks and being rewarded for correct results. This ran for about four weeks on roughly 10,500 NVIDIA GB300 chips and produced over 100 million practice attempts. The authors report test scores still rising at the end of the run.
For serving, Reflection released smaller compressed versions. The 4-bit version fits a full-length conversation on half as many chips and scores within about half a point of the full model on average. A small helper model that guesses upcoming words speeds up text generation by roughly two to three times.
On the authors' own safety suite, Beam refuses or redirects about 89% of harmful requests and answers about 82% of harmless ones. Reflection rates the model's risk as Limited in four areas: biological weapons, chemical weapons, loss of control and cyber attacks. That rating uses the company's own framework.
Our reading
In our reading, the main news is not a new technique. It is a large, well-documented open-weight model aimed squarely at coding agents, released with weights the authors say anyone can use under the Apache 2.0 licence. That places it at level III on our Readiness Scale, open source available, although we could not confirm the download ourselves because the paper gives no link.
The headline claim of "frontier-level reasoning with three to four times less compute" needs care. The paper supports it mainly by pointing to the small number of active parameters. It gives no direct cost test against a named rival under the same conditions. The throughput gains it does measure compare Beam's own compressed versions with its own full version.
A concrete example: a bank or software firm that cannot send code to an outside service could run Beam's 4-bit version on four high-end GPUs and handle long coding sessions. On the authors' numbers it would fix most bugs of the SWE-bench kind. It would still trail the best open models on the hardest agent tasks.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | Training recipe, ablations and scaling-ladder experiments are described in detail. |
| II Benchmarked | Met | Results on public benchmarks including SWE-bench Verified (81.6), SWE-bench Pro, Terminal-Bench v2.1 (80.1), GPQA Diamond (90.6), LongBench v2 and AIME 2026, compared with named models. These are mostly run by the authors. |
| III Open source | Met | The conclusion states the weights are released under Apache 2.0 in BF16, FP8 and NVFP4, plus a drafter model. The paper gives no direct download link and we could not check the release; the main text points to a model card and usage policy without a URL. |
| IV In production | Not yet | No named organisation is documented using Beam in production. The vending-business deployment is an internal test with Andon Labs and is ongoing. The infrastructure was used in house for training only. |
What would move it up
A named company or product running Beam in production, with usage or outcome data, would give level IV. Independent reruns of the headline coding and agent benchmarks, and a controlled test of the claimed 3-4x inference saving against a stated baseline, would firm up level II and III. A checkable public repository link would help confirm level III.
The study
Reflection Team, Adam Abdulhamid, Osama Abuelsorour et al.. Beam Model Technical Report. arXiv, 19 November 2026. Read the paper
Notes
- Reflection Team et al., Beam Model Technical Report, arXiv preprint 2611.26358, 19 November 2026. Not peer reviewed. Source
- Citation URL given by the authors for the report on Reflection's site. Source
- Readiness: level III, open source available, based on the paper's statement that weights are released under Apache 2.0 in BF16, FP8 and NVFP4 with a drafter model. We have not verified the release. Level IV is not met: no named outside organisation uses the model in production.
- Exact headline figures at reasoning effort 512, temperature 0.7: SWE-bench Verified 81.6, Terminal-Bench v2.1 80.1, GPQA Diamond 90.6, BrowseComp 76.8, LongBench v2 67.5. Kimi K3: Terminal-Bench 88.3, GPQA 93.5, BrowseComp 91.2. DeepSeek V4.1 Flash: DeepSWE 74.2 vs Beam 43.3, Terminal-Bench 90.6.
- Safety figures: 88.9% refuse-or-pivot on harmful requests, 81.5% compliance on harmless; agentic dynamic harmful compliance 14.5%; prompt-injection attack success 3.1% and 3.6% without defence, 0.3% and 0% with it.
- Serving: NVFP4 on 4 GPUs gives 6.0x the per-GPU throughput of BF16 on 8 GPUs at 50 tokens per second per user (8K in, 1K out). FP8 recovers 99.93% and NVFP4 98.98% of median benchmark scores. Drafter speedup 1.9x to 3.0x.
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections
