AI that writes, tests, reviews and repairs software: research on coding agents and code models, with the benchmarks and limits that matter in practice.
A preprint follows AI coding agents as they port a quantum-safe encryption chip design to new hardware. The hardware works, but the agents' own tests and reviews proved weaker than they looked.
A preprint tests short code-quality reports injected into coding agents' context during long runs on Commit0, SlopCodeBench and SWE-EVO. The agents produced structurally simpler code, but held-out tests did not show more correct code.
A preprint re-ran the released Rust from eleven AI translation tools against the original C programs. Every tool shipped faulty code, and most faults sat on inputs the tools' own checks never tried.
A Duke University team built a simulated workbench where AI coding agents can change sensor readings and press virtual buttons while testing code for small chips. With it, agents passed 89% of tasks, against about 80% with compiler feedback alone.
A new open leaderboard, PerfArena, times the correct code of 13 language models in seven programming languages. Passing the tests turns out to say little about how fast or energy-hungry the code is.
A new benchmark from Amazon researchers chains 20 coding requests together. Ten agents complete about three quarters of the single requests but only about half of the chains, mostly because they build on mistakes they never notice.