# AI agents find bugs alone; CVE records double, fix times hold

A preprint from NIST and university researchers maps what AI agents have shown they can do in software security since 2024. It then checks a decade of public vulnerability records for the effects.

- Category: Safety & evaluation
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-20
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/ai-agents-find-bugs-alone-cve-records-double-fix-times-hold

## The study

SoK: Assessing the Evolution of LLM Capability in Software Vulnerability Research (preprint, not peer reviewed). arXiv 2611.26601. https://arxiv.org/abs/2611.26601

## At a glance

- What it shows: AI systems can now find, prove and fix real software security flaws without human help. In 2026 the number of publicly reported flaws more than doubled, and more of them were serious.
- What it does not show: It does not show that AI caused the jump in reports, since most records do not name the tool used. It also does not show that attackers are using these systems at scale.
- Why it matters: Finding flaws is no longer the slow step. Fixing them and getting the fixes installed now looks like the weak point for anyone who runs software.

## What the study shows

The authors built a rating scale for AI work on software flaws, modelled on the levels used for self-driving cars. It runs from A0, where the system does nothing, to A5, where an agent picks its own targets and works for weeks. Each result also gets a precision grade, meaning how often its findings hold up. It also gets an access tier, from lab-only to open weights (models anyone can download), and an evidence grade.

They rated four tasks: finding a flaw, proving it is real, writing a fix, and checking that a fix is correct. AI agents gathered and scored over 53,000 sources, and two authors reviewed every result. The final map has 128 dated steps, of which 85 rest on direct evidence from real software.

The authors report that agents did all four tasks with no human help by October 2025. That was before restricted frontier models such as Anthropic's Claude Mythos Preview arrived in 2026. Finding and proving flaws has since reached expert-level precision at every access tier. For example, a security firm confirmed 96% of findings from Aikido Attack, a commercial agent anyone can sign up for.

Wider access followed fast. Open-weight models reached the same levels about 10 to 14 months after the first system, on real-software evidence. Across twelve security benchmarks, the gap was about 8 months.

The authors then turned to public records. CVE records, the standard public list of disclosed software flaws, more than doubled in the first ten months of 2026 against the same months of 2025. The share rated Critical or High rose from 46% to 56%. Where the age of the flawed code is known, severe bugs at least ten years old were reported 6 to 11 times as often.

Some things did not change. The median time from report to fix at the Zero Day Initiative, a large disclosure programme, stayed between 96 and 119 days every year since 2016. The median time from a CVE record to a confirmed attack stayed at 2 to 5 days.

The paper is a preprint and has not been peer reviewed. On our readiness scale it sits at level II, benchmarked: it is checked against public records and benchmarks, but we could not find its released data or code.

## Our reading

In our reading, the most useful contribution is the separation of three questions that public debate tends to blur. What has an AI system been shown to do? Who can get hold of it? How good is the evidence? A vendor blog post and an independently confirmed result now sit on the same scale, with the difference made visible.

The headline result is a mismatch. Capability and volume have jumped, but the timelines that matter to defenders have not moved yet. The authors are careful here. They say the records cannot tell whether defenders absorbed the extra load or whether attackers simply have not adopted these tools at scale.

The fix-rate data is the part we would point a product lead to first. Only 5 to 26% of reports sent in bulk to outside maintainers were fixed, against 58 to 94% of curated reports. A flood of valid findings does little good if the people who own the code cannot act on them.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/ai-agents-find-bugs-alone-cve-records-double-fix-times-hold

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
