# Study of 120,000 GitHub bug reports: AI adds detail, not more confirmed bugs

A preprint tracing AI use in bug reports on 101 popular open-source projects, including the Linux kernel, finds AI-involved reports are fuller but no more often confirmed. AI help with reproducing and diagnosing bugs is the use linked to valid findings.

- Category: Code
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-20
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/study-of-120-000-github-bug-reports-ai-adds-detail-not-more-confirmed-bugs

## The study

Copilot or Culprit? How AI Is Changing Open-Source Bug Reporting and Maintenance (preprint, not peer reviewed). arXiv 2611.25605. https://arxiv.org/abs/2611.25605

## At a glance

- What it shows: Bug reports written or found with AI's help are growing fast on popular open-source projects. They are more complete than other reports, but maintainers confirm them as real bugs no more often.
- What it does not show: It does not show that AI causes better or worse reports, because the data are observational. It also misses AI use that reporters did not disclose.
- Why it matters: Open-source maintainers are being asked to check a rising flood of AI-assisted reports. The study points to which kinds of AI help actually lead to confirmed bugs.

## What the study shows

The authors, from the University of Hong Kong and other Chinese institutions, studied 120,918 bug reports on 101 open-source projects. The projects are popular GitHub repositories plus the Linux kernel. Reports were opened between January 2024 and September 2026. The paper is a preprint and has not been peer reviewed.

They sorted reports by AI's role. In 'AI-assisted' reports, a person hit the bug and used AI to help. In 'AI-discovered' reports, an AI tool found the bug, usually by reading source code. A third group was only suspected of AI use. Most labels were applied by a large language model (LLM), an AI system trained on text, and disagreements with a second LLM were settled by human reviewers.

AI use rose quickly. About 1 in 140 bug reports involved AI in 2024. By mid 2026 it was about 1 in 12. Reports where AI found the bug grew from a small fraction to about a fifth of AI-involved reports.

AI-involved reports were more complete. They more often included reproduction steps, triggering conditions and the expected behaviour. Comparing reports within the same project and quarter, about 9 percentage points more were rated high quality.

Completeness did not mean correctness. Overall, maintainers confirmed AI-involved reports as real bugs at about the same rate as other reports. Reports where AI found the bug did worse, and most of them never got a clear verdict at all.

The type of AI help mattered. The authors report that AI help with reproducing a bug roughly doubled the odds of a valid verdict. AI help with diagnosis also helped. AI help with writing the report showed no link to validity.

AI-generated materials still needed checking. Maintainers rejected about a quarter of explicitly AI-generated reproducers, the small scripts or inputs that trigger a bug, against about 1 in 20 others. That figure rests on only 34 AI-generated reproducers.

Maintainers are also starting to use AI themselves, mostly to read code and suggest fixes. AI-involved maintainer replies rose to about 3% of replies in 2026, from almost none in 2024. Most came from bots, and a handful of projects accounted for most of them.

## Our reading

In our reading, the central result is a gap between polish and truth. AI makes reports look complete. It does not, on its own, make them right. The uses of AI that track with confirmed bugs are the ones that test something: building a reproducer, or digging into the cause. Writing nicer prose is not one of them.

A practical example: a developer whose program crashes asks an AI assistant to write up the bug. The result reads well, but the evidence suggests it is no more likely to be confirmed. If the developer instead has the AI build a small script that triggers the crash, and runs it, the report is more likely to land.

The split between AI-assisted and AI-discovered reports is the second lesson. When a person hit the bug, AI help drew more maintainer work, not less. When an AI found the bug by reading code, maintainers mostly did not engage. Fewer than one in five such reports got any reply. That is a lack of attention, not proof the findings were wrong.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/study-of-120-000-github-bug-reports-ai-adds-detail-not-more-confirmed-bugs

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
