One fake news article can flip AI forecasts, and accuracy hides it
A preprint finds that language-model forecasters usually follow a single planted news story once they read it. The multi-agent systems that search most widely are the ones most likely to find it.

AI illustrationWhat the study shows
This is a preprint and has not been peer reviewed. The authors tested AI forecasters that use retrieval, meaning they search a news archive and read the top results before answering. Questions came from stock returns and from resolved prediction-market questions on Polymarket, Kalshi and the ForecastBench benchmark. Forecasts were scored against what actually happened.
The attacker is deliberately weak. They know only the question and which way they want the forecast to go. They cannot see the model or its searches. Their only move is to publish a plausible fake news story before the forecast date.
When the authors placed the fake article directly in the model's reading list, it reversed 82% of opposing stock forecasts and 74% of opposing prediction-market forecasts. This held across five frontier models. As a control, a genuine news article in the same slot changed decisions about a third as often as the fake one.
Whether the article gets retrieved decides the outcome. A simple single-search forecaster found it in about one run in seven. The authors' rebuilds of two leading multi-agent forecasters, the AIA Forecaster and xAI's Grok ForecastBench system, found it in about nine runs in ten. Many agents running many differently worded searches cover the angles one search misses.
The models cannot check the story. Asked whether a passage was fabricated, they flagged nearly all fake articles but also almost half of the genuine ones. Their written reasoning rarely doubted the fake event. In the AIA rebuild, a supervising agent sometimes noted the weak sourcing but had no rule that let it act.
Average accuracy did not drop. On the curated stock questions it stayed around half. When the fake article pointed the wrong way, accuracy fell from about 50% to 15%. When it happened to point the right way, accuracy rose to 86%. The two cancel out.
A second attack published two fake articles arguing opposite ways, hoping to make the model unsure. It did not hedge. It gave a confident forecast in whichever direction it weighed more heavily, so the attacker lost control of the direction but the user still got a confident wrong signal.
Two published defences each stopped the one-article attack. Passage filtering uses a model to remove passages it thinks are fake. Partitioned retrieval splits the archive into separate parts, forecasts from each, and takes the median. Each was cheaply bypassed. Together they held against four articles but not sixteen edited copies, while changing 29% of decisions on days with no attack.
Our reading
In our reading, the central point is not that models are gullible. It is that the systems built to search most thoroughly are the ones most likely to pick up a planted story. Breadth of search, usually a strength, becomes the entry point.
The accuracy finding is the most useful one for anyone running these systems. A team that monitors only hit rates would see nothing wrong. A trading desk could be taking the attacker's bets for weeks while its dashboard looks normal. The authors argue that checks must look at the evidence the system read, not at its outputs.
On our readiness scale we rate this level II, benchmarked. The attack was measured on public benchmark questions and real market questions with pre-set rules. The code is promised but not yet released, so it does not reach level III.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | Controlled experiments on frozen archives with re-implemented systems and author-written attack articles. |
| II Benchmarked | Met | Evaluated on questions from ForecastBench, Polymarket and Kalshi, scored against published resolutions, with pre-fixed rules and bootstrap intervals. Reproduction of the attack on questions the authors did not write is reported. The attack and defences are not themselves a standard benchmark entry. |
| III Open source | Not yet | The paper says code, configurations and result tables 'will be released on publication' and refers to an anonymised repository. No public link is given. Fabricated articles will not be posted publicly. |
| IV In production | Not yet | No documented use by a named organisation. No live-web test was run. The defences were only tested on authors' own archives. |
What would move it up
Releasing the code, prompts, configurations and result tables at a public link would meet level 3. For level 4, a named deployer would need to run the filter or partitioned defence in production, or a live-web test would need to show the attack and defences against a real search system.
Checked automatically every week, last on 16 November.
The study
Zeyu Zhang, Bradly C. Stadie. One Fabrication Is Enough: A Fake News Article Changes the Forecasts of LLM Forecasting Systems. arXiv, 11 November 2026. Read the paper
Notes
- Zhang and Stadie, 'One Fabrication Is Enough: A Fake News Article Changes the Forecasts of LLM Forecasting Systems', arXiv preprint, November 2026. Source
- Code, configurations and result tables are described as in an anonymised repository, to be released on publication. No public link was available at the time of writing.
- Forced-context reversal (Table 2): 82.2% on 183 stock questions, three flash models pooled; 74.0% on 308 prediction-market questions, four models. Clean re-run placebo 16.7%; floor with article not retrieved 4.8%.
- Exposure under own retrieval (Table 6, 60 stock questions): hybrid single retrieval 16.7%, AIA Forecaster rebuild 88.3%, Grok ForecastBench rebuild 90.0%.
- Verification (Table 3): fabricated passages flagged 97.9%, genuine passages flagged 45.7%, four models pooled.
- Accuracy (Table 8, curated): 47.9% clean against 52.2% attacked; 50.4% to 14.9% when the article was wrong; 45.6% to 86.0% when right.
- Defences (Tables 9 to 11): filtering 74.4% to 23.3%; partitioning 82.0% to 14.2%; combination 7.1% at four distorted articles, 17.9% at sixteen; combination changes 28.8% of clean decisions and removes 39.0% of genuine passages.
- AIA Forecaster technical report, of which Bradly Stadie is a co-author. Source
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections

