# One fake news article can flip AI forecasts, and accuracy hides it

A preprint finds that language-model forecasters usually follow a single planted news story once they read it. The multi-agent systems that search most widely are the ones most likely to find it.

- Category: Safety & evaluation
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-14
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/one-fake-news-article-can-flip-ai-forecasts-and-accuracy-hides-it

## The study

One Fabrication Is Enough: A Fake News Article Changes the Forecasts of LLM Forecasting Systems (preprint, not peer reviewed). arXiv 2611.15102. https://arxiv.org/abs/2611.15102

## At a glance

- What it shows: AI systems that forecast markets and events by reading the news can be steered by one invented article. Once the article is in front of the model, about three quarters or more of the forecasts that disagreed with it switch sides.
- What it does not show: It does not show the attack on the real, live products. The authors rebuilt two well-known forecasting systems from public descriptions and ran them over a small, frozen news archive, not the open web.
- Why it matters: The damage is invisible in average accuracy: the forecaster becomes right whenever the attacker happens to be right. The defences that work best are costly and need control over the news index, which web-searching systems do not have.

## What the study shows

This is a preprint and has not been peer reviewed. The authors tested AI forecasters that use retrieval, meaning they search a news archive and read the top results before answering. Questions came from stock returns and from resolved prediction-market questions on Polymarket, Kalshi and the ForecastBench benchmark. Forecasts were scored against what actually happened.

The attacker is deliberately weak. They know only the question and which way they want the forecast to go. They cannot see the model or its searches. Their only move is to publish a plausible fake news story before the forecast date.

When the authors placed the fake article directly in the model's reading list, it reversed 82% of opposing stock forecasts and 74% of opposing prediction-market forecasts. This held across five frontier models. As a control, a genuine news article in the same slot changed decisions about a third as often as the fake one.

Whether the article gets retrieved decides the outcome. A simple single-search forecaster found it in about one run in seven. The authors' rebuilds of two leading multi-agent forecasters, the AIA Forecaster and xAI's Grok ForecastBench system, found it in about nine runs in ten. Many agents running many differently worded searches cover the angles one search misses.

The models cannot check the story. Asked whether a passage was fabricated, they flagged nearly all fake articles but also almost half of the genuine ones. Their written reasoning rarely doubted the fake event. In the AIA rebuild, a supervising agent sometimes noted the weak sourcing but had no rule that let it act.

Average accuracy did not drop. On the curated stock questions it stayed around half. When the fake article pointed the wrong way, accuracy fell from about 50% to 15%. When it happened to point the right way, accuracy rose to 86%. The two cancel out.

A second attack published two fake articles arguing opposite ways, hoping to make the model unsure. It did not hedge. It gave a confident forecast in whichever direction it weighed more heavily, so the attacker lost control of the direction but the user still got a confident wrong signal.

Two published defences each stopped the one-article attack. Passage filtering uses a model to remove passages it thinks are fake. Partitioned retrieval splits the archive into separate parts, forecasts from each, and takes the median. Each was cheaply bypassed. Together they held against four articles but not sixteen edited copies, while changing 29% of decisions on days with no attack.

## Our reading

In our reading, the central point is not that models are gullible. It is that the systems built to search most thoroughly are the ones most likely to pick up a planted story. Breadth of search, usually a strength, becomes the entry point.

The accuracy finding is the most useful one for anyone running these systems. A team that monitors only hit rates would see nothing wrong. A trading desk could be taking the attacker's bets for weeks while its dashboard looks normal. The authors argue that checks must look at the evidence the system read, not at its outputs.

On our readiness scale we rate this level II, benchmarked. The attack was measured on public benchmark questions and real market questions with pre-set rules. The code is promised but not yet released, so it does not reach level III.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/one-fake-news-article-can-flip-ai-forecasts-and-accuracy-hides-it

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
