# True posts, false conclusions: AI agents fill gaps with rumours

A preprint shows that language-model agents can be steered towards rumours using only truthful posts, arranged to hint at a link that none of them states. A simple support check helps, but only partly.

- Category: Safety & evaluation
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-17
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/true-posts-false-conclusions-ai-agents-fill-gaps-with-rumours

## The study

Language-model agents' beliefs can be manipulated via truthful evidence in open channel (preprint, not peer reviewed). arXiv 2611.21235. https://arxiv.org/abs/2611.21235

## At a glance

- What it shows: When true social-media posts are carefully selected and ordered, AI assistants often accept a rumour that the posts never actually state. Asking several assistants and taking a vote, or adding a reviewing AI, did not fix most of these errors.
- What it does not show: It does not show this happening on live platforms; all tests ran in a simulated feed. It does not show that the proposed safeguard reliably tells true claims from false ones.
- Why it matters: Products already use AI to summarise social media for the public. Checking that each fact is true is not enough if the AI invents the connections between facts.

## What the study shows

The researchers, from the University of Liverpool and MBZUAI, study what they call narrative overfitting. This is when an AI agent links separate facts with a relationship it has assumed, then treats that assumption as established. The work is a preprint and has not been peer reviewed.

To test it, they built an attack pipeline called Generative Montage. Three AI roles, all run on GPT-4.1-mini, pick true posts from real Twitter coverage, cut them into short posts and reorder them. The aim is to suggest a rumour without ever stating it. A second AI role checks that every post stays factually accurate.

They ran this on 1,090 rumour claims from six 2014 to 2015 news events, including the Charlie Hebdo attack and the Sydney siege. Fourteen models from the GPT, Claude, Qwen and DeepSeek families acted as analysts. Twelve of them accepted the rumours in more than 65% of runs. The most resistant, Claude-4.5-Haiku, still accepted about four in ten.

One example: posts mentioned hostages in a Sydney cafe and, separately, that 13 people had been counted. The analyst concluded there were 13 hostages. Nothing in the feed linked the two.

Group checks did not solve the problem. A majority vote across five analysts still accepted rumours in roughly 60% to 75% of cases, depending on model family. An AI judge that could see the original posts and all five reports accepted them in roughly 58% to 70% of cases.

The authors then built a proof-of-concept safeguard, Grounded Monitor. It labels each post by whether it directly establishes the claim or merely makes it plausible. If a feed pushes hard towards a claim but no single post establishes it, the monitor raises a warning. Given that warning plus an 'uncertain' answer option, four small open models made more correct accept-or-reject decisions, by about 11 to 29 percentage points on the original feeds.

## Our reading

In our reading, the most useful contribution is the framing. Most defences against AI misinformation check whether individual statements are true. This study shows the error can live entirely in the gaps between true statements. A fact-checker that verifies each sentence would pass every post in these attack feeds.

The finding about review is the one that should worry system designers most. Five analysts reading the same feed are not five independent sources. Their agreement looks like corroboration but adds no new evidence. The authors report that a reviewing AI with access to the original posts still let most rumours through. We note, though, that the judge was a fairly small open model, and stronger judges were not tested.

On our four-level readiness scale, we place this at level III, open source available. The paper reports results on a public dataset and links code and data on GitHub, though we have not checked the repository's contents or licence. There is no documented use by any organisation, and the authors themselves call the monitor a proof of concept.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/true-posts-false-conclusions-ai-agents-fill-gaps-with-rumours

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
