GPT-5.5, Grok and Llama rarely catch distress shared gradually in everyday chats

An Oxford preprint tested six AI assistants on the same five signs of distress, given all at once or spread through a casual conversation. Most models responded far more weakly when the signs were spread out. The two Claude models held up best.

6 min readIIBenchmarkedPreprint, not peer reviewed
Save
A photograph of a folded cream paper strip stretching across a warm off-white background, with five small grey pebbles connected by a delicate blue thread.AI illustration
Illustration: AI generated

What the study shows

Researchers at the Oxford Internet Institute wrote one ordinary conversation of 21 messages. A young person asks an assistant for help replying to a friend, writing a photo caption, cooking with little food, checking a budget and planning a weekend away.

Into this chat they added five short remarks hinting at distress. In one version, all five came together in the last message. In the other, one remark appeared every fifth message. A third version had no remarks at all. The remarks covered depression, psychosis or domestic abuse, each at low, medium and high strength, giving nine scenarios.

Three community mental health support workers from a London service helped write the remarks. Two of them set the reply they would expect for each scenario, on a five-step scale. The scale runs from ignoring the distress, through sympathy and suggesting support, to pausing the task to deal with safety first. They almost never lowered their expectations when remarks were spread out.

The team ran more than 7,400 conversations through six models: GPT-5.5, Grok 4.3, DeepSeek V4, Llama 4 Maverick, Claude Sonnet 5 and Claude Opus 5. Another AI model graded each final reply against the scale. The first author checked a sample of 80 grades and agreed closely with them.

The authors report that every model responded more weakly when the remarks were spread out. For GPT-5.5, Grok 4.3, DeepSeek V4 and Llama 4 Maverick, replies met the support workers' standard in roughly a quarter to three fifths of conversations when the remarks came together. When they were spread out, that fell to about one in twenty or fewer. Grok 4.3 ignored the distress entirely in more than half of the spread-out chats.

The two Claude models showed a much smaller gap. Claude Opus 5 met the standard in nearly all packed conversations and in 87% of spread-out ones. Even so, it fell short in about one in eight spread-out chats.

A re-run tested whether the wording of the packed message drove the result. The researchers copied the five remarks word for word into one message. The gap remained in all six models, though it shrank somewhat for Grok 4.3 and GPT-5.5. The paper is a preprint and has not been peer reviewed.

Our reading

In our reading, the core finding is about how the test is built, not just which model scores best. The researchers held everything fixed except where the remarks appeared. That makes the gap hard to explain away. A model that responds well to a list of struggles can still say nothing when the same struggles turn up between a pasta recipe and a rent calculation.

Picture a student who uses one assistant for coursework, meal ideas and budgeting. Over an evening they mention not sleeping, not eating and avoiding friends. Each remark is small. Together, a support worker would want to check in. On these results, most of the tested assistants would simply help with the budget.

The result also puts developers' own safety numbers in context. Companies have reported large gains on internal mental health tests. The authors point out that those tests mostly state distress plainly. This study suggests that measure and the real-world pattern of disclosure can diverge sharply.

Free membership

Read the rest of this story

Membership is free. You get every article in full and the Issue every Sunday.

We send a sign-in link, no password needed. Already a member? Sign in

Readiness assessment

II
Benchmarked
LevelStatusEvidence
I  Lab onlyMetA controlled lab-style study with scripted conversations and expert-defined standards.
II  BenchmarkedMetSix named models were tested on 7,400+ conversations with a clear scoring method, expert gold standards and a sensitivity analysis. This is a bespoke benchmark by the authors, not an established public one, so the fit to level II is loose.
III  Open sourceNot yetThe paper links no code, scripts, data or judge outputs. It mentions AI-assisted analysis code but gives no repository. The full scripts and rubric are in the appendix, so the study could be rebuilt by hand.
IV  In productionNot yetNo documented use by a named organisation. This is an evaluation of existing models, not a deployed tool.

What would move it up
Releasing the conversation scripts, cue sets, judge prompt, raw replies and analysis code would reach level III. Independent replication, with other clinicians, real users, system prompts and longer chats, would strengthen level II. Level IV would need a developer adopting this test in its safety checks and reporting results.

The study

Charlotte Peart, Franziska Sofia Hafner, Luc Rocher. Evaluating safeguarding in multi-turn conversations with AI assistants. arXiv, 19 November 2026. Read the paper

Preprint, not peer reviewed · Licence arXiv (no reuse)

Notes

  1. Peart, Hafner and Rocher, 'Evaluating safeguarding in multi-turn conversations with AI assistants', Oxford Internet Institute, arXiv preprint 2611.26009, 19 November 2026. Not peer reviewed. Source
  2. No code or data repository is linked in the paper. The full conversation script, all 45 remarks and the grading questions are in appendices D and E.
  3. Share of replies meeting the support workers' standard, packed against spread-out remarks: GPT-5.5 59% to 3%; Grok 4.3 41% to 3%; DeepSeek V4 39% to 6%; Llama 4 Maverick 26% to 4%; Claude Opus 5 97% to 87%; Claude Sonnet 5 68% to 59%.
  4. Effect size (Cliff's delta, where 0 means no change and positive means weaker replies when remarks are spread out) ranged from 0.09 for Claude Sonnet 5 to 0.75 for Grok 4.3, p<0.01 for every model. Grok 4.3 gave no acknowledgement of distress in 57% of spread-out chats, against 1% when packed.
  5. Models were called through OpenRouter in June and July 2026, temperature 0.7, no system prompt, reasoning off. Paid models: 30 runs per condition; cheaper models: 100. Grader: Claude Sonnet 4.6; human agreement on 80 replies, quadratic-weighted kappa 0.93.
  6. Word-for-word re-run: effect held in all six models; Grok 4.3 fell from 0.75 to 0.66 and GPT-5.5 from 0.61 to 0.52.

Edited and signed by

Stefan PerdalEditor and publisher86 stories edited

Stefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.

How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

More on Safety & evaluation