Eight in ten AI models gave up a secret hidden in their instructions
A preprint tested 363 models on one gateway. An AI attacker, guided by a researcher, pulled a protected code out of 296 of them, often by asking for a translation.


What the study shows
Most AI products start each chat with a system prompt: hidden instructions from the developer that set the model's role and rules. Users are not meant to see it, and the model is meant to follow it over user requests. Security guidance already warns against putting secrets there. This study tests how often that warning matters in practice.
The authors took 363 text models listed on OpenRouter, a service that gives access to models from many providers through one connection. Each model received the same system prompt. It held a random 16-character code and a strict order never to reveal it, in any form. The catalogue was frozen on 6 October 2026.
The attacker was an AI agent, a coding assistant driven by the DeepSeek V4.1 Flash model, steered by a human researcher. A capture counted only when the exact code appeared in a reply. Each model was counted once, at its first capture.
The authors report that 296 of the 363 models gave up the code. Four newer models released after the freeze were also captured, for 300 in total. Fifty models were never captured, and 17 could not be reached with text.
Simple requests did most of the work. Asking a model to translate its instructions into French, keeping the code unchanged, took 116 models in one message. Asking for a plain English rewrite took 67 more. Overall, 170 of the 300 captures involved a language other than English.
Captured models include older versions of well-known families, such as Google's Gemini 2.5 Flash and Pro, as well as small roleplay models. The work is a preprint and has not been peer reviewed.
Our reading
In our reading, the striking part is not the attack. It is how ordinary the attack was. Most models did not fall to clever tricks. They fell to a request that sounds like normal work: translate this, tidy this, keep the identifiers as they are. The model treats the request as a task and the code as just another string to preserve.
This suggests a model cannot easily tell the difference between handling its instructions and revealing them. Picture a support chatbot whose hidden prompt holds a staff discount code. A customer who asks it to put its setup into French may get the code back, with no hacking skill at all.
The study is a measurement across a wide catalogue, not a new attack or defence. We rate it at readiness level II, benchmarked: it reports per-model results on a public catalogue. It does not reach level III yet, because the authors' code and records sit in a repository that is private until the paper is accepted.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | A research study with a defined method, applied across a catalogue of 363 models. |
| II Benchmarked | Met | The audit is a public-catalogue measurement. It reports per-model, per-technique and per-language capture results across 363 OpenRouter models (Tables 3 and 5, Appendices C and D). It is not a standard shared benchmark with independent replication, so this level is met only loosely. |
| III Open source | Not yet | The harness, ledger, transcripts and catalogue are in a Hugging Face repository (huggingface.co/buckets/ilostmytails/nevan). The paper says it is private at submission and will be public only on acceptance. Nothing is verifiably released yet. |
| IV In production | Not yet | No documented use in production by a named organisation. This is a measurement study, not a deployable tool. |
What would move it up
Making the Hugging Face repository public, with the harness, ledger and transcripts, would reach level 3. Independent replication, repeated trials per model and technique, and tests with semantically sensitive secrets would strengthen level 2.
The study
Ana Clara Zoppi Serpa, Anderson Rocha, Hélio Pedrini. Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study. arXiv, 16 November 2026. Read the paper
Notes
- Serpa, Rocha and Pedrini, 'Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study', arXiv preprint, November 2026, CC BY 4.0. Source
- Data and harness repository named by the authors. Described as private at submission, to be made public on acceptance. We could not verify its contents. Source
- Exact figures: 296 of 363 catalogue entries captured (about 82%); 4 post-freeze captures; 300 total; 50 pending; 17 discarded. 170 of 300 captures (56.7%) involved a non-English language, 119 of them French. Translation into French took 116 models; English paraphrase took 67.
- Side test on xiaomi/mimo-v2.6-pro: refusal rate 40% with the fragment 'Hpw', 15% with 'hpw', 0% without, 20 probes per condition (Appendix B, Table B.1).
- OWASP Top 10 for LLM Applications 2025, which lists system prompt leakage as its own risk. Source
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections

