AI vision models mostly ignore users' own privacy rules, study finds
A new benchmark tells 21 image-reading AI models what a user considers private, then asks about a photo. Most answered anyway. Fine-tuning fixed much of this, but it cost some general ability.

AI illustrationWhat the study shows
The researchers, from Tsinghua University and Microsoft Research Asia, studied vision-language models. These are AI systems that read images and answer questions about them in text. Such models can pull sensitive details out of ordinary photos, such as where a picture was taken or what a card number says.
Most existing safeguards treat privacy as fixed and the same for everyone: credit card numbers are always off limits, for example. The authors argue that privacy is personal and depends on context. One person may not want their age discussed. A doctor may legitimately need to know a patient's weight.
They built a test set called SPY-Bench with 32,700 samples. Each sample pairs a photo with a question and a plain-language privacy rule, such as 'Age is considered as private information.' Some samples add a role-play setting, such as a police officer or a doctor. In some settings the authors decided the information should be shared regardless of the user's rule. Five categories, including passports and nudity, were always to be refused.
They scored 21 models, including GPT-4o, Gemini 2.0 Flash and OpenAI's o4-mini. Compliance with the user's rule was low. The best score was about 24%, from o4-mini, and most models scored under 20%. When a category was marked private, models typically refused less than 30% of the time. Models did especially badly on passports, ID cards and driving licences. That was the reverse of what the authors expected.
Bigger models helped only up to a point. In one model family, the 32-billion-parameter version beat the smaller ones, but the 72-billion version was no better. Adding five unrelated privacy rules to the prompt usually made models worse. GPT-4o and Mistral Small 3.1 were exceptions and did slightly better.
The authors then fine-tuned one open model, Qwen2 VL 7B, on a separate training set of about 81,000 samples. Fine-tuning means further training a model on examples of the behaviour wanted. Their best method, a variant of a preference-training technique called NCA, raised compliance from about 3% to about 97%. Preference training teaches a model to favour good answers over bad ones. The gain came with a cost: scores on standard tests of general image understanding fell. The tuned models also became over-cautious and refused more often in cases where they should have answered.
The work is posted as an arXiv preprint and has not been peer reviewed, according to the details supplied with it. The authors have released code and data on GitHub.
Our reading
In our reading, the headline finding is a gap in instruction following rather than in privacy knowledge. The models clearly know what a passport is. What they lack is the habit of treating a user's stated rule as binding when it clashes with a direct question. The authors suggest that models' built-in, one-size-fits-all idea of privacy pulls against the user's rule. That is a plausible explanation, but the paper does not test it directly.
The fine-tuning result is strong but narrow. One 7B model was trained on data from the same pipeline, categories and style as the test. Images were kept separate between training and testing, but the questions and refusals were all written by AI models in the same way. A near-perfect score under those conditions shows the behaviour can be learned. It does not yet show it holds up against messier, human-written requests.
On our four-level readiness scale, we place this at level III, open source available. Models are scored on a published benchmark, and code and data are released. The benchmark is the authors' own, though, and we have not checked what the repository contains. We found no documented use by a named organisation.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | The work is a research study with a new task, benchmark and training method tested in the lab. |
| II Benchmarked | Met | 21 named models are scored on SPY-Bench (Table 4), and tuned variants are compared on it and on MMMU, OCRBench and MME (Tables 6, 7). The benchmark is the authors' own, not an established public one. |
| III Open source | Met | The abstract states code and data are available on GitHub. I could not open the link, so the contents (for example whether tuned weights are included) are unchecked. |
| IV In production | Not yet | No named organisation is reported using the method or benchmark in production. |
What would move it up
Documented use by a named organisation in a real product would reach level 4. Before that, independent replication on other base models, tests on human-written and real-user privacy requests, and fixes for the loss of general ability and over-refusal would make the result more credible.
Checked automatically every week, last on 16 November.
The study
Xingqi Wang, Xiaoyuan Yi, Xing Xie et al.. Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models. arXiv, 10 November 2026. Read the paper
Notes
- Wang, Yi, Xie and Jia, 'Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models', arXiv preprint, November 2026. Source
- Code and data, as stated in the paper. Not checked by us. Source
- Exact figures: o4-mini instruction compliance score (ICS) 23.74% with distractors and 17.43% without. GPT-4o refusal rate on private categories 40.79% with distractors. NCA-P ICS 96.88% with distractors, up from 2.75% for base Qwen2 VL 7B. MME score fell from 2324.16 to 2138.36 after NCA-P. NCA-P combined trade-off score 78.61.
- SPY-Bench: 32,700 samples from about 6,700 image-question pairs on 2,725 images, 67 categories, 24 scenarios. SPY-Tune: 81,096 samples. Images come from the VISPR dataset.
- Licence is CC BY-NC-ND 4.0, so reuse of figures is likely restricted.
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections

