# Driving AI models see hazards in their path but barely brake

A new benchmark pastes pedestrians and cars into real driving scenes. Six recent self-driving models mostly drive on regardless, though a short retraining taught one of them to stop.

- Category: Safety & evaluation
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-17
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/driving-ai-models-see-hazards-in-their-path-but-barely-brake

## The study

ReactBench-AD: Do Driving VLAs and World-Action Models React to Imminent Hazards in Their Path? (preprint, not peer reviewed). arXiv 2611.20011. https://arxiv.org/abs/2611.20011

## At a glance

- What it shows: Six recent AI driving models were shown real road scenes with a person or car suddenly placed in their path. They mostly kept their planned route and speed and would have hit the obstacle. A short extra training round taught one model to brake much more often.
- What it does not show: It does not show how these models behave on a real road, where the car moves and the model can correct itself. Every test scored a single planned route on recorded data.
- Why it matters: These models are marketed on their ability to handle rare situations. This study suggests that ability does not yet extend to emergency braking, the most basic safety reaction.

## What the study shows

The researchers, from the University of Limerick and Valeo, built a test set called ReactBench-AD. It uses real recorded driving scenes from two public datasets. Into each scene they paste a 3D model of a crossing pedestrian, a standing pedestrian or a stopped car. The object sits on the exact path the car really drove, between half a second and two seconds ahead.

Only the pixels of the object and its shadow change. So each edited scene has a perfect twin without the hazard. Any change in the model's planned route can be put down to the object alone. An off-the-shelf object detector found the pasted objects in about 99% of front-camera images, so they are clearly visible.

The team tested six openly available models. Four are vision-language-action models: language models adapted to output driving routes. Two are world-action models: video-generation models adapted the same way. All six reacted weakly. For four of them, the share of collision-free plans rose by more than one percentage point only for stopped cars, with one small exception. For about a third of AutoVLA's crossing-pedestrian scenes, its plan moved by less than 5 cm.

NVIDIA's two Alpamayo models did best, mainly when the hazard was two seconds away. Even in Alpamayo 1.5's best case, a standing pedestrian, most of its plans were still unsafe. The Alpamayo models write out their reasoning before driving. For pasted pedestrians, that reasoning almost always said to slow or stop and named the person. The planned route mostly did not follow through.

Finally, the authors retrained only the part of one model, ReCogDrive, that turns its understanding into a route. They used about 23,000 edited scenes with hand-built braking routes, for a few minutes' worth of updates. Its collision-free gain on the test set rose from almost nothing to about 41 points. Its score on ordinary driving barely moved. It also braked more for animals it had never seen and for real stopped cars. The paper is a preprint and has not been peer reviewed.

## Our reading

In our reading, the most useful finding is the gap between seeing and doing. The Alpamayo models describe the pedestrian and say they should stop, then plan a route that hits them. Even when the authors replaced the reasoning with an explicit order to stop immediately, Alpamayo 1.5 still collided in more than half of cases. The weak link appears to sit between the model's understanding and the part that draws the route. That reading rests on the two Alpamayo models only.

The retraining result points the same way. Only the route-drawing part was trained. The rest of ReCogDrive was frozen, yet it learned to brake for objects it had not been trained on. The authors take this as a sign the model already knew the path was blocked. It simply had never been taught what to do about it. Ordinary driving logs rarely contain a hard stop for a person in the road.

A practical example: a fleet testing one of these models on normal city routes could see excellent benchmark scores and smooth driving. The same model might still plan to drive through a pedestrian who steps out at close range. Standard driving scores would not reveal that, because they are built mostly from uneventful scenes.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/driving-ai-models-see-hazards-in-their-path-but-barely-brake

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
