# Trimming overlooked steps brings AI video generation to interactive speed

An Apple and Cornell team found that most of the delay in fast video generators sits outside the generation step itself. Replacing those parts with small networks brings response times under 50 milliseconds on top-end GPUs, in a preprint that has not been peer reviewed.

- Category: Efficiency
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-18
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/trimming-overlooked-steps-brings-ai-video-generation-to-interactive-speed

## The study

FastFrames: Video Generation at Interactive Latency (preprint, not peer reviewed). arXiv 2611.23226. https://arxiv.org/abs/2611.23226

## At a glance

- What it shows: AI systems that make video one frame at a time can now react to an input in about 36 thousandths of a second. That is fast enough that people should not notice a delay. The gain comes mostly from replacing slow supporting parts, not from changing the core generator.
- What it does not show: It does not show this speed on ordinary hardware, at high resolution or in real products. Quality is judged mainly by one automated benchmark and a few hand-picked examples.
- Why it matters: Interactive uses such as games, simulators and robot planning need video that reacts at once. This work suggests the main obstacle was engineering overhead, not the generation step itself.

## What the study shows

The study looks at autoregressive video models. These make a video piece by piece, so each new frame can respond to a new input, such as a steering command. The authors argue that frames per second is the wrong measure for this. A model can produce many frames per second yet still make a person wait a second for any reaction. They measure instead the delay from an input to the first frame that shows its effect. Their target is 50 milliseconds, a threshold taken from studies of how people perceive delay with touch and mouse input.

They first timed the fastest existing model of this kind, Causal Forcing++, on one Nvidia B200 GPU. It took 272 milliseconds per step at 480p resolution. Only about 30% of that went on generation. More than half went on the decoder, which turns the model's compressed internal output into pixels. Another big share went on an extra full pass of the model that tidies its memory of past frames, known as the KV cache.

The authors report three changes. A small network replaces the decoder, cutting it from 147 milliseconds to about 1. A second small network repairs the frame memory, taking about 2 milliseconds instead of a full model pass. Systems tuning then makes each remaining model pass about two and a half times faster on one GPU.

The 1.3 billion parameter version, FastFrames-S, responds in 36 milliseconds on one B200 GPU, about 7.6 times faster than the baseline. The 14 billion parameter version, FastFrames-L, takes 45 milliseconds but needs eight B200 GPUs. On VBench, a public automated test of video quality, both score about the same as or slightly above their closest rivals. The authors also applied the methods to two existing interactive models, one steered by camera moves and one by physical actions. Both fell from over 800 milliseconds to about 50, using eight GPUs. The authors say code and model weights are on their project page.

## Our reading

In our reading, the most useful finding is the profiling itself. Teams have spent years cutting the number of generation steps. This paper shows that, for the fastest models, the generation step is no longer the main cost. The decoder and the memory refresh were treated as fixed plumbing. Measured honestly, they were the bottleneck.

The quality claim is weaker than the speed claim. On VBench, FastFrames-S beats its baseline by about a tenth of a point, and FastFrames-L leads its rival by two hundredths. The paper gives no measure of spread and no user study. We read this as "no visible loss on this benchmark", not as a quality gain. The speed numbers are clearer, because the authors break each step down and measure every model the same way.

On our readiness scale, we place this at level III, open source available. Public benchmark results are reported, and the authors say code and weights are released. We could not confirm the licence or the exact contents from the paper. There is no documented production use by a named organisation.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/trimming-overlooked-steps-brings-ai-video-generation-to-interactive-speed

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
