# Kyutai streams talking-face video from live speech on one H100 GPU

Researchers at Kyutai shrank a slow talking-head video model into one that keeps pace with speech. The smaller version runs faster than real time on a single GPU. The work is a preprint and has not been peer reviewed.

- Category: Vision & multimodal
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-20
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/kyutai-streams-talking-face-video-from-live-speech-on-one-h100-gpu

## The study

AURA: AUdio-dRiven streaming Avatar (preprint, not peer reviewed). arXiv 2611.26540. https://arxiv.org/abs/2611.26540

## At a glance

- What it shows: A video model can turn one photo and a stream of speech into a talking face as the words arrive, on one graphics card. The smaller version keeps up with playback, and the larger one gives sharper video about 17 times faster than the model it learned from.
- What it does not show: It does not measure the delay between a word being spoken and the matching face appearing. It also shows no use in a real product, and does not test streams longer than one minute.
- Why it matters: Voice assistants can already talk in real time, but have no face. This work suggests a talking face could be added on a single card rather than a rack of them.

## What the study shows

Talking-head models take a reference photo and a speech recording and produce video of that person speaking. The best open ones, such as TalkVerse and Wan-S2V, are slow. They generate several seconds of video at once, over dozens of refinement passes, and need to hear audio ahead of time. That rules them out for live conversation.

The authors, mostly from the Paris lab Kyutai, built a family of models they call AURA. It produces video in short chunks of about two thirds of a second. Each chunk needs only the audio for that same stretch, not future speech. It uses four refinement passes per chunk instead of 50, and keeps a small fixed memory of recent chunks plus the very first one, so memory use does not grow as the video gets longer.

They made it by distillation, where a fast 'student' model is trained to copy a slow 'teacher'. The larger AURA 5B (5 billion parameters, the adjustable weights in a model) learned from TalkVerse. The smaller AURA 1.3B learned from a teacher the authors trained themselves. Training used a set of 30,000 synthetic video clips the teacher generated from computer-made speech.

On one Nvidia H100 GPU, the authors report that AURA 5B makes 720p video at 9.7 frames per second, about 17 times faster than TalkVerse. AURA 1.3B makes lower-resolution video at about 21 frames per second, above its 16 frames per second playback rate, so it runs in real time. Both were faster than every rival tested, including Live Avatar, a 14-billion-parameter streaming model, which managed 1.9 frames per second.

On quality, results are mixed. AURA 5B beat TalkVerse on a standard video realism score, and kept the person's identity closest to the photo of all models tested. The older Hallo3 model scored better on the realism scores. In a human study with 27 raters and 400 side-by-side comparisons, both AURA models were preferred over Hallo3 on all four measures. AURA 5B was on par with TalkVerse on quality, lip-sync and naturalness. AURA 1.3B ranked below both on those three.

## Our reading

In our reading, the main result is the smaller model running faster than playback on one card. Earlier streaming talking-head systems of this kind used models of 14 to 18 billion parameters and, the authors say, needed four or five GPUs. Going down to one card changes who can afford to run a live face. A customer-service voice bot could show a talking agent next to its replies, without a cluster behind each conversation.

The quality picture needs care. Standard video scores favour Hallo3, yet human raters preferred AURA, and the authors point to hand and mouth artefacts the scores miss. That is a useful warning about automatic metrics in general. But it also means AURA's quality case rests heavily on a small human study the authors ran themselves, on their own test sets.

On our readiness scale we place this at level III, open source available, provisionally. The paper says weights for both models, the dataset and the small teacher are released, with a pointer to the project page. We have not yet confirmed the download or the licence terms. There is no documented use by an outside organisation, so level IV is not met.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/kyutai-streams-talking-face-video-from-live-speech-on-one-h100-gpu

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
