NTH

TaoMate-HD: Progressive-Resolution Autoregressive Audio-Video Generation

AuthorsQijun Gan, Chenwei Zhang, Xinyang Li, Meiguang Jin, Junfeng Ma, Shen Qiu

AffiliationsNov 2026 1 2 Alibaba Group Nanjing University

November 14, 2026 2 min read
Watch on YouTube
The one-line take

TaoMate-HD generates high-resolution video and speech in streaming chunks, aiming to make digital humans faster and more coherent as text updates.

Key results

3
Denoising evaluations per chunk

Two low-resolution evaluations followed by one high-resolution refinement.

1.60
Progressive speedup

Speedup over direct high-resolution generation.

1.92
Stage-parallel speedup

Speedup over serial progressive generation.

11.18
Character error rate reduction

Percentage-point reduction, from 30.75% to 19.57%.

60
Demonstrated stream duration

Minutes of audiovisual generation.

What the paper found

TaoMate-HD, initialized from MiniMax-H3 FL2VA, targets high-resolution audio-video streaming without making each chunk wait for full visual refinement. Its 3-step denoising trajectory uses two low-resolution video evaluations at 480×864, followed by one refinement at 768×1376, while audio remains part of the joint generation process. At every stage, clean audio-video predictions enter a stage-local key-value history, allowing the next chunk to begin early. To preserve speech across changing text prompts, asymmetric distribution-matching distillation trains the generator on updated dialogue while sequence-level scores retain the opening text context; the model uses audio history to infer what has already been spoken, without ASR or an explicit text pointer. Progressive generation is 1.60× faster than direct high-resolution generation, and stage-parallel deployment adds a 1.92× speedup over serial progressive generation, reaching 8.99 FPS on eight NVIDIA H20 GPUs. Continuation training lowers character error rate from 30.75% to 19.57%, an 11.18-percentage-point reduction, and the system demonstrates audiovisual generation lasting 60 minutes.

Original abstract

Streaming digital humans produce detailed video while continuing speech as new text arrives. High-resolution denoising is costly, and text updates can cause repeated or interrupted speech. We present TaoMate-HD, an autoregressive model for high-resolution audio-video streaming. For each short segment, or chunk, two denoising steps establish coarse video structure at low resolution. A final step refines the video to high resolution, with audio generated throughout the same trajectory. Each stage passes its audiovisual predictions to the same stage of the next chunk as history. Across text updates, the model retains this history to infer what has already been spoken. Continuation training utilizes asymmetric text conditioning, with separate text windows for generation and distillation. Experiments show a 1.6x speedup over direct high-resolution generation with the same model and hardware. Optimized stage-parallel deployment provides a further 1.9x overall speedup over serial progressive generation. Our continuation training reduces character error rate by 11.18 percentage points relative to the baseline. The model also supports hour-long audiovisual generation. Our project page is https://taoliveaigc.github.io/TaoMate-HD.

Read the original paper

More in Multimodal AI

Browse all 79 papers →
01Multimodal

ALIVE: Making Streaming Video Models Interactive

Jinzhao Li, Yuan Xu, Jinxuan Wu, Yukun Wang, Yijia Lei, Hanyu Ji, Yinuo Chen, Zining Huang, Chi Zhang, Hang Ye, Yiwu Zhong, Honglei Yan, Panwang Pan, Yizhou Wang, Yin Li, Miao Liu

ALIVE aims to make video-streaming AI an interactive partner that can monitor events, coordinate tasks, and ask for feedback.

Read analysis