TaoMate-HD: Progressive-Resolution Autoregressive Audio-Video Generation
AuthorsQijun Gan, Chenwei Zhang, Xinyang Li, Meiguang Jin, Junfeng Ma, Shen Qiu
AffiliationsNov 2026 1 2 Alibaba Group Nanjing University
TaoMate-HD generates high-resolution video and speech in streaming chunks, aiming to make digital humans faster and more coherent as text updates.
Key results
Two low-resolution evaluations followed by one high-resolution refinement.
Speedup over direct high-resolution generation.
Speedup over serial progressive generation.
Percentage-point reduction, from 30.75% to 19.57%.
Minutes of audiovisual generation.
What the paper found
TaoMate-HD, initialized from MiniMax-H3 FL2VA, targets high-resolution audio-video streaming without making each chunk wait for full visual refinement. Its 3-step denoising trajectory uses two low-resolution video evaluations at 480×864, followed by one refinement at 768×1376, while audio remains part of the joint generation process. At every stage, clean audio-video predictions enter a stage-local key-value history, allowing the next chunk to begin early. To preserve speech across changing text prompts, asymmetric distribution-matching distillation trains the generator on updated dialogue while sequence-level scores retain the opening text context; the model uses audio history to infer what has already been spoken, without ASR or an explicit text pointer. Progressive generation is 1.60× faster than direct high-resolution generation, and stage-parallel deployment adds a 1.92× speedup over serial progressive generation, reaching 8.99 FPS on eight NVIDIA H20 GPUs. Continuation training lowers character error rate from 30.75% to 19.57%, an 11.18-percentage-point reduction, and the system demonstrates audiovisual generation lasting 60 minutes.
Original abstract
Streaming digital humans produce detailed video while continuing speech as new text arrives. High-resolution denoising is costly, and text updates can cause repeated or interrupted speech. We present TaoMate-HD, an autoregressive model for high-resolution audio-video streaming. For each short segment, or chunk, two denoising steps establish coarse video structure at low resolution. A final step refines the video to high resolution, with audio generated throughout the same trajectory. Each stage passes its audiovisual predictions to the same stage of the next chunk as history. Across text updates, the model retains this history to infer what has already been spoken. Continuation training utilizes asymmetric text conditioning, with separate text windows for generation and distillation. Experiments show a 1.6x speedup over direct high-resolution generation with the same model and hardware. Optimized stage-parallel deployment provides a further 1.9x overall speedup over serial progressive generation. Our continuation training reduces character error rate by 11.18 percentage points relative to the baseline. The model also supports hour-long audiovisual generation. Our project page is https://taoliveaigc.github.io/TaoMate-HD.
Read the original paperMore in Multimodal AI
Browse all 79 papers →ALIVE: Making Streaming Video Models Interactive
Jinzhao Li, Yuan Xu, Jinxuan Wu, Yukun Wang, Yijia Lei, Hanyu Ji, Yinuo Chen, Zining Huang, Chi Zhang, Hang Ye, Yiwu Zhong, Honglei Yan, Panwang Pan, Yizhou Wang, Yin Li, Miao Liu
ALIVE aims to make video-streaming AI an interactive partner that can monitor events, coordinate tasks, and ask for feedback.
Let Samples Sing in Chorus: Mitigating Spurious Correlations via Cross-view Consensus
Weiwei Li, Junzhuo Liu, Tong Chu, Hengfu Yu, Yinjie Lei, Wen Li
CHORD combines clues from multiple representation spaces to help models rely less on misleading shortcuts and perform better on underrepresented groups.
LightTAR: Light and Fast Target-Aware Representations for Multimodal Tabular Learning
Eilam Shapira, Alan Arazi, Shoham Grunblat, Roi Reichart
LightTAR adds a small set of target-aware features to standard embeddings, helping tabular models make better predictions with little extra runtime.