NTH

Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

AuthorsDaniel Huang, Qi He, Shaoxiong Duan, Yingjie Li, Qiao Cheng, Yitao Duan

November 15, 2026 2 min read
Watch on YouTube
The one-line take

Confucius4-R2T2 transcribes speech in real time while keeping previously emitted words fixed.

Key results

1.5K
Post-training audio

Hours of Chinese and English audio used to adapt Qwen3-ASR-1.7B.

160
Main streaming chunk size

Milliseconds per chunk for the reported 5.76% macro error.

5.76%
R2T2 macro error

English and Chinese benchmark average at 160 ms chunks.

21.01%
Base model macro error

Unadapted Qwen3-ASR-1.7B at the same 160 ms chunk size.

5.15%
Full-context base macro error

Offline Qwen3-ASR-1.7B reference score.

0.5
Mean emission-lag threshold

Seconds; R2T2 stays below this at 80 ms chunks in both languages.

What the paper found

Confucius4-R2T2 turns Qwen3-ASR-1.7B into a streaming recognizer whose committed words are never revised. Its central technique, Longest Stable Prefix learning, labels each partial-audio input with the longest earlier model output that is also a prefix of the model’s full-audio transcript; the labels are monotone, and an explicit Wait token teaches the model to defer uncertain text. The method post-trains on 1.5K hours of Chinese and English audio using supervised fine-tuning followed by Group Relative Policy Optimization, while freezing the audio encoder and projector and making no architectural changes. At 160 ms chunks, R2T2 records 5.76% macro error across the English and Chinese benchmarks, compared with 21.01% for the unadapted model at the same chunk size and 5.15% for the base model in full-context mode. It also outperforms evaluated streaming systems including NVIDIA Nemotron 3.5 and Mistral’s Voxtral Realtime on the aggregate cohort. With 80 ms chunks, mean retrospective emission lag stays below 0.5 seconds in both languages; the model’s streaming gains also transfer to 10 additional languages, improving results across 18 test sets. The work shows that stable streaming can be learned through supervision and decoding policy while retaining a full-context encoder, rather than requiring a causalized architecture.

Original abstract

Streaming speech recognition is increasingly consumed by machines, such as translators and voice agents, that act on each token as it appears. A token that is later revised costs wasted work or a wrong action. In this work, we present Confucius4-R2T2 (short for Real Real-Time Transcription), a stable-prefix streaming ASR model that never modifies its earlier output while maintaining high accuracy and low latency. R2T2 is obtained by post-training Qwen3-ASR-1.7B on 1,500 hours of Chinese and English audio while freezing its pre-trained audio encoder and projector. The adaptation is based on the concept of Longest Stable Prefix (LSP), a new notion that we introduce in this work. We prove that LSP is monotone under the prefix order, consistent with the full-audio output, and maximal among labels that equal a whole earlier output. Learning from LSP labels converts a full-context model into a stable-prefix streaming one without any architectural change. Extensive evaluations show that R2T2 brings streaming recognition close to the base model's full-context quality while delivering stable text with low emission lag. Its gains extend beyond the post-training languages, demonstrating a practical path from full-context ASR to stable streaming. We open-source the model weights and code at https://huggingface.co/netease-youdao/Confucius4-R2T2 and https://github.com/netease-youdao/Confucius4-R2T2.

Read the original paper

More in Speech AI

Browse all 48 papers →
02Speech

Marimba: A System One Model for Speech

Solano Amadori

Marimba reuses a speech recognition model’s features to answer questions about words, speakers, language, and turn endings.

Read analysis
03Speech

VoxMel TTS Technical Report

Wenhao Guan, Jiaxin Ye, Haitao Li, Yifan Duan, Junxi Liu, Wangjin Zhou, Lin Li, Qingyang Hong, Xie Chen

VoxMel generates speech by autoregressively modeling transformed mel-spectrograms, avoiding a learned tokenizer while aiming for high fidelity and faster inference.

Read analysis