Large Flow Language Models
AuthorsArnav Chavan, Chanhyuk Lee, Udbhav Bamba, Jaehoon Yoo, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim
AffiliationsUC Berkeley · Amazon · KAIST · Carnegie Mellon University
Lafa turns large language models into fast continuous-flow generators, reporting up to 2.54× throughput at comparable quality.
Key results
Lafa is trained at scales up to this parameter count.
Lafa uses fewer than this many tokens.
Lafa single-token decoding macro-average.
7B Lafa at block size 16 versus autoregressive decoding, at comparable accuracy.
Tokens per second for 7B Lafa at block size 16 on an NVIDIA H100.
What the paper found
Large Flow Language Models introduces Lafa, a blockwise continuous-flow language model that keeps causal attention over the prefix while denoising each block with bidirectional attention and learned, asynchronous tokenwise timing. Adapted from pretrained Qwen2.5 and Qwen3 models, Lafa combines autoregressive prediction with flow training, preserving prefix caching and improving the retained autoregressive capability. The models reach 8B parameters with fewer than 30B training tokens on Dolci-Instruct-SFT; distilling the flow into a flow map also enables one-step generation. For faster decoding, Lafa drafts tokens in parallel from the flow and verifies them with its own autoregressive predictor, preserving greedy output. On a NVIDIA H100, 7B Lafa at block size 16 achieves 2.54× the throughput of the autoregressive baseline at comparable accuracy, reaching 100.2 tokens per second versus 39.5. Across six benchmarks, the 8B model scores 63.1% in single-token decoding, comparable to Efficient-DLM despite using at least 18× fewer training tokens; the paper also compares its parallel decoding with DeepSeek-MTP and other methods.
Original abstract
Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.
Read the original paperMore in Large Language Models
Browse all 87 papers →Copy-Paste Language Models
Nathan Godey, Yoav Artzi
Copy-Paste Language Models add a low-overhead copying mechanism that improves language-model training and performance, especially in long-context and retrieval settings.
Retrieval Augmented Generation Compositionality
Ido Levy, Segev Shlomov, Leshem Choshen
A large-scale study finds that RAG techniques that work well alone may not work well together, and the best combination can change as the knowledge corpus grows.
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Junying Chen, Xinyuan Xie, Ziniu Li, Wenyuan Gu, Jianquan Li, Xiang Wan, Guangjun Yu, Ruoyu Sun, Haizhou Li, Benyou Wang
OnePO uses temporary teacher guidance and reinforcement learning alone to turn general LLMs into stronger medical specialists without the usual supervised fine-tuning stage.