dNaViT: Discrete Codes are General-Purpose Visual Representations
AuthorsXiaotong Li, Zhengrong Yue, Xu Huang, Jialv Zou, Minhao Jing, Wenhao Yan, Haodong Li, Hai Huang, Bo Zhao, Yan Feng, Peng Pei, Xunliang Cai
dNaViT turns images into reusable discrete codes designed to work across visual understanding, generation, and editing.
Key results
dB for dNaViT-G64 reconstruction.
Score for dNaViT-8B visual understanding.
Overall score for dNaViT-8B image generation.
Score for dNaViT-8B, above the reported open-weight baselines.
Image-editing score for dNaViT-8B.
What the paper found
dNaViT asks whether a single discrete image vocabulary can preserve both semantic meaning and pixel-level detail. It keeps a pretrained Qwen3-VL vision encoder frozen, enriches its features with a zero-initialized residual modulator, then applies grouped product quantization to expand code capacity without lengthening the spatial token grid. A staged dual-branch objective trains the same codes to reconstruct semantic features and image pixels; a code-conditioned flow refiner sharpens decoded images. At a spatial downsampling factor of 32, dNaViT-G64 reaches 28.12 dB PSNR and 0.81 SSIM at 512×512 on ImageNet-1K. Paired with Qwen3-8B, it scores 885 on OCRBench and 96.2 on DocVQA, putting it near Qwen3-VL-8B on document understanding. For image generation, dNaViT-8B scores 49.5 on GenEval2 and 52.17 on Qwen-Image-Bench, exceeding open-weight baselines on the latter while trailing OpenAI’s GPT Image 1. On GEdit-Bench, it earns an overall score of 7.30. Across 12 visual-understanding benchmarks, 4 generation benchmarks, and an editing benchmark, the results support using one discrete representation across reconstruction, understanding, generation, and editing.
Original abstract
Discrete autoregressive modeling, notably through next-token prediction, has become a cornerstone of modern language models. Extending this paradigm to vision allows visual and language models to converge on a unified framework, thereby fostering shared methodology and infrastructure. This work investigates a central question: how far can a finite discrete vocabulary go in representing the richness of the visual world? We present dNaViT, a Discrete Native-resolution Vision Transformer that produces discrete visual primitives analogous to language tokens. Built upon a frozen pretrained encoder, dNaViT applies lightweight residual modulation with a dual-branch objective to inject reconstruction-aware cues into semantic features. By leveraging grouped product quantization, it scales discrete capacity without inflating spatial sequence lengths, while a balanced training recipe maintains both semantic and pixel-level information. Consequently, dNaViT functions as a versatile discrete visual tokenizer, enabling unified encoding across downstream tasks alongside high-fidelity image decoding. Extensive evaluations show that dNaViT matches or exceeds task-specific baselines across 12 visual understanding benchmarks, 4 generation benchmarks, and an image editing benchmark. These findings demonstrate that a compact discrete vocabulary can serve as a universal visual currency and unlock the full potential of fully unified, native multimodal methodology.
Read the original paperMore in Generative Models
Browse all 87 papers →Lipschitz-Controlled Flow Matching for Robust Image Restoration
Sedjro Salomon Hotegni, Konstantin Sonntag, Paul-Christian Bürkner, Sebastian Peitz
LipsFlower aims to make flow-matching image restorations less sensitive to small input changes without sacrificing perceptual quality.
PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models
Yuxuan Chen, Xiaojie Li, Hao Chen, Juncheng Ma, Kewei Zhang, Zhongjing Du, Guoqiang Wei, Fangfang Wang, Zhen Dong, Zhiyang Dou, Liming Zhao, Yan Zeng, Jianwei Zhang, Daquan Zhou
PhysLoop uses LLMs’ physics knowledge to help video generators create and repair more physically consistent videos.
FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation
Yijing Lin, Zhendong Mao, Mengqi Huang, Yuxin Wang, Yongdong Zhang
FrameForge uses video models’ sense of temporal change to better generate images that preserve a visual input while following new instructions.