NTH

JoVox: Joint Hierarchical Flow Matching for Lightweight Voxel Generation

AuthorsXiangjun Tang, Jianqi Chen, Peter Wonka

AffiliationsKing Abdullah University of Science and Technology (KAUST) · xiangjun.tang@outlook.com windvchen@gmail.com pwonka@gmail.com

November 10, 2026 2 min read
Watch on YouTube
The one-line take

JoVox generates detailed voxel shapes more efficiently by letting fine-scale structure help guide and correct coarse decisions.

Key results

11%
Chamfer distance reduction vs. TRELLIS.2

Lower voxel Chamfer distance on image-conditioned Toys4k generation.

8×
Training throughput gain

Higher throughput than TRELLIS.2 with the same backbone.

0.4158
Voxel IoU

JoVox result on image-conditioned Toys4k generation; TRELLIS.2 scores 0.4017.

29.0
GPU memory at batch size 32

GB used on one NVIDIA A100, compared with 47.3 GB for TRELLIS.2.

4
Training GPUs

H100 GPUs used for JoVox training, versus 32 for TRELLIS.2.

What the paper found

JoVox tackles a weakness in hierarchical voxel generation: early coarse cells can be plausible but wrong, and conventional methods never revisit them. Its discrete flow-matching model learns the joint distribution across all levels, allowing finer cells to inform coarse-cell predictions. Two-level prediction generates occupancy codes two levels down at once, halving the number of levels processed, while an unmasking strategy revisits the oldest predictions as more detail becomes available. On image-conditioned Toys4k generation, JoVox reaches a voxel IoU of 0.4158 versus 0.4017 for TRELLIS.2 and an 11% lower Chamfer distance. It also delivers 8× higher training throughput; on one NVIDIA A100 at batch size 32, it uses 29.0 GB of memory versus 47.3 GB for TRELLIS.2. The model was trained on 4 H100 GPUs, compared with 32 for TRELLIS.2, and uses DINOv3 features for image conditioning. JoVox also outperforms baselines on unconditional ShapeNet generation, while avoiding an autoencoder and processing only cells within occupied regions.

Original abstract

Generating high-fidelity voxels while keeping their lightweight nature remains an open problem in computer vision. Current generators are either high-fidelity but computationally heavy, or lightweight but of lower fidelity. Hierarchical methods keep the lightweight nature of voxels. However, their coarse levels, which decide where all finer cells can appear, are generated with the least information about the shape and are never revisited, so they are often plausible but wrong. Our key insight is to let finer cells take part in deciding coarse cells. To this end, we propose JoVox, a discrete flow matching framework that learns the joint distribution of all levels of the voxel hierarchy, instead of a separate distribution for each level, so that coarse cells can be predicted with finer cells already generated nearby. Within this framework, we further introduce a two-level prediction, which outputs the cells two levels finer at once instead of one level finer, halving the number of levels that the network processes and yielding voxels of higher fidelity. We also introduce an unmasking strategy that revisits early decisions once finer cells are generated and corrects inconsistencies in masked diffusion models. Our framework outperforms the baselines on both unconditional and image-conditioned voxel generation. Compared with the current state-of-the-art TRELLIS.2, we achieve 8x higher training throughput, 1/15 of the FLOPs in a forward pass, train on 4 vs. 32 GPUs, and obtain higher quality with an 11% lower Chamfer distance.

Read the original paper

More in Generative Models

Browse all 87 papers →
01Generative Model

dNaViT: Discrete Codes are General-Purpose Visual Representations

Xiaotong Li, Zhengrong Yue, Xu Huang, Jialv Zou, Minhao Jing, Wenhao Yan, Haodong Li, Hai Huang, Bo Zhao, Yan Feng, Peng Pei, Xunliang Cai

dNaViT turns images into reusable discrete codes designed to work across visual understanding, generation, and editing.

Read analysis