Tetris3D: 3D Scene Generation With Objects That Fit Together
AuthorsJaeyeong Kim, Jinhyuk Jang, Jongmin Lee, Kyehong Park, Seungryong Kim
AffiliationsKAIST AI
Tetris3D generates 3D scenes whose objects physically fit together by conditioning each object on the geometry and interactions of its neighbors.
Key results
Physically simulated scenes in the training dataset.
Annotated samples generated from ComOb scenes.
Tetris3D scene-level F1 score.
Tetris3D inter-object penetration depth.
Tetris3D scene-level F1 score on a real-world benchmark.
Tetris3D inter-object penetration depth on a real-world benchmark.
What the paper found
Tetris3D reconstructs a complete 3D scene from one image by generating objects in sequence, with each new shape guided by neighboring geometry and physical relations such as stacking, leaning, and containment. A vision-language model infers which objects support others and sets their generation order; voxel-aligned conditions encode nearby surfaces, while visible-to-invisible attention helps fill in occluded regions. Built on TRELLIS.2, with image features from DINOv3, the system is trained on ComOb, a physics-simulated dataset containing 1.2M scenes and 3.4M annotated samples. On Toys4K, Tetris3D reports a scene-level F1 score of 0.8407 and penetration depth of 0.0036; on MessyKitchens, it reaches an F1 score of 0.9620 and penetration depth of 0.0011. These results indicate more accurate reconstruction and less interpenetration than the compared methods. The paper also qualitatively compares Tetris3D with an agentic Blender workflow using OpenAI’s GPT-6 Astra, reporting that the agent often misplaces objects or produces floating and intersecting geometry. The key contribution is to use physical interaction as a cue for recovering both hidden object shape and scene arrangement, rather than checking physical plausibility only after generation.
Original abstract
We propose Tetris3D, a generative framework for single-image 3D scene reconstruction that recovers objects which are physically and geometrically coherent as a scene. Existing methods often generate objects independently or couple them implicitly, providing limited guidance for ensuring fine-grained spatial compatibility between neighboring objects that interact with one another. To address this, we explicitly condition the generation of each object on the geometry of surrounding objects and their physical relationships, guiding its shape and pose to remain geometrically and physically plausible within the scene. Moreover, we introduce ComOb, a physics simulation-based dataset of 1.2M scenes featuring physical interactions across diverse object categories, with per-object meshes and pairwise physical relation annotations. Comprehensive experiments on synthetic and realworld scenes show that Tetris3D recovers coherent object shapes and poses even when interacting regions are occluded, and achieves state-of-the-art performance in both generation quality and physical stability.
Read the original paperMore in Computer Vision
Browse all 61 papers →JLD: Perceptual Distance Through A Jacobian Lens
Shreshth Saini, Balu Adsumilli, Alan C. Bovik
JLD uses the Jacobian of a frozen vision model to build a faster, more resolution-robust measure of how visually different two images appear.
LVMT: Video Mask Transformer for Long-term Video Segmentation
Narges Norouzi, Niccolò Cavagnero, Idil Esen Zulfikar, Bastian Leibe, Gijs Dubbelman, Daan de Geus
LVMT makes long-term video object segmentation more capable and much faster by selectively remembering object information while training efficiently on long videos.
All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts
Xingsong Ye, Yongkun Du, Jiaxin Zhang, Zhixian Li, Chong Sun, Chen Li, Jing Lyu, Lianwen Jin, Zhineng Chen
A lightweight script-aware mixture-of-experts model brings more accurate, scalable multilingual scene text recognition to many languages and scripts.