NTH

TTB: Test-time Baking for Efficient Rendering of Decoder-only View Synthesis Models

AuthorsChin-Yang Lin, Ryo Hachiuma, Jaesung Choe, Min-Hung Chen, Yu-Chiang Frank Wang, Yen-Yu Lin, Wei-Chen Chiu, Yu-Lun Liu, Cheng Sun

AffiliationsNVIDIA · Department of Computer Science, National Yang Ming Chiao Tung University

November 13, 2026 2 min read
Watch on YouTube
The one-line take

TTB speeds up novel-view rendering by baking scene-specific cross-view attention into a small MLP.

Key results

23
Rendering speedup over DVSM

× faster at 960p on DL3DV with 64 context views.

31.33
DL3DV PSNR

Local-duplicate TTB result at 960p with 64 context views.

85
Scene-state reduction

× smaller than DVSM’s KV-cache at 64 context views.

What the paper found

Test-time Baking, or TTB, speeds up decoder-only view synthesis by using a fast-weight MLP to learn the outputs of cross-view attention rather than replacing attention outright. During scene prefilling, it pairs each context view’s camera-only tokens with image-and-camera tokens, training the MLP to map between them; novel views then use the same camera-only input, avoiding a mismatch between baking and rendering. This shifts computation to scene setup while giving rendering constant complexity in the number of context views. On DL3DV at 960p with 64 context views, TTB’s local-duplicate variant reaches 31.33 PSNR and renders at 23 FPS—23× faster than DVSM. The duplication strategy groups nearby cameras so each MLP handles fewer views; the resulting scene state is 85× smaller than DVSM’s KV-cache at 64 views. TTB also performs well on Re10K and in-the-wild datasets, while its attention-based baking makes scene prefilling more expensive. The work reports training on NVIDIA A100 GPUs and notes that Claude Code was used for coding assistance, not research ideation or model design.

Original abstract

Decoder-only Large View Synthesis Models (LVSMs) with KV-cache have recently achieved state-of-the-art quality by regressing novel views from neural networks without reconstructing 3D geometry, while Test-Time Training (TTT) layers save computation but sacrifice quality. We propose Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding. The first key is to repurpose the fast-weight MLP of TTT to learn the cross-view attention mapping explicitly from its input-output pairs, rather than replacing attention entirely, achieving the same rendering efficiency as TTT while delivering substantially higher quality. The second key addresses a modality misalignment between context prefilling and novel-view rendering: we disentangle context tokens into camera-only context rendering tokens to serve as TTB input, and context ground-truth tokens with image content to serve as the TTB mapping target, ensuring the baked MLPs are relieved of the modality generalization challenge between context and novel-view processing. Our method achieves up to **23x** rendering speedup with quality on par with the state-of-the-art across several benchmarks. Project page: https://test-time-baking.github.io

Read the original paper

More in Computer Vision

Browse all 82 papers →