NTH

WorldRuler: Interactive World Modeling under Configurable Rules

AuthorsMingliang Zhai, Chuanhao Li, Jiaming Tan, Zhen Li, Yuwei Wu, Yunde Jia, Zian Meng, Yuanyang Yin, Kaipeng Zhang

AffiliationsAlaya Lab · Beijing Institute of Technology · Shanghai Innovation Institute · The University of Tokyo · Shenzhen MSU-BIT University

November 20, 2026 2 min read
Watch on YouTube
The one-line take

WorldRuler aims to make generated interactive worlds obey customizable rules, even after saving and restoring the world.

Key results

3,200
RuleVerse pairs

Paired rule-driven interactions collected in GTA-V and Habitat.

64.99%
Habitat average RSR

WorldRuler’s average Rule Success Rate on the Habitat subset.

44.06%
SWM Habitat average RSR

Baseline average Rule Success Rate on the Habitat subset.

67.42%
GTA-V average RSR

WorldRuler’s average Rule Success Rate on the GTA-V subset.

98.17%
Save/load consistency

Consistency when jointly restoring rules, execution state, and bindings.

What the paper found

WorldRuler targets a key weakness in interactive video world models: a user-defined rule must keep working after a save, while its effects must appear on the correct object—even if that object is initially off-screen. It represents rules, dependencies, object identities, states, and visual bindings in an executable 3D rule graph that can be saved and restored together. During video denoising, object attribute state replacement enforces the rule’s target state; counterfactual reconstruction and attribute swapping train compatible visual changes while limiting edits to unrelated content. Perceptual feedback updates bindings and checks outcomes across generation windows. The team built RuleVerse from paired interactions in GTA-V and Habitat, with 3,200 pairs annotated for states and events. Using MiniMax-H3 Ref2VA Turbo for video generation, alongside Qwen3.8-Max and SAM2 in the observation pipeline, WorldRuler achieved a 64.99% average rule success rate on Habitat, compared with 44.06% for SWM, and 67.42% on GTA-V. Jointly restoring rules, execution state, and bindings yielded 98.17% consistency after loading, showing how explicit rule memory can connect persistent gameplay logic to generated visuals.

Original abstract

Rules govern how player actions change interactive worlds. Different rules create distinct gameplay experiences within similar scenes and action spaces. For customizable gameplay, generative world models must follow user-defined rules beyond real-world behaviors. However, realizing user-defined rules presents two challenges: Rule-State Persistence and Rule-to-Visual Realization. The former requires preserving rule definitions, dependencies, and world states across saving and loading, so that restored worlds follow the established rules. The latter requires accurately realizing rule-induced changes on designated objects and attributes, particularly when affected objects are unseen. In this paper, we propose WorldRuler, which organizes rule definitions, dependencies, and object states in an executable 3D rule graph and connects rule execution, video generation, and perceptual feedback in a closed loop. The graph supports joint saving and restoration of rules and world states, while dynamic object binding grounds rule-specified entities in visual instances and uses stored states to constrain generation from their first appearance. We further introduce object attribute state replacement and counterfactual reconstruction training, enabling accurate attribute control and compatible appearance and motion changes while limiting interference with unrelated content. We develop an automated data collection and annotation pipeline using GTA-V and Habitat to construct RuleVerse, a dataset of rule-driven interactions with detailed state and event annotations, including cases where rules take effect before affected objects become visible. Extensive experiments demonstrate the effectiveness of WorldRuler in preserving rule-state consistency across saving and loading and accurately realizing rule outcomes on designated objects and attributes.

Read the original paper

More in World Models

Browse all 59 papers →
01World Model

Duo: A Reciprocal World Model for Full-Duplex Interaction

Hengyuan Cao, Biao Gong, Guoshan Lu, Qiyuan Zhang, Lin Liu, Yiding Tian, Yilei Jiang, Yuxin Ma, Chengzhang Wu, Kun Chen, Yiqian Xie, Ting Liu, Xin Zheng, Xijun Gu, Tianren Ma, Jiancheng Pan, Chi Zhang, Xuzhe Zheng, Siyi Li, Zhenzhong Lan

Duo aims to create an always-on audiovisual assistant that can perceive, reason, and respond continuously, including during interruptions.

Read analysis
02World Model

PanDWM: Panoramic 4D Driving World Models

Jiajun Sun, Zeyu Zhang, Zeyu Ren, Guangfeng Jiang, Yang Zhao, Hao Tang

PanDWM generates a continuous 360-degree driving view and reconstructs it into a fast 4D scene.

Read analysis