MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

MobileGym introduces a browser-hosted Android-like simulation platform that makes everyday mobile GUI tasks both verifiable and massively parallel without replicating proprietary backends. Its core novelty is a layered structured-JSON state model in which app data, OS state, and device context are readable, writable, snapshotable, and forkable, enabling deterministic state-diff judging, side-effect detection, and exact reset from any snapshot. A single machine can host hundreds of instances at roughly 400 MB each with about 3 s cold start, compared with gigabyte-scale emulator overhead. The accompanying MobileGym-Bench contains 416 parameterized task templates across 28 apps, including 12 everyday apps and 16 system apps, with a typed AnswerSheet protocol that replaces brittle free-text answer matching and yields deterministic success, progress, false-completion, overdue, and unexpected-side-effect metrics. Across 9 evaluated agents, success rate ranges from 9.4% for Qwen3-VL-4B-Instruct to 58.8% for Gemini 3.1 Pro, while a GRPO fine-tuning run on Qwen3-VL-4B-Instruct improves test success from 9.4% to 22.2%, a gain of 12.8 percentage points. In a sim-to-real study on 59 real-device signal tasks, the same training retains 95.1% of the simulation-side gain, rising from 32.2% to 72.9% on device. The paper also reports 10.2% VLM-judge misjudgment on audited trajectories, motivating programmatic verification as the central technical contribution.