Pictura: Perspective-View Self-Play at Scale for Driving

Pictura addresses a central flaw in driving self-play: privileged vector observations expose exact positions, velocities, and occluded agents, creating a gap between training and camera-based deployment. Its GPU-native CUDA rasterizer renders each vehicle’s egocentric perspective view, including vehicles, pedestrians, cyclists, traffic lights, parked cars, walls, and lane geometry, directly inside the reinforcement-learning loop. On a single H100, Pictura reaches 500 K agent-steps per second, or 2 M images per second. Using plain PPO, the Alberti policy jointly learns visual encoding and control from four low-resolution cameras, without privileged observations, a teacher, or distillation. Training covers 50 B agent steps and approximately 35 M km of synthetic driving on CARLA maps. In-domain, Alberti approaches a matched vectorized baseline, reaching 11.21 goals at medium traffic density versus 12.81, although its red-light violation rate is higher at 0.058 versus 0.009. The more important result appears in zero-shot transfer: on layouts re-rendered from the Waymo Open Motion Dataset, Alberti reaches 86% of goals at full traffic density, compared with 72% for the privileged agent, while reducing collisions to 0.006 and off-road events to 0.014. Counterfactual tests show that Alberti becomes insensitive to fully occluded agents and slows at blind corners, demonstrating decisions grounded in visible evidence rather than hidden simulator state.