See2Think: Do Multimodal Models Really Use Intermediate Visual States?

See2Think asks whether multimodal models genuinely use the sketches, highlights, crops, and annotations they generate during reasoning, rather than merely producing plausible visual traces. The authors from Shanghai AI Laboratory, Central South University, and partner universities introduce See2ThinkBench, a benchmark of 1200 visually dependent problems spanning 12 categories across 2D structured reasoning, 3D scenes, and real-world environments, and Visual Action-of-Thought, or VAoT, which records textual thoughts, structured visual actions, externally rendered states, and subsequent reasoning. They compare four systems—OpenAI’s GPT-5.5 and GPT-o3, Google’s Gemini 3.5 Flash, and Alibaba’s Qwen3-VL-32B-Instruct—under text-only CoT, action planning without rendering, closed-loop VAoT, and corrupted-feedback VAoT. No strategy consistently wins: GPT-5.5 reaches 50.2% with CoT but 44.6% with VAoT, while Gemini 3.5 Flash performs best with action planning without rendering at 58.2%. Process diagnosis finds that models usually select relevant operations, with overall Action Relevance at 0.985, but rendering is substantially weaker, with Render Faithfulness at 0.616. Crucially, visual-state utility differs from behavioral dependence: when task-relevant feedback is corrupted, accuracy in 3D scenes falls by 15.5 percentage points for trajectories with maximal Feedback Uptake, and corrupted states change semantic answers in 32.7% to 54.2% of cases across models. The central conclusion is that “thinking with images” is conditional and fragile; faithful rendering and correct downstream use, not action selection alone, determine whether intermediate visual states help.