MemDreamer: Decoupling Perception and Reasoning for Long Video Understanding via Hierarchical Graph Memory and Agentic Retrieval Mechanism

MemDreamer, developed by an Ant Group and Zhejiang University team, reframes long-video understanding as a decoupled perception-and-reasoning system instead of end-to-end token ingestion. A streaming perception model builds a purely textual Hierarchical Graph Memory with three levels—Video Root, Super Events, and Macro Events—plus leaf subgraphs whose entities, micro-events, and edges encode spatial, temporal, and causal structure. At inference, a separate reasoning model uses an agentic Observation-Reason-Action loop with seven tools, including hierarchical navigation, semantic search, time filtering, and graph traversal, so it can retrieve evidence without rereading raw video. On LVBench, the full Gemini-3.1-Pro configuration reaches 90.7, beating the strongest native baseline by 12.5 points and leaving only a 3.7-point gap to human experts; across the four benchmarks it reports 86.3 on LongVideoBench, 92.1 on Video-MME, and 88.2 on EgoSchema. The active reasoning context is only 5.9K to 6.3K tokens, versus 240K to 784K for end-to-end models, and the paper reports a 21.2-point jump for Qwen3-VL, from 63.6 to 84.8, when wrapped in MemDreamer. Ablations show that Hierarchical-Graph memory outperforms Flat-Chunk by 13.3 points, and the full tool suite outperforms vanilla embedding similarity by 20.2 points.