OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

OmniHallu presents a unified claim-level hallucination detector for multimodal large language models across six bidirectional tasks: image-to-text, video-to-text, audio-to-text, text-to-image, text-to-video, and text-to-audio. Its OmniHallu-Bench contains 10,000 human-annotated samples spanning image, video, and audio inputs and outputs. The method decomposes captions or prompts into atomic claims with GPT-4.1, verifies them using modality-specific experts such as Grounding DINO, Qwen2.5-VL-72B, VideoLLaMA3, and Qwen2-Audio-7B-Instruct, then aggregates evidence with GPT-5.2. Compared with Self-Check and UNIHD, it improves claim-level macro-F1 by 3.4 to 8.1 points, reaching 82.95 on image-to-text, 77.14 on video-to-text, and 74.11 on audio-to-text; the largest gains occur in audio, where individual models are weaker. The benchmark also reveals a performance gradient from image to video to audio, and from comprehension to generation. Removing atomic claim decomposition reduces macro-F1 by up to 7.93 points, confirming that fine-grained localization is central. A GRPO-trained verifier initialized from Qwen2.5-VL-7B acts as a low-cost filter, reducing expert calls by 66 percent while preserving performance. The evaluation includes outputs from major systems including OpenAI’s GPT-4.1, Google’s Gemini-2.5-Pro, DALL-E 3, Stable Diffusion 3.5 Large, and Open-Sora, positioning OmniHallu as a cross-modal evaluation framework rather than a detector tied to one model family.