NTH

Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis

AuthorsJulia Herbst, Simon Roschmann, Quentin Bouniot, Zeynep Akata

AffiliationsHelmholtz Munich · Technical University of Munich · Munich Center for Machine Learning · LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France

November 14, 2026 2 min read
Watch on YouTube
The one-line take

The study finds that language and vision models can develop surprisingly similar representations, but still fall short of the alignment seen between models trained on the same kind of data.

Key results

0.143
Peak WIT cross-modal alignment

Mutual k-NN score between DINOv3 and Qwen3-8B at their maximally aligned layers.

100
Effective alignment subspace

Approximate number of language representation dimensions that captures much of the cross-modal alignment.

0.66
DOCCI MMLU correlation

Pearson correlation between MMLU accuracy and alignment on detailed DOCCI captions.

0.73
DOCCI ARC Challenge correlation

Pearson correlation between ARC Challenge accuracy and alignment.

0.5
Same-modality alignment

Approximate mutual k-NN alignment between models operating within the same modality.

What the paper found

This study takes a layer-by-layer look at the Platonic Representation Hypothesis: whether vision and language models develop similar representations of the world. Comparing vision encoders with 21 generative language models—including Meta’s Llama, DeepSeek, Qwen, Mistral, and Gemma—using mutual k-nearest-neighbor overlap on WIT, CC3M, COCO, and DOCCI, the researchers find that cross-modal alignment mostly appears in later layers. Its peak on WIT is 0.143 between DINOv3 and Qwen3-8B, while principal-component analysis shows that alignment can largely be retained in about 100 dimensions, despite language representations spanning 1,536–5,120 dimensions. The link between language-model performance and alignment depends on caption quality: on detailed DOCCI captions, alignment correlates with MMLU accuracy at Pearson r = 0.66, and with ARC Challenge accuracy at r = 0.73, but does not consistently track GSM8K math performance. For context, unimodal vision-language pairs exceed 0.3 alignment, outperforming CLIP and SigLIP 2 on COCO and DOCCI, but remaining below Qwen3-VL-Embedding-8B and the roughly 0.5 alignment seen between models of the same modality. The results suggest that cross-modal alignment captures some shared semantic structure, not a universal measure of model capability.

Original abstract

The Platonic Representation Hypothesis (PRH) suggests that neural networks are converging toward similar representation spaces. As language models become more capable, their representations are aligning with those of vision models. However, recent work has questioned whether this trend continues to hold for newer language models. Moreover, it has been shown that representation similarity metrics can be confounded by model depth and width, requiring appropriate baselines for interpretation. In this work, we first investigate where alignment emerges between unimodal models across model layers and representation dimensions. Next, we find that the PRH capability-alignment trend persists for newer models as long as the captions are sufficiently clean and informative. Finally, we contextualize cross-modal alignment, introducing unimodal and multimodal baselines on various datasets. Our results show that the cross-modal representational alignment of unimodal models can surpass that of explicitly aligned vision-language models, but remains substantially behind the alignment observed for models of the same modality. Code is available at https://github.com/ExplainableML/prh-closer-look.

Read the original paper

More in Multimodal AI

Browse all 79 papers →
01Multimodal

ALIVE: Making Streaming Video Models Interactive

Jinzhao Li, Yuan Xu, Jinxuan Wu, Yukun Wang, Yijia Lei, Hanyu Ji, Yinuo Chen, Zining Huang, Chi Zhang, Hang Ye, Yiwu Zhong, Honglei Yan, Panwang Pan, Yizhou Wang, Yin Li, Miao Liu

ALIVE aims to make video-streaming AI an interactive partner that can monitor events, coordinate tasks, and ask for feedback.

Read analysis