Proving and Improving the Platonic Representation Hypothesis with JEPAs

The paper turns the Platonic Representation Hypothesis - the idea that models trained on different kinds of data eventually learn a shared representation of the world - into a formal result. It proves that separately trained Joint-Embedding Predictive Architectures (JEPAs) learn representations equivalent up to a rotation when their training pairs are “good views,” such as two observations of the same scene; when the views are less aligned, the theorem predicts bounded disagreement. The authors also recast next-token language modeling as a JEPA trained on “bad views”: different parts of a document. This, they argue, explains why language models align with visual representations only after substantial scaling.

Experiments spanning 145,000 captions to text associated with 940 million web images support the theory and reveal a scaling law that predicts alignment in unseen training runs. On the same data, good-view training needs 42 times fewer examples than next-token prediction to reach comparable alignment. A 109-million-parameter text JEPA aligns with DINOv2 and CLIP image encoders better than all tested language models, including models up to 72 billion parameters, and better than CLIP’s text encoder. A single learned rotation can connect separately trained image and text encoders for captioning, achieving a CIDEr score of .838 on COCO. The text JEPA also exceeds reported language-model results on a pair-free zero-shot benchmark, reaching 64% top-5 accuracy on ImageNet-100 versus at most 56% for models up to 8 billion parameters. The central conclusion is that informative paired views, not scale alone, are key to shared representations.