How to win a beer with high-dimensional statistics

A recent experiment challenged a striking finding about embeddings: months of the year map to a circle in PCA space and produce an approximately circulant Gram matrix. Using 10,000-dimensional word2vec embeddings from a 25,000-word vocabulary, ten seemingly unrelated words were selected so their embedding vectors project to a convincing circle and yield a near-circulant Gram matrix. The search progressed from blind random sampling to a “looks circular” objective and then to an iterative pursuit algorithm that drops the worst point and replaces it from the vocabulary, finally optimizing against a target circulant Gram. The embedding dimension never mattered; the whole search ran in an afternoon with a coding agent and succeeded in producing a clear visual circle and matching Gram structure.

The result demonstrates that low-dimensional geometric patterns can be spurious: targeted searches can manufacture circulant structure without the semantic relationship that produced the original months result. Important caveats remain - the sinusoidal amplitude of off-diagonals is weaker than in the months example, the method was validated for ten points and likely fails at much larger sets, and it does not falsify broader findings like year-wise geometry. The practical takeaways are concrete: representation-geometry claims demand stronger statistical controls, and automated feature-finding or interpretable pipelines must account for pursuit-matching false positives.