UMAP 0.6 is a major update to the dimensionality-reduction library, focused on making embeddings faster, more reproducible, and more informative at very large scale. On more than three million 4,096-dimensional arXiv text embeddings, the new pipeline completes in under five minutes, while UMAP 0.5 had not yet finished finding nearest neighbours. Neighbour search is about four times faster, graph construction and initialization are more efficient, and the optimizers can use all available cores while still producing repeatable results with a fixed seed. The release also reports better neighbourhood preservation and finer cluster structure, despite the speed gains.
Two changes underpin much of the improvement. Recursive initialization builds and refines a coarse layout using UMAP itself, giving the optimizer a stronger starting point than spectral or PCA initialization while avoiding some of spectral initialization’s failures on large graphs. The new optimizers use hard-negative mining to focus repulsive sampling on nearby points, where it has more effect, and support much higher repulsion strengths. These changes are opt-in for now: setting `compatibility_layout=False` enables the new pipeline, while the legacy layout remains available for compatibility. The release also adds parallel, reproducible transforms and DensMAP, an optional nearest-neighbour backend, faster inverse transforms, memory and performance improvements, and numerous bug fixes. It requires newer NumPy, SciPy, Numba, and PyNNDescent versions.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.