Timezone: »

 
Poster
On UMAP's True Loss Function
Sebastian Damrich · Fred Hamprecht

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual
in Poster Session 1 »
UMAP has supplanted $t$-SNE as state-of-the-art for visualizing high-dimensional datasets in many disciplines, but the reason for its success is not well understood. In this work, we investigate UMAP's sampling based optimization scheme in detail. We derive UMAP's true loss function in closed form and find that it differs from the published one in a dataset size dependent way. As a consequence, we show that UMAP does not aim to reproduce its theoretically motivated high-dimensional UMAP similarities. Instead, it tries to reproduce similarities that only encode the $k$ nearest neighbor graph, thereby challenging the previous understanding of UMAP's effectiveness. Alternatively, we consider the implicit balancing of attraction and repulsion due to the negative sampling to be key to UMAP's success. We corroborate our theoretical findings on toy and single cell RNA sequencing data.
Keywords: Optimization · Self-Supervised Learning · Graph Learning

Author Information

Sebastian Damrich (Heidelberg University)
Fred Hamprecht (Heidelberg University)

More from the Same Authors