CCA with Shared Weights for Self-Supervised Learning

In this paper, we explore SSL-EY (Self-Supervised Learning with an Eckhart-Young characterization), a novel self-supervised learning loss function directly inspired by Deep Canonical Correlation Analysis (DCCA). Our key insight is that maximizing the correlation of learned representations can serve as both an effective and interpretable objective in self-supervised learning. We demonstrate that SSL-EY not only strengthens the theoretical underpinning of existing methods, such as Barlow Twins and VICReg, but also performs competitively on benchmark datasets.