VISTA: A Panoramic View of Neural Representations

We present VISTA (Visualization of Internal States and Their Associations), a novel pipeline for visually exploring and interpreting neural network representations. VISTA addresses the challenge of analyzing vast multidimensional spaces in modern machine learning models by mapping representations into a semantic 2D space. The resulting visual collages visually reveal patterns and relationships within internal representations. We demonstrate VISTA's effectiveness by applying it to sparse autoencoder latents uncovering new properties and interpretations. We review the VISTA methodology, present findings from our case study, and discuss implications for neural network interpretability across various domains of machine learning.