Poster
in
Affinity Workshop: Women in Machine Learning

The Representations of Deep Neural Networks Trained on Dihedral Group Multiplication

Sihui Wei · Harley Wiltzer · Zhaoyue(Rebecca) Wang · Irina Rish

Project Page

Abstract

We find coset and approximate coset circuits play a key role in how multilayer perceptrons learn dihedral group multiplication, consistent with recent findings on modular addition. We identify that neural preactivations concentrate on (approximate) cosets and visualize the manifolds distributed across neurons that correspond to the (approximate) coset representations.