Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training

Machine learning systems often acquire biases by leveraging undesired features in the data, impacting accuracy variably across different sub-populations. This paper explores the evolution of bias in a teacher-student setup modeling different data sub-populations with a Gaussian-mixture model, by providing an analytical description of the stochastic gradient descent dynamics of a linear classifier in thissetting. Our analysis reveals how different properties of sub-populations influence bias at different timescales, showing a shifting preference of the classifier during training. We empirically validate our results in more complex scenarios by training deeper networks on real datasets including CIFAR10, MNIST, and CelebA.