Timezone: »

 
Poster
On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification
Sanyam Kapoor · Wesley Maddox · Pavel Izmailov · Andrew Wilson

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #715
in Poster Session 5 »

Aleatoric uncertainty captures the inherent randomness of the data, such as measurement noise. In Bayesian regression, we often use a Gaussian observation model, where we control the level of aleatoric uncertainty with a noise variance parameter. By contrast, for Bayesian classification we use a categorical distribution with no mechanism to represent our beliefs about aleatoric uncertainty. Our work shows that explicitly accounting for aleatoric uncertainty significantly improves the performance of Bayesian neural networks. We note that many standard benchmarks, such as CIFAR-10, have essentially no aleatoric uncertainty. Moreover, we show that data augmentation in approximate inference softens the likelihood, leading to underconfidence and misrepresenting our beliefs about aleatoric uncertainty. Accordingly, we find that a cold posterior, tempered by a power greater than one, often more honestly reflects our beliefs about aleatoric uncertainty than no tempering --- providing an explicit link between data augmentation and cold posteriors. We further show that we can match or exceed the performance of posterior tempering by using a Dirichlet observation model, where we explicitly control the level of aleatoric uncertainty, without any need for tempering.

Keywords: Deep Learning · Probabilistic Methods · Bayesian deep learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Sanyam Kapoor (New York University)
Wesley Maddox (Jump Trading)
Pavel Izmailov (New York University)
Andrew Wilson (New York University)
Andrew Wilson

I am a professor of machine learning at New York University.

More from the Same Authors