Timezone: »
Are you using test log-likelihood correctly?
Sameer Deshpande · Soumya Ghosh · Tin Nguyen · Tamara Broderick
Event URL: https://openreview.net/forum?id=6fHJniMUra »
Test log-likelihood is commonly used to compare different models of the same data and different approximate inference algorithms for fitting the same probabilistic model. We present simple examples demonstrating how comparisons based on test log-likelihood can contradict comparisons according to other objectives. Specifically, our examples show that (i) conclusions about forecast accuracy based on test log-likelihood comparisons may not agree with conclusions based on other distributional quantities like means; and (ii) that approximate Bayesian inference algorithms that attain higher test log-likelihoods need not also yield more accurate posterior approximations.
Author Information
Sameer Deshpande (University of Wisconsin--Madison)
Soumya Ghosh (MIT-IBM Watson AI Lab, IBM Research)
Tin Nguyen (MIT)
Tamara Broderick (MIT)
More from the Same Authors
-
2021 : Measuring the sensitivity of Gaussian processes to kernel choice »
Will Stephenson · Soumya Ghosh · Tin Nguyen · Mikhail Yurochkin · Sameer Deshpande · Tamara Broderick -
2022 : Gaussian processes at the Helm(holtz): A better way to model ocean currents »
Renato Berlinghieri · Tamara Broderick · Ryan Giordano · Tamay Ozgokmen · Kaushik Srinivasan · Brian Trippe · Junfei Xia -
2022 : Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem »
Jason Yim · Brian L Trippe · Doug Tischer · David Baker · Tamara Broderick · Regina Barzilay · Tommi Jaakkola -
2022 : Double trouble: Predicting new variant counts across two heterogeneous populations »
Yunyi Shen · Tamara Broderick -
2022 Poster: Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting »
Prasanna Sattigeri · Soumya Ghosh · Inkit Padhi · Pierre Dognin · Kush Varshney -
2021 Workshop: Your Model is Wrong: Robustness and misspecification in probabilistic modeling »
Diana Cai · Sameer Deshpande · Michael Hughes · Tamara Broderick · Trevor Campbell · Nick Foti · Barbara Engelhardt · Sinead Williamson -
2020 Poster: Approximate Cross-Validation for Structured Models »
Soumya Ghosh · Will Stephenson · Tin Nguyen · Sameer Deshpande · Tamara Broderick -
2019 Poster: Statistical Model Aggregation via Parameter Matching »
Mikhail Yurochkin · Mayank Agarwal · Soumya Ghosh · Kristjan Greenewald · Nghia Hoang -
2012 Poster: From Deformations to Parts: Motion-based Segmentation of 3D Objects »
Soumya Ghosh · Erik Sudderth · Matthew Loper · Michael J Black -
2011 Poster: Spatial distance dependent Chinese Restaurant Process for image segmentation »
Soumya Ghosh · Andrei B Ungureanu · Erik Sudderth · David Blei