Timezone: »
Computer simulations have proven a valuable tool for understanding complex phenomena across the sciences. However, the utility of simulators for modelling and forecasting purposes is often restricted by low data quality, as well as practical limits to model fidelity. In order to circumvent these difficulties, we argue that modellers must treat simulators as idealistic representations of the true data generating process, and consequently should thoughtfully consider the risk of model misspecification. In this work we revisit neural posterior estimation (NPE), a class of algorithms that enable black-box parameter inference in simulation models, and consider the implication of a simulation-to-reality gap. While recent works have demonstrated reliable performance of these methods, the analyses have been performed using synthetic data generated by the simulator model itself, and have therefore only addressed the well-specified case. In this paper, we find that the presence of misspecification, in contrast, leads to unreliable inference when NPE is used naïvely. As a remedy we argue that principled scientific inquiry with simulators should incorporate a model criticism component, to facilitate interpretable identification of misspecification and a robust inference component, to fit ‘wrong but useful’ models. We propose robust neural posterior estimation (RNPE), an extension of NPE to simultaneously achieve both these aims, through explicitly modelling the discrepancies between simulations and the observed data. We assess the approach on a range of artificially misspecified examples, and find RNPE performs well across the tasks, whereas naïvely using NPE leads to misleading and erratic posteriors.
Author Information
Daniel Ward (University of Bristol)
PhD student studying at University of Bristol on the COMPASS CDT. Interests include: - Simulation-based/approximate Bayesian inference - Normalising flows - Model misspecification
Patrick Cannon (Improbable)
Mark Beaumont (University of Bristol)
Matteo Fasiolo
Sebastian Schmon (Improbable / Durham University)
More from the Same Authors
-
2022 : Approximate Bayesian Computation for Panel Data with Signature Maximum Mean Discrepancies »
Joel Dyer · John Fitzgerald · Bastian Rieck · Sebastian Schmon -
2022 : SurviVAEl: Variational Autoencoders for Clustering Time Series »
Stefan Groha · Alexander Gusev · Sebastian Schmon