Linear systems are the bedrock of virtually all numerical computation. Machine learning poses specific challenges for the solution of such systems due to their scale, characteristic structure, stochasticity and the central role of uncertainty in the field. Unifying earlier work we propose a class of probabilistic linear solvers which jointly infer the matrix, its inverse and the solution from matrix-vector product observations. This class emerges from a fundamental set of desiderata which constrains the space of possible algorithms and recovers the method of conjugate gradients under certain conditions. We demonstrate how to incorporate prior spectral information in order to calibrate uncertainty and experimentally showcase the potential of such solvers for machine learning.
Jonathan Wenger (University of Tübingen)
Philipp Hennig (University of Tübingen and MPI for Intelligent Systems Tübingen)
More from the Same Authors
2019 Poster: Limitations of the empirical Fisher approximation for natural gradient descent »
Frederik Kunstner · Philipp Hennig · Lukas Balles
2019 Poster: Convergence Guarantees for Adaptive Bayesian Quadrature Methods »
Motonobu Kanagawa · Philipp Hennig