Bayesian Backprop as Belief Propagation: Single-Pass Predictive Uncertainty

Modern ML systems increasingly use uncertainty to decide when to abstain, route to a fallback, retrieve more context, or spend additional computation. Yet many strong Bayesian deep-learning recipes make uncertainty expensive on the serving path: ensembles, posterior samples, and MC dropout require repeated forward passes, while fast deterministic confidence heads are not generally posterior-predictive computations over parameter beliefs. We introduce \emph{Bayesify}, a message-passing view that turns a deterministic neural computation graph into a single-pass posterior-predictive moment graph. Trainable tensors become Gaussian beliefs, activations carry selected moments, and prediction is one deterministic forward propagation of those moments. Theoretically, we show that exact belief propagation is natural-gradient message passing in exponential-family mean space, and that projected EP is implemented by reverse-mode adjoints followed by local Fisher moment updates. This yields a practical wrapper for dense, convolutional, residual, and adapter modules. Experiments evaluate calibrated uncertainty, selective prediction, and OOD behavior at one-pass inference cost, comparing against deterministic calibration, ensembles, dropout, Laplace, SWAG, and variational baselines.