Reliable Uncertainty Quantification of Deep Learning Models for a Free Electron Laser Scientific Facility

Particle accelerators are essential instruments for scientific experiments. They provide different experiments with particle beams of different parameters (e.g. beam energies or durations). This is accomplished by changing a wide variety of controllable settings, in a process called tuning. This is a challenging task, as many particle accelerators are complex machines with thousands of components, each of which contribute sources of uncertainty. Fast, accurate models of these systems could aid rapid customization of beams, but in order to accomplish this reliably, quantified uncertainties are essential. We address the problem of obtaining reliable uncertainties from learned models of a noisy, high-dimensional, nonlinear accelerator system: the X-ray free electron laser at the Linac Coherent Light Source, which is a scientific user facility. We examine the efficacy of Bayesian Neural Networks (BNNs) to reliably quantify predictive uncertainty and compare these with Quantile Regression Neural Networks (QRNNs). The QRNN models provide mean absolute error on predictions that are consistent with the noise of the measured data. We find the BNN is sensitive to outliers and is substantially more computationally expensive, but it still captures the general trend of the target data.