Deep Gaussian Processes on Directed Acyclic Graphs

Many real world processes can be represented as compositions of functions along a Directed Acyclic Graph (DAG). In causal modelling these correspond to the underlying mechanisms, in engineering to the multiple fidelities present, and in gene-regulatory networks to the transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour, and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, satisfies compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theoretical results and our methodology, and model a latent-collider DAG, a protein signalling network, and a multi-fidelity heavy-ion collision task, attaining state-of-the-art performance while recovering the low-fidelity contributions and yielding explainability over the simulator hierarchy.