Variational inference in coupled models of amino acid substitution

We investigate the use of Expectation-Maximization (EM) and variational Bayes for inferring rates and interactions under models of molecular coevolution. We first review EM theory for continuous-time Markov chains (CTMCs) and develop it for coevolutionary models, exploiting exchangeability and reversibility symmetries to constrain the parameter dimension. We fit several paired amino-acid coevolutionary models to structural alignments and compare the results to previous work. Our richest model trained on pooled coevolutionary data has explanatory power comparable to CherryML's Q2 matrix (also trained on pooled data), with half the parameters, exploiting the Klein four-group symmetry of reversibly exchangeable flux. We observe that pooling of training data can lead to a form of Simpson's Paradox: a mixture model, whose components experience Potts energy-modulated substitution modeled via continuous-time Bayesian networks (CTBNs), resolves interaction signals that wash out when a single such component tries to capture everything. The interactions include correlated and anticorrelated hydropathy and volume-packing in coevolving amino-acid pairs, as well as the anticorrelated acid/base compensation that was detected by CherryML's Q2. We next assess an evidence lower bound (ELBO) for CTBNs built from the same EM statistics, in which the coupling contracts onto per-site bridge quantities at a cost polynomial, rather than exponential, in cluster size. The ELBO is the natural homogeneous initializer for the state of the art in variational modeling of CTBNs, the Euler-Lagrange equations derived by Cohn et al. (JMLR, 2010). This ELBO is computed efficiently for a Potts model by Gauss-Legendre quadrature. Compared to integrating the Euler-Lagrange ODEs, it is competitive in accuracy, faster, simpler, and better-conditioned.