Timezone: »
Multiple lines of evidence support the notion that the brain performs probabilistic inference in multiple cognitive domains, including perception and decision making. There is also evidence that probabilistic inference may be implemented in the brain through the (quasi)stochastic activity of neural circuits, producing samples from the appropriate posterior distributions, effectively implementing a Markov chain Monte Carlo algorithm. However, time becomes a fundamental bottleneck in such samplingbased probabilistic representations: the quality of inferences depends on how fast the neural circuit generates new, uncorrelated samples from its stationary distribution (the posterior). We explore this bottleneck in a simple, linearGaussian latent variable model, in which posterior sampling can be achieved by stochastic neural networks with linear dynamics. The wellknown Langevin sampling (LS) recipe, so far the only sampling algorithm for continuous variables of which a neural implementation has been suggested, naturally fits into this dynamical framework. However, we first show analytically and through simulations that the symmetry of the synaptic weight matrix implied by LS yields critically slow mixing when the posterior is highdimensional. Next, using methods from control theory, we construct and inspect networks that are optimally fast, and hence orders of magnitude faster than LS, while being far more biologically plausible. In these networks, strong  but transient  selective amplification of external noise generates the spatially correlated activity fluctuations prescribed by the posterior. Intriguingly, although a detailed balance of excitation and inhibition is dynamically maintained, detailed balance of Markov chain steps in the resulting sampler is violated, consistent with recent findings on how statistical irreversibility can overcome the speed limitation of random walks in other domains.
Author Information
Guillaume Hennequin (University of Cambridge)
Laurence Aitchison (University of Cambridge)
Mate Lengyel (University of Cambridge)
More from the Same Authors

2021 Poster: A universal probabilistic spike count model reveals ongoing modulation of neural variability »
David Liu · Mate Lengyel 
2018 Poster: Exact natural gradient in deep linear networks and its application to the nonlinear case »
Alberto Bernacchia · Mate Lengyel · Guillaume Hennequin 
2016 Poster: Efficient statespace modularization for planning: theory, behavioral and neural signatures »
Daniel McNamee · Daniel M Wolpert · Mate Lengyel 
2014 Poster: Analog Memories in a Balanced RateBased Network of EI Neurons »
Dylan Festa · Guillaume Hennequin · Mate Lengyel 
2014 Poster: A Dual Algorithm for Olfactory Computation in the Locust Brain »
Sina Tootoonian · Mate Lengyel 
2014 Oral: Analog Memories in a Balanced RateBased Network of EI Neurons »
Dylan Festa · Guillaume Hennequin · Mate Lengyel 
2013 Poster: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel 
2013 Oral: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel 
2011 Session: Oral Session 11 »
Mate Lengyel 
2011 Poster: Two is better than one: distinct roles for familiarity and recollection in retrieving palimpsest memories »
Cristina Savin · Peter Dayan · Mate Lengyel 
2011 Poster: Active dendrites: adaptation to spikebased communication »
Balazs B Ujfalussy · Mate Lengyel 
2011 Spotlight: Active dendrites: adaptation to spikebased communication »
Balazs B Ujfalussy · Mate Lengyel 
2009 Workshop: Normative electrophysiology: Explaining cellular properties of neurons from first principles »
JeanPascal Pfister · Mate Lengyel 
2009 Poster: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
JeanPascal Pfister · Peter Dayan · Mate Lengyel 
2009 Oral: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
JeanPascal Pfister · Peter Dayan · Mate Lengyel 
2007 Oral: Hippocampal Contributions to Control: The Third Way »
Mate Lengyel · Peter Dayan 
2007 Poster: Hippocampal Contributions to Control: The Third Way »
Mate Lengyel · Peter Dayan 
2006 Poster: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan 
2006 Talk: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan