Timezone: »
Recent experimental studies have focused on the specialization of different neural structures for different types of instrumental behavior. Recent theoretical work has provided normative accounts for why there should be more than one control system, and how the output of different controllers can be integrated. Two particlar controllers have been identified, one associated with a forward model and the prefrontal cortex and a second associated with computationally simpler, habitual, actor-critic methods and part of the striatum. We argue here for the normative appropriateness of an additional, but so far marginalized control system, associated with episodic memory, and involving the hippocampus and medial temporal cortices. We analyze in depth a class of simple environments to show that episodic control should be useful in a range of cases characterized by complexity and inferential noise, and most particularly at the very early stages of learning, long before habitization has set in. We interpret data on the transfer of control from the hippocampus to the striatum in the light of this hypothesis.
Author Information
Mate Lengyel (University of Cambridge)
Peter Dayan (Gatsby Unit, UCL)
I am Director of the Gatsby Computational Neuroscience Unit at University College London. I studied mathematics at the University of Cambridge and then did a PhD at the University of Edinburgh, specialising in associative memory and reinforcement learning. I did postdocs with Terry Sejnowski at the Salk Institute and Geoff Hinton at the University of Toronto, then became an Assistant Professor in Brain and Cognitive Science at the Massachusetts Institute of Technology before moving to UCL.
Related Events (a corresponding poster, oral, or spotlight)
-
2007 Poster: Hippocampal Contributions to Control: The Third Way »
Wed. Dec 5th 06:30 -- 06:40 PM Room
More from the Same Authors
-
2023 Poster: Bayesian nonparametric (non-)renewal processes for analyzing neural spike train variability »
David Liu · Mate Lengyel -
2022 Poster: Training stochastic stabilized supralinear networks by dynamics-neutral growth »
Wayne Soo · Mate Lengyel -
2021 Poster: A universal probabilistic spike count model reveals ongoing modulation of neural variability »
David Liu · Mate Lengyel -
2018 Poster: Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models »
Amir Dezfouli · Richard Morris · Fabio Ramos · Peter Dayan · Bernard Balleine -
2018 Oral: Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models »
Amir Dezfouli · Richard Morris · Fabio Ramos · Peter Dayan · Bernard Balleine -
2018 Poster: Exact natural gradient in deep linear networks and its application to the nonlinear case »
Alberto Bernacchia · Mate Lengyel · Guillaume Hennequin -
2016 Poster: Efficient state-space modularization for planning: theory, behavioral and neural signatures »
Daniel McNamee · Daniel M Wolpert · Mate Lengyel -
2014 Poster: Bayes-Adaptive Simulation-based Search with Value Function Approximation »
Arthur Guez · Nicolas Heess · David Silver · Peter Dayan -
2014 Poster: Analog Memories in a Balanced Rate-Based Network of E-I Neurons »
Dylan Festa · Guillaume Hennequin · Mate Lengyel -
2014 Poster: A Dual Algorithm for Olfactory Computation in the Locust Brain »
Sina Tootoonian · Mate Lengyel -
2014 Oral: Analog Memories in a Balanced Rate-Based Network of E-I Neurons »
Dylan Festa · Guillaume Hennequin · Mate Lengyel -
2014 Poster: Fast Sampling-Based Inference in Balanced Neuronal Networks »
Guillaume Hennequin · Laurence Aitchison · Mate Lengyel -
2013 Invited Talk: Neural Reinforcement Learning »
Peter Dayan -
2013 Poster: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel -
2013 Oral: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel -
2012 Poster: Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search »
Arthur Guez · David Silver · Peter Dayan -
2011 Session: Oral Session 11 »
Mate Lengyel -
2011 Poster: Two is better than one: distinct roles for familiarity and recollection in retrieving palimpsest memories »
Cristina Savin · Peter Dayan · Mate Lengyel -
2011 Poster: Active dendrites: adaptation to spike-based communication »
Balazs B Ujfalussy · Mate Lengyel -
2011 Spotlight: Active dendrites: adaptation to spike-based communication »
Balazs B Ujfalussy · Mate Lengyel -
2009 Workshop: Normative electrophysiology: Explaining cellular properties of neurons from first principles »
Jean-Pascal Pfister · Mate Lengyel -
2009 Poster: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
Jean-Pascal Pfister · Peter Dayan · Mate Lengyel -
2009 Oral: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
Jean-Pascal Pfister · Peter Dayan · Mate Lengyel -
2009 Poster: Statistical Models of Linear and Nonlinear Contextual Interactions in Early Visual Processing »
Ruben Coen-Cagli · Peter Dayan · Odelia Schwartz -
2009 Spotlight: Statistical Models of Linear and Nonlinear Contextual Interactions in Early Visual Processing »
Ruben Coen-Cagli · Peter Dayan · Odelia Schwartz -
2008 Oral: Load and Attentional Bayes »
Peter Dayan -
2008 Poster: Load and Attentional Bayes »
Peter Dayan -
2008 Poster: Depression: an RL formulation and a behavioural test »
Quentin J Huys · Joshua T Vogelstein · Peter Dayan -
2008 Poster: Bayesian Model of Behaviour in Economic Games »
Debajyoti Ray · Brooks King-Casas · P. Read Montague · Peter Dayan -
2006 Poster: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan -
2006 Talk: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan