Timezone: »
Neuroscience studies of human decision-making abilities commonly involve subjects completing a decision-making task while BOLD signals are recorded using fMRI. Hypotheses are tested about which brain regions mediate the effect of past experience, such as rewards, on future actions. One standard approach to this is model-based fMRI data analysis, in which a model is fitted to the behavioral data, i.e., a subject's choices, and then the neural data are parsed to find brain regions whose BOLD signals are related to the model's internal signals. However, the internal mechanics of such purely behavioral models are not constrained by the neural data, and therefore might miss or mischaracterize aspects of the brain. To address this limitation, we introduce a new method using recurrent neural network models that are flexible enough to be jointly fitted to the behavioral and neural data. We trained a model so that its internal states were suitably related to neural activity during the task, while at the same time its output predicted the next action a subject would execute. We then used the fitted model to create a novel visualization of the relationship between the activity in brain regions at different times following a reward and the choices the subject subsequently made. Finally, we validated our method using a previously published dataset. We found that the model was able to recover the underlying neural substrates that were discovered by explicit model engineering in the previous work, and also derived new results regarding the temporal pattern of brain activity.
Author Information
Amir Dezfouli (Data61, CSIRO)
Richard Morris (U Sydney)
Fabio Ramos (University of Sydney)
Peter Dayan (Gatsby Unit, UCL)
I am Director of the Gatsby Computational Neuroscience Unit at University College London. I studied mathematics at the University of Cambridge and then did a PhD at the University of Edinburgh, specialising in associative memory and reinforcement learning. I did postdocs with Terry Sejnowski at the Salk Institute and Geoff Hinton at the University of Toronto, then became an Assistant Professor in Brain and Cognitive Science at the Massachusetts Institute of Technology before moving to UCL.
Bernard Balleine (UNSW)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Oral: Integrated accounts of behavioral and neuroimaging data using flexible recurrent neural network models »
Thu. Dec 6th 03:05 -- 03:20 PM Room Room 220 CD
More from the Same Authors
-
2022 : Variance Reduction in Off-Policy Deep Reinforcement Learning using Spectral Normalization »
Payal Bawa · Rafael Oliveira · Fabio Ramos -
2022 : Learning Successor Feature Representations to Train Robust Policies for Multi-task Learning »
Melissa Mozifian · Dieter Fox · David Meger · Fabio Ramos · Animesh Garg -
2023 Poster: The contextual lasso: Sparse linear models via deep neural networks »
Ryan Thompson · Amir Dezfouli · robert kohn -
2022 Workshop: 5th Robot Learning Workshop: Trustworthy Robotics »
Alex Bewley · Roberto Calandra · Anca Dragan · Igor Gilitschenski · Emily Hannigan · Masha Itkina · Hamidreza Kasaei · Jens Kober · Danica Kragic · Nathan Lambert · Julien PEREZ · Fabio Ramos · Ransalu Senanayake · Jonathan Tompson · Vincent Vanhoucke · Markus Wulfmeier -
2022 Spotlight: Batch Bayesian optimisation via density-ratio estimation with guarantees »
Rafael Oliveira · Louis Tiao · Fabio Ramos -
2022 Poster: Batch Bayesian optimisation via density-ratio estimation with guarantees »
Rafael Oliveira · Louis Tiao · Fabio Ramos -
2021 Poster: TacticZero: Learning to Prove Theorems from Scratch with Deep Reinforcement Learning »
Minchao Wu · Michael Norrish · Christian Walder · Amir Dezfouli -
2020 : Invited Talk - "RL with Sim2Real in the Loop / Online Domain Adaptation for Mapping" »
Fabio Ramos · Anthony Tompkins -
2020 : Discussion Panel »
Pete Florence · Dorsa Sadigh · Carolina Parada · Jeannette Bohg · Roberto Calandra · Peter Stone · Fabio Ramos -
2020 : Bayesian optimization by density ratio estimation »
Louis Tiao · Aaron Klein · Cedric Archambeau · Edwin Bonilla · Matthias W Seeger · Fabio Ramos -
2020 Poster: Sparse Spectrum Warped Input Measures for Nonstationary Kernel Learning »
Anthony Tompkins · Rafael Oliveira · Fabio Ramos -
2019 : Poster Session »
Lili Yu · Aleksei Kroshnin · Alex Delalande · Andrew Carr · Anthony Tompkins · Aram-Alexandre Pooladian · Arnaud Robert · Ashok Vardhan Makkuva · Aude Genevay · Bangjie Liu · Bo Zeng · Charlie Frogner · Elsa Cazelles · Esteban G Tabak · Fabio Ramos · François-Pierre PATY · Georgios Balikas · Giulio Trigila · Hao Wang · Hinrich Mahler · Jared Nielsen · Karim Lounici · Kyle Swanson · Mukul Bhutani · Pierre Bréchet · Piotr Indyk · samuel cohen · Stefanie Jegelka · Tao Wu · Thibault Sejourne · Tudor Manole · Wenjun Zhao · Wenlin Wang · Wenqi Wang · Yonatan Dukler · Zihao Wang · Chaosheng Dong -
2019 Poster: Disentangled behavioural representations »
Amir Dezfouli · Hassan Ashtiani · Omar Ghattas · Richard Nock · Peter Dayan · Cheng Soon Ong -
2018 : Fabio Ramos (Uni. of Sydney): Learning and Planning in Spatial-Temporal Data »
Fabio Ramos -
2018 Workshop: Modeling and decision-making in the spatiotemporal domain »
Ransalu Senanayake · Neal Jean · Fabio Ramos · Girish Chowdhary -
2016 Poster: Spatio-Temporal Hilbert Maps for Continuous Occupancy Representation in Dynamic Environments »
Ransalu Senanayake · Lionel Ott · Simon O'Callaghan · Fabio Ramos -
2015 Poster: Scalable Inference for Gaussian Process Models with Black-Box Likelihoods »
Amir Dezfouli · Edwin Bonilla -
2014 Poster: Bayes-Adaptive Simulation-based Search with Value Function Approximation »
Arthur Guez · Nicolas Heess · David Silver · Peter Dayan -
2014 Poster: On Integrated Clustering and Outlier Detection »
Lionel Ott · Linsey Pang · Fabio Ramos · Sanjay Chawla -
2013 Invited Talk: Neural Reinforcement Learning »
Peter Dayan -
2013 Poster: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel -
2013 Oral: Correlations strike back (again): the case of associative memory retrieval »
Cristina Savin · Peter Dayan · Mate Lengyel -
2012 Poster: Efficient Bayes-Adaptive Reinforcement Learning using Sample-Based Search »
Arthur Guez · David Silver · Peter Dayan -
2011 Poster: Two is better than one: distinct roles for familiarity and recollection in retrieving palimpsest memories »
Cristina Savin · Peter Dayan · Mate Lengyel -
2009 Poster: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
Jean-Pascal Pfister · Peter Dayan · Mate Lengyel -
2009 Oral: Know Thy Neighbour: A Normative Theory of Synaptic Depression »
Jean-Pascal Pfister · Peter Dayan · Mate Lengyel -
2009 Poster: Statistical Models of Linear and Nonlinear Contextual Interactions in Early Visual Processing »
Ruben Coen-Cagli · Peter Dayan · Odelia Schwartz -
2009 Spotlight: Statistical Models of Linear and Nonlinear Contextual Interactions in Early Visual Processing »
Ruben Coen-Cagli · Peter Dayan · Odelia Schwartz -
2008 Oral: Load and Attentional Bayes »
Peter Dayan -
2008 Poster: Load and Attentional Bayes »
Peter Dayan -
2008 Poster: Depression: an RL formulation and a behavioural test »
Quentin J Huys · Joshua T Vogelstein · Peter Dayan -
2008 Poster: Bayesian Model of Behaviour in Economic Games »
Debajyoti Ray · Brooks King-Casas · P. Read Montague · Peter Dayan -
2007 Oral: Hippocampal Contributions to Control: The Third Way »
Mate Lengyel · Peter Dayan -
2007 Poster: Hippocampal Contributions to Control: The Third Way »
Mate Lengyel · Peter Dayan -
2006 Poster: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan -
2006 Talk: Uncertainty, phase and oscillatory hippocampal recall »
Mate Lengyel · Peter Dayan