Timezone: »

 
Associative memory via covariance-learning predictive coding networks
Mufeng Tang · Tommaso Salvatori · Yuhang Song · Beren Millidge · Thomas Lukasiewicz · Rafal Bogacz

@
in Memory in Artificial and Real Intelligence (MemARI) »
Event URL: https://openreview.net/forum?id=7mgtCKm8k_ »

Classical models of biological memory assume that associative memory (AM) in the hippocampus is achieved by learning a covariance matrix of simulated neural activities. However, it has been also proposed that AM in the hippocampus could be explained in the predictive coding framework. These two seemingly disparate computational principles pose difficulties for developing a unitary theory of memory storage and recall in the brain. In this work, we address this dichotomy using a family of covariance-learning predictive coding networks (covPCNs). We show that earlier predictive coding networks (PCNs) explicitly learning the covariance matrix perform AM, but their learning rule is non-local and unstable. We propose a novel model that implicitly learns the covariance matrix with Hebbian plasticity and stably converges to the same memory retrieval as the earlier models. We further show that this model can be combined with hierarchical PCNs to model the hippocampo-neocortical interactions. In practice, our models can store a large number of memories of structured images and retrieve them with high fidelity.

Keywords: associative memory · machine learning · predictive coding · computational neuroscience

Author Information

Mufeng Tang (University of Oxford)
Mufeng Tang

DPhil (PhD) student in computational neuroscience @ University of Oxford

Tommaso Salvatori (University of Oxford)
Yuhang Song (University of Oxford)
Beren Millidge (University of Oxford)
Thomas Lukasiewicz (University of Oxford)
Rafal Bogacz (University of Oxford)

More from the Same Authors