Information bottleneck dynamics during learning across artificial and biological neural systems
Abstract
The information bottleneck offers a candidate general framework for learning, defined by two axes: how much a representation preserves about the external world and how much it carries about the relevant target. However, empirical evidence for information-plane dynamics in realistic-scale artificial networks and in biological systems remains limited. We address this by tracking information-plane trajectories during learning in three systems with different substrates, using a matched analytical pipeline. First, in a gradient-trained spiking ResNet-18 on CIFAR-10 --- a biologically plausible artificial substrate --- where a two-phase fitting-then-compression trajectory emerges specifically in the deep layers; the first information-plane analysis of a spiking network at this scale. Second, on the open-source dataset of mouse visual cortex during multi-week category learning, where our pipeline recovers the published cohort-level effect now reformulated in IB-plane terms. Third, in our own single-photon miniscope calcium imaging data of mouse hippocampus during multi-day place learning in open field, where the IB axes map onto the well-established egocentric-to-allocentric transformation: the input axis becomes egocentric sensorimotor state, the output axis becomes allocentric position. Behavioral and decoder-baseline controls confirm the hippocampal trajectory reflects experience-driven learning. Across all three systems, we observe coherent motion along the relevance axis with learning, while compression emerges only where task structure demands it --- most pronounced in the SNN. The information bottleneck plane therefore offers a substrate-independent coordinate for representation learning, even as the strength of compression along it remains task-dependent rather than universal.