Skip to yearly menu bar Skip to main content


Poster

NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping

Yamin Li · Ange Lou · Ziyuan Xu · Roza Bayrak · Shengchao Zhang · Shiyu Wang · Dario Englot · Soheil Kolouri · Daniel Moyer · Catie Chang

[ ]
Fri 13 Dec 4:30 p.m. PST — 7:30 p.m. PST

Abstract:

Functional magnetic resonance imaging (fMRI) is an indispensable tool in modern neuroscience, providing a non-invasive window into whole-brain dynamics at millimeter-scale spatial resolution. However, fMRI is constrained by issues such as high operation costs and immobility. With the rapid advancements in cross-modality synthesis and brain decoding, the use of deep neural networks has emerged as a promising solution for inferring whole-brain, high-resolution fMRI features directly from electroencephalography (EEG), a more widely accessible and portable neuroimaging modality. Nonetheless, the complex projection from neural activity to fMRI hemodynamic responses and the spatial ambiguity of EEG pose substantial challenges both in modeling and interpretability. Only a few studies to date have developed approaches for EEG-fMRI translation, and although they have made significant strides, the inference of fMRI signals has been limited to a small set of brain areas, and exclusively to task conditions. The ability to generalize to other brain areas, as well as to other conditions (such as resting state), remain critical gaps in the field. To tackle these challenges, we introduce a novel and generalizable framework: NeuroBOLT, a transformer-based model that leverages multi-dimensional representation learning from temporal, spatial, and spectral domains to translate raw EEG data to the corresponding fMRI activity signals across the brain. Our experiments demonstrate that NeuroBOLT effectively reconstructs resting-state fMRI signals from primary sensory, high-level cognitive areas, and deep subcortical brain regions, achieving state-of-the-art accuracy and significantly advancing the integration of these two modalities.

Live content is unavailable. Log in and register to view live content