Contextualized Self-supervised Learning with data2vec in P300 Brain-Computer Interfaces

While the current standard for training machine learning models for brain-computer interface (BCI) applications is individual, there is high interest in leveraging generic models to minimize BCI calibration time. data2vec is a generalized transformer-based framework for contextualized self-supervised learning and has been applied to text, image, and speech data. In this work, we apply data2vec self-supervised learning as pretraining for developing generic models for the P300-based BCI. Results show initial encoder only models were sufficient for downstream fine-tuning for P300 classification. While data2vec pretraining was beneficial to graph encoders, the impact was minimal for convolutional encoders. Overall, generic P300 classifier models are beneficial in scenarios with low performing user-specific linear baselines and data2vec pretraining did not provide an advantage over other representative generic deep learning models.