Timezone: »
Self-supervised learning (SSL) methods aim to exploit the abundance of unlabelled data for machine learning (ML), however the underlying principles are often method-specific. An SSL framework derived from biological first principles of embodied learning could unify the various SSL methods, help elucidate learning in the brain, and possibly improve ML. SSL commonly transforms each training datapoint into a pair of views, uses the knowledge of this pairing as a positive (i.e. non-contrastive) self-supervisory sign, and potentially opposes it to unrelated, (i.e. contrastive) negative examples. Here, we show that this type of self-supervision is an incomplete implementation of a concept from neuroscience, the Efference Copy (EC). Specifically, the brain also transforms the environment through efference, i.e. motor commands, however it sends to itself an EC of the full commands, i.e. more than a mere SSL sign. In addition, its action representations are likely egocentric. From such a principled foundation we formally recover and extend SSL methods such as SimCLR, BYOL, and ReLIC under a common theoretical framework, i.e. Self-supervision Through Efference Copies (S-TEC). Empirically, S-TEC restructures meaningfully the within- and between-class representations. This manifests as improvement in recent strong SSL baselines in image classification, segmentation, object detection, and in audio. These results hypothesize a testable positive influence from the brain's motor outputs onto its sensory representations.
Author Information
Franz Scherr (Huawei Technologies Ltd.)
Qinghai Guo (Huawei)
Timoleon Moraitis (Huawei Technologies Ltd.)
More from the Same Authors
-
2022 Poster: Differentiable hierarchical and surrogate gradient search for spiking neural networks »
Kaiwei Che · Luziwei Leng · Kaixuan Zhang · Jianguo Zhang · Qinghu Meng · Jie Cheng · Qinghai Guo · Jianxing Liao -
2022 : Leveraging Episodic Memory to Improve World Models for Reinforcement Learning »
Julian Coda-Forno · Changmin Yu · Qinghai Guo · Zafeirios Fountas · Neil Burgess -
2022 Spotlight: Lightning Talks 4A-3 »
Zhihan Gao · Yabin Wang · Xingyu Qu · Luziwei Leng · Mingqing Xiao · Bohan Wang · Yu Shen · Zhiwu Huang · Xingjian Shi · Qi Meng · Yupeng Lu · Diyang Li · Qingyan Meng · Kaiwei Che · Yang Li · Hao Wang · Huishuai Zhang · Zongpeng Zhang · Kaixuan Zhang · Xiaopeng Hong · Xiaohan Zhao · Di He · Jianguo Zhang · Yaofeng Tu · Bin Gu · Yi Zhu · Ruoyu Sun · Yuyang (Bernie) Wang · Zhouchen Lin · Qinghu Meng · Wei Chen · Wentao Zhang · Bin CUI · Jie Cheng · Zhi-Ming Ma · Mu Li · Qinghai Guo · Dit-Yan Yeung · Tie-Yan Liu · Jianxing Liao -
2022 Spotlight: Differentiable hierarchical and surrogate gradient search for spiking neural networks »
Luziwei Leng · Kaiwei Che · Kaixuan Zhang · Jianguo Zhang · Qinghu Meng · Jie Cheng · Qinghai Guo · Jianxing Liao -
2022 Poster: DevFly: Bio-Inspired Development of Binary Connections for Locality Preserving Sparse Codes »
Tianqi Wei · Rana Alkhoury Maroun · Qinghai Guo · Barbara Webb