Timezone: »

Convergent Representations of Computer Programs in Human and Artificial Neural Networks
Shashank Srikant · Ben Lipkin · Anna Ivanova · Evelina Fedorenko · Una-May O'Reilly

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #738

What aspects of computer programs are represented by the human brain during comprehension? We leverage brain recordings derived from functional magnetic resonance imaging (fMRI) studies of programmers comprehending Python code to evaluate the properties and code-related information encoded in the neural signal. We first evaluate a selection of static and dynamic code properties, such as abstract syntax tree (AST)-related and runtime-related metrics. Then, to learn whether brain representations encode fine-grained information about computer programs, we train a probe to align brain recordings with representations learned by a suite of ML models. We find that both the Multiple Demand and Language systems--brain systems which are responsible for very different cognitive tasks, encode specific code properties and uniquely align with machine learned representations of code. These findings suggest at least two distinct neural mechanisms mediating computer program comprehension and evaluation, prompting the design of code model objectives that go beyond static language modeling.We make all the corresponding code, data, and analysis publicly available at https://github.com/ALFA-group/code-representations-ml-brain

Author Information

Shashank Srikant (MIT)
Ben Lipkin (Massachusetts Institute of Technology)
Anna Ivanova (Massachusetts Institute of Technology)
Evelina Fedorenko (MIT)

Evelina (Ev) Fedorenko is an Assistant Professor in the Department of Brain and Cognitive Sciences and an Associate Member of the McGovern Institute for Brain Research. She also holds an appointment in the Department of Psychiatry at Massachusetts General Hospital and is affiliated with the Harvard-MIT Program in Speech and Hearing Bioscience and Technology. Prior to joining MIT, she spent 5 years as faculty at MGH and Harvard Medical School, supported by an NIH Pathway to Independence K99/R00 award. She received a Ph.D. in Cognitive Science from the MIT Department of Brain and Cognitive Sciences in 2007 and an A.B. in Psychology and Linguistics from Harvard University in 2002.

Una-May O'Reilly (Massachusetts Institute of Technology)

More from the Same Authors

  • 2019 : Panel Discussion »
    Theodore Willke · Evelina Fedorenko · Kenton Lee · Paul Smolensky
  • 2019 : Ev Fedorenko - Composition as the core driver of the human language system »
    Evelina Fedorenko
  • 2018 : Posters (all accepted papers) + Break »
    Jianyu Wang · Denis Gudovskiy · Ziheng Jiang · Michael Kaufmann · Andreea Anghel · James Bradbury · Nikolas Ioannou · Nitin Agrawal · Emma Tosch · Gyeongin Yu · Keno Fischer · Jarrett Revels · Giuseppe Siracusano · Yaoqing Yang · Jeff Johnson · Yang You · Hector Yuen · Chris Ying · Honglei Liu · Nikoli Dryden · Xiangxi Mo · Yangzihao Wang · Amit Juneja · Micah Smith · Qian Yu · pramod gupta · Deepak Narayanan · Keshav Santhanam · Tim Capes · Abdul Dakkak · Norman Mu · Ke Deng · Liam Li · Joao Carreira · Luis Remis · Deepti Raghavan · Una-May O'Reilly · Amanpreet Singh · Mahmoud (Mido) Assran · Eugene Wu · Eytan Bakshy · Jinliang Wei · Michael Innes · Viral Shah · Haibin Lin · Conrad Sanderson · Ryan Curtin · Marcus Edel