Jeffrey Bowers: Researchers Comparing DNNs to Brains Need to Adopt Standard Methods of Science.

The claim that DNNs and brains represent information in similar ways is largely based on the good performance of DNNs on various brain benchmarks. On this approach, the better DNNs can predict neural activity, the better the correspondence between DNNs and brains. But this is at odds with the standard scientific research approach that is characterized by varying independent variables to test specific hypotheses regarding the causal mechanisms that underlie some phenomenon; models are supported to the extent that they account for these experimental results. The best evidence for a model is that it survives “severe” tests, namely, experiments that have a high probability of falsifying a model if and only if the model is false in some relevant manner. When DNNs are assessed in this way, they catastrophically fail. The field needs to change its methods and put far more weight into falsification to get a better characterization of DNN-brain correspondences and to build more human-like AI.