05.12.23 - Martin Schrimpf

Research in the brain and cognitive sciences attempts to uncover the neural mechanisms underlying intelligent behavior. Due to the complexities of brain processing, studies necessarily had to start with a narrow scope of experimental investigation and computational modeling. I will argue that it is time for our field to take the next step: build system models that capture neural mechanisms and supported behaviors in entire domains of intelligence. To make progress on system models, we are developing the Brain-Score platform which, to date, hosts over 50 benchmarks of neural and behavioral experiments that models can be tested on. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (~50% explained variance), but also discover key relationships: Models’ brain scores are predicted by their object categorization performance in vision and their next-word prediction performance in language. The better models predict internal neural activity, the better they match human behavioral outputs, with architecture substantially contributing to brain-like representations. Using the integrative benchmarks, we develop improved state-of-the-art system models that more closely match shallow recurrent neuroanatomy, predict primate temporal processing, and are more robust to image corruptions. Finally I will argue that the newest generation of models can be used to predict the behavioral effects of neural interventions, and to drive new experiments.