Shallow Unsupervised Models Best Predict Neural Responses in Mouse Visual Cortex

Task-optimized deep convolutional neural networks are the most quantitatively accurate models of the primate ventral visual stream. However, such networks are implausible as models of the mouse visual system because mouse visual cortex has both lower retinal resolution and a shallower hierarchy than the primate. Moreover, the category supervision deep networks typically receive is neither ethologically relevant to the mouse in semantic content, nor realistic in quantity. As a result, standard supervised deep neural networks have proven quantitatively ineffective at modeling mouse visual data. Here, we develop and evaluate models that remedy these structural and functional gaps. We first demonstrate that shallow hierarchical architectures applied to lower resolution images improve match to neural responses, both in electrophysiological and calcium imaging data. We then show that networks trained using contrastive embedding methods, a recent unsupervised learning objective that requires no semantic labeling, achieve neural prediction performance that substantially exceed that of the same architectures trained in a supervised manner, across a wide variety of architecture types. Combining these better structural and functional priors yields models that are the most quantitatively accurate match to mouse visual responses to natural scenes, significantly surpassing that of prior attempts using primate-specific models, and approaching the inter-animal consistency level of the data itself. We further find that these shallow unsupervised models transfer to a wide variety of non-categorical visual tasks better than categorization-trained models. Taken together, our results suggest that mouse visual cortex is a low-resolution, shallow network that makes best use of the mouse9s limited resources to create a light-weight, general-purpose visual system -- in contrast to the deep, high-resolution, and more task-specific visual system of primates.