The neural code for face memory

Abstract The ability to recognize familiar visual objects is critical to survival. A central assumption of neuroscience is that long-term memories are represented by the same brain areas that encode sensory stimuli (1). Neurons in inferotemporal (IT) cortex represent the sensory percept of visual objects using a distributed axis code (2–4). Whether and how the same IT neural population represents the long-term memory of visual objects remains unclear. Here, we examined how familiar faces are encoded in face patch AM and perirhinal cortex. We found that familiar faces were represented in a distinct subspace from unfamiliar faces. The familiar face subspace was shifted relative to the unfamiliar face subspace at short latency and then distorted to increase neural distances between familiar faces at long latency. This distortion enabled markedly improved discrimination of familiar faces in both AM and PR. Inactivation of PR did not affect these memory traces in AM, suggesting that the memory traces arise from intrinsic recurrent processes within IT cortex or interactions with downstream regions outside the medial temporal lobe (5, 6). Overall, our results reveal that memories of familiar faces are represented in IT and perirhinal cortex by a distinct long-latency code that is optimized to distinguish familiar identities.