Feature Selection Methods for Zero-Shot Learning of Neural Activity

Dimensionality is a serious challenge in human neuroimaging. Across imaging modalities, large pools of potential neural features (e.g. responses from particular voxels, electrodes, and temporal windows) have to be related to typically limited sets of stimuli and samples. To deal with limited stimulus sets, zero-shot encoding and decoding models have been introduced to classify classes outside the training set. However, these models have been found to be particularly susceptible to the curse of dimensionality and the risk of over-fitting. While no systematic study of feature selection for these models has been reported, a relatively straightforward feature stability approach has been successfully applied across models and imaging modalities. To characterize the tradeoffs in feature selection for encoding and decoding models, we compared correlationbased stability to several other feature selection techniques on comparable data sets from two distinct imaging modalities: functional Magnetic Resonance Imaging and Electrocorticography. Semantic attribute encoding and decoding models were trained and tested with five distinct feature selection methods: correlation-based stability, a wrapper method, Fisher’s Method, Mutual Information-based stability, and a novel attribute/feature correlation technique. Results demonstrate that correlation-based stability performs well across imaging modalities, but other methods result in qualitatively different patterns of selected features while achieving similar levels of performance, and performance can be maintained with fewer features. Choices about feature selection can have a substantial impact when closed-set accuracy, open-set robustness, and runtime are competing design objectives.