Mutual-Information-based Feature Selection for Facial Emotion Recognition on Light-Weight Devices

Light-weight devices have become ubiquitous in our daily life, such as smartphones, smart monitors, and other smart devices in our home. As light-weight devices are becoming popular, the demand for sophisticated human-computer interaction (HCI) applications for light-weight devices is also increasing. One particularly promising HCI application for light-weight devices is facial expression recognition (FER), since it may open up possibilities of various medical, psychological or psychiatric monitoring. However, its high computational demand has prevented widespread adoption of FER on light-weight devices. To address this issue, here we aim at decreasing computational overhead of FER by reducing the number of facial landmarks. We calculated mutual information of facial landmarks’ movements and detected their clusters using hierarchical agglomerative clustering (HAC). We also applied a genetic algorithm (GA)-inspired landmark selection method to filter out low-utility features from each facial landmark cluster. The selected features were provided to a support vector machine (SVM) classifier to classify facial expressions, and its performance was compared among several different algorithm settings. Results showed that our proposed method achieved classification accuracy similar to the classifier that used the original full-featured dataset, with improved performance robustness and computational time reduced by 63.5%.