Semi-supervised clustering of unknown expressions

Abstract Automated facial expressions recognition is an important area in computer vision and machine learning due to its important role in socially aware human-computer interaction and affective analysis. However, expression recognition is not trivial as it may vary across different genders, age groups and ethnicities, despite the universality of basic expressions. We propose here a semi-supervised learning algorithm with the motivation to cluster known, as well as unknown facial behavior. We detect and align faces and then reduce their dimensionality using various linear and non-linear, supervised and unsupervised methods. The transformed faces in new dimensions are then clustered with various methods, including but not limited to, consensus clustering, which is also used for automated class discovery in genetics. The linear dimensionality reduction methods, that we employ, include principal component analysis and linear discriminant analysis; while the non-linear methods include learning embeddings with Deep Convolution Neural Networks (CNNs). Deep CNNs can learn highly complicated non-linear transformations in a complex feature space. We get the best results using embeddings that we learn using deep convolution neural networks with consensus clustering. The novelty of our work is to show that we can cluster facial expressions, which were not included while learning the embeddings in the first place. This shows that we can generalize our work to non-standard expressions and can learn expression classes from the datasets themselves.