Classification of User Satisfaction Using Facial Expression Recognition and Machine Learning

In the current design processes, it has been often needed to use a level of final user satisfaction to evaluate products or services. Evaluation of the final user satisfaction on products and services has been considered an interesting challenge because it is difficult to measure the final user satisfaction according to products and services. Several papers and articles regarding the measurement of UX (user experience) as the satisfaction have been published. However, in the most approaches, UX was measured by questionnaire or survey collection method, which may lead to bias and a lack of exact feeling data of the target users. On the other hand, soft biometric data such as gender, age and facial expression can be used as the essential data for the user satisfaction analysis. In this research, we assume that the facial expression is essential in physical expressions and can be used as the accurate satisfaction data. It may be possible to capture the user’s facial expression during the particular use of products or services without users’ consciousness. However, in general cases, it is difficult to get the final user satisfaction.This study aimed to propose a framework to classify the final user satisfaction of products or services by the facial expression recognition and machine learning. The proposed framework consists of the three main steps. First, the data of facial expression, gender, age and the final user satisfaction are experimentally collected. Second, classification models are built by machine learning algorithms using the data. Finally, the model evaluation is employed to verify the accuracy of the model. After making the classification model, it is possible to classify the final user satisfaction only from the data of facial expression, gender and age.