Linearly Augmented Real-Time 4D Expressional Face Capture

Abstract Personalised 3D face creation has always been a hot topic in the computer vision community. Many methods have been proposed including the statistic model, the non-rigid registration and high-end depth acquisition equipment. However, in practical applications, those existing methods still have their own limitations. For example, the performance of the statistic model-based methods highly depends on the generality of the pre-trained statistic model; the non-rigid registration based methods are sensitive to the quality of input data; the high-end equipment-based methods are less able to be popularised due to the expensive equipment costs; the deep learning-based methods can only perform well if proper training data provided for the target domain, and require GPU for better performance. To this end, this paper presents an adaptive template augmented method that can automatically obtain a personalised 4D facial modelling only using a consumer-grade device. The noisy data from such a cheap device are well handled. The whole process consists of a series of linear solutions and can be achieved in real-time for online processing only based on the CPU computation on a laptop. There is no constraint nor complex operation required by the proposed method. No additional time-consumptive pre- or post-processing for the personalisation is needed. Comparisons against several existing methods demonstrate the superiority of the proposed method.