Learning a gaze estimator with neighbor selection from large-scale synthetic eye images

Abstract Appearance-based gaze estimation works well in inferring human gaze under real-world condition. But one of the significant limitations in appearance-based methods is the need for huge amounts of training data. Eye image synthesis addresses this problem by generating huge amounts of synthetic eye images with computer graphics. To fully use the large-scale synthetic eye images, a simple-but-effective appearance-based gaze estimation framework with neighbor selection is proposed in this paper. The proposed framework hierarchically fuses multiple k -NN queries (in head pose, pupil center and eye appearance spaces) to choose closest samples with more relevant features. Considering the structure characters of the closet samples, neighbor regression methods then can be applied to predict the gaze directions. Experimental results demonstrate that the representative neighbor regression methods under the proposed framework achieve better performance for within-subject and cross-subject gaze estimation.