A new verification approach for subjective evaluation of actions in HMD-VR with EEG

We propose a new verification approach for subjective evaluation of actions and reactions in virtual reality. We have been working on subject behavior analysis in VR. A head tracking HMD allows users to move around and make actions up to a room size, and their experiences can be measured by their head position, motion estimation, and gaze tracking when they are available. But in most cases, subjective evaluation for total VR experience are conducted after their experiences are over. We have succeeded in integrating a new EEG device that can measure EEG and mind indexes in real time with gaze-trackable and head-trackable HMD. By referring the gaze tracking results and EEG analysis and/or mind indexes when a situation is presented in VR, we can check the subject is surely watching the objects with their attention in the situation and their attention level in his/her mind. It may support the reliability of the score of subjective evaluation. This is very important when we need to conduct a number of subjective evaluations with a fewer number of subjects. We have developed our preliminary system for two applications – sport action analysis and traffic safety evaluation – and confirmed that it is a promising approach.