Evaluating the effectiveness of biometric sensors and their signal features for classifying human experience in virtual environments

Abstract Built environments play an essential role in our day-to-day lives since people spend more than 85% of their times indoors. Previous studies at the conjunction of neuroscience and architecture confirmed the impact of architectural design features on varying human experience, which propelled researchers to study the improvement of human experience in built environments using quantitative methods such as biometric sensing. However, a notable gap in the knowledge persists as researchers are faced with sensors that are commonly used in the neuroscience domain, resulting in a disconnect regarding the selection of effective sensors that can be used to measure human experience in designed spaces. This issue is magnified when considering the variety of sensor signal features that have been proposed and used in previous studies. This study builds on data captured during a series of user studies conducted to measure subjects’ physiological responses in designed spaces using the combination of virtual environments and biometric sensing. This study focuses on the data analysis of the collected sensor data to identify effective sensors and their signal features in classifying human experience. To that end, we used a feature attribution model (i.e., SHAP), which calculates the importance of each signal feature in terms of Shapley values. Results show that electroencephalography (EEG) sensors are more effective as compared to galvanic skin response (GSR) and photoplethysmogram (PPG) (i.e., achieving the highest SHAP values among the three at 3.55 as compared to 0.34 for GSR and 0.21 for PPG) when capturing human experience in alternate designed spaces. For EEG, signal features calculated from the back channels (occipital and parietal areas) were found to possess comparable effectiveness as the frontal channel (i.e., have similar mean SHAP values per channel). In addition, frontal and occipital asymmetry were found to be effective in identifying human experience in designed spaces.