INTOSIS: Interactive Observation of Smartphone Inferred Symptoms for In-The-Wild Data

Current research in passive health monitoring utilizes machine learning methods to infer users’ symptoms and health status from smartphone-sensed data, which can be gathered on a large scale. However, reasoning about smartphone-sensed health behaviors should engage health experts who may not want to solely rely on computational approaches as they provide limited insights. We designed and proposed a visualization framework for the INTeractive Observation of Smartphone-Inferred Symptoms (INTOSIS), that supports contextualization of symptomatic days by presenting a holistic picture of complex smartphone data for analysts to find c oncerning b ehavior patterns. For instance, while sedentary behavior caused by the flu is concerning, sedentary behaviors on holidays are non-concerning. INTOSIS visualizes multiple smartphone sensor data channels such as geo-location, app usage and screen usage. It uses visual metaphors to effectively represent the data to help analysts derive important human-understandable spatio-temporal contexts and assign health consequences with corresponding semantic labels. INTOSIS provides timeline visualizations of contextual clues such as screen and app usage at night that support an analyst in being able to reason about and then extract plausible explanations for the occurrence of certain symptoms, such as, sleep problems. We validate INTOSIS with intuitive use cases, using a real-world smartphone-sensed dataset, along with expert evaluation.