Towards Sensor-based Learning Analytics: A Contactless Approach

Learning analytics is an emerging field in which sophisticated analytic tools are used to improve students’ learning. Driven by the data from heterogeneous resources and the latest data mining techniques, learning analytics mainly attempts at creating a more integrated and personalized learning experience for each student. Most of the studies in the field rely exclusively on keyboard-mouse interactions. Though these interactions are effective in capturing overall sentiments and reactions, they do not provide enough granularity to conduct detailed analyses about students’ learning patterns. This dissertation will provide an interdisciplinary perspective on sensor-based learning analytics by discussing data-science techniques, HCI-based solutions in relation to educational theories. The thesis begins by providing an overview of the field of learning analytics and discusses the limitations of traditional contact-based sensor technologies in education research. We then discuss how data generated from contactless sensors such as an eye-tracker and a thermal camera can provide an informative window on students’ learning experiences. By conducting two lab-based studies, we analysed students’ eye movements and facial temperature to answer our four research questions related to predicting students’ desktop activities, evaluating learning task design, measuring students’ cognitive load, and measuring students’ attention patterns while students’ perform a learning task in a digital learning environment. The first study investigates the role of gaze-based features in predicting the desktop activities of the students. The outcomes of the study reveal that gaze-based features can be used to predict desktop activities of the students, and the design of a novel set of features (mid-level gaze features) can improve the accuracy of the prediction. The study lacks strong ground truth and contains a small sample size with only one sensor data. The limitations are addressed by the second study, which is a more extensive study and utilises a multi-sensor setup consisting of an eye-tracker, a thermal camera, a web camera and a physical slider. The study focuses on understanding students’ learning process in video-based learning. The data is analysed at different granularity, and results are present as separate chapters in the thesis. The first analysis compares two types of instructional designs of the video lectures (text vs. animation) using a physical slider. The results suggest that continuous evaluation of video lectures using a slider provides deeper insight into how students experience video lectures. To confirm these findings using a physiological marker, we again compare different instructional designs of the video lecture by measuring the cognitive load of the students using a thermal camera. The results suggest that thermal imaging is a reliable psycho-physiological indicator of cognitive load, and text-based video lectures induce higher cognitive load than animation-based video lectures. The final analysis focuses on measuring the coattention between students’ attention and instructor’s dialogues by measuring how much student follows instructor dialogues in a video lecture using an eye-tracker. The results suggest that students’ attention mostly follows the instructor’s dialogues on the screen, and their direction of attention is influenced by their prior knowledge. The thesis contributes to the fields of learning analytics by combining knowledge from different disciplines – HCI-design based on contactless sensors, video lectures derived from educational psychology, and novel data science techniques to extract features from eye movements and facial temperature to understand students’ learning process in a digital learning environment. The thesis’s final chapter discusses these contributions and provides future directions for sensor-based learning analytics using contactless sensors.