Cognitive Load Measurement through Multimodal Behaviour Patterns - eScholarship

Cognitive Load Measurement through Multimodal Behaviour Patterns Natalie Ruiz NICTA, UNSW Ronnie Taib NICTA, UNSW Fang Chen NICTA, UNSW Abstract: Our research focuses on extending the accepted benefits of multimodal computer interaction by using the paradigm to detect fluctuations in cognitive load. The advantage of this approach is that cognitive load can be determined implicitly by monitoring variations in specific multimodal input features executed during day to day tasks using a computer interface. Such unobtrusive measures may help determine the user’s cognitive load in real-time, and achieve the ultimate goal of adapting information selection and presentation in a dynamic computer interface with reference to load. We identified some correlations between the communicative structure of combined speech and manual gesture input and high levels of cognitive load. The results suggest that semantic multimodal communicative structures are sensitive to cognitive load variations, with multimodal communication becoming half as redundant in high load than in low load. The feasibility of using rates of redundant constructions or complementary constructions in multimodal input as an index of cognitive load is supported by the results of our study.