Learning Analytics on Structured and Unstructured Heterogeneous Data Sources: Perspectives from Procrastination, Help-Seeking, and Machine-Learning Defined Cognitive Engagement

Abstract Statistics is one of the most challenging courses for university students. The personal learning environment (PLE) pedagogical design was introduced to assist students’ Statistics learning. With the PLE pedagogy, this study examined learners’ demographic backgrounds, motivational measures (i.e., help-seeking and academic procrastination due to the use of Information and Communication Technologies, ICT), and ill-structured data (i.e., Facebook posts and comments) to understand what student demographic information, how they feel, and what they do can impact their statistics learning performance. Seventy-eight people joined Facebook groups to form statistics learning communities. Using weakly supervised machine learning (ML), we categorized students’ Facebook messages into statistics-relevant and statistics-irrelevant. Results of the learning analytics on multimodal sources of student data showed that help-seeking positively predicted statistics achievement. In contrast, academic procrastination with ICT negatively predicted statistics achievement, controlling for students' demographics information, including age, gender, prior knowledge, and Internet/social media use. Moreover, the ensemble ML classified messages constructed by taking the sum of relevance coding (0 or 1) across three selected ML algorithms was highly aligned with the human coded message in terms of the degree of relevance to statistics. The ensemble ML classified messages were conceptualized as students’ cognitive engagement in statistics learning due to their high consistency with the human-labeled relevance coding and were positively associated with statistics achievement with a large effect size. The study contributed to developing an integrated learning analytics framework with the PLE pedagogical design encompassing learner backgrounds and unstructured learner artifacts.