Using Synthetic Datasets to Hone Intuitions Within an Adaptive Learning Environment

Adaptive learning typically focuses on the challenge of adjusting the presentation of instructional material based on an automated assessment of real-time student performance. The goal is to move away from a one-size-fits-all approach to learning toward a more personalized experience. While we support this goal, in this paper we focus on a related but logically prior challenge, namely, the development of an adaptive environment that helps the instructional designer build better learning materials in the first place. Rather than focus on the automated assessment of student performance, we have instead focused on mining data generated by the instructional designers as they developed new course materials. Working on the assumption that the discovery of patterns in past design decisions will inform better future design decisions, we have applied an off-the-shelf machine learning technique to explore associations between learning contexts and the selection of specific learning activities. Although machine learning techniques have become commodities, practical guidance regarding their expected performance is harder to come by. We have filled this gap by systematically generating our own synthetic data sets that represent notional histories of user interactions to hone our own intuitions about the performance of the Bayesian network that underpins our adaptive environment. Our intent is not to add yet one more benchmark data set to promote Bayesian approaches but, rather, to describe by way of example a method for generating such data sets that will help the practitioner understand what an otherwise black box is doing.