Timed Report Measures Learning: Game-Based Embedded Assessment

Is the CyGaMEs (cyberlearning through game-based, metaphor enhanced learning objects) Timed Report tool (embedded assessment for game-based instructional environments derived from analogical reasoning and instructional design theory) a sensitive measure of learning? Using gameplay assessed every 10 s from 221 volunteers (primarily 13–18-year-olds distributed within and outside the United States playing the online videogame Selene), an algorithm identified a targeted learning moment, designated Timed Reports as “before” or “at and after” learning, and calculated Pearson correlation coefficients and slopes for both learning trajectories. Three studies compared the strength and quality of the Timed Report: a multilevel model using raw Timed Report values, repeated measures using slopes, and repeated measures using Pearson correlation coefficients. The three studies supported the same conclusions and replicate, elaborate, and generalize earlier findings. Timed Report successfully distinguishes player progress before and after learning. The strong effect supports the CyGaMEs approach to instructional game design and Timed Report embedded assessment for educational data mining. Timed Report also distinguishes between players who are Always Progressing and those who experience a learning moment. Participants who watch gameplay before playing have early success; however, participants who play without first watching gameplay make the largest gains in performance, and their after-learning performance equals that of watchers. Watching instructional videos about targeted game content in addition to watching gameplay before playing did not enhance performance.