Caliper: Measuring success in STEM learning ecosystems

Abstract Ecologized learning situated in diverse in-school, out-of-school, and after-school contexts poses daunting challenges to measuring, communicating, managing, and improving outcomes. Formal and informal educators, policymakers, funders, and other key stakeholders engaged in cross-sector collaborations defined five pillars of STEM learning ecosystem success: partnerships, measures, administrative systems, teaching and learning practices, and workforce development connections. Variation in the ecosystem success construct was mapped, existing items from an earlier ecosystem assessment were revised, four parallel survey forms with common items were devised, and an online administration system was created in early 2020. Data from over 400 stakeholders in dozens of ecosystems were fit to a probabilistic model of measurement capable of supporting metrological quality assurance standards.