Expanded Abstract
Experimental research is the gold standard for establishing causal links between instructional practices and student outcomes. However, controlled experiments are rare in education due to their prohibitive cost, logistical complexity, and difficulty in naturalistic school settings. Meanwhile, the Learning Management System (LMS) has become central digital infrastructure in K-12 and higher education, directly shaping students' learning experiences (Davis et al., 2009; Hill, 2020 Lonn & Teasley, 2009; Mervosh & Swales, 2020; Pomerantz & Brooks, 2017). Current guidance for teachers on designing effective LMS assignments is minimal and often lacks a strong evidence base. We advocate for leveraging existing LMS infrastructure to lower the barriers to rigorous experimental research by manipulating LMS assignments. We present Terracotta, a novel, open-source LMS plugin that automates a complete, rigorous, and ethically responsible experimental research workflow (Motz et al., 2023).
Terracotta creates a seamless integration into existing LMS platforms using the Learning Tool Interoperability (LTI) standard, enabling experimental manipulation of assignment content and random assignment of students to different treatment conditions. The tool automates four requisite functions for conducting high-quality embedded experiments: random assignment, informed consent (with concealed student responses), mapping of outcomes to assignments, and de-identified data export. Terracotta supports various designs, including simple A/B tests and complex within-subject crossover designs (AB/BA) with pretest/posttest measures, which are otherwise notoriously difficult and expensive to implement manually.
Terracotta makes large-scale, individually randomized experiments feasible and cost effective within standard educational settings. By automating the most challenging logistical components, Terracotta bypasses difficulties that have hindered such studies in the past. The LMS provides a ready-made, large-scale platform for non-intrusive experimental manipulation of core instructional elements. While Terracotta has always enabled multiple choice, short answer, and file-upload assignments, now researchers can embed a Qualtrics survey (or any external web activity) in an LMS assignment and send personalized messages through the LMS (or via email). Automation drastically reduces the labor and errors associated with complex experimental designs like crossover studies, making them accessible to a wider research audience.
Using Terracotta, researchers can reverse the decline in experimental education studies while also casting wide nets to sample from more classes and institutions, enabling robust testing of the generalizability of findings across diverse student populations and pedagogical contexts (Slavin, 2002; Sullivan, 2011; Motz et al., 2025). Teachers and researchers can collaborate to embed research into routine learning activities, increasing teachers’ involvement in research, and leading to more relevant insights about what works in practice. In total, the goal of Terracotta is to lower the barriers to easy, accessible, responsible, and rigorous experimental research across education levels, student populations, and learning materials.
EdTech Archives