Introduction
REAL CHEM is a comprehensive courseware environment for general chemistry (Carnegie Mellon University, 2022) being iteratively refined through multiple cycles of Learning Engineering (Craig et al., 2025). It is built around the OpenStax Chemistry 2e textbook (Flowers et al., 2009) to ensure alignment with widely used open-source instructional materials, particularly in two-year institutions where consistent, scalable support is often lacking yet critical. Designed for hybrid use, REAL CHEM blends online and in-person instruction, integrating content and assessment into a coherent instructional thread that supports both teaching and learning. The platform places a strong emphasis on meeting the needs of at-risk students through scaffolded practice, timely feedback, and tools that help instructors target support. Every student interaction is instrumented to generate rich data that flows back to instructors to inform teaching, to developers to improve content, and to researchers working to advance the science of learning. In this way, REAL CHEM functions not just as a tool for teaching and learning, but as a continuously evolving platform for instructional improvement and research.
To engineer REAL CHEM as a learning experience that is accessible and usable by a growing community of education researchers, the technical infrastructure has been expanded with targeted support for instructors and researchers, including streamlined onboarding, implementation guides, and structured opportunities for collaboration. This includes the expansion of existing training programs with dedicated tracks to prepare faculty and early-career researchers to contribute effectively to courseware-based research. One such example is the support of action research by REAL CHEM instructors through the LearnLab Summer School.
Given both the Learning Engineering (Craig et al., 2025) and practitioner focus of this manuscript, we frame our method as a description of how instructors were supported through the LearnLab Summer School to conduct action research with data from their courses that incorporate the REAL CHEM platform.
The LearnLab Summer School (LearnLab) is an intensive one-week program focused on supporting technology-enhanced learning environments and analyzing data generated in those environments (LearnLab, 2025). All LearnLab participants take part in daily whole-group activities where they attend talks on topics such as intelligent tutoring systems, computer-supported collaborative learning, and educational data mining. In addition to whole-group activities, LearnLab also offers programming in multiple subject tracks. REAL CHEM involvement in the 2025 LearnLab fell under the general chemistry subject track.
Track-specific activities were supported by mentors involved in educational research surrounding REAL CHEM. Mentors would help instructors develop action research questions based on the available data and provide guidance for the corresponding data transformation and analysis necessary to pursue those questions.
Each REAL CHEM instructor at LearnLab was provided a personalized dataset based on their students’ engagement and performance within the REAL CHEM platform. Examples of action research based on this data included exploration of how different grading policies might associate with different levels of REAL CHEM student engagement, as well as how different paths by which students navigate through REAL CHEM potentially associate with chemistry exam performance.
With supporting REAL CHEM instructors through LearnLab serving as the method of this Learning engineering and practitioner-focused manuscript, the corresponding results are the instructor projects exploring their action research questions that emerged from LearnLab. We highlight one of these projects now.
One REAL CHEM instructor utilized LearnLab to investigate how engagement with REAL CHEM is potentially associated with outcomes on the American Chemical Society (ACS) general chemistry exam (Bass, Carpenter, & Henderson, 2025). In large enrollment gateway courses (CHEM 101), the instructor examined whether progress completion, checkpoint assessments, semester timing, and prior knowledge influenced chemistry exam performance. Results showed that students who took the course in the spring (off-track) performed about 5.6 points lower than those in the fall (on-track). This finding suggests that targeted interventions could help close performance gaps between semesters and across student groups, and the instructor is now exploring possible interventions to reduce the gap their action research has identified.
Supporting REAL CHEM instructors through LearnLab provides an example of how to make action research with big data more tenable for practitioners, both in terms of access to the data and mentor support with data transformation and analysis. While some of the LearnLab participants had limited experience with data-driven educational research, by the end of the week, multiple REAL CHEM instructors were analyzing learning growth curves, as well as conducting regression and principal component analyses on their personalized student data. Surveys of LearnLab participants suggest interest in conducting future data-driven reforms of their teaching. This continued interest is promising, and might not be the case if structured supports were not provided for data mining, transformation, and analysis. Curating big data for instructors and providing them personalized mentoring can potentially instill an improved sense of self-efficacy when it comes to their likelihood of engaging with data-driven instructional reform. This makes for a potentially valuable cycle of Learning Engineering (Craig et al., 2025).
We would like to acknowledge the organizers and participants of LearnLab for an engaging week of professional learning. We gratefully acknowledge the financial support from The Bill & Melinda Gates Foundation for this research effort. Opinions and conclusions expressed in this article do not necessarily reflect the views of this funding agency.
EdTech Archives