Online Curriculum Design and Development for High School STEMM Education: A Systematic Review from 2012 to 2024

Introduction

The online curriculum has become a vital resource for high school STEMM education (Abichandani et al., 2022; Gamage et al., 2022), with students increasingly relying on digital platforms to enhance educational accessibility and engagement (Chiang et al., 2022; Su et al., 2022). Meanwhile, high school teachers are integrating these curricula into their teaching practices to meet diverse learning needs and seek digital solutions in educational settings (Erickson et al., 2020; Yıldırım, 2022). Especially after the COVID-19 pandemic, both teachers and students have become more familiar with and dependent on online learning environments for STEMM education, accelerating the adoption of digital teaching tools and methodologies (DeCoito & Estaiteyeh, 2022). This shift has highlighted the critical role of well-designed online curricula in supporting effective learning and teaching processes. 

The design and development of online curricula critically determine their quality and effectiveness, influencing student engagement and how teachers integrate these resources into their instructional practices (Anwar et al., 2022; Yang, 2017). Thoughtful curriculum development is important because well-designed curricula can be adapted to varied teaching methods and learning needs (Meyers & Nulty, 2009; Viana & Peralta, 2021). For example, through collaboration with various stakeholders, Lesiak and colleagues (2024) applied inquiry-based learning approaches to design a student-directed online module. This initiative helped students make connections across disciplinary concepts. The curriculum was successfully delivered without placing an undue burden on teachers, highlighting the benefits of well-thought-out design and development in facilitating effective education. 

Curriculum design papers are invaluable for both researchers and practitioners for several reasons (McKenney & Reeves, 2018). First, they provide a detailed articulation of the development process, offering insights into specific cases and demonstrating how curricula achieve targeted educational outcomes. These documents not only supply valuable findings but also inspire future curriculum development efforts. Additionally, design papers often include evaluations of the curricula, allowing them to pinpoint challenges and pitfalls encountered during the development process. Such evaluations are crucial for continuous improvement, making these papers essential resources for both refining existing educational strategies and informing future developments. Through identifying common challenges and highlighting successful practices, curriculum design papers contribute significantly to educational design. 

Despite the critical role of curriculum design papers, there is a notable gap in systematic reviews that synthesize findings of online curricula development for high school STEMM education. A comprehensive analysis that integrates diverse design experiences and methodologies is necessary to advance the understanding of online curriculum design practices (Kitchenham, 2004; Mulrow, 1994). This gap highlights the need for systematically exploring and synthesizing the curriculum design papers to enhance both the theoretical understanding and practical application of online learning within high school STEMM disciplines. 

This paper seeks to bridge this literature gap by systematically reviewing the design and development of online curricula in high school STEMM education. The review will highlight contributions to both theoretical understanding and practical applications, focusing on identifying key design features, theoretical considerations, practical implementations, and evaluation of the modules. Furthermore, it includes a special inquiry into how online curricula address social justice and equity, aligning with the needs for socially responsible educational development (Reeves, 2000). This comprehensive approach ensures that the review not only adds to the academic discourse but also aids practitioners in designing effective and equitable educational solutions. More specifically, the following Research Questions (RQs) guided the review:

1. What pedagogies and design elements have been utilized to enhance online learning?
2. What innovative learning activities have been developed for online curricula?
3.  What theoretical considerations or frameworks have been used in developing online curricula? 
4. What specific goals have been aimed at by online learning curricula? 
5.  How have online learning curricula been evaluated? 
6.  In what ways have social justice and equity issues been addressed within online curricula?

Methodology

This study was conducted following the methodological guidelines for systematic reviews outlined by Page et al. (2021), which emphasize the importance of clear research questions, defined eligibility criteria, transparent study selection, and systematic data extraction. Using this reporting guideline helps ensure comprehensive synthesis of results and reproducibility of findings. Specifically, the selection of articles from January 2012 to January 2024 deliberately encompasses the most recent 12 years of research, capturing the dynamic evolution of online learning environments since 2012, famously known as the Year of the MOOC (Pappano, 2012). This period highlights significant developments in the popularity and implementation of online learning, particularly during the COVID-19 pandemic, which saw a dramatic increase in online instruction. The chosen timeframe also ensures that the scope of the review remains manageable and systematic. Figure 1 illustrates the PRISMA data collection and analysis process employed in this study.

Figure 1

PRISMA Flow Diagram for the Systematic Review 

Initial Search

The initial literature search was conducted across three research databases: APA PsycINFO, PubMed, and the Education Resources Information Center (ERIC). We utilized a series of search strings composed of four clusters of key phrases to conduct the searches. The first cluster targeted online learning contexts with key phrases such as ‘online learn*,’ ‘e-learning,’ ‘virtual learn*,’ ‘digital education,’ and ‘remote learn*.’ The second cluster focused specifically on ‘high school’ and ‘secondary education.’ The third cluster included terms related to curriculum design, such as 'curriculum design', ‘curriculum develop*’, ‘curricula design’, and ‘curricula develop*’. The fourth cluster encompassed signifiers of STEMM subjects, including ‘STEM*,’ ‘science education,' 'technology education,' 'engineering education,' 'mathematics education,' and 'medicine education.' The search yielded a total of 113 records initially. 

Inclusion and Exclusion Criteria 

At this stage, we manually screened the 113 articles obtained from the initial literature search. We included studies that specifically focused on the development of online curricula for high school STEMM education. Given that the review focused on identifying pedagogical and design elements, studies were not required to report specific outcome measures or evidence of curriculum effectiveness to be included. Although we noted whether each curriculum included an evaluation component (RQ 5), we did not assess the outcomes of these evaluations, as outcome analysis was beyond the scope of this review. 

We excluded unpublished research, conference abstracts or posters, literature reviews, editorials, letters, and conceptual pieces. Additionally, articles not written in English, duplicate records, and those lacking full-text availability were also excluded from further consideration. After applying these criteria, 11 journal articles remained and were selected for detailed analysis. 

Analytical Review 

To analyze the articles in the dataset, we first conducted a within-case analysis (Miles & Huberman, 1994). Each article was individually analyzed and summarized in an initial spreadsheet, which was organized into six categories that align with our six research questions (RQs). The first category, Pedagogies and Design Elements (QR 1), covered various instructional design practices that enhanced online learning, ranging from general practices ensuring the curriculum quality to strategies that supported student engagement (Smith & Ragan, 2004). The second category, Learning Activities (QR 2), focused on the interactive components of the online curriculum and examined how students engage with the online material. The third category, Theoretical Considerations (QR 3), identified the theoretical frameworks underpinning curriculum design. The fourth category, Curricula Goals (QR 4), aimed to clarify the intended impacts of the online curricula. The fifth category, Evaluation Methods (QR 5), specified the assessment techniques employed in the review studies. Finally, the sixth category addressed Social Justice and Equity Considerations (QR 6), highlighting how online curricula engaged underserved communities and provided equitable learning opportunities (Calabrese Barton & Tan, 2020). Table 1 presents the findings from the reviewed studies according to these categories. 

Subsequently, we performed a cross-case analysis (Miles & Huberman, 1994), where instructional practices identified across articles were reorganized. Similar elements were grouped together to establish a foundation to answer the research questions. Finally, we synthesized key findings by comparing the results from the reviewed studies.

Table 1

Overview of the review (n=11)

Results

RQ 1: Pedagogies and Design Elements

We identified 11 pedagogies and design elements from the reviewed articles, which were further categorized into three main groups: Comprehensive Curriculum Design, Innovative Pedagogical Approaches, and Learning Support Strategies, as summarized in Figure 2. The Comprehensive Curriculum Design category refers to an integrative approach to curriculum development, which enhances learning outcomes through collaborative efforts among multiple stakeholders. This includes achieving interdisciplinary connections and incorporating various educational standards. The second category, Innovative Pedagogical Approaches, includes the instructional strategies and learning theories employed to guide curriculum design and enhance student engagement. Examples of these approaches include project-based learning and inquiry-based learning. In contrast, Learning Support Strategies consist of specific mechanisms and strategies designed to assist learners in online environments. These strategies aim to enhance student engagement, understanding, and overall educational experience. 

Specifically, the Comprehensive Curriculum Design category includes six studies. For instance, three articles featured curriculum co-design involving various stakeholders, such as in the works of Jones et al. (2022) and Lesiak et al. (2024). Four articles focused on curricula aligned with science content standards like the Next Generation Science Standards (NGSS) (e.g., Marsteller & Bodzin, 2017; Solon et al., 2021). Additionally, three articles developed integrated curricula that emphasized interdisciplinary connections, including those by Astuti et al. (2020) and Sadler et al. (2013). Some studies encompassed all three of these design features. For example, Lesiak and colleagues (2024) crafted a biology curriculum in collaboration with teachers, scientists, and program staff. This curriculum not only merged various subjects and disciplines to promote a holistic understanding but also aligned with the NGSS to reflect real-world applications. 

The second category focuses on pedagogical approaches. We identified six articles that explicitly mention learning theories or pedagogical considerations guiding their curriculum design. For instance, Jones et al. (2022) implemented project-based learning to engage students in virtual environments, providing opportunities for them to create community-serving projects. Similarly, Lesiak et al. (2024) and Sung et al. (2021) utilized inquiry-based learning to promote active participation in interactive learning activities. Sadler et al. (2013) employed game-based learning to boost engagement in science education. Notably, three articles emphasized the use of real-world problem-solving as a pedagogical strategy to design relevant curricula, enabling students to develop transferable skills and enhance their scientific literacy for practical application in daily life (Jones et al., 2022; Solon et al., 2021; Zacharova & Sokolova, 2015). 

The third category focuses on learning support features. We found six articles that emphasized the significance of support mechanisms in online learning. For instance, Heinert et al. (2021) introduced digital badges to encourage learners to engage in positive educational behaviors. Ceviker et al. (2022) underscored the importance of providing instant feedback through online homework systems to enhance student learning. Additionally, the option for students to progress at their own pace was highlighted as a crucial support feature in studies by Lesiak et al. (2024) and Solon et al. (2021). Moreover, the vital role of scaffolding in facilitating online learning was discussed in three articles (Astuti et al., 2020; Lesiak et al., 2024; Sadler et al., 2013), highlighting its effectiveness in supporting student engagement and understanding.

Figure 2

A summary of the pedagogies and Design Elements

Notably, the categorization of design features exhibits a degree of interrelation and overlap, as depicted in Figure 2. This Venn diagram illustrates that while certain studies are distinct in their primary focus, sharing no common features with others (e.g., Marsteller & Bodzin, 2017), several studies bridge two or more categories. Specifically, there are two studies that incorporate elements from both Comprehensive Curriculum Design and Innovative Pedagogical Approaches, three studies that blend Innovative Pedagogical Approaches with Learning Support Strategies, and another three that merge Comprehensive Curriculum Design with Learning Support Strategies. Moreover, a unique intersection is found where three studies embody design features from all three categories, reflecting a multifaceted approach to online curriculum design in high school STEMM education. This overlap highlights the complexity and multifunctional nature of designing online educational resources.

RQ 2: Innovative Learning Activities

From the review, we found various learning activities have been designed to enhance the interactivity and engagement of online curricula (see Figure 3). The most frequently designed learning activity is online lessons with interactive media, with five studies applying this approach (e.g., Astuti et al., 2020; Lesiak et al., 2024). For example, Solon and collaborators (2021) developed a series of lessons for a biomedical science curriculum targeted at 10th to 12th graders, focusing on the science and public health strategies to combat COVID-19. 

Another common activity is group projects or discussions, which were employed in three studies to foster collaborative learning environments where students can participate in remote discussions and group-based tasks (Solon et al., 2021; Sadler et al., 2013; Tsai et al., 2021). Virtual or remote labs also play a significant role in online curricula, adopted by three studies to enhance student engagement in scientific exploration (Sadler et al., 2013; Sung et al., 2021; Tsai et al., 2021). For example, Sadler and colleagues (2013) employed synchronous remote labs to increase high school students’ cognitive and behavioral engagement in science education.

Figure 3

Online Learning Activities

The use of online homework and project-based tasks, though less frequent, still plays a critical role in online curriculum development. Specifically, two reviewed papers applied online homework as one essential learning activity for the online learning curriculum (Ceviker et al., 2022; Tsai et al., 2021). Additionally, two curricula designed specific tasks for students to accomplish (Heinert et al., 2021; Jones et al., 2022). For example, Jones and colleagues (2022) developed an online curriculum assigning students projects they need to accomplish, offering opportunities to students of color to serve their communities with products they made. 

Other innovative learning activities include the use of video games to increase students’ engagement in science learning (Sadler et al., 2013) and the creation of educational comic movies to represent real-life stories (Zacharova & Sokolova, 2015). Notably, some curricula combined multiple learning activities to foster online learning (Sadler et al., 2013; Solon et al., 2021; Tsai et al., 2021). For example, Tsai et al. (2021) implemented a flipped teaching approach, using virtual labs for experiments followed by group discussions to facilitate student reflection. 

RQ 3: Theoretical Considerations 

In the analysis of curriculum development and evaluation processes, we found six studies involved theoretical considerations, while five did not specify any theoretical underpinnings. As Figure 4 indicates, three studies applied a Design-Based Research (DBR) approach, focusing on enhancing educational practices through iterative phases of analysis, design, development, and implementation. For example, Ceviker and collaborators (2022) developed an online homework system for high school physics classes based on DBR, aiming to address existing problems and contribute to a theoretical understanding of online homework practice (McKenney & Reeves, 2018). 

In addition, two studies adopted the Technology Acceptance Model (TAM) as a framework to understand how students and teachers adapt to and use online learning curricula. Specifically, Astuti and colleagues (2020) developed an educational website for science learning on human body topics, using TAM to investigate the perceptions and behavioral intentions of teachers and students toward online resources. Similarly, Tsai and collaborators (2021) conducted experimental teaching with a developed course for senior high school AI deep learning, and the TAM was used to explore learning effectiveness and satisfaction with the teaching.

Figure 4

Theoretical Consideration

Furthermore, one study combined multiple theoretical considerations to craft a design framework guiding curriculum development. Marsteller and Bodzin (2017) synthesized the key components from the Attention, Relevance, Confidence, Satisfaction (ARCS) motivation model, social learning theory, and situated learning theory to support the development of an online module on biological evolution. For example, they incorporated social discourses, an essential element based on social learning theory, emphasizing the importance of learner interaction and communities of practice.

RQ 4: Curricula Goals

The review identified various goals targeted by different curricula, as indicated in Figure 5. Specifically, two studies focused on enhancing scientific literacy through their curriculum design and implementation (Solon et al., 2021; Zacharova & Sokolova, 2015). For example, Solon and collaborators (2021) created an online curriculum about COVID-19 with the overarching goal of promoting scientific and health literacy. Meanwhile, three curricula aimed to improve students’ knowledge or skills. For example, Marsteller and Bodzin (2017) designed an online curriculum specifically to support evidentiary reasoning and self-regulation in lessons on biological evolution. 

A significant number of curricula (seven) targeted increasing high school students' motivation, interest, and engagement in STEMM (Science, Technology, Engineering, Mathematics, and Medicine) learning. For example, Heinert et al. (2021) incorporated digital badging to incentivize learners to participate in positive learning behavior with continued engagement. Sadler et al. (2013) developed a game-based curriculum to boost students’ engagement in science learning. Similarly, Sung et al. (2021) aimed to engage students in online inquiry-based laboratories through a virtual chemistry curriculum.

Figure 5

Targeted Outcomes of the Online

Additionally, two curricula were designed to support problem-solving in STEMM learning. For example, Ceviker et al. (2022) developed an online homework system to enhance STEM learning and problem-solving, specifically in a physics class. Jones et al. (2022) designed their curriculum to provide students with opportunities to tackle real-world problems. Furthermore, two studies in the review highlighted the importance of developing integrated curriculum to support interdisciplinary learning (Astuti et al., 2020; Lesiak et al., 2024). 

Notably, four curricula aimed to achieve more than one of the aforementioned goals (Astuti et al., 2020; Jones et al., 2022; Solon et al., 2021; Zacharova & Sokolova, 2015). For example, Zacharova and Sokolova (2015) developed an interactive psychology course to enhance general science literacy while also promoting knowledge and skills acquisition. Astuti et al. (2020) encouraged students to see the interconnectedness between different subjects while simultaneously boosting student motivation. 

RQ 5: Curricula Evaluation

Different approaches have been used to evaluate online curricula in the reviewed studies (see Table 2). Noticeably, all the reviewed studies examined student perceptions of the developed curriculum. For example, Ceviker et al. (2022) and Marsteller and Bodzin (2017) used student surveys to assess students’ perception of the curriculum, while Heinert et al. (2021) employed student focus groups to understand students’ learning experience with the program. 

Table 2

Overview of the Evaluation of Curricula in Each Paper

In addition to evaluating student perceptions, the reviewed studies employed various evaluation approaches. For example, Astuti et al. (2020) included expert judgment to examine the content, language, and media of the developed website education. Meanwhile, Jones et al. (2022) and Sung et al. (2021) emphasized post-implementation reflections from the curriculum development team as an evaluation approach. Additionally, six curricula involved high school teachers in the evaluation process to gather their perceptions. For example, Lesiak et al. (2024) surveyed teachers on how they taught and rated the curricula. 

Beyond student perceptions, this review identified three types of student-level evaluation approaches to assess the effectiveness and impact of the developed curricula. Specifically, two studies evaluated students’ content-based knowledge acquisition, such as learning gains (Marsteller & Bodzin, 2017; Sung et al., 2021). Two studies examined students’ skill acquisition, such as self-efficacy (Lesiak et al., 2024) and evidence-based reasoning (Marsteller & Bodzin, 2017). Furthermore, online learning provided opportunities to record and analyze students’ online behaviors, allowing two studies to evaluate the curricula based on students’ online learning behaviors recorded in the Learning Management System (Solon et al., 2021; Sung et al., 2021). 

RQ 6: Social Justice and Equity 

Two studies addressed social justice and equity issues in their curriculum development. Specifically, Jones and colleagues (2022) developed a public health curriculum to empower high school students to address racial disparities during the COVID-19 pandemic. The curriculum introduced students to core ideas of health equity and social justice and provided opportunities for them to collaborate with medical professionals to tackle racial and ethnic disparities. As part of the curriculum, students completed projects aimed at serving their communities, such as conducting community empathy interviews and producing social media content to disseminate information on preventing infection from COVID-19. 

Heinert and collaborators (2021) contributed another curriculum that addressed social justice and equity. Recognizing the challenges of reaching underserved communities with health education and the lack of health resources in these areas, their curriculum engaged youth through digital badging to enhance their health knowledge. The curriculum created engaging online experiences that formed a playlist, encouraging students to complete challenges and earn badges, thus fostering their involvement and learning in a motivating manner. 

Discussion and Implications 

The systematic review examined the design and development of online curricula in high school STEMM education, covering studies from 2012 to 2024. Guided by six research questions, this review identified common practices and existing gaps in online curriculum development, including their design features, theoretical frameworks, curriculum goals, evaluation approaches, and considerations for social justice and equity. By answering the RQs through analyzing the findings, we identified patterns and insights that inform further discussion. 

The findings of the review imply the important role of teachers in online curriculum development and implementation. Specifically, findings from RQ 1 highlighted the importance of incorporating diverse pedagogies and design elements in online curricula. This finding reveals the complex and multifunctional nature of creating online educational resources, which requires not only technical and instructional design but also collaboration among various stakeholders, such as teachers, designers, and content experts. Such a comprehensive approach enhances curriculum adaptability across different teaching methods and learning needs (Meyers & Nulty, 2009; Viana & Peralta, 2021), suggesting the significant role teachers play in the development and implementation process. Meanwhile, the review examined learning activities in RQ 2, revealing online lessons with interactive media were the most frequently used. This finding reinforces the essential role of teachers in implementing these designs effectively through teacher-student interactions. Interactive features, such as virtual labs and multimedia-based assignments, offer learning potential, yet active guidance and feedback from teachers remain central to fostering engagement and comprehension (e.g., Tsai et al., 2021). This highlights the importance of defining teachers’ roles in facilitating meaningful interactions within online curricula. 

In addition, the review exposed some notable gaps in online curriculum development practices. Findings from RQ 3 revealed that nearly half of the reviewed studies did not explicitly incorporate theoretical frameworks in their curriculum development processes. This indicates the need for more thorough analysis and planning throughout the curriculum development, recognizing the importance of theoretical foundations in instructional design (Smith & Ragan, 2004). However, DBR emerged as the most frequently used framework among studies that involved theoretical considerations, providing a valuable model for integrating theory with practical applications to evaluate and refine online curricula (e.g., Ceviker et al., 2022). Furthermore, results from RQ 6 revealed a gap in addressing social justice and equity within online curriculum development for high school STEMM education. The lack of focus on these issues indicates the need for intentional strategies that ensure all students have equitable access to quality online resources. This gap raises the question of how we can develop online curricula that address social justice and equity issues. 

Furthermore, the review findings from RQs 4 and 5 provided insights into curriculum goals and evaluation methods. Results from RQ 4 highlighted a range of targeted curriculum goals, ranging from knowledge acquisition to interdisciplinary learning. Many curricula aimed to enhance motivation and engagement in STEMM subjects by leveraging the unique features of online environments and interactive materials (e.g., Heinert et al., 2021; Sadler et al., 2013). These goals indicate the potential of online curricula to support diverse learning needs and emphasize student-centered design as a priority for curriculum developers. In RQ 5, we identified varied evaluation methods across studies, while most studies relied on student perceptions to assess the effectiveness of the curriculum. This finding reflects the value placed on student perceptions to assess whether educational goals are being met and to identify areas for improvement (Hussain, 2011; Nouraey et al., 2020). However, this raises questions about which metrics might enhance online curriculum evaluation, especially given the distinctive goals of online curricula. 

Based on these findings, we raise three questions for further discussion: What roles should teachers play in online curriculum development and implementation? What needs to be considered for effective online curriculum evaluation? and How can online curricula address social justice and equity issues? These questions reflect the critical areas identified in our review and aim to foster further exploration into creating online curricula that are more effective, inclusive, and adaptable.

What Roles Should Teachers Play in Online Curriculum Development and Implementation?

Teachers are not invisible in online curriculum development; they play essential roles that go beyond merely using the developed materials (Lesiak et al., 2024). The review revealed three distinct roles that high school teachers played in the development and implementation of online curricula. Firstly, teachers acted as designers, participating in the co-design process to ensure the content was relevant and practical, drawing from their classroom experience to shape the curriculum effectively. For example, four studies in the review involved teachers in the curriculum co-design process (e.g., Jones et al., 2022; Solon et al., 2021). Secondly, teachers served as facilitators during the implementation phase, engaging students and fostering interaction using online tools (e.g., Sung et al., 2021; Tsai et al., 2021). Finally, teachers provided ongoing feedback on the curriculum's effectiveness, contributing to its iterative refinement and improvement. Specifically, six studies in the review involved teachers in the evaluation process to gather their opinions (e.g., Astuti et al., 2020; Ceviker et al., 2022). These roles suggest the multifaceted involvement of teachers, highlighting their indispensable contribution to the success of online curricula.

Despite the essential roles teachers play in online curriculum development, their involvement comes with several challenges. One significant issue is time constraints. Teachers often have demanding schedules filled with instructional responsibilities, grading, and administrative tasks, leaving limited time for additional activities (Gutierez & Kim, 2017). This time pressure can hinder their ability to fully engage in the curriculum co-design process, potentially affecting their experiences and contributions. For example, providing continuous feedback requires extra effort and time, which can be challenging for teachers to manage alongside their primary teaching duties. Another challenge is the lack of long-term partnerships between teachers and development teams. Effective curriculum development requires ongoing collaboration and continuous feedback, while many projects are short-term and do not establish sustained relationships (Ernest et al., 2013). This lack of continuity might lead to inconsistencies in curriculum implementation and hinder the iterative improvement process. Furthermore, teachers may struggle to access professional development opportunities necessary for staying updated with the latest technological advancements and pedagogical strategies (Teräs, 2016). Without adequate training, teachers might find it difficult to effectively use online tools and integrate innovative practices into their teaching. 

These challenges imply the need to support teachers in online curriculum development and implementation. Several potential solutions can be considered to increase the involvement of teachers in online curriculum development. In particular, building long-term partnerships between teachers and online program development teams is a practical approach (Ernest et al., 2013). These sustained collaborations can facilitate continuous feedback and iterative improvements, enhancing the curriculum’s effectiveness and consistency (Goodyear, 2017). Meanwhile, establishing a community of practice among teachers who are involved in online curriculum development can foster shared learning and support. This community can serve as a platform for exchanging ideas, best practices, and solutions to common challenges, thus enriching the overall development process (Tseng & Kuo, 2014). Furthermore, incorporating mechanisms for recognizing teachers' contributions can motivate and validate their involvement (Bano, 2022; Rimal, 2018). Acknowledging teachers’ efforts through professional recognition, incentives, or formal acknowledgments within the educational community can enhance their commitment and engagement. 

What Needs to Be Considered for Online Curriculum Evaluation? 

Curriculum evaluation is an essential component of the development process (Hussain, 2011). The primary purpose of curriculum evaluation is to assess whether the educational goals and objectives are being met and to identify areas for improvement (Nouraey et al., 2020). The results provide valuable feedback to instructional designers, educators, administrators, and policymakers, guiding them in making informed decisions about curriculum design and implementation (e.g., Padwick et al., 2023; Wu et al., 2021). Through systematic evaluation, the strengths and weaknesses of the curriculum can be identified. Accordingly, curriculum evaluation promotes accountability and transparency, demonstrating to stakeholders that the curriculum is achieving its intended outcomes and providing a basis for continuous improvement. 

The review identified a range of methods used for curriculum evaluation, recognizing four key participant groups: students, teachers, design teams, and experts. These groups provide diverse insights regarding the impact of the developed curriculum. Accordingly, we synthesize a framework for online curriculum evaluation that incorporates multiple perspectives and offers a reference for future evaluation projects. The framework consists of contributions from different participant groups (see Figure 7). From the left to right side, the design team's reflection on the development process helps identify challenges and make necessary adjustments to enhance the curriculum’s quality and coherence (Ernest et al., 2013), while the student feedback provides direct measures of the curriculum’s effectiveness and engagement from the primary beneficiaries of the developed product (Ufnar & Shepherd, 2018). Notably, there are different aspects regarding student evaluation, including assessment of learning gain to measure knowledge and skill acquisition, investigation of student perception to gauge engagement and satisfaction, and analysis of online behavior data to reflect student interaction and engagement. 

Furthermore, teacher and expert feedback are two important elements of the holistic evaluation process. From the top to bottom as Figure 7 shows, the teacher feedback offers practical insights into the usability and effectiveness of the curriculum in real-world teaching environments. Teachers’ experiences and reflections are crucial for understanding the curriculum’s impact on daily teaching practices and student engagement (Gutierez & Kim, 2017). Meanwhile, expert feedback involves different types of experts (e.g., subject matter experts and online technology specialists) evaluating the curriculum's content, structure, and alignment with educational standards. Experts ensure that the material is accurate, relevant, and comprehensive, providing an authoritative perspective on the curriculum’s academic rigor and technological integration (Cordingley, 2015). 

By combining diverse perspectives from these four participant groups, the framework presents a holistic evaluation process that addresses various aspects of the curriculum. Depending on the desired outcomes from the evaluation, future studies can choose to work with different groups. For example, to generate lessons for iterative improvements or design principles to inform other studies, involving the design teams in the evaluation process is essential. Likewise, to find out end-user experience and issues to refine the curriculum, student groups should be involved.

Figure 7

An Online Curriculum Evaluation Framework

How Do We Develop Online Curriculum That Address Social Justice and Equity Issues?

 Social justice in education refers to the fair distribution of resources and opportunities to all students, ensuring every individual can achieve their fullest potential regardless of their background (Sensoy & DiAngelo, 2017). Meanwhile, equity goes beyond equality by recognizing the different needs of students and providing the necessary support to address those needs (Gorski, 2013). Most commonly, equity is framed around the extension of rights, questioning who has access to high-quality learning opportunities. However, this framing can be problematic as it does not automatically translate into actual equitable outcomes in the classroom (Calabrese Barton & Tan, 2020). Instead, equity should be viewed through the lens of "rightful presence," which emphasizes creating learning environments where all students feel they belong and can see their identities and experiences reflected in the curriculum (Calabrese Barton & Tan, 2019). 

The review finding suggested a lack of consideration for social justice and equity in the development of online curricula. Only two studies explicitly addressed these issues in the context of online education. Specifically, Jones et al. (2022) developed a public health curriculum to empower high school students to address racial disparities during the COVID-19 pandemic. This curriculum exposed students to health equity and social justice issues and invited them to address the disparities in their communities through project-based learning. Heinert et al. (2021) created a curriculum for underserved communities, using digital badging to engage youth and improve their health knowledge. This curriculum recognized the challenges of reaching underserved populations and sought to provide access to health education resources. Through the lens of rightful presence, Jones and collaborators developed a curriculum for students to find their positions and address the racial and ethnic disparities in their communities. Heinert and collaborators aimed to create a curriculum that extends rights to online settings by providing access to essential resources. 

While online curricula can be viewed as an approach to social justice and equity by extending learning opportunities, it is insufficient if justice-centered considerations are not included. Open-access materials alone do not ensure equity unless they address the specific needs and contexts of diverse learners. The review identified a couple of frameworks for addressing social justice and equity in online curriculum development. Specifically, Research-Practice Partnerships (RPPs) can be an effective approach. RPPs involve collaboration between researchers and practitioners to design, implement, and evaluate educational interventions. This partnership ensures that the curriculum is grounded in both theory and practice, addressing real-world needs and challenges (Coburn & Penuel, 2016; Marshall et al., 2021). By working together, researchers and practitioners can create curricula that are not only academically rigorous but also socially and culturally relevant. 

Another approach is DBR, which involves iterative cycles of design, implementation, analysis, and redesign (Design-Based Research Collective, 2003). DBR was identified in our review as the most frequently used framework, highlighting its suitability for addressing complex challenges in online curriculum development. The iterative nature of DBR fosters collaboration among diverse stakeholders—such as educators, curriculum designers, subject experts, and students—throughout each stage of curriculum development, ensuring that multiple perspectives shape and continuously refine the curriculum (Anderson & Shattuck, 2012). Through continual refinement based on empirical data and feedback, DBR facilitates the development of online curricula that are not only pedagogically effective but also accessible and equitable. As a result, DBR offers a promising approach to addressing complex educational challenges, including equity and social justice. Additionally, DBR can overlap and integrate with RPPs, allowing stakeholders to take dual roles as both practitioners and researchers. The combination of these two approaches supports active collaboration rather than merely implementing researcher-developed interventions, thus enhancing the practical relevance and impact of the curriculum (Ceviker et al., 2022; Penuel et al., 2013). 

Incorporating social justice and equity into online curriculum development requires intentionality and collaboration. Viewing equity through the lens of rightful presence emphasizes the importance of creating learning environments where all students feel they belong and can see their identities and experiences reflected in the curriculum. By using frameworks like Research-Practice Partnerships and Design-Based Research, educators and curriculum developers can create online learning environments that are inclusive, responsive, and equitable for all students. These approaches ensure that multiple perspectives are considered and that the curriculum is continuously refined to meet the diverse needs of learners, ultimately fostering a rightful presence for every student. 

Limitations and Directions for Future Research

This review focused on identifying common design features and existing gaps in online curriculum development for high school STEMM education, analyzing 11 studies published between 2012 and 2024. However, some limitations of this study need to be considered when interpreting the findings. First, while this review highlighted common practices in online curriculum development, it did not assess the quality or effectiveness of these practices. As a result, our findings should not be interpreted as a showcase of best practices for online curriculum design; instead, they represent commonly observed practices across the selected studies. Future studies could consider conducting meta-analyses or evaluative studies to determine which practices yield the most positive impacts on online learning outcomes for high school STEMM education. 

Another limitation is the relatively small sample size of 11 studies, which may not fully capture the range of practices in online STEMM education. This review focused specifically on high school education, which limits the scope of the findings for online curriculum development. Future research could expand the search range to include K-12 education or higher education to capture a broader range of practices and insights across different educational levels. Additionally, the studies included in this review did not consistently specify whether the curricula were implemented in public or private school settings. This distinction is important since differences in resources and student populations between public and private schools could influence the curriculum design, implementation, and its effectiveness (Frenette & Chan, 2015; Sarwar & Jabbar, 2024). Therefore, future studies could explore whether and how school type impacts the design and success of online curricula. 

List of Abbreviation

• ARCS: Attention, Relevance, Confidence, Satisfaction (a motivational design model)
• DBR: Design-Based Research ERIC: Education Resources Information Center
• NGSS: Next Generation Science Standards
• PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
• RPPs: Research-Practice Partnerships
• STEMM: Science, Technology, Engineering, Mathematics, and Medicine
• TAM: Technology Acceptance Model

Declaration

Availability of Data and Materials

All data generated or analyzed during this study are included in this published article.

Competing Interests

The authors declare that they have no competing interests.

Funding

No funding was received for this study.

Authors’ Contributions

Author 1 led the systematic review process and wrote the manuscript. Authors 2 and 3 contributed to the data analysis and provided revisions to the manuscript. All authors read and approved the final manuscript.

Acknowledgments

The authors declare that they have nothing to acknowledge.

Reference

2. Abichandani, P., Sivakumar, V., Lobo, D., Iaboni, C., & Shekhar, P. (2022). Internet-of-things curriculum, pedagogy, and assessment for stem education: A review of literature. IEEE Access, 10, 38351-38369. https://ieeexplore.ieee.org/abstract/document/9749084 
3. Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16-25. https://doi.org/10.3102/0013189X11428813 
4. Anwar, S., Menekse, M., Guzey, S., & Bryan, L. A. (2022). The effectiveness of an integrated STEM curriculum unit on middle school students' life science learning. Journal of Research in Science Teaching, 59(7), 1204-1234. https://doi.org/10.1002/tea.21756 
5. Astuti, L., Wihardi, Y., & Rochintaniawati, D. (2020). The Development of web-based learning using interactive media for science learning on levers in human body topic. Journal of Science Learning, 3(2), 89-98. https://ejournal.upi.edu/index.php/jslearning/article/view/51 
6. Bano, R. (2022). Examine the importance of teachers’ participation in curriculum development. Voyage Journal of Educational Studies, 2(1), 1-12. https://doi.org/10.58622/vjes.v2i1.9 
7. Calabrese Barton, A., & Tan, E. (2019). Designing for rightful presence in STEM: The role of making present practices. Journal of the Learning Sciences, 28(4-5), 616-658. https://doi.org/10.1080/10508406.2019.1591411 
8. Calabrese Barton, A., & Tan, E. (2020). Beyond equity as inclusion: A framework of “rightful presence” for guiding justice-oriented studies in teaching and learning. Educational Researcher, 49(6), 433-440. https://doi.org/10.3102/0013189X20927363 
9. Ceviker, E., Strycker, J., & Moody, A. (2022). Development and evaluation of an online homework system for high school physics classes. Journal of Learning Spaces, 11(2), 11-23. https://libjournal.uncg.edu/jls/article/view/2189 
10. Chiang, F. K., Zhang, Y., Zhu, D., Shang, X., & Jiang, Z. (2022). The influence of online STEM education camps on students’ self-efficacy, computational thinking, and task value. Journal of Science Education and Technology, 31, 461–472. https://doi.org/10.1007/s10956-022-09967-y 
11. Coburn, C. E., & Penuel, W. R. (2016). Research–practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48-54. https://doi.org/10.3102/0013189X16631750 
12. Cordingley, P. (2015). The contribution of research to teachers’ professional learning and development. Oxford Review of Education, 41(2), 234-252. https://doi.org/10.1080/03054985.2015.1020105 
13. DeCoito, I., & Estaiteyeh, M. (2022). Online teaching during the COVID-19 pandemic: Exploring science/STEM teachers’ curriculum and assessment practices in Canada. Disciplinary and Interdisciplinary Science Education Research, 4(1), 8. https://doi.org/10.1186/s43031-022-00048-z 
14. Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. https://doi.org/10.3102/0013189X032001005 
15. Erickson, M., Erasmus, M. A., Karcher, D. M., Knobloch, N. A., & Karcher, E. L. (2020). High school student and teacher perceptions of an online learning experience integrating STEM and Poultry Science. Journal of Agricultural Education, 61(2), 20-43. https://doi.org/10.5032/jae.2020.02020 
16. Ernest, P., Guitert Catasús, M., Hampel, R., Heiser, S., Hopkins, J., Murphy, L., & Stickler, U. (2013). Online teacher development: Collaborating in a virtual learning environment. Computer Assisted Language Learning, 26(4), 311-333. https://doi.org/10.1080/09588221.2012.667814 
17. Frenette, M., & Chan, P. C. W. (2015). Academic outcomes of public and private high school students: What lies behind the differences? (Analytical Studies Branch Research Paper Series). Statistics Canada. https://files.eric.ed.gov/fulltext/ED585228.pdf 
18. Gamage, S. H., Ayres, J. R., & Behrend, M. B. (2022). A systematic review on trends in using Moodle for teaching and learning. International Journal of STEM Education, 9(1), 9. https://doi.org/10.1186/s40594-021-00323-x 
19. Goodyear, V. A., Casey, A., & Kirk, D. (2017). Practice architectures and sustainable curriculum renewal. Journal of Curriculum Studies, 49(2), 235-254. https://doi.org/10.1080/00220272.2016.1149223 
20. Gorski, P. C. (2013). Reaching and teaching students in poverty: Strategies for erasing the opportunity gap. Teachers College Press.
21. Gutierez, S. B., & Kim, H. B. (2017). Becoming teacher-researchers: Teachers’ reflections on collaborative professional development. Educational Research, 59(4), 444-459. https://doi.org/10.1080/00131881.2017.1347051 
22. Heinert, S. W., Quasim, N., Ollmann, E., Socarras, M., & Suarez, N. (2021). Engaging youth through digital badges to promote health in underserved communities. Health Promotion Practice, 22(5), 631-637. https://doi.org/10.1177/1524839920934798 
23. Hussain, A., Dogar, A. H., Azeem, M., & Shakoor, A. (2011). Evaluation of curriculum development process. International Journal of Humanities and Social Science, 1(14), 263-271. https://www.ijhssnet.com/journals/Vol_1_No_14_October_2011/34.pdf 
24. Jones, N., Cohen, J., & Chapman, J. I. (2022). Empowering high school students to address racial disparities during the COVID-19 pandemic. Pediatrics, 149(1), e2021050483. https://doi.org/10.1542/peds.2021-050483 
25. Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele University. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=29890a936639862f45cb9a987dd599dce9759bf5 
26. Lesiak, A., Griswold, J. C., Moylan, A., & Starks, H. (2024). Development and evaluation of integrated diabetes curricula for teaching gene by environment concepts to high school health and biology students. Journal of STEM Outreach, 7(1). https://www.jstemoutreach.org/article/115429-development-and-evaluation-of-integrated-diabetes-curricula-for-teaching-gene-by-environment-concepts-to-high-school-health-and-biology-students. 
27. Marshall, S. L., Nazar, C. R., Ibourk, A., & McElhaney, K. W. (2021). The role of collective sensemaking and science curriculum development within a research–practice partnership. Science Education, 105(6), 1202-1228. https://doi.org/10.1002/sce.21676 
28. Marsteller, R. B., & Bodzin, A. (2019). Examining the implementation of an online curriculum designed with the person theoretical framework on student’s evidentiary reasoning and self-regulated learning. The Electronic Journal for Research in Science & Mathematics Education, 23(3), 73-95. https://files.eric.ed.gov/fulltext/EJ1228448.pdf 
29. McKenney, S., & Reeves, T. (2018). Conducting educational design research. Routledge.
30. Meyers, N. M., & Nulty, D. D. (2009). How to use (five) curriculum design principles to align authentic learning environments, assessment, students’ approaches to thinking and learning outcomes. Assessment and Evaluation in Higher Education, 34(5), 565-577. https://doi.org/10.1080/02602930802226502 
31. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. SAGE Publications.
32. Mulrow, C. D. (1994). Systematic reviews: Rationale for systematic reviews. BMJ, 309(6954), 597-599. https://doi.org/10.1136/bmj.309.6954.597 
33. Nouraey, P., Al-Badi, A., Riasati, M. J., & Maata, R. L. (2020). Educational program and curriculum evaluation models: A mini systematic review of the recent trends. Universal Journal of Educational Research, 8(9), 4048-4055. https://doi.org/10.13189/ujer.2020.080930   
34. Padwick, A., Dele-Ajayi, O., Davenport, C., & Strachan, R. (2023). Evaluating a complex and sustained STEM engagement programme through the lens of science capital: Insights from Northeast England. International Journal of STEM Education, 10(1), 33. https://doi.org/10.1186/s40594-023-00421-y 
35. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., ... & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372. https://doi.org/10.1136/bmj.n71 
36. Pappano, L. (2012, November 2). The Year of the MOOC. The New York Times. https://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html 
37. Penuel, W. R., Coburn, C. E., & Gallagher, L. P. (2013). Negotiating problems of practice in research-practice partnerships: Learning as we go in design-based implementation research. Yearbook of the National Society for the Study of Education, 112(2), 237-256. https://doi.org/10.1177/016146811311501404 
38. Reeves, T. C. (2000). Socially responsible educational technology research. Educational Technology, 40(6), 19-28. https://www.jstor.org/stable/44428634 
39. Rimal, K. (2018). Teacher: An important but less recognized actor of school curriculum development in Nepal. Dhaulagiri Journal of Sociology and Anthropology, 12, 66-71. https://doi.org/10.3126/dsaj.v12i0.22181 
40. Sadler, T. D., Romine, W. L., Stuart, P. E., & Merle‐Johnson, D. (2013). Game‐based curricula in biology classes: Differential effects among varying academic levels. Journal of Research in Science Teaching, 50(4), 479-499. https://doi.org/10.1002/tea.21085 
41. Sarwar, S. & Jabbar, M. N. (2024). Comparison between public and private secondary school teachers about the involvement in administrative role. Contemporary Journal of Social Science Review, 2(04), 190-197. https://stars.library.ucf.edu/etd/621 
42. Sensoy, O., & DiAngelo, R. (2017). Is everyone really equal? An introduction to key concepts in social justice education. Teachers College Press.
43. Smith, P. L., & Ragan, T. J. (2004). Instructional design. John Wiley & Sons.
44. Solon, V., Gao, Y. Z., Jacque, B., & Meiri, K. (2021). Designing, Disseminating and Evaluating a New Online High School Curriculum about COVID-19. Journal of STEM Outreach, 4(3), n3. https://doi.org/10.15695/jstem/v4i3.10   
45. Su, Y. S., Chang, C. Y., Wang, C. H., & Lai, C. F. (2022). A study of students’ learning perceptions and behaviors in remote STEM programming education. Frontiers in Psychology, 13, 01-10. https://doi.org/10.3389/fpsyg.2022.962984 
46. Sung, S. H., Li, C., Huang, X., & Xie, C. (2021). Enhancing distance learning of science—Impacts of remote labs 2.0 on students' behavioural and cognitive engagement. Journal of Computer Assisted Learning, 37(6), 1606-1621. https://doi.org/10.1111/jcal.12600 
47. Teräs, H. (2016). Collaborative online professional development for teachers in higher education. Professional Development in Education, 42(2), 258-275. https://doi.org/10.1080/19415257.2014.961094 
48. Tsai, C. C., Cheng, Y. M., Tsai, Y. S., & Lou, S. J. (2021). Impacts of AIOT implementation course on the learning outcomes of senior high school students. Education Sciences, 11(2), 82. https://doi.org/10.3390/educsci11020082 
49. Tseng, F. C., & Kuo, F. Y. (2014). A study of social participation and knowledge sharing in the teachers’ online professional community of practice. Computers and Education, 72, 37-47. https://doi.org/10.1016/j.compedu.2013.10.005 
50. Ufnar, J., & Shepherd, V. (2018). Interdisciplinary science and research: A novel program to advance student preparation in STEM. Journal of STEM Outreach, 1(1), 1-10. https://doi.org/10.15695/jstem/v1i1.6 
51. Viana, J., & Peralta, H. (2021). Online learning: From the curriculum for all to the curriculum for each individual. Journal of New Approaches in Educational Research, 10(1), 122-136. https://doi.org/10.7821/naer.2021.1.579 
52. Wu, X. B., Sandoval, C., Knight, S., Jaime, X., Macik, M., & Schielack, J. F. (2021). Web-based authentic inquiry experiences in large introductory classes consistently associated with significant learning gains for all students. International Journal of STEM Education, 8, 1-18. https://doi.org/10.1186/s40594-021-00290-3 
53. Yang, D. (2017). Instructional strategies and course design for teaching statistics online: perspectives from online students. International Journal of STEM education, 4, 1-15. https://doi.org/10.1186/s40594-017-0096-x 
54. Yıldırım, B. (2022). MOOCs in STEM education: Teacher preparation and views. Technology, Knowledge and Learning, 27(3), 663-688. https://doi.org/10.1007/s10758-020-09481-3 
55. Zacharova, Z. J., Sokolova, L., & Lemesova, M. (2015). Interactive teaching of psychology: Using design-based research in developing an interactive psychology course. In I. Maleńczyk (Ed.), The 11th International Scientific Conference eLearning and Software for Education (pp. 186-193). https://doi.org/10.12753/2066-026X-15-118