Design of Problems in a PBL-Based Makerspace–A Delphi Technique

Introduction

The widespread popularity of makerspaces has boosted the rise of making practices in pre-K12. More schools are incorporating making activities into their curricula, either through informal or formal mediums of instruction. Despite the wide acceptance of makerspaces in schools, makerspaces lack a pedagogical approach focusing on making activities that can be aligned with the school curriculum. While the notion of makerspace seems modern, its framework is woven through educational learning theories of constructivism and constructionism, both emphasizing providing conducive learning environments to learners by engaging in hands-on learning activities through making (Hmelo-Silver et al., 2007; Palincsar, 1998). Makerspace practices are grounded within the theoretical tenets that led to the belief and have proven to develop problem-solving and critical thinking skills among learners (Karagiorgi & Symeou, 2005). Such potential in makerspaces has fuelled the momentum of the maker movement in many schools to incorporate making in their schools with a strong belief that making can teach STEM concepts in more concrete and innovative ways (Bevan, 2017; Honey & Kanter, 2013; Martin, 2015; Quinn & Bell, 2013). However, a lack of a pedagogy is one of the key quests to formalize the makerspace culture in schools that still needs investigation.

In this paper, we present a design of problems to anchor a makerspace learning environment woven in a problem-based learning framework for catalyzing the learning process. Problem-based learning approaches harmonize with makerspace cultures that foster learning environments by engaging learners in hands-on learning; both emphasize student-centered learning goals where learners construct their own knowledge through active participation in meaningful activities. Problem-based learning seamlessly integrates well with makerspaces, as both align with theoretical notions. To date, only a handful of studies have utilized the PBL approach in makerspace learning. In addition, the use and implementation of the PBL approach in makerspace activities remain unclear. Many studies have integrated makerspace learning into existing school curricula or capstone projects. However, there is a need to understand the impact of maker activities when integrated within the school settings (Schlegel et al., 2019). One of the challenges in implementing makerspace activities successfully is the distinct difference between the learning environments of makerspaces and schools. Makerspaces are known for their informal structure, where learning is a byproduct of hands-on activities such as artifact creation and discovery-based learning. In contrast, schools typically follow traditional, standardized instruction and assessment methods, which may lead to conflicts in pedagogical approaches in Makerspaces.

While the capability of makerspaces is widely acknowledged in transforming education, the design of problems suggested in this paper based on PBL frameworks may attempt to enhance the actual potential of makerspace learning environments through guided instruction, which can benefit learners in constructing their knowledge by solving real-world problems and challenges.

Design Team

The design and development of problems and challenges for integration into a PBL-based makerspace were carried out by a diverse team that brought unique skills and expertise to create a robust design. This design phase involved a team of four members and was part of a larger project aimed at understanding the role and effectiveness of PBL-supported makerspaces in enhancing middle school students' critical thinking skills. The interdisciplinary expertise of the team members significantly contributed to the development of problem tasks for implementing the project in middle school after-school science programs. The team included one doctoral student, two faculty experts, and an undergraduate engineering student, all of whom collectively contributed their distinct knowledge and expertise.

The first author is a doctoral student and is in the third year of her PhD studies in instructional technology at a public university in the Midwest U.S. in the Education Technology Research and Assessment (ETRA) department. Her focus areas include problem-based learning in makerspace learning environments. She (a doctoral student) served as a research assistant under the supervision of a faculty expert (the second author). She developed and designed the problem tasks to integrate them into makerspace learning environments to understand how problem tasks based on PBL frameworks can encourage problem-solving skills and enhance learning among middle schoolers.

The second author is a faculty expert and a professor in the Education Technology Research and Assessment department at a public university in the Midwest U.S. for the past 19 years. Her expertise is developing intelligent tutoring systems to promote scientific argumentation, equity pedagogy in teaching and learning, emerging technology integration in teaching and learning, and computer-supported collaborative learning. Faculty experts guided and mentored in reviewing and refining the design at various levels. Faculty experts provided guidance right from the ideation of the project, starting with offering real-world challenges that can mimic realistic problems. The focus was placed on the alignment of theory and activities to offer learning opportunities to middle school students. Further, another faculty expert from Mechanical engineering and an undergraduate engineering student offered insights and guidance from the engineers’ perspective, which helped in refining the problems to make a better relevance for STEM education and connect with STEM concepts.

Design Process

The design process went through several stages and a period of a semester, beginning with the initial idea of making learning relevant for middle school standards.

Challenge: Crafting Problem Scenarios for Makerspace Learning Environments

One of the initial challenges that our design team experienced is finding relevant pedagogy to teach problem-solving to middle school students that can mirror real-world issues. Moreover, the problem should be relevant to incorporating STEM-related disciplines within makerspace learning environments. Identifying and crafting the problem scenarios was the initial struggle that our team experienced in integrating into the maker space learning environment. Thus, the very first question was that is our problem is well-crafted or not? Within that purview, the central notion behind designing the problems was developing and replicating real-world issues that could challenge students and enable them to tackle such tasks. Another association that we aimed to look out for is connecting problems with community issues. Providing opportunities that allow students to solve problems that exist in their communities could offer a great sense of achievement among students. Furthermore, students can find an affiliation with their communities which is a motivating factor for engaging them in exploring the solutions. Design experts advocate that engaging students in communities related contextual learning helps students build a deeper understanding of course content or topic because of real-world exposure of meaningful local or global issues through hands-on learning (Hou, 2009; Butin, 2006).

Moreover, while designing high-quality problem-solving tasks that effectively stimulate learning, it presents a significant challenge for instructors. In problem-based learning (PBL), well-crafted problems serve as the catalyst for driving inquiry and encouraging deeper exploration, which demands careful planning and thoughtful implementation. To successfully integrate PBL frameworks into instructional design, it is essential to incorporate key PBL characteristics that promote engagement, challenge students, and motivate them to actively pursue solutions. The problem itself acts as a learning stimulus, sparking inquiry and critical thinking. Therefore, developing an effective problem task requires a structured approach aligned with the core learning objectives of PBL, ensuring that the task is both engaging and educationally rigorous. Table 1 summarises the problem features that will be utilized as a basis for framing a well-crafted problem inspired by Tan-Oon-Seng and Tee's (2003) criteria, which is used as a questionnaire to gain the expert’s opinion. Tan-Oon-Seng and Tee (2003) outlined the features of design or crafting problems in Problem-based Learning (PBL), which provides a foundational framework for designing effective problems and challenges that can promise in enhancing student’s learning. A well-designed problem should reflect real-world relevance, connecting learners to authentic challenges that make their learning meaningful and applicable. Problems are typically open-ended, allowing for multiple solutions and encouraging critical thinking and creativity. They must also align with specific learning outcomes, ensuring that students achieve both cognitive and skill-based objectives. A key feature is the cognitive challenge posed by the problem, which motivates learners to analyze, synthesize, and evaluate information actively. Additionally, the problem designs should be crafted, maintaining progress from beginners’ level to advanced challenges through embedding simpler tasks in the complex ones, ensuring gradual and incremental success and learning outcomes. Finally, embedding reflective thinking is an integral part of PBL design, ensuring reinforcement of metacognitive skills, enabling students to evaluate their own progress over the learning period and improve their cognitive abilities.

This framework is widely used across many educational contexts for designing problem challenges in PBL curricula, including STEM education and makerspace environments, to ensure that learning is engaging, rigorous, and contextually meaningful.

Table 1

Rubric/Criteria for the Design of PBL Framework in Makerspace Science Class Activity

Note: Adapted from Tan-Oon-Seng (2003), Problem-Based Learning Innovation: Using Problems to Power Learning in the 21st Century, pp. 87

Conceptualizing Design of Problems for Makerspace Learning Environments

The foundational principles of PBL are that learning is situated in a problem, and the problem navigates through the solution finding by identifying the information that they know, how new information is associated with prior knowledge, and finding out the viability of new conceptions. This is one of the essential aspects of PBL that offers holistic learning by engaging students in an active role and encouraging their thinking skills (Radcliffe & Kumar, 2016; Drake & Long, 2009; Boud & Feletti, 1991; Schmidt, 1983; Barrows & Tamblyn, 1980). The nature and characteristics of PBL approaches center on problems, which are the starting point of learning, and problem-solving involves the application of cross-disciplinary knowledge, which is another key feature in PBL curricula (Seng, 2000).

We selected a real-world problem scenario involving power outages and dark residential communities, which can increase the risk of theft and crime. The scenario was broken down into smaller, interconnected episodes, each leading to the final challenge. Students address these episodes step by step, allowing them to gradually grasp related concepts without feeling overwhelmed by a single large task. This episodic approach enables progressive learning and enhances their problem-solving abilities. The instructional design integrates smaller, interconnected episodes, each contributing to a larger problem-solving challenge. This episodic approach not only scaffolds learning but also prepares students to tackle the final project, a comprehensive problem-solving task requiring the application of all previously learned concepts.

The four-day after-school makerspace program integrates Problem-Based Learning (PBL) by engaging students in hands-on problem-solving, collaborative exploration, and reflective practices. On Day 1, students are introduced to the overarching problem scenario, "Moving into a new community." They begin by sketching their dream house and transforming their designs into 3D prototypes using materials like cardboard. This initial activity encourages creativity, teamwork, and idea-sharing, culminating in reflective journaling. On Day 2, the focus shifts to addressing a power outage by designing a basic lighting system using single-color LEDs, progressively advancing to programming light sensors to control multiple LEDs. This introduces students to foundational programming concepts. On Day 3, the challenge intensifies with the task of creating an intruder alert system. Students incorporate a distance sensor to trigger LED lights based on the object's proximity, further refining their programming and circuit-building skills. Day 4 builds on this by tasking students with developing an escape plan for emergencies. This involves integrating a servo motor, a sound sensor, and LED lights to create a functional alarm system with audio and motion features, simulating a complete safety solution.

Each day includes scaffolded practice and exploration tasks, allowing students to master concepts such as basic circuits and sensor integration before applying their skills to increasingly complex challenges. Students reflect daily on their learning experience, highlighting problem-solving strategies, teamwork, and creativity. This iterative design ensures both cognitive (e.g., learning STEM concepts) and affective (e.g., building self-efficacy) goals are achieved, aligning the program with makerspace objectives. The incorporation of Snap! Programming throughout helps students link theoretical concepts with practical robotic solutions (See Appendix - Table 2 at the end summarizes the four-day problem task activities in makerspace after school science program).

Rationale for Design of Problems

This design of problems adopts a structured approach to teaching middle school students’ robotic concepts in a Problem-Based Learning (PBL) environment, gradually increasing task difficulty to align with Bloom's Taxonomy. Initially, foundational concepts, such as using single-color LED lights, are introduced to establish basic understanding (Remember and Understand levels of Bloom's Taxonomy). As students gain proficiency, more complex tasks, including programming TRI-Color LED lights, sensors, motors, and servos from the Hummingbird robotics toolkit, are introduced. These components are presented as inputs and outputs, with each session building on previous knowledge via developing higher-order thinking skills. By gradually advancing from simpler to more challenging activities, students’ progress through Bloom's levels, from applying and analyzing individual components to evaluating and creating complex robotic solutions, such as students developing automated home security systems in this study.

The design of problems also utilizes a gradual approach of scaffolding as guided instructions within a student’s zone of proximal development (Vygotsky, 1978) that offer students resources and knowledge that are required for them to increase their skills to address the problem task. To further support students within their zone of proximal development (ZPD), each lesson episode includes scaffolded tutorials and guided support. These instructional aides are designed to provide students with just enough assistance to complete tasks that they might struggle to accomplish independently, while still encouraging autonomy and exploration. By guiding students through smaller episodes that build upon each other, we ensure they are working at the edge of their capabilities, gradually mastering each component before moving to the next level of difficulty. This aligns with Vygotsky's ZPD by allowing students to achieve learning outcomes through structured support, reducing assistance as their competence grows. This approach supports students as they progress from simple to complex concepts, fostering both skill development and confidence in their problem-solving abilities.

Design of Robotic Activities

As a team, we aimed to develop the design of problems that enable students to navigate through complex problems in the STEM area by providing them with the opportunity to apply problem-solving and critical thinking skills. Papert (1993) advocates that the use of educational robotics in education can offer hands-on learning opportunities to learners. Moreover, research demonstrates that the positive impact of integrating robotics challenges can improve students’ cognitive abilities, maths, and science content knowledge as well (e.g., Darmawansah et al., 2023; Zhang et al., 2024; Arafat et al., 2024; Hiskens et al., 2011; Nugent et al., 2009). We employed block-based coding using Snap! Programming alongside the Hummingbird robotic kit to introduce the basics of robotics to middle school students in an after-school science program. The Hummingbird robotic kit is a product of Birdbrain Technologies, which originated at Carnegie Mellon University and was funded by the National Science Foundation (NSF). Hummingbird robotic kit offers enormous and fascinating science projects from the age of 8 years and onwards. Moreover, Hummingbird course contents encourage interdisciplinary projects that allow students to gain learning and get familiar with concepts from multiple domains. Also, pertaining to meet the standards of Next Generation Science Standards (NGSS). Hummingbird robotic kit contains LED lights, servos, motors, sensors, etc. Figure 1 displays the hummingbird robotic kit components, and Figure 2 shows the hummingbird controller, which is connected via USB to the computer device.

Figure 1

Hummingbird Kit Content

Figure 2

Hummingbird Controller

To activate Hummingbird robotic technology, a programming language Snap! is used. Snap! is a visual programming language that uses a drag-and-drop interface with visual block-based coding, making learning more intuitive, as whatever changes are made to the code can instantly reflect in the hummingbird robot functionality. Also, it’s a web-based browser which makes it convenient for users, especially a hassle-free access to school students (See Figure 3., showing Snap! block-based coding interface). The right-hand panel with “blocks” displays various colors of block codes, which are used to make coding run the command and develop a program. These blocks are simply dragged to the Script area, where the code can be written. The stage area is used when a user wants to see the movement or motions of a character, also called Sprites (animated characters), such as a boy playing football or a cat walking.

Figure 3

Snap! Programming Interface

To start the hummingbird robotic kit, it requires connection to the computer device using a USB cable, which then opens the BirdBrain Robotic Server application to launch the Snap! programming. Once the Snap! programming interface is open for access, which allows users to create block-based coding to control hummingbird outputs (LED lights and motors) and inputs (sensors). Snap! programming offers different categories in the Menu section (see Figure 4. Menu of Snap! programming) for selecting block codes for output and input devices, such as the motion category for servos, motors for movement direction, Look blocks for LED and TRI-color LED lights, with additional alternatives for sprite (animated character) and speech. The scripts in Snap! Programming sends commands to output devices to perform or make something happen, such as turn on the LED lights to maximum brightness. On the other hand, input devices (sensors) send information to Snap! for decision making or controlling output devices (LED lights and motors). Figure 5 demonstrates the light sensor controlling the two different single-color LED lights, and fig 6 shows the Snap! programming code for light sensors controlling LED lights.

Figure 4

Menu Categories in Snap! Programming

Figure 5

Two Single-Color LED

Figure 6

Snap! Code for Light Sensor Controlling LED Lights

Post Design Evaluation of the Design of Problems

The Delphi technique was employed as a systematic approach to gather and integrate expert feedback, which is especially relevant when refining instructional designs for complex educational frameworks like PBL. Therefore, we applied the Delphi technique after developing the final PBL design but before its implementation in a pilot program.

Rationale for Adopting the Delphi Technique Post Design Phase

Applying the Delphi technique to refine the design of problems for PBL-based makerspaces in the post-design phase and prior to pilot implementation can provide valuable insights through expert feedback. While there are various ways to utilize the Delphi method, this approach enables us to systematically refine and enhance our PBL design, ensuring its effectiveness and readiness before launching the pilot program in middle school.

The outset of the Delphi technique began in the 1930s in academia and has been found lucrative in educational settings, such as preparing guidelines and standards. Educators found the Delphi technique very useful in developing curricula that can emphasize enhancing learners’ experience and providing leverage to prepare students for their future careers. The goal of adapting Delphi techniques will encourage educators to align their educational goals with curriculum development (Green, 2014). Scholars, for instance, Judd (1972) also underscore the value of the Delphi technique in many areas such as curriculum planning and educational objectives in education. The method used in Delphi is via collecting data such as feedback from an expert panel seeking their opinions to make informed decisions, and crucial insights can help in determining future plans (Helmer, 1966).

Expert Panel

To implement the Delphi technique in this design study, a panel of four experts was carefully selected. The panel included a professor specializing in education and technology, an assistant professor from the engineering field, a junior high school teacher with expertise in STEM, and a teacher with experience in STEM and medical science education for grades 6-8. Each expert was highly skilled and experienced in the STEM field, with extensive involvement in research projects focused on problem design within STEM education. Additionally, all panel members had substantial experience in designing PBL units and developing problem-solving cases for academic settings.

Delphi Methodology: Overview

In this design study, we applied the Delphi technique to reach the expert consensus on the design of problems in PBL-based makerspace learning environments. Expert feedback and opinions were collected over one round to refine the design of the problem until reaching an acceptable level of agreement. The literature suggests that the Delphi process can be continued theoretically with multiple rounds until consensus is reached or achieved. As a result, Delphi analysts can conduct more rounds if they believe they need more iterations in the feedback for further clarity or to refine the consensus. In this study, before starting the Delphi rounds, we defined a priori that achieving a 75% level of agreement among experts would indicate sufficient consensus to proceed without additional rounds. This threshold aligns with prior studies, where similar consensus levels were set to ensure reliability while acknowledging expert diversity (Hsu & Sandford, 2007; Diamond et al., 2014). The acceptable threshold varies widely considering as low as 50% to as high as 97%. On average, many studies consider 75% as an acceptable agreement as a practical standard for reaching a consensus (Diamond et al., 2014; Foth et al., 2016). There is a significant variation in what is accepted as consensus, which may also depend on the context’s settings and the subjective nature, which are deciding factors in establishing the threshold for consensus in advance.

We observed during the first round that the expert panel reached a consensus threshold of 52% for strong agreement, with the remaining 48% falling within moderate consensus. One of the key considerations in the Delphi process is establishing a priori criteria to determine when to terminate further iterations (Diamond et al., 2014). Although we did not initially set a strict predefined criterion for limiting the number of rounds, the results of the first round provided a meaningful level of agreement. While 52% of the criteria met strong consensus, the remaining 48% showed moderate consensus, indicating areas for refinement rather than the need for additional rounds to seek further convergence. Given that expert feedback did not present major disagreements or conflicting viewpoints, further rounds were deemed unnecessary, as they were unlikely to yield significant shifts in agreement. Additionally, research suggests that moderate consensus can still provide actionable insights for implementation (Keeney et al., 2010). Therefore, we determined that the primary objectives of the Delphi process had been met, allowing us to proceed with integrating these findings into the pilot study phase.

Data Collection and Analysis

In this study, standard deviation is applied to understand the level of dispersion and allows for measuring the level of consensus variation among the expert panel on various criteria to design problems framed on the PBL framework. Standard deviation is considered the most common approach of descriptive statistics and as a measure of dispersion (Holey, 2007; Von Der, 2012; Giannarou & Zervas, 2014). Mean rank was employed to calculate the standard deviation as a means of assessing the consensus among expert panels. While the literature demonstrates the use of mean, median, and mode as the most appropriate measures of central tendency in Delphi studies, the median is considered the most recommended due to its robustness against the outliers (Shang, 2023). Despite these recommendations, we opted for the mean rank to calculate standard deviation, which allowed for a more precise measurement of variation in expert rankings. Although the mean is typically suited for interval rather than ordinal data, it provided a clear, quantitative insight into the extent of agreement, with low standard deviations indicating strong consensus. Thus, while the median is the standard approach for Delphi studies, the mean and standard deviation offer a practical and interpretable way to assess agreement among experts and guide further refinements in the problem design.

Further, to calculate the mean rank, first, we applied a ranking approach to prioritize PBL features based on expert votes. Each expert was asked to evaluate a list of PBL criteria through a questionnaire used as features in crafting PBL problems, and their rankings were determined according to the number of votes each criterion received. The criterion with the highest number of votes was assigned rank 1, indicating it as the most preferred by the experts, followed by the criterion with fewer votes, which were assigned ranks in descending order (2, 3, 4, etc.). This voting and ranking system provided a clear structure for identifying the most valued criterion as perceived by the expert panel. This process gave us the mean rank for each criterion. To further quantify the level of consensus among the experts, we calculated the standard deviation of the ranks for each criterion. Low standard deviation values indicated strong agreement among the experts, while higher values suggested more variability in the rankings, highlighting areas where perspectives diverged. Table 3 displays standard deviations for each criterion, and Figure 7 shows the visual graphs of standard deviations for each criterion and consensus agreement.

Table 3

Standard Deviation for Each Criterion/Characteristic Based on Mean Ranking

Figure 7

Bar Chart Depicting the SD by Criterion Showing Distribution of Consensus Levels

In addition to analyzing mean ranking and standard deviation, we applied Interquartile Range (IQR) to further assess the dispersion of expert consensus on each PBL design criterion. While mean and standard deviation provide a general sense of agreement and variability, IQR is particularly useful in Delphi studies as it helps identify the spread of responses around the median, minimizing the influence of extreme values or outliers (Keeney et al., 2010). By applying IQR, we gained a more nuanced understanding of how experts' opinions varied across the 21 criteria, distinguishing between highly stable areas of agreement and those requiring further refinement.

Upon performing the Interquartile Range (IQR) analysis, we validated the consensus distribution across the 21 criteria used for refining problem scenarios in the PBL-supported makerspace (See Table 4). The key findings are:

1. Strong Consensus (IQR ≤ 0.5): 18 out of 21 criteria met a strong consensus, confirming their reliability and agreement among experts.

2. Moderate Consensus (IQR > 0.5): Only 3 criteria exhibited moderate consensus, indicating areas where further refinement and validation through pilot testing are necessary

Table 4

IQR Assessment–Supporting Table

The discrepancy between the SD (standard deviation) and IQR (interquartile range) occurs because SD and IQR measure consensus differently. While the SD accounts for the spread of all expert rankings, giving a general idea of agreement levels. IQR focuses on the middle 50% of the data, depicting the spread of the middle half of the distribution. Thus, SD showed that 10 criteria had moderate consensus, while IQR refined this by revealing that only 3 of those 10 had significant dispersion in expert ratings. The other 7 criteria had some variation but were still within acceptable agreement ranges when assessed using IQR.

Furthermore, previous research supports the use of IQR in Delphi studies as an alternative or complementary measure to standard deviation, particularly when sample sizes are small, and data distribution may not be perfectly normal (Keeney et al., 2010). Given our study's focus on refining problem scenarios in PBL-supported makerspaces, IQR provided a more reliable statistical measure to interpret consensus trends, ensuring that the derived criteria are both rigorous and practically applicable for future instructional design.

Results and Discussion

In this study, the Delphi technique was utilized to establish expert consensus on the design criteria for problem scenarios in a PBL-supported makerspace learning environment, grounded on the Tan-Oon-Seng (2003) PBL framework. A panel of four experts participated in the process by ranking and voting on 21 design criteria, after which mean values and standard deviations were calculated to assess consensus, and additionally, IQR was performed to assess the spread of responses for each PBL criterion.

The initial threshold for consensus was set at 75%, in line with existing literature, where consensus thresholds typically range between 50% and 97% depending on the context (Diamond et al., 2014; Foth et al., 2016). However, based on the first round of analysis, 52% of the criteria (11 out of 21) achieved complete agreement (SD = 0). These criteria include critical aspects such as real-world relevance, problem-solving, team skills, moderate difficulty, medium complexity, and integration of multiple knowledge domains. The remaining 48% of the criteria demonstrated moderate variation (SD ≤ 0.5), suggesting areas for potential refinement.

This refinement process, which combined ranking, standard deviation, and IQR, provided a structured approach to determine expert consensus levels. Applying IQR demonstrated that IQR was a valuable addition in validating and strengthening the consensus findings. IQR provided a more reliable statistical measure to interpret consensus trends, ensuring that the derived criteria are both rigorous and practically applicable for future instructional design.

Given the time constraints and the immediate need to implement the refined criteria in the pilot program for middle school, the decision was made not to proceed with further Delphi rounds. Instead, the 52% consensus achieved in the first round is considered acceptable for this phase, particularly as it surpasses the minimum thresholds reported in studies that have utilized the Delphi method in educational research. The feedback provided by the panel during the first round will be used to address the moderate consensus criteria, ensuring that the PBL scenarios are adequately structured and aligned with the intended learning outcomes.

Implications and Next Steps

The findings indicate a solid foundation for the pilot program, with over half of the design criteria achieving strong consensus and the remainder offering actionable insights for improvement. The next phase will involve a small-scale pilot implementation, integrating both expert-driven refinements and student feedback. Since the moderate consensus criteria require further validation, the pilot study will focus on evaluating these areas through direct student engagement, gathering qualitative and quantitative insights to assess their feasibility and effectiveness in real-world learning environments.

Following the pilot, a larger-scale study will be conducted, incorporating both a pre-test and post-test design to quantitatively measure learning gains and assess the impact of the refined problem scenarios on students' knowledge acquisition and problem-solving skills. By integrating expert consensus with student-centered evaluation, this two-phase approach ensures that the final PBL problem scenarios are both pedagogically robust and practically effective for makerspace learning environments.

Furthermore, findings also suggest bridging the gaps for STEM participation by ensuring that problem-based tasks are engaging, appropriately challenging, and interdisciplinary. The criteria that met strong consensus emphasize real-world relevance, inquiry-driven learning, and interdisciplinary knowledge application. These elements scaffold learning for students with varying abilities, ensuring that even average students can meaningfully engage with STEM problem-solving (Hmelo-Silver, 2004).

Additionally, research suggests that problem complexity must be structured appropriately to maintain cognitive challenge without giving an overwhelming feeling among learners with different skill levels (Jonassen, 2011).

By adopting a pragmatic approach that leverages the first-round findings and integrates student feedback in the second phase, the study ensures a balance between methodological rigor and practical implementation needs. This iterative refinement process aims to enhance the alignment of PBL design with both expert recommendations and student experiences, ultimately strengthening the learning outcomes in makerspace education.

Limitations

A potential limitation of this study is the variation in expert backgrounds and their experiences with different student populations. The diversity of the expert panel provided valuable insights but also introduced potential variations in ratings due to differences in teaching experiences and student populations. One expert, having worked extensively with gifted students, may have evaluated problem scenarios with a higher expectation for complexity and independent problem-solving. Conversely, the other experts, who primarily worked with students from socioeconomically disadvantaged backgrounds, may have prioritized accessibility, structured guidance, and differentiated instruction when assessing the criteria. These differences in perspective could have influenced the ratings and consensus thresholds, particularly for criteria related to problem difficulty and scaffolding. While the diversity of expertise enriched the findings, future studies should consider a more balanced expert panel to ensure that the consensus represents a broader spectrum of educational contexts.

References

1. Arafat, M. H., Budiyanto, C. W., Yuana, R. A., & Fenyvesi, K. (2024). Implementation of integrated STEM learning in educational robotics towards 21st century skills: A systematic review. International Journal of Education in Mathematics, Science and Technology, 12(5), 1127-1141.
2. Barrows, H. S., Tamblyn, R. M. (1980). Problem-Based Learning: An Approach to Medical Education. Springer, New York.
3. Bevan, B. (2017). The promise and the promises of making in science education. Studies in Science Education, 53(1), 75-103.
4. Boud, D. & Feletti, G. (1991). The Challenge of Problem-Based Learning. London. Kogan Page.
5. Butin, D. W. (2006). Introduction: Future Directions for Service Learning in Higher Education
6. Darmawansah, D., Hwang, G. J., Chen, M. R. A., & Liang, J. C. (2023). Trends and research foci of robotics-based STEM education: a systematic review from diverse angles based on the technology-based learning model. International Journal of STEM Education, 10(1), 12.
7. Diamond, I. R., Grant, R. C., Feldman, B. M., Pencharz, P. B., Ling, S. C., Moore, A. M., & Wales, P. W. (2014). Defining consensus: a systematic review recommends methodological criteria for reporting of Delphi studies. Journal of Clinical Epidemiology, 67(4), 401-409.
8. Drake, K. N., & Long, D. (2009). Rebecca’s in the dark: A comparative study of problem-based learning and direct instruction/experiential learning in two 4th-grade classrooms. Journal of Elementary Science Education, 21(1), 1–16.
9. Foth, T., Efstathiou, N., Vanderspank-Wright, B., Ufholz, L. A., Dütthorn, N., Zimansky, M., & Humphrey-Murto, S. (2016). The use of Delphi and Nominal Group Technique in nursing education: A review. International journal of nursing studies, 60, 112-120.
10. Giannarou, L., & Zervas, E. (2014). Using Delphi technique to build consensus in practice. International Journal of Business Science & Applied Management (IJBSAM), 9(2), 65-82.
11. Green, R. A. (2014). The Delphi technique in educational research. Sage Open, 4(2), 2158244014529773
12. Helmer, O. (1966). The use of the Delphi technique in problems of educational innovations.
13. Hiskens, I. A., Peng, H., & Fathy, H. K. (2011, July). Transportation electrification education for k-12 students. In 2011 IEEE Power and Energy Society General Meeting (pp. 1-5). IEEE.
14. Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: a response to Kirschner, Sweller, and Clark. Educational psychologist, 42(2), 99-107.
15. Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235-266.
16. Hou, S. (2009). Service-learning+ New MPH students= Challenges for the professor. International Journal of Teaching and Learning in Higher Education, 20(2), 292-297.
17. Honey, M., & Kanter, D. E. (2013). Design, make, play: Growing the next generation of science innovators. In Honey, M., & Kanter, D. E. (Eds.), Design, Make, Play: Growing the Next Generation of STEM Innovators (pp. 1–6). New York: Routledge.
18. Holey, E. A., Feeley, J. L., Dixon, J., & Whittaker, V. J. (2007). An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Medical Research Methodology, 7, 1-10.
19. Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: making sense of consensus. Practical assessment, research, and evaluation, 12(1).
20. Jonassen, D. H. (2011). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
21. Judd, R. (1972). Forecasting to consensus gathering: Delphi grows up to college needs. College & University Business, 53(1), 35- 38, 43.
22. Karagiorgi, Y., & Symeou, L. (2005). Translating constructivism into instructional design: Potential and limitations. Journal of Educational Technology & Society, 8(1), 17-27.
23. Keeney, S., McKenna, H. P., & Hasson, F. (2010). The Delphi technique in nursing and health research. John Wiley & Sons.
24. Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), 4.
25. Nugent, G., Barker, B., Grandgenett, N., & Adamchuk, V. (2009, October). The use of digital manipulatives in k-12: robotics, GPS/GIS and programming. In 2009 39th IEEE Frontiers in Education Conference (pp. 1-6). IEEE.
26. Palincsar,A. S. 1998. Social Constructivist Perspective on Teaching and Learning. Annual Review of Psychology, 49, 345-375.
27. Papert, S. (1993). The children's machine: Rethinking school in the age of the computer. New York.
28. Quinn, H., & Bell, P. (2013). How designing, making, and playing relate to the learning goals of K-12 science education. Design, Make, Play (pp. 17-33). Routledge.
29. Radcliffe, P. J., & Kumar, D. (2016). Is problem-based learning suitable for engineering? Australasian Journal of Engineering Education, 21(2), 81-88.
30. Schmidt, H.G. (1983). Problem-based learning. rationale and description. Medical Education, 17, 11-16.
31. Schlegel, R. J., Chu, S. L., Chen, K., Deuermeyer, E., Christy, A. G., & Quek, F. (2019). Making in the classroom: Longitudinal evidence of increases in self-efficacy and STEM possible selves over time. Computers & Education, 142, 103637.
32. Seng, T. O. (2000). Reflecting on innovating the academic architecture for the 21st century. Educational Developments, 1, 8-10.
33. Shang, Z. (2023). Use of Delphi in health sciences research: a narrative review. Medicine, 102(7), e32829.
34. Tan, M. (2019). When makerspaces meet school: Negotiating tensions between instruction and construction. Journal of Science Education and Technology, 28(2), 75-89.
35. Tan, O. S., & Tee, M. Y. (2003). Problem-Based Learning. Using Problems to Power Learning in the 21st Century. Cengage Learning.
36. Tan, O.S. (2000c). Reflecting on innovating the academic architecture for the 21st century. Educational Developments, 1, 8–11.
37. Von Der Gracht, H. A. (2012). Consensus measurement in Delphi studies: review and implications for future quality assurance. Technological forecasting and social change, 79(8), 1525-1536.
38. Vygotsky, L. S., & Cole, M. (1978). Mind in society: Development of higher psychological processes. Harvard University Press.
39. Zhang, D., Wang, J., Jing, Y., & Shen, A. (2024). The impact of robotics on STEM education: Facilitating cognitive and interdisciplinary advancements. Applied and Computational Engineering, 69, 7-12.

Acknowledgement

The author would like to express sincere gratitude to Dr. Pi-Sui Hsu, dissertation advisor, for her continuous mentorship, guidance, and constructive feedback throughout the development of this study. Her support was instrumental in refining the design and development of this study.

Appendix

Table 2

Summary of Problem Task Broken Down into Activities Day Wise

.