EdTech Archives
International Consortium for Innovation and Collaboration in Learning Engineering (ICICLE) 2024 Conference Proceedings: Solving for Complexity at Scale
EdTech Archives
International Consortium for Innovation and Collaboration in Learning Engineering (ICICLE) 2024 Conference Proceedings: Solving for Complexity at Scale
For the ICICLE 2023 conference, the pK-12 Market Interest Group developed a website that contained scenarios, resources, and information on the application of the learning engineering process within the pK-12 educational context. The learning engineering process is a process and process which uses human-centered design, learning sciences, engineering principles, and data to support learners and their learning (Goodell & Kolodner, 2022). The pK-12 MIG prepared materials on how to use and adapt the process for school environments.
The pK-12 MIG had the opportunity to present at the ISTELive 2024 conference on learning engineering. To prepare for the conference, the pK-12 MIG modified and updated the website. Leveraging the insights gained from the ICICLE 2023 attendees and by following the Learning Engineering Evidence and Decision (LEED) structured approach (Totino & Kessler, 2024), the team meticulously documented each decision in systematically redesigning the website for ISTELive 2024. At the ICICLE 2024 conference show and share session, "Using the Learning Engineering Evidence and Decision (LEED) tracker in a pK-12 context: A Metacognitive Journey," the pK-12 MIG presented a case study on how the LEED tracker was applied to redesign scenarios, resources, and information on the website. This process enabled the pK-12 MIG to enhance the website with differentiated levels and develop learning scenarios tailored to reflect diverse contexts and perspectives suitable for pK-12 administrators, teachers, and learners.
The LEED Tracker was developed to support learning engineers and designers by serving as a tool to capture the design decision-making process and to organize the evidence of what may have influenced those decisions. This tool promoted transparency and reflection by enabling designers to document the rationale behind key design choices, the sources of evidence used (e.g., empirical data, user feedback), and whether decisions led to iteration. The benefits of using the LEED tracker for any effort to redesign learning and training programs include concretizing decisions grounded in understanding of design approaches and context, facilitating iteration and improvement based on data, and serving as a communication tool among stakeholders (Totino & Kessler, 2024).
The LEED Tracker was employed to redesign the activities presented during the ISTELive 2024 conference. Design decisions were grounded in various factors such as design principles from instructional design and learning sciences as well as feedback from stakeholders and observations by the research team. Each decision was carefully tracked and revisited as the team iterated the design. This method allowed for a comprehensive exploration of how technology (Google Sites) and contextual factors (presentations and training in the pK-12 settings) can be used to shape the learning environment.
The iterative nature of the LEED Tracker facilitated continuous improvement. Iteration was a key component in the learning engineering process. The process was intended to continually and purposefully address what was happening within learning experiences and environments because it is derived from results, impact, feedback, observations, and any other data collected throughout the learning (Kessler et al., 2022). In this case study, each design iteration was informed by data and observations from the ICICLE 2023 conference active learning session. This iterative process resulted in a refined scenario and learning activities that were better aligned with the learning needs of the pK-12 audience at the ISTELive 2024 conference.
The affordances and constraints framework was used to understand how the properties of a tool or technology could either support (afford) or inhibit (constrain) specific actions and decisions. Affordances refer to what a tool or technology enables a user to do, while constraints represent the limitations or restrictions inherent to the tool or its context of use (Gibson, 1966; Norman, 1988). When applied to learning engineering, this framework helped to identify the alignment between technology capabilities, learning goals, and user needs.
The LEED Tracker, in combination with the affordances and constraints framework, was used to evaluate the redesign of the technology, Google Sites, as a platform for designing learning activities for pK12 educators. The research team identified the following affordances of Google Sites:
Ease of use: Google Sites offered a user-friendly interface that allowed for rapid prototyping and iteration of the learning activities.
Collaboration: The platform supported collaborative content creation, enabling multiple stakeholders to contribute to the design process.
Integration: Google Sites integrated easily with other tools (e.g., Google Forms and Docs) to facilitate data collection and analysis.
However, these constraints were also noted:
Limited customization: The rigidity of Google Sites in terms of design elements restricted the team’s ability to create highly customized interactive features.
Scalability issues: Some features did not scale well when attempting to integrate more complex interactive elements, limiting the depth of the learning experience.
These affordances and constraints were documented using the LEED Tracker, and each design decision was examined in light of these factors. For example, while the affordance of ease of use supported rapid iteration, the constraint of limited customization led the team to make design decisions based on what was possible within the technology.
The LEED Tracker allowed for metacognitive reflection by making the design team’s thought and experience processes explicit. Metacognitive reflection has two main components: deliberately considering “what we know” and “how we know,” along with the ability to self-regulate one's cognitive processes (Miller et al., 2021). Throughout the process, the team critically evaluated the use of LE processes and tools and considered the integration of additional LE methodologies for different scenarios. By systematically recording each decision and its associated evidence, the LEED Tracker served as a repository for shared understanding. This transparency was particularly valuable in a multi-stakeholder context, where decisions needed to be communicated and justified to a diverse audience, including educators, instructional designers, and conference presenters.
The use of the LEED Tracker, combined with the affordances and constraints framework, enabled the pK-12 MIG group to adopt a systematic, evidence-based approach to learning engineering in a pK-12 context. The LEED Tracker provided a structured means to capture design decisions, while the affordances and constraints framework allowed for a nuanced understanding of how technology and context influence learning design. The presentation showcased the MIG’s journey and outcomes of employing a reflective LEED tracker. It also explored the potential for using LE tools to enhance educational experiences.