Introduction
Training medical military personnel is critical to saving lives in emergencies such as a CCP, where rapid and effective care can mean the difference between survival and loss. To support this mission, a virtual reality (VR) experience has been created to provide soldiers with additional means of training (Craig et al., 2024). This training uses the Generalized Intelligent Framework for Tutoring (GIFT), an open-source software platform that enables dynamic training. Designed to enhance the training of combat medics, our team-based training application incorporates tailored instruction and real-time feedback to the learner while providing instructors with metrics on learner performance. However, the creation of such learning materials borrows elements from other forms of pedagogy, and lacks its own distinct identification.
The team-based immersive VR system (Craig et al., 2024). we are developing in GIFT includes an emphasis on short, focused, and repetitive learning exercises. Aspects of gamification, educational burst gaming, gameLets, extended reality, and microlearning aspects are all included within this combined VR and GIFT framework. This review aims to identify the best practices for the components for learning within immersive VR and intelligent tutoring, and propose the creation of a checklist to assess this learning system during development. To better guide development of these areas, we conducted a Learning Strategy Analysis (LSA) (Roscoe & Craig, 2022) to examine similar pedagogical frameworks used in gamification, VR, and shorter lessons that could be applied to guide the development of the team training exercise in GIFT. The goal of LSA is to identify the best practices within learning strategies used to support an educational system and provide a heuristic checklist to guide the system during development (Roscoe & Craig, 2022; Ville et al., 2021).
Short-form interactive learning environments leverage brief, interactive, and often game-inspired activities to support rapid knowledge, skill acquisition, and engagement across digital training platforms (Craig et al., 2022). Gamification introduces game-like mechanics to learning materials that promote motivation and enhance learning, with serious games having long demonstrated effectiveness across domains (Estellés-Arolas & González-Ladrón-De-Guevara, 2012; Kasurinen & Knutas, 2018). Educational Burst Gaming (EBG) and GameLets extend this approach through fast-paced, repetitive tasks designed to build proficiency (Amresh et al., 2024; Baron et al., 2016), incorporating structured role-based interactions similar to thinkLets (Kolfschoten et al., 2005). Extended reality further enhances the impact of these short-form learning modalities by providing scalable, low-risk, and repeatable training opportunities that may be difficult to provide with traditional in-person exercises (Allen et al., 2016; Djukic et al., 2015; Fuentes, 2018; Siegle et al., 2020). Finally, microlearning principles reinforce these structures through targeted units of instruction and feedback (Hug et al., 2006), though challenges remain in sustaining meta-cognitive reflection and higher-order thinking (Jahnke et al., 2019).
Emphasizing the need to balance game design and educational value (Baron, 2017), elements of narration, fidelity, and even the means of skill acquisition all impact learning potential. Lester and colleagues (2013) argue that narrative should be woven into gameplay, even suggesting integrating intelligent tutoring systems (ITS) into a game. ITSs have been known to provide the learner with adaptive content (Graesser et al., 2019). An ITS could develop the story throughout each level without interruption, progressing the narration as the learner reaches key points to feed intrinsic motivation (Lester et al., 2013). Narrative could be further improved by having a virtual human present the narration, which has been shown to increase learning outcomes (Schroeder & Craig, 2021; Siegle et al., 2023). This guidance suggests we could use virtual humans to provide instruction and guidance to trainees in our course, issuing the trainees their standing orders and providing a narrative framework and social component that can guide them through the virtual training environment (Siegle & Craig, 2024).
Simulation fidelity refers to the degree to which a training environment replicates aspects of the real-world system or task it is intended to represent. High-fidelity environments allow for a more realistic experience, fostering deeper learning, motivation, and team performance (Yuzhakov, 2023). Aiming to provide the most realistic virtual experience possible. Within team-based immersive VR systems, fidelity extends beyond environmental realism to include the behavior of non-player characters, teammates, and instructional agents. These elements influence trust, credibility, and social presence, which in turn affect engagement and learning outcomes (Schroeder et al., 2023; Siegle & Craig, 2024).
Gamification and EBG work best when the core gameplay remains the same from level to level, allowing the user to focus on the increased difficulty or new challenge being tested rather than relearning their role or controls with the game (Baron, 2017). Keeping the gameplay repetitive enough to test the skillset multiple times through play, spaced-out repetition is recommended over continuous repetition of the same skill tested (Amresh et al., 2024). The current GIFT experience aims to achieve this, adhering to principles of learner analytics as users engage with the learning experience (Nguyen et al., 2021).
LSA is a heuristic evaluation method for learning technology that combines a review of the system to identify learning supports/scaffolds built into the system, a literature review to identify best practices, and heuristic evaluation to determine overlap and potential improvements. This process has four steps: 1) Identify the system’s learning strategies; 2) Literature review on each strategy for effectiveness; 3) Comparison between best practice and system; and 4) Providing ratings and recommendations. The current paper provides an overview of Steps 1 and 2. For Step 1, the relevant project and GIFT documentation were reviewed, leading to the identification of seven categories of learning strategies. A targeted literature review was conducted for each category to identify best practices for implementation. A heuristic checklist was created based on the review.
The outcome of this review is to understand and apply best practices. Using them to create a heuristic checklist to guide the development of the system. After considering the GIFT-based course experience and reviewing the extant scientific literature we identified seven broad heuristic categories and twenty criteria that should be evaluated across them. Using four different ratings for each criterion, which have been adapted from past heuristic checklists regarding learning technologies (Craig & Schroeder, 2023; Li et al., 2021). Summaries of the heuristics are provided below, with a full list of the rubric located here.
Micro-Learning.
The micro-learning heuristic emphasizes brief, focused instructional units that support efficient skill acquisition without overloading learners. Structured, task-oriented content and repeated exposure foster long-term retention through improved motivation (Nikou & Economides, 2018). Effective micro-learning requires lessons that are engaging, practically relevant, and appropriately scoped to the task at hand (Hug et al., 2006). This approach aligns with cognitive learning principles by reducing overload and maintaining attention by making lessons engaging, practical, and bite-sized (Jahnke et al., 2020).
The instructional flow heuristic refers to the organization and sequencing of learning activities within the immersive environment. Strong implementations provide a clear progression that includes contextual introductions, explicit objectives, opportunities for guided practice, and timely feedback to reinforce performance (Jahnke et al., 2020). In team-based VR scenarios instructional structure should support smooth transitions between phases of action without disrupting immersion, providing structured sequencing through careful guidance (Kirschner et al., 2010). Poorly scaffolded designs may present tasks without sufficient framing or closure, leaving learners uncertain about goals or expectations.
The Game Environment Design heuristic addresses how interface elements and system feedback support learning within the immersive experience. Effective environments include well-timed screen interventions, mission completion indicators, and synchronized task stages (Septiani & Rosmansyah, 2021). These features help learners interpret progress without breaking engagement (de Souza e Silva & Delacruz, 2006). Poor design may overwhelm users or fail to provide meaningful feedback.
The Roles heuristic evaluates how clearly and meaningfully player and non-player roles are defined within the system (de Souza e Silva & Delacruz, 2006). Strong implementation provides well-specified player responsibilities, consistent non-player scripted roles, and an appropriate number of distinct roles to support teamwork (Bots & van Daalen, 2005). Inadequate implementations may blur role boundaries or underutilize non-player characters, reducing opportunities for collaboration and coordinated decision-making.
Plot and Problem Situations.
The plot and problem situations heuristic focuses on the structure and realism of the narrative problem space. Effective implementation presents focused, coherent problem situations that reinforce training objectives and reflect realistic operational constraints (Marocco et al., 2015). Systems benefit when narrative is well integrated with the learning context, and seamlessly with task demands, while not linking plot to situational task may include underdeveloped scenarios that fail to meaningfully support learning or engagement (Bots & van Daalen, 2005).
The Assessment heuristic addresses how learner performance is evaluated and communicated during training. Strong implementations provide real-time feedback aligned with task performance and incorporate structured assessment opportunities (Septiani & Rosmansyah, 2021). Utilizing built in tools to assess the players navigation of the game and measure their skill level (Bots & van Daalen, 2005). Lower-quality implementations may rely on delayed or minimal feedback, limiting learners' ability to adjust strategies during training.
The learner needs heuristic considers whether the system accommodates users’ accessibility, social, and knowledge requirements. High-quality implementations support diverse learners through accessible interfaces, meaningful social interaction, and alignment with learners’ prior knowledge (Jahnke et al., 2020). Examining the individual needs of the learner, providing the necessary scaffolding and feedback to direct their experience (Roscoe & Craig, 2022). In contrast, weaker implementations may neglect accessibility considerations or fail to support team communication, undermining learning.
Team training for medical personnel is critical to improve skill acquisition, speed, and team cooperation. The VR experience in GIFTaims to be both a means of supplemental training and a way to assess complex team dynamics. In order to achieve these goals, it is imperative that the learning materials created be up to the highest standards and best practices. However, training these highly specialized teams means strict adherence to conventional pedagogy approaches may not be the best choice for material creation. The longer scenarios, complex and changing tasks, and serious nature of the scenarios conflict with microlearning, electronic burst gaming, and gamification elements respectively. These conflicts are only partial, meaning that care must be given when applying best practices to the current gift experience. The proposed checklist focuses on broader assessment items. Enabling for the foundation of a new assessment measure for this new wave of potential learning opportunities.
The research described herein has been sponsored by the U.S. Army Combat Capabilities Development Command under cooperative agreement W912CG-23-2-000. The statements and opinions expressed in this article do not necessarily reflect the position or the policy of the United States Government, and no official endorsement should be inferred.
EdTech Archives