Immersive Virtual Reality (IVR) is emerging as a powerful tool for language education. Johnson-Glenberg (2018) defines IVR as an experience that fosters a deep sense of presence through a headset, allowing learners to feel as if they are in a virtual environment that closely mimics reality. The immersive experience is further enhanced through controllers that allow the user to mimic the actions taken in the environment. This sense of presence can directly support critical aspects of successful language learning, including immersion, participation, interaction, and authenticity (Lan, 2020).
There is growing evidence that learners benefit from instruction that combines movement with language learning, particularly in IVR environments (Fuhrman et al., 2021; Lan, 2020). Studies have shown improvement in comprehension, vocabulary, and writing skills through VR-based language instruction (Chen et al., 2020; Chien et al., 2020; Hakim et al., 2022). Other studies highlight IVR’s effectiveness in enhancing motivation and engagement (Chen & Yuan, 2023; Kavanagh et al, 2017), which is important for successful language acquisition. A primary reason for these benefits is that IVR provides a platform for real-time, authentic language use and feedback in a controlled yet dynamic setting that fosters the development of communicative competencies (Ou Yang, 2020).
While increasing research suggests that IVR can improve language learning outcomes, its full potential in K-12 education has yet to be realized (Peixoto et al., 2021). To begin, much of the current research has been experimental, assessing IVR’s effectiveness in isolation from and in comparison to traditional learning methods (e.g., Ahmet & Cavas, 2020; Alfadil, 2020; Wu & Hung, 2022). Furthermore, there is often a lack of detailed instructional design in these studies which makes it challenging to establish the efficacy of the employed instructional strategies. The majority of studies instead employ researcher-developed VR environments, focusing on how elements like games (e.g. Alemi & Khatoony, 2020) and virtual simulations of real-life settings (e.g., Chen & Liao, 2022; Kallioniemi et al., 2015) can improve specific content outcomes. Less explored are hybrid approaches that blend IVR with conventional language learning techniques (Huang et al., 2021; Lan, 2020). Therefore, there is a current need for research that explores practical approaches to language acquisition by integrating off-the-shelf IVR tools with existing pedagogical approaches to enhance learning and engagement.
This case study aims to bridge existing gaps by evaluating the way that IVR can be blended with face-to-face (F2F) Mandarin language instruction within a K-12 context. We position this study as a proof of concept in that our focus was on understanding if and how this instructional approach could support language learning. The research questions guiding this study were:
By exploring these questions, this study provides practical, evidence-based strategies for integrating off-the-shelf IVR tools into K-12 foreign language classrooms. Blending IVR with more traditional F2F approaches offers a practical solution to integrating IVR into language learning that can augment rather than supplant current practices. In IVR, a learner can practice using language in authentic contexts while moving their bodies as they learn to comprehend a new language (Lan, 2020); this is important because language acquisition is enhanced when paired with the sensorimotor system (Glenberg & Gallese, 2012). Likewise, F2F offers learners an opportunity to reinforce pronunciation and receive clear instruction on their language usage (Dressler & Mueller, 2022). Blending IVR and F2F methods has the potential to capitalize on the strengths of both methods to offer a robust language learning experience. This can improve the likelihood that teachers will view IVR as a viable tool for learning in the K-12 classroom (Zhao & Frank, 2003) while addressing the current need for research on blended approaches in IVR learning that are currently absent in the literature.
One reason IVR holds significant potential for enhancing language learning in K-12 education is that it uniquely blends language comprehension with sensorimotor engagement. Off-the-shelf applications such as Language Lab and Noun Town offer a virtual environment where users actively listen to and perform tasks in a foreign language. This active involvement in the IVR setting mirrors the principles of Total Physical Response (TPR), a recognized method in language education. TPR integrates bodily movement with language learning, advocating that physical responses to verbal commands can significantly boost understanding and recall (Asher, 1977). In IVR, this approach is taken a step further. Learners do not just react to language; they immerse themselves in a virtual context where their actions are directly linked to linguistic tasks. This immersive experience can facilitate deeper assimilation of new words and structures, making language learning both interactive and potentially more memorable (Lan, 2020). Thus, the action-based nature of IVR offers a sensorimotor experience that can enhance language acquisition.
However, improving foreign language learning with IVR depends on addressing key gaps in the research base. Current research needs to go beyond a single use of VR to understand more fully the impact on language learning outcomes (Kavanagh et al., 2017). This means involving a wider variety of learners to ensure the findings are widely applicable and reflective of today’s K-12 learning contexts (Alfadil, 2020; Uygun & Girgin, 2022). It is also important to figure out how IVR can be effectively integrated into traditional instruction from both a pedagogical and technical standpoint in K-12 settings (Parmaxi et al., 2021). This study aims to address these gaps and bring greater insight into the ways IVR can support the link between IVR and body-based approaches to language acquisition.
The participants were six 7th-grade students attending a private STEAM school in the US South. Of those, four were video recorded for later analysis. The exclusion of the remaining two students from the video analysis was due to their absence on the days when these recordings were made, with each missing one day of video data. None of the participants had previous experience with speaking Mandarin or with using the Language Lab VR app. These students were drawn from a larger class of 11 students; the class was a social studies class that regularly explored foreign cultures. The research activity in this study was approved by our University’s Internal Review Board.
The lead researcher created a 5-day instructional unit (60 minutes per day) to teach Mandarin; this researcher had expertise and experience in speaking and teaching Mandarin as a foreign language. Table 1 displays the events that took place on each day. For example, Day 1 was non-instructional and focused on the pretest as well as training students to get into and use the IVR environment. Day 5 similarly focused on collecting posttests and conducting interviews with participants. On Days 2 through 4, each session focused on learning Mandarin.
Table 1
Sequence of Instruction Blending IVR with F2F Activities
| Day | Objective | IVR | F2F |
|---|---|---|---|
| 1 | Pretest and IVR orientation | Students used IVR to select a scenario and practiced hearing Chinese while interacting with objects. | Students completed the pretest and then had a teacher-led training session on IVR. |
| 2 | Vocab: Spoon, Fork, Mug, Plate, Bowl | Students completed Lesson 1 in the IVR app; the lesson covered the vocabulary. | Students sat as a whole group and completed teacher-led instruction that included a warm-up and flashcards. They then completed a practice activity (Simon Says) and a peer activity to play a vocabulary game similar to Hot Potato. |
| 3 | Vocab: Pear, Apple, Tangerine, Banana | Students completed Lesson 1 as a review and then the first part of Lesson 2 on the new vocabulary. | Students (whole group) completed teacher-led instruction that included a warm-up and flashcards. They then completed a Hi-Low practice activity and a peer activity where they played a vocabulary game that mimicked Tic-Tac-Toe. |
| 4 | Vocab: Pizza, Hamburger, Carrot, Sandwich, Egg | Students completed the first part of Lesson 2 as a review and then the second part on the new vocabulary. | Students (whole group) completed teacher-led instruction that included a warm-up and flashcards. They then completed the Finger Stretch practice activity and played a competitive vocabulary game that entailed both listening to and speaking Mandarin. |
| 5 | Posttest and final data collection | None. | Students completed the posttest and interview. |
In the first 15 minutes of Days 2 through 4, the teacher gave an introduction to the target vocabulary for the day and reviewed any vocabulary from the previous day. For the next 20-25 minutes, students used the Language Lab VR app to first repeat the previous day’s lesson and then engage with the vocabulary in the new lesson. Language Lab is an IVR environment in which the user is placed in a simulated but familiar environment (e.g., kitchen, living room). In each lesson, the app gives a command in the target language that learners must complete. For example, the Mandarin command “把苹果放在碗里 (bǎ píng guǒ fàng zài wǎn lǐ)” requires students to put an apple in a bowl. To learn the language, students must pick up each object in the room and listen; the app would then say the Mandarin word for each object that was picked up. The goal was for students to match the name of the object with the words being spoken in the command, which meant that they had to recognize the Mandarin word for different objects while also distinguishing those words from other words that were not part of the vocabulary for the day. In this way, students learned to recognize which objects (e.g., an apple) matched the Mandarin word being spoken (e.g., 苹果, píng guǒ). They would then perform the task as directed. This aligned with the principles of TPR in that it challenged the learner to physically interact with and perform actions on the objects while learning the language associated with those objects. Figure 1 displays an apple being placed in a bowl in the Language Lab.
Figure 1
Placing an Apple in the Bowl in the Language Lab App
Note: In the app, picking up an object like an apple results in the app saying the Mandarin word for the object.
After learning in IVR, the students spent the remaining time in teacher-led, F2F instruction. The F2F portion was scaffolded to move from instructor-led to student-led activity. For the instructor-led portion, students gathered as a whole group while the teacher led them through a quick warm-up and then used flashcards to practice the vocabulary for the day. When using the flashcards, the teacher first showed an image of an object on one side and asked students to call out the Mandarin name. The teacher then flipped the card to show the Mandarin characters with phonetic pronunciation on the other side; students again practiced saying each word. This sequence also allowed the students an opportunity to hear and see the instructor pronounce each word as well as practice speaking key vocabulary with immediate feedback from the instructor. This type of modelling from an expert is an essential part of learning new vocabulary in a second language (Dressler & Mueller, 2022). Figure 2 displays the flashcards portion of the F2F instruction.
The F2F instruction then became more student-led. While the type of activity varied, the goal of each was for students to listen to the teacher accurately pronounce each word before trying to pronounce it on their own. Research suggests hearing a word before trying to speak can help alleviate anxiety and make language learners more comfortable when attempting to speak in front of others (Richards & Rodgers, 2001). For example, one practice consisted of a Hi-Low activity where the teacher said the Mandarin name of an object with a high or low level of voice, and the students needed to call out the same word in the opposite level of voice. The teacher then invited two students to lead the same activity. These practice activities then prepared students for a peer activity in which participants played some sort of practice game with peers in small groups. We used games because they give learners an opportunity to repeatedly engage with the new vocabulary with peers, which can make learning more enjoyable (Klimova & Kasset, 2017).
Figure 2
The Flashcards Portion of the F2F Instruction
Note: The flashcards, which appear on the whiteboard (see the red circle), show an image of an object on one side and the Mandarin pronunciations and characters for that object on the other.
For the pretest, each student reviewed an image bank that contained 15 thumbnail images of the objects (e.g., apple, banana, spoon) that would be introduced in the IVR environment. The students were asked to identify any images for which they knew the Mandarin word. They were also given two words spoken in Mandarin and asked to identify the images that those words represented.
The posttest consisted of two types of questions for a maximum score of 10. The first type of question mirrored the pretest in that participants were given four words (one point each) spoken in Mandarin and asked to identify the images from the image bank that represented them. For the second question, students played a video of a speaker giving three commands in Mandarin that were drawn from the IVR environment. They were then asked to translate those commands into English. These items were worth two points each, one point for each of the two vocabulary words spoken in each command. The test was administered using video technology so that they could hear the questions in Mandarin and record themselves speaking in Mandarin.
Structured interviews were conducted after the instruction was concluded. The structure was established through a six-question protocol that sought students’ feedback on their enjoyment and criticisms of the unit, their assessment of IVR’s effectiveness in learning Mandarin, challenges faced and their solutions, interest in other IVR-learnable subjects, and any additional insights not covered by the survey or previous conversations. This approach aimed to gather comprehensive insights into the educational potential of blending IVR and F2F instructions.
One researcher from the team created a coding scheme that was used for the video data analysis; this researcher spoke both English and Mandarin and had a background in foreign language learning. The goal of the scheme was to bring uniformity to the process of reviewing the video data. The scheme was developed to help determine how student experiences in IVR were similar to or different from the F2F portion of the instruction. To do that, the scheme first characterized the learning strategies that the students employed in IVR for each of the commands spoken in each lesson. It also tracked the number of attempts each learner made and the time spent completing each command. In this way, the scheme captured the way learning took place in IVR and the time spent on that learning. The scheme was then applied to the F2F video data to evaluate similar features (i.e., opportunities to use Mandarin and time spent on each portion).
Descriptive statistics were calculated on the pre and post-test items to observe language gains over time. Interview data were analyzed for themes using Maguire and Delahunt (2017) as a guide. Specifically, two members of the research team independently read through the interview transcripts to identify responses that were similar across participants or that were unique to a specific experience. The researchers then met to generate broad themes that described those similarities and unique features.
IVR Video Footage
IVR footage was recorded by the participants using the recording feature in the headset and then transferred from the VR headset by the researchers. The data were analyzed for four participants using our coding scheme (see Instruments). We specifically focused on footage taken on Days 3 and 4 of the instructional unit because these days integrated vocabulary from the previous day with new vocabulary, offering the most robust perspective of the IVR learning experience. In total, we analyzed 1 hour and 50 minutes of footage in IVR; this represented the portion of instruction in which our participants engaged with new learning. We also analyzed 26 minutes of F2F instruction. The responsibility of analyzing the video was divided among members of our research team.
To analyze the IVR footage, the Mandarin-speaking members of the team first identified the commands that were given in each of the lessons. The footage associated with each command was viewed twice. In the first viewing, the researcher coded the segment for the overall strategy that was employed to complete the command. This took one of four forms. The first, called 1st try, was when the participant successfully completed the task on the first try. The second, called Listening, was when the participant took the time to listen to the command and then explored the names of each object until they matched the object name with the words in the command. The third, called Random, was when a participant completed the task by randomly combining one object with another until the correct combination was found; this was in contrast to Listening to the names of each object and associating them with the given command. The fourth, called Hint, was when a participant used the hint feature in the app, which provided them with the English version of the Mandarin command. In several cases, participants blended Listening and Random, meaning the participant recognized the name of one of the two objects in the command and randomly tried combining it with other objects until the task was completed.
Once the first viewing was completed, the researchers watched the segment a second time to track the start/stop times for each command, as well as confirm the type of strategy used. Inter-rater reliability was established by having another team member who has not watched the segments review the findings to confirm the accuracy and resolve any differences in coding. The data collected on each participant was then converted into descriptive statistics. Frequency counts were made on the number of times each strategy was applied. Regarding time in IVR, the results were calculated as a percentage that compared the amount of time employing each strategy (e.g., listening, random, hint) to their total amount of time in IVR. This allowed us to compare the amount of time per strategy across participants in a way that addressed the fact that each participant spent a different total amount of time in IVR each day.
F2F Video Footage
The F2F videos were recorded by a camera positioned in the corner of the classroom. Analysis of the F2F sessions for Days 3 and 4 was conducted using a coding scheme based on the scheme developed for IVR video footage; the goal was to systematically quantify the different types of interactions and responses observed in the video footage (see Instruments). The interactions were broken down into the activity types in the F2F instruction, which included: warm-ups, flashcards, practice, peer activities, and wrap-ups. Within each activity type, the researcher counted the opportunities where participants engaged in listening and speaking Mandarin, as well as physical responses such as pointing to images after hearing the words and responding verbally to flashcards. Special attention was given to repeating words or phrases that the students found more challenging. The total footage analyzed amounted to 27 minutes, with one team member responsible for this task.
Similar to the IVR video footage coding, each segment of the F2F video was reviewed twice by a researcher, which promoted attention to in-depth details and accurate frequency counts. These included the duration of each activity and the frequency of opportunities to use Mandarin within each type of activity. The gathered data for each activity were then quantified into frequency counts and descriptive statistics so we could gain an understanding of students’ language use and engagement within each type of activity.
With regard to the effect of IVR blended with traditional teaching on our participants’ ability to comprehend and speak Mandarin (RQ1), the pretest scores for the seven participants were 0. In other words, none of the participants were able to say the Mandarin word for an object or identify the object after hearing the Mandarin word prior to instruction. The mean posttest score was 7.86 (out of 10) with a standard deviation of 2.27. This marked increase in post-test scores demonstrates a positive change in the participants’ proficiency in Mandarin vocabulary and recognition as a result of the blended instructional approach.
The instructional elements did vary between instruction delivered in the IVR and the traditional approach to learning Mandarin (RQ2). One way was the amount of time the participants spent on different tasks. In IVR, the participants completed an average of 22 (max 29) commands over 391 seconds (6.51 min) on Day 3 and an average of 32 (max 39) commands over 612 seconds (10.20 min) on Day 4. Overall, the participants most frequently accomplished their task on the 1st Try on both Day 3 (M = 12.50) and Day 4 (M = 19.50); this represented 37% (2.41 min) and 62% (6.32 min) of their time in IVR, respectively. On Day 3, the next most frequent strategies were Listening and Hints (M = 3.50 for both), representing 31% (2.02 min) and 11% (0.71 min) of the time in IVR. On Day 4, Listening and Hints (M = 4.75 and 5.25) were also the next most frequent, representing 15% (1.52 min) and 17% (1.73 min) of the time in IVR. Table 2 displays the average frequency and amount of time spent on each strategy by day.
Table 2
Time and Frequency of each Strategy Type in IVR by Day
| Day | 1st Try | Listening | Random | Hint | Listening & Random | Totals |
| Day 3 | ||||||
| Time | 37% | 31% | 20% | 11% | 0% | 6m 31s |
| Freq. | 12.50 | 3.50 | 2.50 | 3.50 | 0.00 | 22 |
| Day 4 | ||||||
| Time | 62% | 15% | 8% | 17% | 5% | 10m 12s |
| Freq. | 19.50 | 4.75 | 1.75 | 5.25 | 1.00 | 32 |
In the F2F instruction, participants spent an average of 12 minutes and 14.25 minutes on Days 3 and 4, respectively. On Day 3, the most frequent opportunities for using Mandarin took place during peer activity (57) and practice (18), representing 52% (6.24 min) and 36% (4.32 min) of the time in F2F instruction. These times were longer than the warm-up (4% or 30 sec) and flashcards (9% or ~1 min), which were more teacher-led. On Day 4, the total number of opportunities to use Mandarin increased to 107 from 96 on Day 3. The most frequent opportunities for using Mandarin took place during peer activity (22) and practice (38), representing 48% (6.84 min) and 28% (3.99 min). Table 3 displays the percent of time and frequency of opportunity to use Mandarin in F2F instruction by day.
Table 3
Time and Frequency of Opportunities to Use Mandarin in F2F Instruction by Day
| Day | Warm-up | Flashcards | Practice | Peer Activity | Wrap-up | Totals |
| Day 3 | ||||||
| Time | 4% | 9% | 36% | 52% | 0% | 12m |
| Freq. | 5 | 16 | 18 | 57 | 0 | 96 |
| Day 4 | ||||||
| Time(%) | 8% | 4% | 28% | 48% | 12% | 14m14s |
| Freq. | 5 | 20 | 38 | 22 | 22 | 107 |
Regarding the aspects of the experience that the participants attributed to their learning (RQ3), several themes emerged from the interview data. The first was that IVR was fun and interactive, allowing them to be creative in learning a new language. Three specifically noted how the IVR environment offered an activity that supported in-class learning but in a way that was active and allowed for exploration, making it more interesting than sitting and listening in the classroom. One stated, “I think that it is easier for kids to learn new languages through virtual reality because it makes it fun and like, not boring [like in the classroom] when learning a new language.”
The participants also found the immersive nature of IVR overwhelming initially. Five specifically noted how it was difficult to learn the new words because they were being spoken quickly and they didn’t recognize all the words in the command. Three of those five further noted how it was difficult and overwhelming at first to distinguish the key vocabulary from other words being spoken in the IVR app. One described,
The [commands] had extra words in Chinese that they didn’t teach us using the items. I had to grab an item and wait for the sentence to figure out what the words were. It was confusing that I would hear stuff that wasn’t part of [the instruction].
Three noted how it helped them to learn the Mandarin names of each object before interacting with those objects in IVR. One noted how “it was all about learning the names, really,” meaning that his experience in IVR became more manageable as he learned the vocabulary. Another described how she had to slow down and listen to each Mandarin word in order to recognize it in the commands in IVR. She stated, “It was very hard learning Mandarin and some challenges that I faced was not knowing what it said all the time ... I overcame that by picking up the items and learning the words of those items.”
The improvement from the pretest (0%) to the posttest (78.6%) suggests that students’ understanding of Mandarin improved as a result of learning Mandarin through a combination of IVR and F2F instruction. This positive shift is likely attributable to the participants benefitting from the dual approach to language learning. For example, the IVR environment we used is in accordance with TPR theory, which advocates for the physical association of language concepts. Previous studies have supported the efficacy of IVR in enhancing new language introduction by connecting physical movement with language acquisition (Fuhrman et al., 2021; Repetto et al., 2021). Likewise, our F2F environment emphasized listening to and speaking key vocabulary in isolation from context, which aligns with recommended practices for early language learning (Dressler & Mueller, 2022; Richards & Rogers, 2014). The increase from pretest to posttest supports the idea that learners may experience the benefits of both modalities when blending IVR with more traditional F2F methods.
The results also suggest how the strengths of one approach can address the weaknesses of another. For example, our participants reported that the IVR environment was more enjoyable and fun than the F2F instruction because they could interact with and openly explore objects as they learned the vocabulary associated with those objects. Others have similarly found that students find IVR more motivating and more engaging than traditional F2F instruction because it offers the freedom to explore which is absent in many F2F environments (Lan, 2020).
At the same time, our interview data suggests that participants in our study initially found the IVR experience confusing and overwhelming, primarily because the IVR environment immersed them in hearing key vocabulary as part of a full sentence that contained words they did not recognize. Our participants also noted how learning the names of each object in the F2F instruction helped them overcome their initial confusion in IVR because it focused more on listening to and speaking key vocabulary with feedback from the instructor as well as peers. In their interviews, they described how, as they learned key vocabulary, they became more successful at discerning that vocabulary from the other words in the commands spoken in IVR. The video analysis suggests that learning the words in isolation from the IVR environment helped with this. The number of tasks completed on the 1st try increased over time while the use of Listening and Random strategies decreased over time, indicating that our participants did become more successful with completing tasks in IVR as they learned key vocabulary. This suggests how each instructional modality helped strengthen the other approach in a way that supported our participants in learning Mandarin vocabulary in a relatively short period of time.
However, it is important to note that these findings are derived from a small sample size and are based on a short period of instruction. Therefore, while promising, they may not be broadly generalizable. The limited number of participants and the brief instructional duration restrict the ability to definitively conclude the effectiveness of the blended instructional approach and its applicability to a wider population. Further research with a larger sample size and extended instruction periods is essential to validate these preliminary results and to refine the instructional design for diverse learning contexts.
One immediate implication is that learners in IVR can benefit from some sort of complementary or additional training that can help make the VR experience less overwhelming. In our study, students reported that the IVR experience was initially overwhelming for them but this improved as they learned to recognize key vocabulary in the F2F instruction. This suggests that our F2F instruction served as a form of training that prepared learners for the VR environment. This is consistent with Meyer et al.’s (2019) research, which suggested that additional training can help improve the quality of learning in VR environments. In our case, traditional F2F instruction served as that additional training - it was a way of preparing our participants to recognize key vocabulary and distinguish it from other spoken words in the VR environment. This ultimately supported their success in the VR environment and made the experience more enjoyable.
Another implication is that designers interested in using IVR for learning should look for ways to use F2F instruction to offset the limitations of off-the-shelf VR. In our case, the IVR didn’t offer students an opportunity to speak Mandarin. In addition, learners could complete the tasks without ever learning the vocabulary. Some of our participants began using the built-in ‘hint’ feature while others simply combined random objects until they found the correct combination. We intentionally structured the F2F to ‘fill in the gaps’ that may have occurred when participants used hints and random strategies in IVR. It was also structured to move from teacher-led to more student-led so that students could gain confidence in speaking and hearing the language independently of the instructor. Other designers who use off-the-shelf VR apps should take the time to learn the affordances and shortcomings of a VR app before using it with learners. This will help ensure that the use of IVR complements F2F and vice versa, creating a more balanced set of learning opportunities for participants.
Immersive Virtual Reality (IVR) is an exciting new technology that has the potential to enhance language learning in K-12 settings, particularly when it blends language comprehension with sensorimotor engagement. Although the literature has yet to reach a consensus on the superiority of either IVR or F2F in language learning, our study suggests that a blended approach can harness the strengths of both. The results suggest that an integrative strategy can not only enhance the overall learning outcomes but also provide a language learning experience that covers a wider range of learning needs in the K to12 classroom. It is our hope that these findings offer other designers a strong foundation upon which to build their own instructional activities that blend IVR with F2F approaches to learning.
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
There were no ethical issues related to the selection and treatment of subjects associated with this paper. The research activity was approved by the University’s Internal Review Board.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
EdTech Archives