The Human-Computer Interaction in the Educational Use of Spherical Video-based Virtual Reality

Introduction

Spherical video-based virtual reality (SVVR), a subset of immersive VR (Jong, 2022), is also known as spherical video, panoramic video, 360-degree video (360°video), or omnidirectional video in the literature (Li et al., 2023). 360-degree video technology is a method of capturing the real environment with special cameras that can record the entire surrounding scene either from a static position or from a dynamic movement (Ranieri et al., 2022; Rosendahl & Wagner, 2024). SVVR can be conveniently utilized with a mobile phone and inexpensive cardboard goggles (Jong, 2022). In other words, SVVR has the advantages of being cheap and user-friendly. It is suitable for teachers who may not be "tech-savvy" but want to offer immersive learning experiences with technological support for their students during teaching (Chien et al., 2020; Jong et al., 2020). SVVR finds extensive application in education in recent years (Wu et al., 2023). To date, SVVR is used for delivering content and factual knowledge through direct instruction, and its instructional use has been designed with limited interaction as providing scaffolding and assessment within the video were lacking (Li et al., 2023).

Interactivity is a crucial aspect of VR experiences (Roussou et al., 2006). It involves the ability to freely navigate a virtual environment, experience the “reality” firsthand from various perspectives, make modifications, control parameters, and respond to cues, feedback, and affordances within the environment. According to Nelson and Erlandson (2012), the design of educational virtual worlds should prioritize the facilitation of learner intentionality. Merely creating a virtual world and placing students within it without a clear learning objective is insufficient. Each virtual world should be meticulously crafted to integrate curricula, tasks, and interactions that enable learners to actively and consciously pursue clearly defined goals. Essentially, this involves aligning the virtual world and its embedded activities with the desired learning outcomes, thereby necessitating learners to make thoughtful and deliberate choices as they engage with the educational content. To foster authenticity, virtual reality must be responsive to user actions (Sherman & Craig, 2018). Sharp et al. (2023) provide a detailed description of the five types of interaction, noting that they are not mutually exclusive. Specifically, the first type is instructing, where users issue instructions to the system through various means. The second type is conversing, where users engage in a dialogue with the system. The third type is manipulating, where users interact with objects in virtual or physical spaces by performing actions like opening, holding, closing, or placing them. This allows users to apply their existing knowledge of interacting with objects. The fourth type is exploring, where users navigate through virtual environments or physical spaces. Virtual environments include 3D worlds, augmented reality, and virtual reality systems, while sensor-based technologies enable navigation in physical spaces like smart rooms and ambient environments. This type of interaction allows users to utilize their familiarity with the environment. The fifth type is responding, in which the system initiates the interaction and the user decides whether to respond.

Literature Review

The number of reviews pertaining to the application of SVVR in education has been steadily increasing in recent years, with Snelson & Hsu (2020) being the earliest (Schroeder et al., 2023). These reviews exhibit a certain degree of overlap in research objectives, primarily focusing on exploring the scope and characteristics of research related to 360°video (Snelson & Hsu, 2020), examining its applications in general or specific educational contexts (Li et al., 2023; Rosendahl & Wagner, 2024), and analyzing the effects of educational SVVR (Ranieri et al., 2022; Schroeder et al., 2023). Upon further analysis of these reviews, it is evident that these reviews highlighted the significance of interactivity in SVVR-based educational learning systems but did not provide any substantial descriptions of how interactivity was achieved. Meanwhile, the interactive features of SVVR, which are primarily limited to changing observational directions and angles through users' head movements, present constraints that hinder its effectiveness in facilitating teaching strategies or supporting learning exercises. As Jong (2023) claims, future research is needed to explore the feasibility of integrating effective learning scaffolds and enhanced interactivity in SVVR for a constructivist-oriented educational shift.

Method

This scoping review adhered to the PRISMA-ScR guidelines (Tricco et al., 2018) and the methodological framework for scoping reviews proposed by Arksey & O'Malley (2005) to interpret the evidence. Peer-reviewed journal papers were included if they were: published between 2019 to 2023, written in English, clearly mentioned the ways that SVVR is used in learning content, and related to the education field. Searches used two electronic databases: Scopus and Web of Science. Thirty-eight articles were identified for this review. Search terms included SVVR terms ‘spherical video based virtual reality’, ‘svvr’, ‘360-degree video’, ‘360° video’, and ‘panoramic video’ with “OR” (Boolean relationship), and Education terms ‘education’, ‘learning’, ‘school’, ‘primary school’, ‘elementary school’, ‘secondary school’, ‘middle school’, ‘high school’, ‘higher education’ and ‘professional training’ with “OR” to ensure that the studies met the inclusion criteria captured.

Discussion and Conclusion

While Sharp et al. (2023)'s summary covers general forms of interaction, it does not account for the specific patterns of HCI usage in the context of current SVVR learning, nor does it consider the intended effects associated with interaction design. Consequently, this paper integrates the findings from the reviewed literature to provide more concrete summary and definition of the three primary categories of HCI usage within SVVR learning (see Table 1).

Table 1

Types of HCI interaction usage in the SVVR learning

Research on VR training has demonstrated the effectiveness of interactive activities, particularly in enhancing spatial knowledge acquisition and recall (Arthur & Hancock, 2001). Similarly, Burdea and Coiffet (2003) summarized that the user's sense of immersion and feeling like they are a part of the action on the screen is facilitated by interaction and its mesmerizing power. Moreover, the presence experienced by users in VR has often been associated with the level of interaction and feedback provided, highlighting the significance of user control over the virtual environment (Jelfs & Whitelock, 2000). In conclusion, this study stands out from other reviews in the educational SVVR field by specifically examining HCI in the educational SVVR use. Object Manipulation, Test Scaffold, and Voice Scaffold, are summarized to emphasize the significance of interactive use in SVVR-based learning and promote the exploration of new interaction types to enhance the SVVR experience. Theoretically, it responds to the gaps that future research should emphasize the importance of add-ons for optimal SVVR learning experiences claimed by Evens et al. (2023). Practically, the findings enable a comprehensive understanding of SVVR interactions and offer valuable references for front-line teachers designing SVVR materials and preparing SVVR-supported classes.

References

1. Arksey, H., & O' Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. https://doi.org/10.1080/1364557032000119616
2. Arthur, E. J., & Hancock, P. A. (2001). Navigation Training in Virtual Environments. International Journal of Cognitive Ergonomics, 5(4), 387-400. https://doi.org/10.1207/S15327566IJCE0504_2
3. Burdea, G., & Coiffet, P. (2003). Virtual reality technology (2nd ed.). John Wiley.
4. Chien, S. Y., Hwang, G. J., & Jong, M. S. Y. (2020). Effects of peer assessment within the context of spherical video-based virtual reality on EFL students’ English-Speaking performance and learning perceptions. Computers and Education, 146, 103751. https://doi.org/10.1016/j.compedu.2019.103751
5. Evens, M., Empsen, M., & Hustinx, W. (2023). A literature review on 360-degree video as an educational tool: towards design guidelines. Journal of Computers in Education, 10(2), 325-375. https://doi.org/10.1007/s40692-022-00233-z
6. Jelfs, A., & Whitelock, D. (2000). The notion of presence in virtual learning environments: what makes the environment "real". British Journal of Educational Technology, 31(2), 145-152. https://doi.org/10.1111/1467-8535.00145
7. Jong, M. S. Y. (2022). Flipped classroom: motivational affordances of spherical video-based immersive virtual reality in support of pre-lecture individual learning in pre-service teacher education. Journal of Computing in Higher Education, 1-22. https://doi.org/10.1007/s12528-022-09334-1
8. Jong, M. S. Y. (2023). Pedagogical adoption of SVVR in formal education: Design-based research on the development of teacher-facilitated tactics for supporting immersive and interactive virtual inquiry fieldwork-based learning. Computers and Education, 207, 104921. https://doi.org/10.1016/j.compedu.2023.104921
9. Jong, M. S. Y., Tsai, C. C., Xie, H., & Wong, K. K. F. (2020). Integrating interactive learner‐immersed video‐based virtual reality into learning and teaching of physical geography. British Journal of Educational Technology, 51(6), 2063-2078. https://doi.org/10.1111/bjet.12947
10. Li, W., Qian, L., Feng, Q., & Luo, H. (2023). Panoramic video in education: A systematic literature review from 2011 to 2021. Journal of Computer Assisted Learning, 39(1), 1-19. https://doi.org/10.1111/jcal.12730
11. Nelson, B. C., & Erlandson, B. E. (2012). Design for learning in virtual worlds. Routledge.
12. Ranieri, M., Luzzi, D., Cuomo, S., & Bruni, I. (2022). If and how do 360° videos fit into education settings? Results from a scoping review of empirical research. Journal of Computer Assisted Learning, 38(5), 1199-1219. https://doi.org/https://doi.org/10.1111/jcal.12683
13. Rosendahl, P., & Wagner, I. (2024). 360° videos in education-A systematic literature review on application areas and future potentials. Education and Information Technologies, 29(2), 1319-1355. https://doi.org/10.1007/s10639-022-11549-9
14. Roussou, M., Oliver, M., & Slater, M. (2006). The virtual playground: an educational virtual reality environment for evaluating interactivity and conceptual learning. Virtual Reality, 10(3), 227-240. https://doi.org/10.1007/s10055-006-0035-5
15. Schroeder, N. L., Siegle, R. F., & Craig, S. D. (2023). A meta-analysis on learning from 360° video. Computers and Education, 206, 104901. https://doi.org/10.1016/j.compedu.2023.104901
16. Sharp, H., Rogers, Y., & Preece, J. (2023). Interaction design: beyond human-computer interaction (Sixth edition). John Wiley & Sons.
17. Sherman, W. R., & Craig, A. B. (2018). Understanding virtual reality: Interface, application, and design. Morgan Kaufmann.
18. Snelson, C., & Hsu, Y. C. (2020). Educational 360-Degree Videos in Virtual Reality: a Scoping Review of the Emerging Research. TechTrends, 64(3), 404-412. https://doi.org/10.1007/s11528-019-00474-3
19. Tricco, A. C., et al. (2018). PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Annals of Internal Medicine, 169(7), 467-473. https://doi.org/10.7326/m18-0850
20. Wu, W. L., Hsu, Y., Yang, Q. F., Chen, J. J., & Jong, M. S. Y. (2023). Effects of the self-regulated strategy within the context of spherical video-based virtual reality on students’ learning performances in an art history class. Interactive Learning Environments, 31(4), 2244-2267. https://doi.org/10.1080/10494820.2021.1878231