Motivational Design for Inclusive Digital Learning Innovation: A Systematic Literature Review

Introduction

As a result of COVID-19, recent education and learning and development operations have shifted the delivery of learning and teaching activities to online environments due to limited face-to-face interactions in physical workplaces and school settings (Means & Neisler, 2020). As an unintended consequence, critical limitations in online learning environments are highlighted owing to the increasing deployment of online learning and technology-enriched and -enabled learning environments (TEELE) (Huang, 2021; 2022) across various learning and development contexts. For instance, disparities among students’ access to computers and the internet continue to remain a significant barrier (Bacher-Hicks et al., 2021; McGuire et al., 2021). In addition to the access barrier, the most challenging aspects of employing online learning include: (1) maintaining students’ motivation with this learning format (Zaccoletti et al., 2020) and (2) dealing with the diverse demographics of online learners (Conto et al., 2020). These challenges highlight the importance of applying motivational design strategies for online learning environments by understanding the roots of learners’ motivation. Online learning has played a vital role in the dissemination of education during the pandemic. To sustain the innovative features of online learning systems, diverse learners’ motivational needs should be considered.

The Role of Motivational Design in Instructional System Design

This rapid shift in the delivery mode of instruction from face-to-face to online may have led learners across contexts to experience considerable challenges in maintaining their motivation with online learning (Huang, 2013; Park & Choi, 2009; Zaccoletti et al., 2020). This present study, grounded in prior inquiries (e.g., Hartnett, 2016; Huang, 2013; Keller, 2010; Ryan & Deci, 2000), considers online learning motivation an ongoing social process that dictates learners’ decisions to interact with intended online learning processes. Further, online learning motivation is largely localized to individual learners’ early responses to intended learning processes, and it contributes to “learning engagement” that aims at sustaining meaningful and long-term online learning processes. The Engagement Theory (Kearsley & Shneiderman, 1998) emphasizes that engagement is different from interaction in the context of online learning, which may consist of learners’ cognitive processes as well as perceived motivational support. Therefore, focusing only on cognitive and behavioral interactions in online learning environments is insufficient to fully motivate learners (Huang, 2013). In addition, learners in online learning contexts are more likely to have control over what to learn, when to learn, and how to learn. Learners have the flexibility to learn anytime and at different locations (Dhawan, 2020). Even if flexibility is one of the strengths of online learning, the metacognitive and meta-social control a learner has to implement for online coursework depends on learners’ motivational status (i.e., volitional control) (Keller, 2008). Providing learners with motivating online learning processes via systematic design approaches, however, has often been overlooked in mainstream instructional system design processes and models. Although research has argued that systematic approaches to address learners’ motivational needs (motivational design) is critical to ensure effective online learning (Huang, 2013, 2018; Keller, 2018).

The definition of ‘motivational design’ adopted in this study stems from John Keller’s scholarship with decades of conceptual and empirical findings (Keller, 1987, 1988, 2008, 2010). In terms of design process, motivational design refers to “the process of arranging resources and procedures to bring about changes in people’s motivation” (Keller, 2010, p. 22). Motivational design in this study is focused on the design and development of motivational support in learning environments. It involves systematic processes and motivational strategies that help learners sustain their behaviors of achieving learning goals. The systematic motivational design process includes ten steps: (1) obtain course information, (2) obtain audience information, (3) analyze audience, (4) analyze existing materials, (5) list objectives and assessments, (6) list potential tactics, (7) select and design tactics, (8) integrate with instruction, (9) select and develop materials, and (10) evaluate and revise (Keller, 2010, p.57).

Diversified Online Learner Populations

Online learner populations are becoming significantly diverse due to the ongoing systematic interruption (i.e., COVID 19) as it necessitates the expansion of online learning across various learning and development contexts. Such diversity among learners not only is manifested by their access to and prior learning experiences in online learning environments, but also it is grounded in learners’ racial, social, and cultural backgrounds. All the demographic, educational, and social backgrounds among online learners are the foundation to form their unique motivational needs and therefore, influence engagement with online learning. As an example, studies have shown that ethnically underrepresented students in STEM fields tend to struggle with having motivation for online courses (Asgari et al., 2021; Cromley & Kunze, 2021; Walsh et al., 2021). In contrast, an alternative study (Amina, 2021) reports that women’s capabilities are increased through expanded access to online learning by having more opportunities to be involved in their STEM-related jobs during the pandemic. These studies show that learners’ social and cultural background impact their learning motivation when they learn through online learning.

Conto and colleagues (2020) reported that in recent school shutdowns around the world due to limiting face-to-face interactions, lower-income nations show the least utilization of online platforms and take-home materials (64%) and are alternatively relying on television (92%) and radio (93%). In comparison, higher-income nations show the most utilization for online platforms (95%) while relying the least on television (63%) and radio (22%).

Prior to the school closures, online learning was generally more adopted for training returning adults and transfer students where online learning programs were focused on primarily adults returning to school from an absence. For K-12 students, very few teachers and students had extensive experience with online learning before the mandatory school closures by the pandemic (Barbour & LaBonte, 2017; Barbour & Reeves, 2009).

As of late March 2020, UNESCO projected that more than 190 countries in the world closed schools. As a result, this pandemic context affected 1.6 billion students’ learning experiences (Conto et al., 2020). While the emerging challenges brought by the pandemic could be less relevant years from now, they have offered impetus to respond now to the changing demographics of online learners and the accompanied diverse learners’ needs for long-term success in online learning or digital learning environments.

In the context of this present study, the diversity of learners, manifested by their motivational needs, highlights a focus on physical access to online learning environments or digital learning innovations, but learners’ motivational needs for achieving intended online learning processes and outcomes must be addressed. In particular, designing motivational support in online learning environments should be the priority. Our rationale is threefold. First, motivational support is the foundation of learning engagement (Huang, 2013; Kearsley & Shneiderman, 1998). “Learning motivation” that is largely localized to individual learners’ early responses to intended learning processes can lead to long-term “learning engagement” in online learning environments. Second, motivational support has been largely overlooked by prominent instructional design processes and models. As learner motivation drives learners’ early cognitive, affective, and behavioral efforts during online learning processes, instructional design effort should purposefully be a part of learners’ motivational analysis and motivational design. Third, as online learner populations are increasingly diverse in their racial, social, cultural, linguistic, and educational backgrounds, the design of motivational support for learning should no longer be based on outdated assumptions (e.g., all learners have equitable access to internet connections, learners’ skill levels in using online content are the same) (Ragnedda, 2019). A dedicated motivational design analysis is needed to reveal the fundamental causes of learners’ motivational barriers created by learners’ social and cultural backgrounds. Grounded in the aforementioned reasons, we are advocating for inclusive digital learning innovation that is focused on addressing learners’ diverse motivational needs with systematic motivational design processes.

Purpose of the Study

Current societal and social phenomena show the importance of motivational design for diverse learners as the first step towards inclusive digital learning innovation in the context of online learning. This systematic literature review study surveys the landscape of motivational design research between 2010 and 2021 to understand the recent trends of how motivational design has been investigated and what types of learners have been included in online and digital learning environments. The definition of ‘motivational design’ helps this study focus on the systematic motivational strategies and methods to enable changes in people’s motivation rather than the broadly defined instructional design strategies.

Research Questions

This review aims to answer the following questions:

1. What are the roles of motivational design in online and digital learning environments?
2. What are the demographic and contextual criteria considered for motivational design in online and digital learning environments?

Method

This research was carried out by following the systematic literature review key steps laid out by Pati and Lorusso (2018).

Selection criteria

The following criteria were applied to identify the literature to be reviewed:

1. The literature was selected if it was published in peer-reviewed journal articles and proceedings in the English language. This provides easy access to the majority of scientific publication readers around the world as most scientific publications are in English (Gordin, 2015; Montgomery, 2013; Ramírez-Castañeda, 2020).
2. The literature was included if the studies provided empirical data and interpretation of data analysis. Literature reviews, proposals, and conceptual papers were excluded.
3. The literature was included if studies were conducted for online courses, blended courses, and digital learning applications.
4. The literature was included if the studies were conducted in teaching and learning environments. Studies for patients, the general public, and employees without teaching and learning goals were excluded.

Search and selection process

The literature search and selection process is listed below.

1. SCOPUS was used as the literature database for two reasons. First, Shah and colleagues (2017) reported that on the topic of inclusive education research, SCOPUS could retrieve publications from influential research journals more effectively than automated academic databases (e.g., Google Scholar). Second, the use of SCOPUS allows the study to be differentiated from previous literature review search strategies on motivational design based on a broad range of academic databases (e.g., Li & Keller, 2018).
2. The keywords “motivational design”, “motivation" and “instructional design" and "online learning”, “motivation" and "instructional design" and "blended learning”, and “motivation" and "instructional design" and "digital" were used based on the literature review. These keywords are used to keep the focus of this study on systematic motivational strategies in the instructional design process and methods, specifically in the context of online learning, blended learning, and digital learning environments.
3. The literature was limited to published peer-reviewed journal articles and proceedings between January 2010 and December 2021.

The search process yielded a total of 58 publications. The volume is insignificant in comparison with the volume of peer-reviewed publications with keywords of “learning technology (n=3,814), “educational technology” (n=3,978), or “instructional design” (n=1,358) during the same publication period (2010 - 2021) on SCOPUS. All 58 articles were reviewed by two researchers to enhance validity and reliability. Only 29 articles met the mentioned four selection criteria and were included in the analysis.

Data Analysis

Based on the nature of the research questions and the amount of literature, content analysis (Hsieh & Shannon, 2005) was conducted for this review. The two research questions served as the initial coding categories for the intended content analysis. That is, all 29 articles were reviewed and categorized based on the research questions. In addition, considering the essential role of ARCS motivational design in the field of learning system design (Keller, 2018; Li & Keller, 2018), the literature was divided based on whether or not the study adopted the ARCS model to guide the study. A discussion on the ARCS model will follow. To answer the first research question, all 29 articles were categorized by “research goals” and “roles of motivational design”. Second, to answer the second research question, all 29 articles were compared based on the “locations of the research”, “learning environment”, “target learners”, and “studied demographic factors” to reveal demographic and contextual factors applied in reviewed studies. The demographic factors in this study refer to the target audiences’ socio-demographic factors (e.g., age, gender, race, education, and prior experience), which were either identified by the study participants or were applied to interpret the findings. Both researchers were able to achieve a high level of inter-rater reliability at 96% (Drost, 2011; Frey, 2018) prior to analyzing all 29 publications.

Findings

Although all 29 publications studied some aspect of “motivational design”, 17 studies applied the ARCS model (Keller, 1987) to their inquiries. The ARCS motivational design model was developed for creating effective ways to identify major influences on the motivation to learn, and for adopting systematic methods to diagnose and address learners’ motivational needs. This model articulates concepts and variables that characterize learning motivation and implements strategies that enhance the motivational appeal of instruction. The model defines four major motivational conditions (i.e., Attention, Relevance, Confidence, and Satisfaction) that must be met for learners to become and remain motivated. Also, it proposes a systematic motivational design process (i.e., Define, Design, Develop, and Evaluate), which can be used with typical instructional system design and development models (Huang, 2013; Keller, 1987, 2010).

ARCS Model-Grounded Studies

Studies grounded in ARCS model can be categorized by “research goals”, “roles of motivational design”, “locations of the research”, “learning environment”, “target learners”, and “studied demographic factors”. The roles of motivational design are depicted in study findings by explaining the impact of motivational design on various learning outcomes and learners’ attitudes. 11 studies applied the ARCS model to design and evaluate new instructional tools; another six studies applied the ARCS model only for evaluating existing educational tools with the focus on learner’s motivation status; eight studies applied the ARCS model to measure learner’s motivation along with learners’ learning outcomes, confidence, interests, tendency to use technology, and engagement (see Table 1).

Table 1

Goals of research and roles of motivational design of reviewed ARCS model studies

Studies that developed educational tools by applying the ARCS model describe the role of motivational design as it plays an effective part in developing learners’ motivation in regard to the new learning environments (e.g., Open Learner Model and blended learning environment) (Durrani & Kamal, 2020; Sek et al., 2015), towards their interests/attitudes toward mathematics with better learning outcomes (Hodges & Kim, 2013), and the audience’s inspiration for future technology use (Huang, 2014). In addition, one study showed how the combination of another instructional design model/feature (e.g., ADDIE model and gamification) and motivational design improved learners’ motivation and learning process (Vagianou et al., 2021). On the other hand, studies that evaluated existing educational tools based on the ARCS model were focused on the roles of motivational design based on the motivational factors such as ‘Attention, Relevance, Confidence, and Satisfaction’. For instance, augmented reality (AR) functionality in physical puzzle-type games did support a comparatively lower confidence level among K-12 students (Lu et al., 2020). The learners’ motivation progress was mostly measured by using the validated Instructional Materials Motivation Survey (IMMS) (Keller, 1987) or the Course Interest Survey (CIS) (Keller & Subhiyah, 1993). Learning outcomes, learners’ interests, and tendency to use technology were measured by learners’ post-course test scores and other instruments, such as the Fennema-Sherman Mathematics Attitudes (FSAMA) (Fennema & Sherman, 1976). The analysis implies that the motivational design strategies are applied to improve not only learners’ motivation but also learners’ confidence and familiarity with using technology.

The geographical locations of the 17 studies using the ARCS model include Australia, Byzantine, China, Iran, Malaysia, Taiwan, Turkey, and the U.S. There are ten studies conducted outside of the U.S., while seven studies were conducted in the U.S. (see Table 2).

Table 2

Locations of the Reviewed ARCS Model Studies

The studied learning environment grounded in the ARCS model consisted of blended learning, digital application/web 2.0 (social media), e-learning/online learning, Massive Open Online Course (MOOCs), and virtual reality (see Table 3). Digital applications include music instrument practice and augmented reality function puzzle games to motivate learners. While the online and e-learning environments were studied the most, the ARCS model was applied to diverse learning environments. 

Table 3

Learning Environment of reviewed ARCS Model Studies

In terms of target audience, only four out of 17 studies based on the ARCS model targeted the K-12 learning setting, while 13 studies were situated in higher education (see Table 4). For studies in K-12, learners’ age was mainly considered as a demographic factor. One of the studies developed a new motivational design framework and this framework was evaluated not by students but K-12 teachers (Vagianou et al., 2021). In this study, teachers’ field of study, working experience, and gender were considered during the data collection process. Studies in the higher education setting addressed many socio-demographic factors of learners including academic level, age, gender, marital status, learning preference, and prior experience with technology (or online learning). One study mentioned the efforts of including diverse students’ groups and indicated that there were no participants from the special needs group (Durrani & Kamal, 2020). Participants demographic factors were described in the methodology section but none of these studies showed how the findings were related to the participants’ socio-demographic factors. For example, how learners’ motivation and learning outcomes were different based on their demographic factors was not addressed.

Table 4

Target learners and studied demographic factors of reviewed ARCS Model Studies

Other Motivational Design Studies

There were 12 reviewed studies that did not use the ARCS model for the motivational design inquiries. Among these 12 studies, four studies either did not measure learners’ motivation or did not directly discuss learners’ motivation in their findings (Casimiro, 2011; Joo et al., 2015; Ng & Przybyłek, 2021; Rosenberger, 2019). These studies were excluded from Tables 5, 6, and 7. One study applied modality and radiance design principle (Mayer, 2014) to design and evaluate adult learners’ situational interest in the online learning environment (Dousay, 2016). Motivational design in this study was implemented by avoiding needless multimedia methods to teach learners so that learners can sustain their interests for better learning outcomes. Another study applied the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003) to investigate how learners might be motivated to engage with the Open Learning Environment System (Huang, 2017) (see Table 5 and 6). The main role of the motivational design in this study was to examine the Open Learning System to address learners’ motivational challenges relevant to learning goals and self-efficiency. Also, there was one study which measured how the motivational design influenced learners’ attitude toward the blended learning format and the results showed that students highly rated this format since it helped the learners to stay on track (Gawlik-Kobylińska et al., 2021). In this study, participants were asked about their prior experience with the learning format. Similar to the aforementioned studies with the ARCS model, none of these studies connected participants’ socio-demographic factors to study findings (see Table 7).

However, there is one study connecting learners’ gamification user types to online learning activities to understand how learners are motivated differently based on their types (Bovermann & Bastiaens, 2020). This study suggests that it is important to understand target learner groups with their own leaning types and use a systematic approach to conduct meaningful online learning design. 

Table 5

Goals of research and roles of motivational design studies without ARCS model (n=4)

Table 6

Learning environment of reviewed studies without ARCS model (n=4)

Table 7

Target audience, demographic factors, and locations of reviewed studies without ARCS model

Discussion

The findings highlight several emerging needs in order to address motivational needs of diverse online learner populations. First, this review study suggests the need for applying systematic design processes to improve motivational support as merely half of the reviewed studies (11 out of 25) applied a systematic process (i.e., ARCS model) to design and evaluate corresponding motivational support. Many studies have not applied systematic design methods or have not appropriately measured learners’ motivation progress. Even for studies applying the ARCS model to design new learning tools, the effectiveness of the motivational strategies was assessed by learners’ assessment scores or other non-motivational achievements. According to Keller (1987), it is an important fact to base evaluation of the instructional materials primarily on motivational and learning outcomes since learning achievements (e.g., scores) could be affected by many other circumstances. Learners’ persistence, intensity of effort, emotion, and attitude should be considered to understand the effectiveness of motivational strategies to address learners’ diverse motivational needs.

Second, K-12 learners and teachers, by comparison with other learning and development contexts (e.g., higher education, workplaces), have not been exposed to the online learning environment extensively. Consequently, there is a lack of motivational design studies that are focused on K-12 learners’ online learning environment for formal learning purposes. Motivational design studies that targeted K-12 learners are also limited to the shorter-term use of digital applications as part of some learning activities. A comprehensive and longitudinal approach to diagnose and address young learners’ and their teachers’ motivational needs in online learning environments is in dire need.

Third, the findings show the diversity of learning environments (blended learning, e-learning, mobile applications, and virtual reality) and many geographic locations (Australia, China, Malaysia, and U.S.) of the reviewed motivational design studies. However, there is a noticeable absence of studies investigating influences of social experiences, cultural affiliation, economic status, and prior educational struggles of learners in a time when online learning is becoming increasingly diverse. In other words, learners’ diverse backgrounds and thus their impact on learners’ motivational needs have been excluded from the majority of reviewed motivational design studies. As online learners’ motivational needs are the product of constant social interactions with systemic barriers (access barriers), considering online learners’ vibrant and diverse experiences based on sex, age, race, ethnicity, socio-economic status, languages, and culture is essential to fully understand the root causes of their motivational problems. By extension, diversity-driven motivational design approaches could help us address the impact of digital divides derived from current and future digital learning innovations.

Fourth, for a deeper understanding of diverse learners’ motivational needs, an expanded inquiry of motivational support using various methodological approaches is needed. In addition to cross-sectional studies, longitudinal research design should be adopted more frequently to contribute to the field of motivational design with time-based evidence to document online learners’ fluctuating motivational needs during learning processes.

Finally, this study recognizes the limitation of sourcing the reviewed studies from one scientific and academic database. Our goal is to provide a focused and differentiated perspective derived from impactful peer-reviewed research publications.

Conclusion

To address the need of applying motivational design as the first step towards an inclusive digital learning innovation, the keywords of “motivational design”, “motivation” and “instructional design” and "online learning”, “motivation" and "instructional design" and "blended learning”, and “motivation" and "instructional design" and "digital" were used to retrieve 29 peer-reviewed journal articles published in English from 2010 to 2021. These papers were reviewed based on research goals, research locations, learning environments, and targeted audience. The findings suggest:

1. Applying a systematic design process to improve motivational support is needed
2. There is a lack of motivational studies for K-12 online learners
3. There is a lack of effort to study the impacts of the learners’ diverse backgrounds on their motivational needs in the context online learning
4. Various methodological approaches for a deeper understanding of diverse learners’ motivational needs are required.

A collaborative approach of these efforts would enhance our understanding on how to make the motivational design process more systematic and inclusive. 

References

1. Amina, T. (2021, August 31). Online education and women’s empowerment. Oxford Research Encyclopedia of Education. https://doi.org/10.1093/acrefore/9780190264093.013.1592
2. Asgari, S., Trajkovic, J., Rahmani, M., Zhang, W., Lo, R. C., & Sciortino, A. (2021). An observational study of engineering online education during the COVID-19 pandemic. PLOS ONE, 16(4), e0250041. https://doi.org/10.1371/journal.pone.0250041
3. Bacher-Hicks, A., Goodman, J., & Mulhern, C. (2021). Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. Journal of Public Economics, 193, 104345. https://doi.org/10.1016/j.jpubeco.2020.104345
4. Barbour, M. K. , & LaBonte, R. (2017). State of the nation: K‐12 e‐learning in Canada, 2017 edition. http://k12sotn.ca/wp‐content/uploads/2018/02/StateNation17.pdf
5. Barbour, M. K. , & Reeves, T. C. (2009). The reality of virtual schools: A review of the literature. Computers & Education, 52(2), 402–416. https://doi.org/10.1016/j.compedu.2008.09.009
6. Bovermann, K., & Bastiaens, T. J. (2020). Towards a motivational design? Connecting gamification user types and online learning activities. Research and Practice in Technology Enhanced Learning, 15(1), 1. https://doi.org/10.1186/s41039-019-0121-4
7. Casimiro. (2011). Effective online instructional design as perceived by teachers and students in selected private colleges and universities. Proceedings of the IADIS International Conference e-Learning 2011, Part of the IADIS Multi Conference on Computer Science and Information Systems 2011, MCCSIS 2011, 1, 275–282.
8. Colakoglu, O. M., & Akdemir, O. (2010). Motivational measure of the instruction compared: Instruction based on the ARCS motivation theory vs traditional instruction in blended courses. Turkish Online Journal of Distance Education, 11(2), 73-89. https://dergipark.org.tr/en/pub/tojde/issue/16908/176336
9. Conto, C. A., Akseer, S., Dreesen, T., Kamei, A., Mizunoya, S., Rigole, A., & Unicef. (2020). COVID-19: Effects of school closures on foundational skills and promising practices for monitoring and mitigating learning loss (pp. 1-30). UNICEF Office of Research-Innocenti. https://www.unicef-irc.org/publications/1144-covid19-effects-of-school-closures-on-foundational-skills-and-promising-practices.html
10. Cromley, J., & Kunze, A. (2021). Motivational resilience during COVID-19 across at-risk undergraduates. Journal of Microbiology & Biology Education, 22(1), ev22i1.2271. https://doi.org/10.1128/jmbe.v22i1.2271
11. Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 Crisis. Journal of Educational Technology Systems, 49(1), 5–22. https://doi.org/10.1177/0047239520934018
12. Dias, A.L., Silva, M.A.R., Anacleto, J.C., Silveira, L.M. and Penteado, R.A.D. (2010). A case study that shows the importance of color in web collaborative educational environment. CSEDU 2010 - 2nd International Conference on Computer Supported Education, Proceedings, 2, 226–231.
13. Dousay, T. A. (2016). Effects of redundancy and modality on the situational interest of adult learners in multimedia learning. Educational Technology Research and Development, 64(6), 1251–1271. https://doi.org/10.1007/s11423-016-9456-3
14. Drost, E. A. (2011). Validity and reliability in social science research. Education Research and perspectives, 38(1), 105-123.
15. Durrani, U., & Kamal, M. M. (2020, December). Towards applying ARCS model for a blended teaching methodologies: A quantitative research on students’ motivation amid the COVID-19. In International Conference on Design, Learning, and Innovation (pp. 198-207). Springer.
16. Fennema, E., & Sherman, J. A. (1976). Fennema-Sherman mathematics attitudes scales: Instruments designed to measure attitudes towards the learning of mathematics by males and females. JSAS Catalog of Selected Documents in Psychology, 6(1), 31-32. https://doi.org/10.2307/748467
17. Frey, B. B. (Ed.). (2018). The SAGE encyclopedia of educational research, measurement, and evaluation. Sage Publications.
18. Gawlik-Kobylińska, M., Domalewska, D., Maciejewski, P. (2021). How to motivate students? The four dimensional instructional design approach in a non-core blended learning course. In M. Auer & D. May (Eds.), Cross Reality and Data Science in Engineering (pp. 782–794). Springer. https://doi.org/10.1007/978-3-030-52575-0_64
19. Gordin, M. D. (2015). Scientific babel. University of Chicago Press.
20. Hamzah, W. M. A. F. W., Ali, N. H., Saman, M. Y. M., Yusoff, M. H., & Yacob, A. (2015). Influence of gamification on students’ motivation in using e-learning applications based on the motivational design model. International Journal of Emerging Technologies in Learning (iJET), 10(2), 30-34. http://dx.doi.org/10.3991/ijet.v10i2.4355
21. Hartnett M. (2016). Motivation in Online Education. Springer, Singapore. https://doi.org/10.1007/978-981-10-0700-2_2
22. Hawlitschek, A., & Joeckel, S. (2017). Increasing the effectiveness of digital educational games: The effects of a learning instruction on students’ learning, motivation and cognitive load. Computers in Human Behavior, 72, 79-86. https://doi.org/10.1016/j.chb.2017.01.040
23. Hodges, C. B., & Kim, C. (2013). Improving college students’ attitudes toward mathematics. TechTrends, 57(4), 59–66. https://doi.org/10.1007/s11528-013-0679-4
24. Huang, W. H. D. (2013). Online learning engagement system (OLES) design framework for postsecondary online learning environments: A synthesis on affordances from game-based learning, social media-enabled learning, and open learning. In V. Wang (Ed.), Handbook of research on teaching and learning in K-20 education (pp. 182-200). IGI Global. https://doi.org/10.4018/978-1-4666-4249-2
25. Huang, W. D., Hood, D. W., & Yoo, S. J. (2014). Motivational support in Web 2.0 learning environments: A regression analysis based on the integrative theory of motivation, volition and performance. Innovations in Education and Teaching International, 51(6), 631–641. https://doi.org/10.1080/14703297.2013.796718
26. Huang, W. D., & Wu, C.-G. (2017). Understanding motivational system in open learning: Learners’ engagement with a Traditional Chinese-based open educational resource system. Educational Technology Research and Development, 65(6), 1495–1521. https://doi.org/10.1007/s11423-017-9529-y
27. Huang, W. D., & Oh, E. G. (2018). Motivational support from digital game-based learning environments (DGBLEs) for scientific topics designed by novice end users. Educational Media International, 55(2), 123–136. https://doi.org/10.1080/09523987.2018.1484043
28. Huang, W. D., Shackelford, L., Craig, A., Merrill, C., & Chen, D. (2019). Relationships between motivational support and game features in a game-based virtual reality learning environment for teaching introductory archaeology. Educational Media International, 56(3), 183-200. https://doi.org/10.1080/09523987.2019.1669946
29. Huang, W. D., & Sung, J. S. (2021, November 2-6). Inclusive digital learning innovation: Preliminary systematic literature synthesis on motivational design from 2010-2020 [Conference session]. Association for Educational Communications & Technology (AECT) 2021 Convention, Chicago, IL United States.
30. Huang, W. (2022, July). Learning technologies foundations and applications. Coursera. https://www.coursera.org/learn/learning-technologies-foundations-applications.
31. Hui, Y. K., Li, C., Qian, S., & Kwok, L.-F. (2018). Cultivating situational interest in blended learning environment in Cheung, S., Kwok, Lf., Kubota, K., Lee, LK., Tokito, J. (Eds.), International conference on blended learning (pp.81-92). Springer.
32. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
33. Iwasaki, P. Y. (2021). Five tips from filmmakers: An online instructional module for documentary film research. IAFOR Journal of Education, 9(4), 63-82.
34. Joo, Y. J., Oh, E., & Kim, S. M. (2015). Motivation, instructional design, flow, and academic achievement at a Korean online university: A structural equation modeling study. Journal of Computing in Higher Education, 27(1), 28-46. https://doi.org/10.1007/s12528-015-9090-9
35. Kearsley, G., & Shneiderman, B. (1998). Engagement theory: A framework for technology-based teaching and learning. Educational Technology, 38, 20–23. https://www.jstor.org/stable/44428478
36. Keller, J. M. (1987). Development and use of the ARCS Model of instructional design. Journal of Instructional Development, 10(3), 2–10. https://doi.org/10.1007/BF02905780
37. Keller, J. M. (1988). Motivational design. In R. McAleese & U. C. (Eds.), Encyclopedia of educational media communications and technology (2nd ed., pp. 406 - 409). Greenwood Press.
38. Keller, J. M. & Subhiyah, R. (1993). Manual for the course interest survey (CIS) Tallahassee, FL: Instructional Systems Program, Florida State University.
39. Keller, J. M. (2008). An integrative theory of motivation, volition, and performance. Technology, Instruction, Cognition, and Learning, 6(2), 79-104.
40. Keller, J. M. (2010). Motivational design for learning and performance: The ARCS Model approach. Springer.
41. Keller, J. M. (2018). Where there's a will . . . motivation and volition in college teaching and learning: New directions for teaching and learning (Number 152). Wiley.
42. Königschulte, A. (2015). Sound as affective design feature in multimedia learning--benefits and drawbacks from a cognitive load theory perspective. Proceedings of the 12^th International Conference on Cognition and Exploratory Learning in the Digital Age, 75-83.
43. Li, K., & Keller, J. M. (2018). Use of the ARCS model in education: A literature review. Computers & Education, 122, 54-62. https://doi.org/10.1016/j.compedu.2018.03.019
44. Lu, S.-J., Liu, Y.-C., Chen, P.-J., & Hsieh, M.-R. (2020). Evaluation of AR embedded physical puzzle game on students’ learning achievement and motivation on elementary natural science. Interactive Learning Environments, 28(4), 451–463. https://doi.org/10.1080/10494820.2018.1541908
45. Ma, L., & Lee, C. S. (2020). A motivational design approach to integrate MOOCs in traditional classrooms. In E. Ishita, N. L. S. Pang, & L. Zhou (Eds.), Digital Libraries at Times of Massive Societal Transition (Vol. 12504, pp. 187–195). Springer International Publishing. https://doi.org/10.1007/978-3-030-64452-9_16
46. Mayer, R. E. (2014). Multimedia instruction. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 385–399). Springer International Publishing.
47. McGuire, D., Germain, M. L., & Reynolds, K. (2021). Reshaping HRD in light of the COVID-19 pandemic: An ethics of care approach. Advances in Developing Human Resources, 23(1), 26–40. https://doi.org/10.1177/1523422320973426
48. Means, B., & Neisler, J. (2020). Suddenly online: A national survey of undergraduates during the COVID-19 pandemic. Digital Promise. https://doi.org/10.51388/20.500.12265/98
49. Montgomery, S. L. (2013). Does science need a global language?: English and the future of research. University of Chicago Press.
50. Ng, Y. Y., & Przybyłek, A. (2021). Instructor presence in video lectures: Preliminary findings from an online experiment. IEEE Access, 9, 36485-36499. https://doi.org/10.1109/ACCESS.2021.3058735
51. Omrani, S., Fardanesh, H., Hemmati, N., Hemmati, N. (2012). Exploring an appropriate instructional design model for continuing medical education. Turkish Online Journal of Distance Education, 13(3), 347-361.
52. Park, J. H., & Choi, H. J. (2009). Factors influencing adult learners' decision to drop out or persist in online learning. Journal of Educational Technology & Society, 12(4), 207-217.
53. Pati, D., & Lorusso, L. N. (2018). How to write a systematic review of the literature. HERD: Health Environments Research & Design Journal, 11(1), 15-30. https://doi.org/10.1177/1937586717747384
54. Pittenger, A., & Doering, A. (2010). Influence of motivational design on completion rates in online self‐study pharmacy‐content courses. Distance Education, 31(3), 275–293. https://doi.org/10.1080/01587919.2010.513953
55. Ragnedda, M. (2019). Conceptualising the digital divide. In M. Ragnedda & B. Mutsvairo (Eds.), Mapping digital divide in Africa: A mediated analysis (pp. 27–44). Amsterdam University Press. https://doi.org/10.2307/j.ctvh4zj72.6
56. Ramírez-Castañeda V. (2020) Disadvantages in preparing and publishing scientific papers caused by the dominance of the English language in science: The case of Colombian researchers in biological sciences. PLoS ONE 15(9): e0238372. https://doi.org/10.1371/journal.pone.0238372
57. Rosenberger, K. (2019). Designing digital badging programs: Findings from an interview-based study with instructional designers. TechTrends, 63(4), 477-484. https://doi.org/10.1007/s11528-018-0349-7
58. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. The American Psychologist, 55, 68–78. https://doi.org/10.1037/0003-066X.55.1.68
59. Sek, Y., McKay, E., & Deng, H. (2015). The effect of learning preferences on learners’ motivations: Towards an ARCS motivational design in open learner models. 2015 IEEE Conference on E-Learning, e-Management and e-Services (IC3e), 52–57. https://doi.org/10.1109/IC3e.2015.7403486
60. Shah, S. R. U., Mahmood, K., & Hameed, A. (2017). Review of Google scholar, Web of Science, and Scopus search results: The case of inclusive education research. Library Philosophy and Practice. http://digitalcommons.unl.edu/libphilprac/1544
61. Stockdale, J., Hughes, C., Stronge, S., & Birch, M. (2019). Motivating midwifery students to digitalise their enquiry-based learning experiences: An evaluative case study. Studies in Educational Evaluation, 60, 59–65. https://doi.org/10.1016/j.stueduc.2018.11.006
62. Thompson, L., & Carrier, H. S. (2016). Scalable equals asynchronous and asynchronous equals boring. Or does it? Internet Reference Services Quarterly, 21(3–4), 81–92. https://doi.org/10.1080/10875301.2016.1241202
63. UNESCO (2021). Education: From disruption to recovery. https://en.unesco.org/covid19/educationresponse#schoolclosures
64. Vagianou, M., Paraskeva, F., Karampa, V., & Bouta, H. (2021, July). Applying motivational techniques and gamified elements on instructional design models for effective instruction in secondary education. In L. Uden & D. Liberona (Eds.), International workshop on learning technology for education challenges (pp. 111-123). Springer.
65. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Towards a unified view. MIS Quarterly, 27, 425–478. https://edtechbooks.org/about:blank
66. Walsh, B. A., Woodliff, T. A., Lucero, J., Harvey, S., Burnham, M. M., Bowser, T. L., Aguirre, M., & Zeh, D. W. (2021). Historically underrepresented graduate students’ experiences during the COVID-19 pandemic. Family Relations, 70(4), 955–972. https://doi.org/10.1111/fare.12574
67. Wan, L. A., & Gregory, S. (2018). Digital tools to support motivation of music students for instrumental practice. Journal of Music, Technology and Education, 11(1), 37–64. https://doi.org/10.1386/jmte.11.1.37_1
68. Yurdaarmagan, B., Melek, C. G., Merdenyan, B., Cikrikcili, O., Salman, Y. B., & Cheng, H. I. (2015). The effects of digital game-based learning on performance and motivation for high school students. ICIC Express Letters, 9(5), 1465-1469.
69. Zaccoletti, S., Camacho, A., Correia, N., Aguiar, C., Mason, L., Alves, R. A., & Daniel, J. R. (2020). Parents’ perceptions of student academic motivation during the COVID-19 lockdown: A cross-country comparison. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.592670