For this study, researchers examined an online Instructional Design class to determine whether instruction on using artificial intelligence (A.I.) in Instructional Design practices affected students' grades and perceptions of performance in the course. Studies that integrated virtual reality (VR) into classes observed increased sense of belonging, greater collaboration among peers, increased learner engagement, greater motivation, and greater support for differentiated learning styles (DELTA, n.d.). The question researchers had for this study was whether integrating A.I. would produce similar effects on student performance as VR integration.
Researchers examined the effects of A.I. on student performance by adding instruction on A.I. design techniques to class lectures. The study took place in a month-long, accelerated online course in the Instructional Design and Technology Master’s in Science program at Full Sail University, which repeats each month with a new student cohort. The full name of the course is Digital Media and Learning Applications, and its course code is IDT574. In the first week of the course, students learn about the various types of learning applications used in Instructional Design and the role they play within different learning environments. In the second week, students learn about best practices and the benefits of using audio in instructional design. In the third week, students learn about the best practices and benefits of using video when designing instructional materials. Finally, in the fourth week of the course, students learn about the best practices and benefits of using interactive media when designing instructional materials.
In the previous iterations of the IDT574 course, prior to July 2024 and before the incorporation of A.I., students explored how audio, video, and interactive media contributed to teaching and training in a learning environment but did not learn about strategies for using A.I. to create these pieces of media. Starting in July 2024, the course gradually but intermittently introduced strategies for using A.I. to create training materials, such as instructional audio. For example, in the second week of the course, the instructor demonstrated in the online class lecture how students could use ChatGPT and Suno.ai to create music based on any topic of their choosing. The course did not require students to use A.I. but allowed them to utilize it for their assignments. The full integration of A.I. strategies into the course lectures did not begin until May 2025, when the instructor began demonstrating how to use various types of Generative A.I. to complete most of the course assignments.
Researchers then specifically focused on the following research questions, which are based on the IDT574 course learning outcomes.
Does including A.I. in an online Instructional Design class help students consider designing and delivering instructional content using a variety of software applications?
Can incorporating A.I. in an online class encourage students to effectively present information and data visually and verbally?
Does using A.I. in an online class motivate students to better assess the effectiveness of media used in an instructional module?
Currently, a limited amount of research exists that examines student performance in relation to their exposure and use of artificial intelligence (A.I.) in online learning environments (Lin, et al., 2024). In one study investigating A.I.’s effect on student performance in higher education classes, the findings indicate that the use of A.I. composition tools led to improved student work (Zhang et al., 2024). However, in a study involving younger students, results indicate that “students [who] rely heavily on artificial intelligence to complete their everyday homework tasks…[impede] their learning process and skills acquisition” (Tamimi et al., 2024). Due to conflicting results from research on students’ use of A.I., researchers felt the need to revisit a previous study they conducted on mastery and performance in a previous version of the IDT574 course (Meeder & McBride, 2023).
In that study, researchers from Full Sail University examined the effects of virtual reality on the mastery and performance of Instructional Design Master's students in the online IDT574 course. For the most part, they observed that integrating VR into course assignments improved student performance. As stated in that study, many learners conveyed the benefits VR provided in their learning (Meeder & McBride, 2023). In this study, the same researchers from Full Sail University hope to add to this body of knowledge, determining A.I.’s effect on student performance (Reiser & Dempsey, 2017).
Researchers used Action Research, starting in April 2025, to reflect, plan, and have cycles of modifications to the course (Dick, 1999; Zuber-Skerritt, 2002) over a 7-month period, ending in November 2025 (see Figure 1). is The researchers chose this methodology due to the fact that this approach produced effective results when integrating VR into the IDT574 course two years ago (Meeder & McBride, 2023). With this study, researchers hoped to produce similar results by integrating A.I.
Figure 1
Timeline for the action research of the study
Researchers used a mixed-methods approach to gather data (Booren, 2024). They collected qualitative data via surveys and student-to-student communication over Zoom, and quantitative data through student grades and course evaluations. Researchers then employed the use of the constant comparative method to analyze the qualitative data to identify themes from the students’ use of A.I. to determine if the method corroborated with the differences in grades and course ratings between the classes that were exposed to A.I. from May to November 2025 and the classes that were not exposed to A.I. in their class lectures from January to June 2024 (see Table 1).
Table 1
Data sources
Type | Explanation |
Past and current grades | Grade averages of students from IDT574 courses from January 2024 to June 2024 and May 2025 to October 2025. |
Course evaluation ratings | Average of positive instructor experience ratings on a scale of 1-5 from January 2024 to June 2024 and May 2025 to October 2025. |
Surveys | Distributed at the end of each month, starting from January 2024 to June 2024 and May 2025 to October 2025, to students with open-ended questions on A.I. |
Student-student communication | Zoom sessions (recorded), from January 2025 to October 2025. |
The population of this study included Instructional Design and Technology Master students enrolled in the IDT574 course at Full Sail University. A total of 66 students participated in the study. Thirty of the students were from the January to June 2024 IDT574 classes, which did not include instruction on A.I. design techniques in class lectures. Thirty-six of the students were from the May 2025 to October 2025 IDT574 classes, which incorporated A.I. design techniques into the lectures.
A comparison of students' past and current grades before and after the addition of instruction on A.I. design techniques to class lectures showed a decrease in grades. The average grade decreased from 95.85 to 90.64. Similarly, course satisfaction ratings within the course evaluations decreased slightly. The positive instructor experience rating went from 4.92 to 4.67. The overall positive experience rating dropped from 4.8 to 4.7.
However, when researchers used the constant comparative method to analyze the qualitative data from the student-to-student communication over Zoom and open-ended surveys, they noticed that the qualitative data contradicted the quantitative data they received. After coding the qualitative data and analyzing it, themes of apprehensiveness, being scared, curiosity, motivation, engagement, helpfulness, and empowerment appeared when students discussed the effects of A.I. on their performance in the class. A few indicated that they felt apprehension when using A.I. prior to the class, but that feeling of apprehension also pushed them towards trying A.I. One student said, “I was like, okay, I need to try all these things that I've been scared to do. And that's also why I used a lot of A.I. [in this class] because I haven't before. I've always like steered away from it in the past. But yeah. And partially because of the frustrations I was having.” Another student mentioned that his past failures when using A.I. in his assignments in other classes helped him focus on using A.I. ethically in the IDT574 class. “I’m disappointed I failed my first class [in the program] due to AI misuse—especially at a tech school. But it was a hard lesson learned, and now I’m focused on using AI ethically and effectively." Other students felt that using A.I. in the IDT574 class helped them not “hate” A.I. as much and they are now more open to using it as a tool when creating instructional materials.
Further, students provided useful suggestions in their feedback that was later incorporated into the course during the Action Research process. “I do think some ethical discussion on A.I. focused around the source of A.I.'s data would be useful to add. Outside of the ‘will A.I. replace me’ discussion, I think the next biggest hinderance for A.I. use is ‘am I hurting someone's work/business/art?’.” Another student stated, “[I’d recommend] just having like the resource or just having that as maybe in the resource for the week of the list of like AI tools.” These and other suggestions from the qualitative data were added over seven months of the Action Research process to further develop the course.
Course satisfaction numbers and student grades decreased after the instructor added instruction on A.I. design techniques to the class lectures. The increase in course content may have hindered student performance. “Cognitive overload occurs when the combination of intrinsic, extraneous, and germane loads becomes overwhelming for the learner” (Medical College of Wisconsin, 2022). It is plausible that cognitive overload occurred among participants in the study when additional instruction was added to the course lectures, thereby causing a decrease in student performance. A further look into the qualitative data indicates that students found the addition of instruction in A.I. design techniques helpful. As one student said, “I really appreciate the [A.I.] challenges that [the instructor] did within each of these classes....” In other words, students found the content helpful, but the amount of content possibly caused the decrease in performance. However, researchers believe that more research and participants are needed to explore whether cognitive overload was the main cause of the decrease, and that other factors should be examined, such as the students' demographic makeup.
EdTech Archives