Among the most influential AI-driven innovations are large language models (LLMs), which utilize vast datasets and advanced machine learning techniques to generate interactive responses in conversational settings (Barrot, 2024). LLMs have many educational applications, including automated grading and advanced analytics, allowing educators to focus on higher-order teaching while providing more accurate and timely feedback that supports continuous learning (Herb & Lloyd, 2024; Ji et al., 2024; Lyanda et al., 2024; Miller, 2024; Mohammadi et al., 2024). From an educational perspective, generative and LLMs can enhance student engagement by offering interactive and responsive learning experiences, such as interactive chatbots in STEM education, which can promote critical thinking, collaborative learning, and a deeper understanding of complex concepts (Adiguzel et al., 2023; Liu, 2024; Palmer et al., 2023). Additional studies have highlighted the potential of LLMs to significantly enhance educational practices, especially in medical and language education. Another notable application is in assessment, where LLMs can provide sophisticated feedback mechanisms, as illustrated by Tomova's and colleagues' (2024) research on generating content-based feedback from medical multiple-choice questions, which improves the summative assessment process. This capability is further explored by Fagbohun et al., who suggest that LLMs could enhance grading practices by analyzing and grading a variety of student responses—from short answers to complex essays—thus offering detailed insights beyond simple correctness (Fagbohun et al., 2024), indicating a shift towards more targeted and nuanced evaluation systems. From an instructional design perspective, LLMs offer dynamic capabilities to learners, such as instant feedback, personalized learning experiences, and interactive environments that simulate real-world conversations.
While considerable research has focused on LLMs to support assessment, additional discourse highlights how these emerging technologies can be used to support the design and development of learning technologies, especially open-educational resources (OERs). Indeed, OERs have seen increased interest since the UNESCO forum, which highlighted the importance of free, open learning resources (Smalley, 2024). Towards further advancement of OER, Wiley and colleagues (2021) suggested that OERs can advance through their ability to retain, revise, remix, reuse, and redistribute materials in line with licensing agreements. While OERs were seen as an advancement towards equitable education, research suggests challenges related to perceived availability and quality of learning materials. To that end, AI-driven platforms that support education can address the challenges of OERs by analyzing student performance in real time and tailoring instructional materials and feedback to accommodate various learning contexts. As it relates specifically to design, artificial intelligence and machine learning can play a crucial role in the design and development of adaptive learning systems that support the diverse needs of learners.
There has been considerable emphasis on developing OER tools for learners, especially as it relates to contextualized problem-solving (Wijnia et al., 2024). However, one challenge is how to adapt content to meet the needs of a specific context in which the learning resource is adopted. To that end, AI tools position leverage LLMs to adapt educational materials and OERS in ways that align with student interests and learning contexts (Mohammadi et al., 2024; Ye et al., 2024). Indeed, LLMs are increasingly recognized as valuable for personalized learning and instructional support, with Zhui highlighting their transformative role in medical education by optimizing teaching and assessment processes and supporting ongoing education (Zhui et al., 2024). For example, positive student attitudes toward LLMs have been documented, with Biri et al. noting that undergraduate medical students view these personalized approaches as beneficial for their learning experiences, reflecting a growing acceptance of AI tools in educational settings (Biri et al., 2023). This sentiment is echoed by Sarangi et al (2024)., who found similar engagement among postgraduate students and emphasized the role of LLMs in addressing faculty shortages and enhancing critical thinking skills.(Sarangi et al., 2024). In language education, LLMs are similarly utilized to enhance teaching strategies and outcomes. Li's (2024) study demonstrates how ChatGPT and AI can facilitate lesson preparation and enhance student engagement across various language domains , underscoring the broader trend of leveraging digital tools to improve language proficiency and enrich the educational experience. The adaptability of LLMs in personalizing educational content to individual learner needs marks a significant advancement in instructional methodologies; however, a gap exists in applying this approach across other areas of design, especially for open educational resources (OERs).
While OERs are often accessible, one challenge is adapting content to meet the needs of a specific learning context and audience. As such, the goal of this case study was to demonstrate how datasets can be personalized using AI and LLMs to better adapt to cases found in open educational resources, especially for data science education. As part of our initial project efforts, we have created 230 hours of data science training materials for post-secondary students. In prior years, the project team recruited individuals from across the country for a 5-week summer program, where they learned a variety of data science concepts (e.g., Naive Bayes, K-Nearest Neighbors) and then applied these principles to an authentic problem from a community partner. These OER training materials are primarily in the form of Jupyter notebooks, which are "documents that can be read like a journal paper and run like a computer program" (O’Hara et al., 2015). These OER Jupyter notebooks are interactive online resources that combine text, mathematical equations, code, and graphs, making them an ideal medium for worked examples pedagogy(Atkinson et al., 2000), which illustrates step-by-step problem solutions.
Figure 1
Data Science Conceptual Content in Jupyter Notebooks
Figure 2
Blocks to Python Code
An important element of the materials are the datastets, which are the “raw materials of statistics and require specific properties in order to demonstrate an analytic technique well. Unfortunately, it is not easy to find small datasets that align with the right properties to demonstrate data science techniques effectively. To support data science education, this project and other data science initiatives often follow standard practices in teaching the subject(James et al., 2013) by using well-known datasets from the UCI Machine Learning Repository (https://archive.ics.uci.edu/) and similar sources. For example, the iris dataset was introduced by Fisher and is perhaps the most widely used dataset in statistics education (Unwin & Kleinman, 2021). Datasets like the iris dataset are frequently used as examples because they are small and straightforward, and instructors are familiar with their properties and the kinds of statistical phenomena they can demonstrate.
As the project progressed, the team discussed ways to adapt the data science training materials to be more accessible, notably for a grades 6-12. This presented considerable design challenges, mainly because there are no established learning standards for data science compared with those in other STEM areas. Through the lens of adaptability in OER, the redesign focused on interdisciplinary data science concepts (e.g, mathematics and computer science), integration of new media, alternative scaffolding, and others. At the outset, an important design consideration was a scenario-based strategy and data science activities to engage younger learners. Based on the problem-solving literature, the design team's goal was to develop cases that would enable the application of data science concepts while also facilitating learner engagement. After considering different options, the design team aligned the curriculum around an after-school club focused on reducing CO2 emissions in their community. These data science OER notebooks provide case studies that position learners to solve problems using data on car types, emissions levels, temperature, and other factors related to human impacts on climate change.
The original data science notebooks (see Figures 1 and 2) relied on pre-determined datasets, such as the classic Iris dataset, which potentially limited student engagement by not directly addressing their interests. As such, the goal was to customize and repurpose the data science notebooks to create more relevant and captivating learning experiences for these OERs. To support our goal, personalized datasets involve repurposing existing data to align with new thematic perspectives of the cases found within the data science OER, such as transforming vehicle attributes into emissions-related characteristics. Our process begins by connecting to a large language model (LLM) to generate a mapping from the original dataset schema to a new schema.
As we considered how to leverage LLMs to personalize datasets, we tested the idea with other topics that were rich in data, preserving structural integrity. We initially tested our LLM idea using air quality indicators, given its emphasis on data science principles. Specifically, the ChatGPT-4 was prompted as follows:
Given the dataset with the following columns: "sepal.length", "sepal.width", "petal.length", "petal.width", "variety". Imagine this dataset is related to Air Quality. Suggest 4 new numerical and one new categorical variable matching exactly the number of columns on the original dataset, ensuring the same number of levels for each categorical variable as in the original dataset. Return a Python dictionary where each original column name is a key, mapped to a tuple with three elements: A flag (is_categorical) indicating if the variable is categorical. The new column name. A vector: For numerical columns, a vector with two values: the minimum and maximum values. For categorical columns, a vector with the new levels. Do not explain, just output the dictionary, do not even mention the name of the language.
The LLM processes this prompt and outputs a Python dictionary that maps each original Iris column to its transformed counterpart, aligned with the theme of air quality. Specifically, the numerical columns, "sepal.length", "sepal.width", "petal.length", and "petal.width” are reimagined as "TrafficVolume", "AverageSpeed", "CO2Emissions", and "NoiseLevel", respectively, with new ranges specified in the dictionary. The categorical column "variety," originally containing three levels (Setosa, Versicolor, Virginica), is transformed into "TrafficCondition" with three traffic conditions: Free Flow, Heavy, or Regular, as shown in the Figure below. The data transformation then uses the LLM-provided mapping to convert the Iris dataset to the new schema. For numerical variables, the original values are rescaled linearly to fit the specified new ranges while preserving their relative distribution. For the categorical variable, "variety" is transformed into "TrafficCondition" using a dictionary-based replacement. The resulting transformed dataset retains the same number of rows (150) and columns (5) as the original Iris dataset. The transformed dataset now features new column names, with numerical values rescaled to their new ranges and the categorical values remapped to air quality levels. This method ensures the reusability of existing data, enhances analytical insights, and improves consistency and accuracy, making the process scalable across different datasets and domains.
Fig. 3
Using AI output of the Personalized Dataset from the Iris Dataset
As we redesigned key aspects of the case, we also considered how other contexts might adopt other thematic areas for their data science cases. To expand the topics and datasets available in the data science curriculum, we aligned with issues identified on Google Trends, including Arts & Entertainment, Business & Industrial, Games, and others. Within each Jupyter notebook, we now embed a Google Trends dropdown list, allowing users to select options based on their interests (see Figure 4). For example, if they select ‘Finance’, a dataset could be automatically loaded from the server's folder structure, which includes data on salaries, job types, and other relevant information. Alternatively, selecting ‘Food’ prompts the LLM to generate a dataset containing nutritional information and calories. In doing so, the emerging technology enables individuals to load datasets aligned with their selected interests, thereby supporting OER adaptation and expanding data science education to broader audiences.
Figure 4
Dropdown to Dynamically Change Iris Dataset Based On Student Interest
|
|
Research and practitioners highlight the challenges associated with designing inquiry-based instruction that leverages contextualized cases. These instructional strategies often require learners to engage in considerable self-directed learning as they represent and solve detailed ill-structured problems. In contrast to the didactic approach, in which all learners are presented with the same information via lecture, research suggests that teachers and instructional designers in inquiry-based learning must provide appropriate scaffolding, given the varying student trajectories pursued during problem-solving (Long & Koehler, 2024). Additionally, educators must often curate resources that support the information-seeking skillset needed for problem-solving. Although inquiry-based learning yields higher learning outcomes, considerable preparatory work is required from an instructional design and curriculum planning perspective (Tawfik et al., 2021).
Another important instructional design implication for this case study relates to the design of open educational resources. In the context of this study, the OERs are the Jupyter notebooks developed for the data science training materials, which are offered under an open license to enable educators to retain, reuse, revise, remix, and redistribute. While much of the discourse on OERs often focuses on their potential for accessibility, researchers have cited considerable barriers, and adoptions have usually lagged among educators because “many OER are not designed to help students learn or do not match the learning needs for a given grade level”(Mishra, 2017). This OER adaptability challenge can be especially difficult for inquiry-based learning, which often leverages cases set within a specific domain and is thus highly contextualized. For example, an OER curriculum that teaches students biology by solving a case involving pollution in a dense urban setting may have limited applicability for learners in a rural context. Based on the details of inquiry-based and preparation work, OERs that use cases have considerable design challenges to be truly accessible to a broader audience.
Generative AI, particularly through LLMs, can potentially address this issue as it enables personalized and adaptive learning experiences, enhancing instruction and assessment, and necessitating a re-evaluation of traditional learning models to incorporate human-computer interaction (HCI) elements (Akavova et al., 2023; Page et al., 2024; Vasconcelos & Santos, 2023). Personalizing data science education significantly enhances its effectiveness by tailoring learning experiences to individual students' interests and needs. A curriculum that adapts to learners' backgrounds and preferences can significantly enhance engagement and understanding, particularly for case-based instruction and other problem-solving educational strategies that require learners to solve contextualized problems. By utilizing adaptive learning technologies and authentic data projects, educators can create customized learning paths that resonate with students, fostering a deeper connection to the case materials.
Recent literature has discussed how AI and LLMs can create opportunities and pose challenges, especially in higher education. More recently, there has been discourse to use AI and LLMs as a way to make K-12 education more “accessible and understandable, [so] schoolteachers and student teachers can use this tool to enhance efficiency, effectiveness and openness in education”(van den Berg & du Plessis, 2023, p. 2). Specific examples (Kasneci et al., 2023) include both teacher tasks (such as lesson planning and assessment) and student activities (including research and writing). As the applications have grown, there has also been a call to support data personalization in education (Ayeni et al., 2024).
To clarify the nature of our open educational resource, the OERs in this study are the Jupyter notebooks containing the data science training materials. That is, the data personalization methodology supported by AI serves as a crucial design strategy to significantly enhance the adaptability and relevance of OERs. This method directly addresses a well-documented barrier to OER adoption: a lack of contextual fit with diverse learning contexts. By using LLMs to scalably transform the underlying datasets into student-selected thematic areas, our approach strengthens the OER principles of revise, remix, and reuse, allowing designers strategies to revise and adapt learning resources. While the initial proof-of-concept utilized a proprietary Large Language Model (GPT-4) to demonstrate this mapping capability, the methodology's value lies in the automated data transformation process, which could be implemented with open-source LLMs in future iterations to maintain full openness of the entire resource development pipeline.
This project addresses advancements in the strategies of designing contextually aligned materials for inquiry-based learning to accommodate a wide array of audiences, particularly as they relate to OERs. This is particularly challenging for STEM-related cases that require considerable technical expertise due to the ill-structured nature of the case (Gómez & Suárez, 2020). To accomplish personalization in data science, the described methodology leverages the capabilities of Large Language Models (LLMs) to generate mappings between original and target schemas, enabling automated and scalable data transformation. Our evaluation of the process reveals several key implications for instructional design. First, the LLM demonstrated strong performance in generating new variable names relevant to the target theme that is coherent within the new schema. The LLM effectively identified appropriate ranges for numerical variables in the new schema. This is crucial for maintaining data integrity and ensuring that the rescaled values are meaningful in the new context. Similarly, the LLM successfully generated appropriate levels for categorical variables (e.g., flower type or car type), facilitating accurate mapping and preserving semantic relationships within the data. The LLM also effectively identified appropriate ranges for numerical variables in the new schema, which is crucial for maintaining data integrity and ensuring that the rescaled values are meaningful within the new context. A key strength of this rescaling process is its ability to preserve the original data's underlying correlation structure. By maintaining the relative distribution of numerical variables during rescaling, the relationships between variables are largely retained, ensuring that analyses performed on the transformed data reflect the original data's inherent patterns.
As the design and development team used LLMs to adapt OER materials for data science education, several challenges emerged throughout the adaptation process. For example, the flexibility of this approach can lead students to draw contextual conclusions that are not sound within the chosen theme and domain. Because the underlying data relationships are preserved, spurious correlations may arise when the context shifts to different contexts. Moreover, transforming iris data (flower characteristics) into an air quality dataset might lead a student to conclude that petal size (transformed to something like "NoiseLevel") affects traffic conditions. While the numerical relationship might be maintained, the contextual link may not transfer across domains.
To mitigate the challenges of designing with AI, it is crucial to emphasize critical thinking and contextual awareness throughout the design and development process. Instructional design teams that apply this process must understand that while correlations are preserved, contextual relevance is not automatically guaranteed. Rather than accepting the output as is, instructional designers should be prepared to question its contextual validity and critically evaluate the assumptions underlying the data transformation. As instruction designers help craft the cases, they should also be encouraged to compare their findings with existing knowledge and research in the target domain to ensure their conclusions are contextually sound. While flexibility is supported, subject matter experts should provide guidance on selecting appropriate themes and ensuring that the transformed data is suitable for addressing specific research questions. Successfully integrating AI into education requires a strategic, ethical approach that maximizes its benefits, addresses challenges, and prepares all stakeholders to navigate this transformative shift in teaching and learning (Miller, 2024).
Although the Iris dataset is a common tool in data science education, it may not be as engaging as topics that reflect students' interests. To increase engagement, the instructional design team used LLMs to generate customized datasets for the OER Jupyter Notebooks, allowing students to choose from themes informed by current Google Trends, such as gaming, technology, and food. Using this approach, it is possible to transform from any original dataset and its details into one that aligns with student interests, such as food (attributes like cuisine, calories, and preparation time), games (genre, platform, and price); or sports (position, matches, and scores). The dynamic nature of this dataset personalization has considerable potential for case-based learning approaches, especially when applied to data science and analytics education. One of the cited challenges in case-based learning is the ‘breadth vs. depth’ problem; that is, the cases are often focused on a singular context, and learners may thus struggle to transfer their knowledge to other contexts(Hung, 2006; Koehler & Vilarinho-Pereira, 2023). To address this gap, we can generate customized datasets that reflect individual interests and align with problem-centered approaches in statistics, data science, business, and related domains. For instance, a student interested in pop culture might prefer to explore data on a musician, while a sports enthusiast might opt to see examples of football scores. This personalization extends beyond just the data to include real-world examples, considering the student’s interests and applying them to case studies (Prakash et al., 2024).
EdTech Archives