With the rapid advancement of generative AI, prompts—natural language expressions used to elicit specific outputs—have become a critical component of effective human–AI interaction. Originally understood as simple command inputs, prompts have evolved into a form of intent expression requiring strategic articulation, reasoning, and refinement. Consequently, prompt literacy has emerged as an essential competency, broadly encompassing learners’ ability to generate, revise, and evaluate prompts to obtain meaningful AI-assisted outcomes.
Existing research has identified various dimensions of prompt literacy, including knowledge, processes, and attitudes necessary for constructing effective prompts, and has proposed frameworks to guide prompt usage, evaluation, and design (Kim, 2023; Kim, 2024; Gattupalli et al., 2023; Hwang et al., 2023; Maloy & Gattupalli, 2024). However, relatively few studies have examined how learners actually use prompts while engaging in authentic, AI-supported problem-solving tasks (Abdelghani et al., 2025). Empirical understanding of learners’ real prompt behaviors remains limited, particularly regarding how these behaviors may differ by performance level.
To address this gap, the present study analyzed learners’ prompts during design thinking using a generative AI chatbot, focusing on identifying prompt types and sequential patterns associated with higher performance. This work contributes to prompt literacy education by illustrating how learners' strategic and metacognitive engagement with AI manifests in authentic tasks.
We adopted design thinking as the primary framework for learners’ problem-solving. Participants consisted of 20 female undergraduate and graduate students, all of whom had prior experience with both design thinking and generative AI. Learners conducted their projects following the Stanford d.school model. To identify patterns in learners’ prompt use, we developed a coding scheme through content and qualitative analysis, classifying prompts into four overarching categories—information acquisition, output-oriented interaction, responses to AI’s answers, and others—and 11 specific prompt types (see Table 1).
We then analyzed the problem-solving reports submitted by the learners and categorized them into high- and low-achievement groups based on the median score. Sequential pattern analysis was applied to examine the temporally ordered prompt sequences used by each group. Sequential pattern analysis, a data-mining method used to detect frequently occurring ordered patterns (Agrawal & Srikant, 1995), enabled us to identify the prompt types and interaction patterns associated with higher academic achievement.
Table 1
Coding Scheme
Category | Prompt Type | Code | Definition |
Information Acquisition | Knowledge of Design thinking | KDT | Requests for explanations or knowledge about design thinking, including goals, stages, and outputs. |
Information Request | IR | Requests for concepts, contextual information, or data needed to investigate problems or generate outputs | |
Output-Oriented Interaction | Training Output | TO | Providing the chatbot with updates on the learner’s progress or outputs. |
Output Request | OR | Requests for generating outputs at each stage of design thinking. | |
Output Feedback | OF | Requests for feedback or evaluation on outputs generated during the process. | |
Condition | CN | Specification of conditions or response styles for output generation. | |
Response to AI’s Answer | Acceptance | AC | Accepting the AI’s responses. |
Reject | RJ | Rejecting or disagreeing with AI’s responses. | |
Reflection | RF | Prompts for self-reflection or evaluation during project execution. | |
Others | Social Discourse | SD | Prompts unrelated to the project, including casual or social interaction. |
Other | OH | Prompts that do not fit any of the above categories. |
Analysis of the distribution of prompt types between the high- and low-performance groups revealed notable differences in their interaction with generative AI. The high-performing group frequently used prompts coded as Output Request, Acceptance, Output Feedback, and Reject. The frequent use of OR prompts indicates that high performers tended to request concrete outputs, positioning AI as a collaborative partner. Moreover, their frequent use of AC, OF, and RJ prompts reflects a critical and self-regulatory engagement with the AI’s responses. In contrast, the low-performance group employed Condition, Reflection, and Training Output prompts more frequently. The frequent use of CN and RF prompts suggests a strong focus on setting conditions and reaffirming their own ideas. Additionally, their greater reliance on prompts suggests repetitive generation and refinement of learner- or AI-generated outputs. Overall, these distributions demonstrate that the high-performing learners perceived the AI as a cognitive partner, engaging in meaning negotiation and cognitive refinement throughout the problem-solving.
Figure 1
The Heatmap of Sequential Pattern Analysis
The result of sequential pattern analysis revealed significant distinctions in the prompt patterns of the two groups (see Figure 1). To examine these patterns in greater detail, we focused on the top five support values. Firstly, the high-performing group exhibited repetitive and exploratory patterns of OR prompts, with a strong support value for the OR to OR pattern, indicating an iterative deepening of output generation. They also integrated patterns of CN, IR, and OR, such as CN to OR, CN to IR, IR to CN, and OR to CN patterns. These patterns suggest that high performers strategically linked prompts to maintain contextual relevance and coherence in their results. Additionally, high performers consistently displayed patterns of IR to IR and KDT to KDT, reflecting their efforts to broaden contextual understanding and apply design thinking knowledge to ensure methodological alignment. Conversely, the low-performance group exhibited output-oriented prompt patterns, such as CN to OR, OR to OR, and IR to OR. Additionally, they demonstrated conditional-output cycles, including patterns CN to OR and OR to CN, which indicate reactive exchanges between conditions and outputs that were used to control the style, format, and scope of AI’s responses. The low performers also displayed repetitive inquiry and localized iteration, characterized by IR to IR and OR to OR patterns, suggesting sustained engagement and exploratory persistence in their interactions with the AI. Lastly, they exhibited OR to KDT, KDT to CN, and KDT to IR patterns, though these appeared with relatively low support values, indicating less frequent occurrences in their prompt sequences.
To summarize, both groups shared common prompt patterns—OR to OR, CN to OR, IR to IR, and OR to CN—which represent fundamental cycles of output repetition and occasional information utilization. Nevertheless, the high-performance group’s patterns reflected a more interconnected and cyclical process linking conceptualization, information requests, and output generation, facilitating deeper cognitive refinement. In contrast, the low-performance group exhibited more surface-level and output-driven patterns with limited evidence of knowledge transformation.
From this analysis, the high-performance group exhibited three key characteristics: (1) active and extensive prompt use, (2) perceiving AI as a collaborative cognitive partner, and (3) strategic and metacognitive integration of prompts. The high performers demonstrated higher support values across major prompt patterns, indicating consistent, intentional, and purposeful engagement with generative AI. Moreover, they positioned AI as a collaborator, engaging in critical reflection and regulatory interactions to refine and evaluate AI-generated responses. These patterns suggest that the high-performance group produced more coherent, refined, and contextually grounded outcomes through deep reasoning and metacognitive engagement.
This study emphasizes that prompt literacy extends beyond linguistic proficiency, encompassing procedural, strategic, and metacognitive competencies that are essential for effective human–AI collaboration. The development of prompt literacy relies on the cultivation of iterative prompting, a process that prioritizes continuous refinement, adaptive questioning, and responsive regulation. Furthermore, it entails fostering critical collaboration with AI, wherein learners actively evaluate and regulate AI-generated responses. By nurturing these dimensions, educators can facilitate a transformation in learners’ use of AI—shifting from surface-level to context-aware, reflective, and creative problem solving.
In conclusion, prompt literacy should be regarded as an essential competency for enabling metacognitively grounded, purposeful, and meaningful engagement with generative AI in educational contexts.
EdTech Archives