Collaborative learning, particularly in an online environment, has been widely adopted across different educational settings (Zheng et al., 2023). The quality of online collaborative learning can be influenced by different factors. Among these factors, teachers’ facilitation often impacts the dynamics of the collaborative environment, which is associated with the quality of collaboration. To evaluate the knowledge construction processes, the researchers applied the interaction analysis framework (Gunawardena et al., 1997; Ye et al., 2022) and coded each comment in a conversation into one of five scaffolding phases. By analyzing the transitions among the scaffolding phases, this proposal aims to address the following two research questions:
RQ1: How to apply the interaction analysis framework in performing transitional probability analysis on the math discussion exchange data?
RQ2: What are the different transitional patterns between the groups with or without teachers' facilitation in the math discussion forum?
The data were collected from an online collaborative mathematics learning platform, Math Nation (2024), which includes 13,807 conversations related to ninth-grade algebra involving 21,262 students and 143 teachers. These conversations were divided into two groups: 9,385 conversations with teacher facilitation and 4,422 without teacher facilitation. Typically, a conversation begins with a question posted by a student, after which teachers and peers voluntarily engage in the discussion thread to address the inquiry.
Figure 1
A Diagram to Show the Workflow of LLM-Based Coding
For RQ1, the researchers leveraged the GPT API (OpenAI, 2024) to facilitate the coding process by following a prompt design framework consisting of four components: role setting, explicit instruction, a few-shot learning examples, and the task (Tabatabaian, 2024). Each comment within a conversation was coded into one of the five scaffolding phases shown in Figure 1.
The five scaffolding phases are:
Phase 1: Sharing/comparing of information
Phase 2: Discovery and exploration of dissonance
Phase 3: Negotiation of meaning and/or co-construction of knowledge
Phase 4: Testing and modification of proposed synthesis or co-construction
Phase 5: Phrasing of agreement, statement(s), and applications of the newly constructed meaning.
After coding the conversation (see Figure 1), the researchers computed transitional probabilities among scaffolding phases for the groups with and without teachers’ facilitation using a bootstrapping process (randomly sampling 100 conversations 20 times):
For RQ2, the Shapiro-Wilk test is first conducted for each group to examine normality. If both distributions met the assumption of normality, an independent t-test was applied to determine whether a statistically significant difference existed between the two groups. Otherwise, the Mann-Whitney U test is used to compare the distributions without assuming normality.
For RQ1, the LLM-coded results were triangulated by the researchers with a high inter-rater reliability (>90% rate of agreement). Based on the coded results, the 25-phase transitional probabilities for the two groups (with and without teachers’ facilitation) are computed.
For RQ2, among the 25 phase transitional probabilities, 9 transitions demonstrated a significant difference between the two groups (with and without teachers’ facilitation), as shown in the following diagram:
Figure 2
Transitional Probabilities with Significant Differences between Two Groups.
On one hand, teacher facilitation increases the transitional probability from Phase 1 to Phase 3, from Phase 3 to Phase 1, and from Phase 5 to Phase 1. These patterns suggest that teacher facilitation encourages students to more actively engage in a deeper iterative meaning-making and revisit foundational information even after reaching a resolution, promoting ongoing reflection and refinement.
On the other hand, teacher facilitation decreases the transitional probability from Phase 1 to Phase 2, and 3 to 5 Phase 2 to Phase 2, from Phase 3 to Phase 4, and from Phase 5 to Phase 5. These decreased transitional probability to Phase 5 suggests that teachers prevent discussions from skipping crucial intermediate phases of knowledge co-construction and validation. For example, teachers frequently redirect students to relevant materials (e.g., “Sections 4, Topics 3 and 4 will help”) and prompt them to revisit prior concepts before advancing the discussion.
Overall, in teacher-facilitated discussions, co-constructed ideas are subjected to more iterative cycles rather than being quickly synthesized.
The results imply that teacher facilitation helps prevent premature closure in discussions. In many online learning environments, students may be inclined to quickly reach consensus (Phase 5) without fully engaging in deeper co-construction of knowledge (Phases 2–4). By strategically intervening, teachers can slow this process, ensuring that discussions remain dynamic and iterative rather than converging too early on a final answer. The findings inform instructional design by emphasizing the need for balanced teacher intervention.
Note. Given the required word limit, not all analyses are included here.
EdTech Archives