Studying psychomotor skills involves an intricate interplay of mental and motor processing that is critical in fields such as exercise science, sports, physical therapy, procedural tasks, etc. To fully understand how psychomotor skills are acquired, the researcher must be able to observe not only visible physical behavior but also uncover the invisible mental processes that guide and support one’s physical behaviors or actions. While the motor processing (i.e., physical behaviors) is overtly observable, the mental processing (i.e., thinking) remains intangible, as it is internally processed. Necessitating the use of specific techniques such as ‘Think-aloud’ to capture and externalize the mental processing involved during PSL, enabling a comprehensive analysis of the interplay between thought and physical actions. The purpose of this study is to evaluate the effectiveness of the TAP training, focusing on training design, instructional materials, resources used, feedback mechanism, facilitator preparedness, whether training facilitated participants’ thinking and verbalization, and participant engagement during think-aloud tasks.
Thinking involves mental processing, primarily a sequential process wherein one thought yields to another (Ericsson & Crutcher, 1991). The term ‘think-aloud’ is defined as the concurrent verbalization of thoughts during task performance (Ericsson & Simon, 1993), widely recognized as a fundamental approach to uncover mental processing (Crutcher, 1994; Eccles & Arsal, 2017; Nielsen et al., 2002; Payne, 1994; Shapiro, 2014; Wilson, 1994). Think-aloud protocol (TAP) reveals a spontaneous stream of consciousness (Garg et al., 2025), participants’ voluntary reporting of thoughts as they engaged in a specific task, such as solving puzzles or performing physical movements (Cowan, 2017; Ericsson &Simon, 1993; Fox et al., 2011). This approach provides a window into how individuals process, monitor, shift, and regulate their thinking (Ericsson, 2017). Several psychomotor skill learning studies, such as a golf-putting task (Calmerio & Tenenbaum, 2011), a long-distance runner (Samson et al., 2015), and cycling (Whitehead et al., 2018), have employed think-aloud, further suggesting this approach as one of the most effective ways to assess higher-order thinking requiring deeper levels of mental processing (Gass & Mackey, 2000, p.13; Olson et al., 2018). Besides uncovering the sequential flow of thoughts during task execution, the verbalization uncovers planning, goal-setting, reasoning, error detection and correction, decision-making, etc., which all contribute to the higher levels of performance (Ericsson & Lehman, 1996).
The training design plays a critical role in determining its effectiveness. The training was conducted individually for each participant. The design encompasses a series of simple to complex puzzle tasks and psychomotor tasks (15 distinct steps of upper body right arm and head movements), progressively building on complexity, to ensure that learners can articulate their thought process while smoothly carrying out given tasks.
The pre-task briefing commenced with an MS PowerPoint presentation, introducing the ‘Think-aloud’ concept with everyday examples, followed by outlining theoretical foundations. Followed by two video demonstrations: video 1: Explanation and Demo of think-aloud [link] and video 2:Basketball free throw using think-aloud [link]. The precondition was to have participants maintain an uninterrupted focus on completing the presented puzzle and psychomotor performance tasks. Hence, participants were provided with explicit instructions to focus on the tasks throughout. A ‘KEEP TALKING’ sign was flashed to remind participants in case they were found to have stopped verbalizing their thoughts… reducing the researcher’s use of speech prompts that may interfere with participants' thought process. The training was structured to last for 1 hour and 30 minutes, with each task allocated for 15 minutes. During each task, participants were required to verbalize their thought process in real-time, followed by writing a reflection of their experience, and recording any thoughts they were unable to articulate while they carried out the task. Sufficient time was allocated for repeated practice, progressing from simple to complex puzzle tasks.
Training materials such as task instructions, visual aids used were MS PowerPoint presentations, video demonstration of think-aloud, and guided practice provided structure, uniformity, and clarity during training. The puzzle activity 1 consisted of blocked plastic of two different shapes – trapezoid and triangle puzzle pieces in two colors – black and white that were used to create a geometric shape pattern jigsaw in 3D. The puzzle activity 2 involved the Coogan wooden tangram puzzle featuring 16 blocks in different hexagon shapes in 6 different colors. Followed by a psychomotor task involving 15 distinct steps of upper-body right-arm and head movements, presented using MS PowerPoint with visual, written, and audio instructions for each step.
This study adopted a combination of a convenient and purposeful sample of participants recruited from graduate students enrolled in education or related fields at a northeastern university. The participants included varying levels of familiarity with psychomotor tasks but no prior exposure to the specific puzzle or psychomotor task chosen for the study. A total of 5 participants were recruited to ensure sufficient data for qualitative analysis. The study was conducted in a controlled environment equipped with the necessary audio-visual recording equipment. All 5 participants went through the same structured TAP training process, utilized the same instructional resources and materials to ensure standardized implementation of the TAP training.
The researcher acts as an observer and facilitator, providing guidance during the pre-task orientation and ensuring participants adhere to the think-aloud protocol. During task execution, the researcher maintains a non-intrusive presence to avoid influencing participants’ thoughts or actions. The researcher will take note of behavioral and non-verbal cues.
An observation checklist was used to track various aspects of participant engagement while they carried out tasks, any other verbal or non-verbal facial gestures or expressions noticed, along with their reflections after each task. The reflection task allowed participants to make a note of any thoughts they could not verbalize while carrying out the task, level of engagement in hands-on activities. These materials provided valuable data for evaluating the overall success of the training and for identifying areas of improvement or adjustments.
A de-brief interview was conducted, including 12 questions related to participants’ overall experience, task-specific experience, cognition-related experience, and emotion-specific experiences.
The data collection was structured to capture the participant’s voluntary verbalizations, the researcher’s observation during puzzle and psychomotor activities, the participant’s written reflection after each task, and responses to debrief interview questions. These interview questions covered various aspects of the ‘think-aloud’ training, including general experience, task-specific insights, mental processing, and emotional reactions. All 5 participants went through the same structured TAP training process & use of instructional resources and materials to ensure consistent execution of the TAP training. These materials were used to assess both the TAP training implementation process and the learning outcomes associated with the TAP training. This approach implemented combined formative and summative evaluation elements to identify effectiveness, instructional implications, and participant experiences.
The verbalization and debrief interviews were transcribed verbatim to capture the participants’ thought processes and feedback accurately. The transcriptions were coded using a thematic analysis approach. Codes were assigned to segments of the test that represented specific themes or patterns related to the think-aloud process during the tasks, capturing mental processing (i.e., planning, reasoning, problem-solving, questions, decision-making, error detection, and correction), emotional responses, and verbal and non-verbal cues. Coding for de-brief interviews captured participants’ overall experience, task-specific experience, cognition-related experience, and emotion-specific experiences. The coded data were analyzed to identify common themes and insights. The analysis focused on understanding how participants engaged with the think-aloud training, the challenges they faced, and the effectiveness of the TAP training.
The analysis of ‘think-aloud’ data from participants performing various puzzles and psychomotor tasks revealed significant insights into participants’ mental processes as they carried out each task. Participants demonstrated higher levels of mental processing during puzzle tasks incorporating goal setting, strategy formulation, and the reasoning process. In contrast, during the psychomotor tasks, participants demonstrated minimal mental engagement with little evidence of higher-order thinking. Re-directing the re-designing instructions specifically for psychomotor tasks, including clear goals and objectives, with a simple animation of each step, and minimal textual instructions.
EdTech Archives