Assessing the black box of feedback neglect in a digital educational game for elementary school

Background Previous research shows that critical constructive feedback, that scaffolds students to improve on tasks, often remains untapped. The paper’s aim is to illuminate at what stages students provided with such feedback drop out of feedback processing. Methods In our model, students can drop out at any of five stages of feedback processing: (1) noticing, (2) decoding, (3) making sense, (4) acting upon, and (5) using feedback to make progress. Eye-tracking was used to measure noticing and decoding of feedback. Behavioral data-logging tracked students’ use of feedback and potential progress. Three feedback signaling conditions were experimentally compared: a pedagogical agent, an animated arrow, and no signaling (control condition). Findings Students dropped out at each stage and few made it past the final stage. The agent condition led to significantly less feedback neglect at the two first stages, suggesting that students who are not initially inclined to notice and read feedback text can be influenced into doing so. Contribution The study provides a model and method to build more fine-grained knowledge of students’ (non)processing of feedback. More knowledge on at what stages students drop out and why can inform methods to counteract drop out and scaffold more productive and fruitful responses.

"In methodological terms, the study contributed with evidence that CCF-neglect can be studied in a more fine-grained way than it has been to date. It demonstrated that it was possible to measure levels of neglect at intermediate stages of CCF-processing, namely noticing, decoding, and acting on the CCF. The study further shows that it was possible to do so in an ecologically valid setting close to the students’ ordinary classroom learning environment and with a game that, like many present day educational games, is rich in content and visual dynamics and offers large degrees of freedom of choice and navigation on the part of students. In terms of intervention, the study confirms the value of having an agent gaze and point toward the CCF that students are intended to read, showing that students were likelier both to notice and to decode (read) the text." (Authors, in "Discussion")