AI Tutoring with Khanmigo in a Two-Year School Experiment

A two-year cluster-randomized trial in 18 Tennessee middle schools tested Khan Academy with Khanmigo, an AI tutor configured to coach rather than supply answers, during existing daily remedial math sessions. Random assignment to the AI-enhanced practice produced modest achievement gains: roughly 1.3 national percentile ranks per term (about 0.06-0.08 standard deviations over a school year), with the implied effect of a full year of active participation near 0.14 standard deviations. Those gains are comparable to results from Khan Academy practice without AI, suggesting limited added impact from the tutoring agent as deployed.

Detailed usage data point to engagement, not technology quality, as the binding constraint. Although 96 percent of students tried Khanmigo at least once, the median student messaged it on only about one-third of practice days and reached out in just 17 percent of exercise sessions where they made mistakes. Most student messages were brief answers or clicks on suggested prompts rather than substantive mathematical dialogue. The study concludes that realizing AI tutoring’s promise will require strategies to increase and deepen student use, not merely providing access to a coach-mode tutor.