Dartmouth's AI-graded quiz platform lifts stats final scores 0.71–1.30 SD

Phosphor, a web learning platform built by Dartmouth researcher Jonah Bard, embeds LLM-graded practice quizzes directly into course readings so that active recall becomes a structural part of the material rather than an add-on. In a Spring 2026 pilot across three sections of introductory statistics (MATH 010, starting at 151 students), completing the full set of optional, ungraded content correlated with final-exam gains of 1.30 SD unadjusted, or 0.71 SD after controlling for prior midterm performance — a large effect by observational education-research standards. Adoption was the headline surprise: 90.2% of enrolled students used the tool despite it being entirely optional, versus baseline textbook-reading compliance estimated at 10–15% for the course.

The most consequential finding concerns question format. Constructed-response questions graded by Claude Sonnet 4.6 against instructor-written rubrics drove measurable learning, while multiple-choice-only quizzes did not. When Module 2 was switched to MCQ-only (after students complained the auto-grader was too rigid), the dosage-to-performance relationship disappeared; it returned when constructed-response items were reintroduced in Module 3. Cumulative ‘Module Review’ quizzes were the single strongest lever — students who passed all three scored 7.1 points higher on the final (d=0.66). A RAG chat assistant bundled into the platform was almost ignored (72 total queries), with students saying general-purpose LLMs were faster and the built-in content was sufficient.

Bard frames the work against evidence that unguarded AI access can backfire — citing an RCT where unrestricted GPT-4 use cut later performance by 17% once the tool was removed — arguing that AI helps most when embedded into content delivery through graded active recall rather than offered as a freestanding chatbot. The paper is candid about its limits: an observational pilot at a single selective institution, no randomized controls, and self-selection as the core confound, since more motivated students both engage more and score higher. The 0.71–1.30 SD figure is explicitly a lower-to-upper bound, not a point estimate. Planned next steps include randomized assignment to quiz formats and replication across other gateway courses and institutions.