StudentSim: Training LLM-based Student Simulators

AI tutors are most useful when they adaptively respond to each student's strengths, weaknesses, and preferred kinds of guidance, but which guidance works for which student is a sparse signal, slow and costly to collect from real students. Student simulators can supply that signal as a proxy, yet existing ones cover only part of what this requires: state-tracking models fit how a student behaves but cannot digest a tutor's explanations or corrections well, while LLMs prompted to role-play a target student follow tutor's guidance fluently but do not reliably reproduce the competence of the student they imitate. We present StudentSim, a training framework that turns sparse per-student data into an individualized simulator for each student through a two-stage pipeline of pooled training followed by per-student specialization, so that the simulator both mirrors the student's own responses and updates them under tutor guidance. To measure these two abilities fairly, we build StudentSimEval, a standardized protocol spanning 60 students across chess, second-language English writing, and mathematics, drawn from public learner datasets whose de-identified student records are shared for research. It scores every method on behavioral fidelity (F↑), how well a simulator matches a student's own responses, and guidance responsiveness (R↑), how readily it updates its response under a tutor's guidance, fitting each method on the same records and scoring it on the same held-out records so results are directly comparable; we release our construction and evaluation code so others can score new methods on the same benchmark and extend it. Across all three domains, our per-student simulators outperform GPT-5.4 on both metrics. In chess, for example, StudentSim reaches F=0.51 and R=0.91, compared with 0.23 and 0.72 for GPT-5.4 and 0.45 and 0.27 for Maia2, a skill-conditioned chess move prediction model. As a proof of concept that the framework also supports AI tutor improvement, a trained StudentSim used as the reward for tutor model reinforcement learning yields a chess tutor that expert humans rate as more accurate, better-guided, and more personalized than both a no-RL baseline and a tutor RL-trained against a GPT-5.4 simulator reward.