It Takes a Village: Open Questions in Evaluating LLM Safety for Children

Safety evaluation for child-facing LLM-based systems has largely focused on modeling the child. Recent benchmarks increasingly account for developmental stage, age cues, psychological characteristics and cultural context. Yet in deployment, the child is not the only one interacting with the system. Caregivers, teachers, peers, and third parties may engage it with distinct goals, permissions, knowledge, and strategies, which can affect the child's safety. Evaluation therefore cannot rest on the child-AI dyad alone. Although some recent benchmarks extend evaluation beyond the child, these efforts remain fragmented across domains and use cases. What is still missing is a systematic methodology for safety evaluation across the different actors and relationships surrounding the child. We frame this as a \emph{multi-actor evaluation} problem. Under this framing, the object of evaluation shifts from individual actor-AI interactions to the system’s behavior across them. We decompose this problem into methodological challenges and open questions, and outline directions for addressing them.