Empirical Design in Reinforcement Learning for Physical AI
Abstract
It is no secret that reinforcement learning algorithms are hard to apply in the real world, but why? The answer is not obvious, so we start with a concrete example: on a realistic industrial simulator, multiple widely used algorithms perform worse than a simple baseline. Attendees then dig into a hands-on session to find out why. Next, we step back and discuss what makes the real world hard. These challenges rarely show up in standard benchmarks, and our current algorithms were never designed for them. The rest of the tutorial is about bringing those concerns into the lab. We cover empirical methodology, new benchmarks, and the many ways researchers accidentally cheat. We also take a hard look at common bad practice in RL: too few random seeds, untuned baselines, and overconfident claims built on incorrect confidence intervals. A second hands-on session shows how easily these mistakes can mislead us about what will work in deployment. We do not offer a single checklist for running RL experiments. Instead, attendees leave with three key messages and practical tools they can use in their own work.