Building close-loop simulation with realtime generative modeling

As end-to-end policies get closer to human-level performance, it becomes an increasingly bigger challenge to identify edge / failure cases in the wild and reproduce / solve them reliably. Simulation that can consistently evaluate the policy's performance against certain edge cases therefore becomes critical for model development. In addition, such a system should be sufficiently versatile to apply to any scenario, support control-in-the-loop for policy fidelity, and have low compute requirement for use in reinforcement learning. Given these requirements, we developed a real-time, close-loop simulation system based on generative modeling, and we will demonstrate how such a system allows us to reproduce real-world interventions in a generated world and solve them in the real world.