Timezone: »

 
Shared Hardware, Shared Baselines: An Offline Robotics Benchmark
Gaoyue Zhou · Victoria Dean

@
in Women in Machine Learning »

Ask 10 robotics researchers what the state-of-the-art learning algorithm is for manipulation, and you'll get 10 different algorithms. Why do we have these fundamental disagreements on which methods work the best? Robotics as a field struggles to compare results between labs due to the wide variety of experimental conditions. In addition, methods are sensitive to specific implementations and hyperparameters, which make it difficult for a researcher to implement competitive baselines in their own setting. Finally, the difficulties of purchasing, building, and installing hardware and software infrastructure make it challenging if not impossible for newcomers to contribute to the field.It is clear that for robotics research to advance we need a way to democratize, benchmark, and pool engineering resources. Our solution is the Offline Robotics Benchmark, which includes not only a large-scale manipulation dataset but also the hardware on which benchmark users can test their own methods now and going forward. Initial benchmark users have contributed open-source implementations, which can be used as baselines in future work without needing to rerun any of these approaches.

[ Poster]  [ Topia] 

Author Information

Gaoyue Zhou (Carnegie Mellon University)
Victoria Dean (Carnegie Mellon University)

More from the Same Authors