An Efficient Open World Benchmark for Multi-Agent Reinforcement Learning

Many challenges remain before AI agents can be deployed in real-world environments. However, one virtue of such environments is that they are inherently multi-agent, and contain experienced expert agents (like humans) who demonstrate useful behaviors. Such behaviors can help an AI agent generalize and apply to new use-cases and scenarios. While this type of social learning ability could improve generalization and human-AI interaction, it is currently difficult to study due to the lack of open-ended multi-agent environments. In this work, we present an environment in which multiple self-interested agents can pursue complex, independent goals. We have developed the first multi-agent version of the Craftax benchmark. Built upon the Craftax-Classic environment in Jax, this extension supports efficient multi-agent training on accelerators. Our experiments reveal that using a 4-agent LSTM model on an Nvidia T4 GPU can complete 100 million steps in approximately one hour. This environment will enable research into improving the social learning capabilities of AI agents in open-ended multi-agent settings, potentially enabling better generalization and faster learning through observing other agents.