Mācistan: Reciprocity in Multi-Agent Reinforcement Learning as a Credit Assignment Problem

Humans and animals have complex social and cultural systems that can span large distances and times. In North America, Plains Indigenous nations like the Métis Nation had a system of reciprocity involving effigies called manitohkan with food, tools and medicine for traveling. Leaving unneeded items at these can be perceived as cooperative actions; a reciprocal sharing of resources or caching of items. Mācistan is a novel MiniGrid environment built off the key-to-door task most common in temporal credit assignment reinforcement learning literature. However, abstracting to multi-agent dimension has led to interesting behaviour and the structural credit assignment problem.