Timezone: »

 
Poster
Self-Predictive Universal AI
Elliot Catt · Jordi Grau-Moya · Marcus Hutter · Matthew Aitchison · Tim Genewein · Grégoire Delétang · Kevin Li · Joel Veness

Thu Dec 14 08:45 AM -- 10:45 AM (PST) @ Great Hall & Hall B1+B2 #1824
in Poster Session 5 »

Reinforcement Learning (RL) algorithms typically utilize learning and/or planning techniques to derive effective policies. The integration of both approaches has proven to be highly successful in addressing complex sequential decision-making challenges, as evidenced by algorithms such as AlphaZero and MuZero, which consolidate the planning process into a parametric search-policy. AIXI, the most potent theoretical universal agent, leverages planning through comprehensive search as its primary means to find an optimal policy. Here we define an alternative universal agent, which we call Self-AIXI, that on the contrary to AIXI, maximally exploits learning to obtain good policies. It does so by self-predicting its own stream of action data, which is generated, similarly to other TD(0) agents, by taking an action maximization step over the current on-policy (universal mixture-policy) Q-value estimates. We prove that Self-AIXI converges to AIXI, and inherits a series of properties like maximal Legg-Hutter intelligence and the self-optimizing property.

Author Information

Elliot Catt (Google Deepmind)
Jordi Grau-Moya (DeepMind)
Marcus Hutter (DeepMind)
Matthew Aitchison (Deepmind)
Tim Genewein (DeepMind)
Grégoire Delétang (DeepMind)
Kevin Li (Google DeepMind)
Joel Veness (Deepmind)

More from the Same Authors