We review the distinction between evidential and causal decision-making and the challenges that this distinction poses to the application of the expected utility principle. We furthermore establish firm connections between causality, information-theory, and game-theoretic concepts. Finally, we show how to use the aforementioned connections to construct adaptive agents that are universal over a given class of stochastic environments - such as Thompson sampling.