Timezone: »
Intelligent agents are continuously faced with the challenge of optimizing a policy based on what they can observe (see) and which actions they can take (do) in the environment where they are deployed. Most policies can be parametrized in terms of these two dimensions, i.e., as a function of what can be seen and done given a certain situation, which we call a mixed policy. In this paper, we investigate several properties of the class of mixed policies and provide an efficient and effective characterization, including optimality and non-redundancy. Specifically, we introduce a graphical criterion to identify unnecessary contexts for a set of actions, leading to a natural characterization of non-redundancy of mixed policies. We then derive sufficient conditions under which one strategy can dominate the other with respect to their maximum achievable expected rewards (optimality). This characterization leads to a fundamental understanding of the space of mixed policies and a possible refinement of the agent's strategy so that it converges to the optimum faster and more robustly. One surprising result of the causal characterization is that the agent following a more standard approach --- intervening on all intervenable variables and observing all available contexts --- may be hurting itself, and will never achieve an optimal performance.
Author Information
Sanghack Lee (Columbia University)
Elias Bareinboim (Columbia University)
More from the Same Authors
-
2021 Spotlight: Double Machine Learning Density Estimation for Local Treatment Effects with Instruments »
Yonghan Jung · Jin Tian · Elias Bareinboim -
2021 : Partition-based Local Independence Discovery »
Inwoo Hwang · Byoung-Tak Zhang · Sanghack Lee -
2022 Poster: Causal Identification under Markov equivalence: Calculus, Algorithm, and Completeness »
Amin Jaber · Adele Ribeiro · Jiji Zhang · Elias Bareinboim -
2022 Poster: Online Reinforcement Learning for Mixed Policy Scopes »
Junzhe Zhang · Elias Bareinboim -
2022 Poster: Finding and Listing Front-door Adjustment Sets »
Hyunchai Jeong · Jin Tian · Elias Bareinboim -
2021 : Panel Discussion »
Elias Bareinboim · Mark van der Laan · Claire Vernade -
2021 : TBD (Elias Bareibnboim) »
Elias Bareinboim -
2021 : Invited Talk: Causality and Fairness »
Elias Bareinboim -
2021 Workshop: Causal Inference & Machine Learning: Why now? »
Elias Bareinboim · Bernhard Schölkopf · Terrence Sejnowski · Yoshua Bengio · Judea Pearl -
2021 Oral: Sequential Causal Imitation Learning with Unobserved Confounders »
Daniel Kumor · Junzhe Zhang · Elias Bareinboim -
2021 Poster: Causal Identification with Matrix Equations »
Sanghack Lee · Elias Bareinboim -
2021 Poster: Nested Counterfactual Identification from Arbitrary Surrogate Experiments »
Juan Correa · Sanghack Lee · Elias Bareinboim -
2021 Poster: Sequential Causal Imitation Learning with Unobserved Confounders »
Daniel Kumor · Junzhe Zhang · Elias Bareinboim -
2021 Poster: The Causal-Neural Connection: Expressiveness, Learnability, and Inference »
Kevin Xia · Kai-Zhan Lee · Yoshua Bengio · Elias Bareinboim -
2021 Poster: Double Machine Learning Density Estimation for Local Treatment Effects with Instruments »
Yonghan Jung · Jin Tian · Elias Bareinboim -
2021 Oral: Causal Identification with Matrix Equations »
Sanghack Lee · Elias Bareinboim -
2020 Workshop: Causal Discovery and Causality-Inspired Machine Learning »
Biwei Huang · Sara Magliacane · Kun Zhang · Danielle Belgrave · Elias Bareinboim · Daniel Malinsky · Thomas Richardson · Christopher Meek · Peter Spirtes · Bernhard Schölkopf -
2020 Poster: Causal Discovery from Soft Interventions with Unknown Targets: Characterization and Learning »
Amin Jaber · Murat Kocaoglu · Karthikeyan Shanmugam · Elias Bareinboim -
2020 Poster: Causal Imitation Learning With Unobserved Confounders »
Junzhe Zhang · Daniel Kumor · Elias Bareinboim -
2020 Poster: General Transportability of Soft Interventions: Completeness Results »
Juan Correa · Elias Bareinboim -
2020 Poster: Learning Causal Effects via Weighted Empirical Risk Minimization »
Yonghan Jung · Jin Tian · Elias Bareinboim -
2020 Oral: Causal Imitation Learning With Unobserved Confounders »
Junzhe Zhang · Daniel Kumor · Elias Bareinboim -
2018 Poster: Structural Causal Bandits: Where to Intervene? »
Sanghack Lee · Elias Bareinboim