Timezone: »
Consider learning an imitation policy on the basis of demonstrated behavior from multiple environments, with an eye towards deployment in an unseen environment. Since the observable features from each setting may be different, directly learning individual policies as mappings from features to actions is prone to spurious correlations---and may not generalize well. However, the expert’s policy is often a function of a shared latent structure underlying those observable features that is invariant across settings. By leveraging data from multiple environments, we propose Invariant Causal Imitation Learning (ICIL), a novel technique in which we learn a feature representation that is invariant across domains, on the basis of which we learn an imitation policy that matches expert behavior. To cope with transition dynamics mismatch, ICIL learns a shared representation of causal features (for all training environments), that is disentangled from the specific representations of noise variables (for each of those environments). Moreover, to ensure that the learned policy matches the observation distribution of the expert's policy, ICIL estimates the energy of the expert's observations and uses a regularization term that minimizes the imitator policy's next state energy. Experimentally, we compare our methods against several benchmarks in control and healthcare tasks and show its effectiveness in learning imitation policies capable of generalizing to unseen environments.
Author Information
Ioana Bica (University of Oxford)
Daniel Jarrett (University of Cambridge)
Mihaela van der Schaar (University of Cambridge)
More from the Same Authors
-
2021 Spotlight: On Inductive Biases for Heterogeneous Treatment Effect Estimation »
Alicia Curth · Mihaela van der Schaar -
2021 Spotlight: Explaining Latent Representations with a Corpus of Examples »
Jonathan Crabbe · Zhaozhi Qian · Fergus Imrie · Mihaela van der Schaar -
2021 : Really Doing Great at Estimating CATE? A Critical Look at ML Benchmarking Practices in Treatment Effect Estimation »
Alicia Curth · David Svensson · Jim Weatherall · Mihaela van der Schaar -
2021 : The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation »
Alex Chan · Ioana Bica · Alihan Hüyük · Daniel Jarrett · Mihaela van der Schaar -
2022 : Dynamic outcomes-based clustering of disease progression in mechanically ventilated patients »
Emma Rocheteau · Ioana Bica · Pietro Lió · Ari Ercole -
2022 Poster: Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability »
Jonathan Crabbé · Alicia Curth · Ioana Bica · Mihaela van der Schaar -
2022 Poster: Transfer Learning on Heterogeneous Feature Spaces for Treatment Effects Estimation »
Ioana Bica · Mihaela van der Schaar -
2022 Poster: Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data »
Nabeel Seedat · Jonathan Crabbé · Ioana Bica · Mihaela van der Schaar -
2021 Poster: Explaining Latent Representations with a Corpus of Examples »
Jonathan Crabbe · Zhaozhi Qian · Fergus Imrie · Mihaela van der Schaar -
2021 Poster: Time-series Generation by Contrastive Imitation »
Daniel Jarrett · Ioana Bica · Mihaela van der Schaar -
2021 Poster: Closing the loop in medical decision support by understanding clinical decision-making: A case study on organ transplantation »
Yuchao Qin · Fergus Imrie · Alihan Hüyük · Daniel Jarrett · alexander gimson · Mihaela van der Schaar -
2021 Poster: DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks »
Boris van Breugel · Trent Kyono · Jeroen Berrevoets · Mihaela van der Schaar -
2021 Poster: MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms »
Trent Kyono · Yao Zhang · Alexis Bellot · Mihaela van der Schaar -
2021 Poster: Conformal Time-series Forecasting »
Kamile Stankeviciute · Ahmed Alaa · Mihaela van der Schaar -
2021 Poster: Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression »
Zhaozhi Qian · William Zame · Lucas Fleuren · Paul Elbers · Mihaela van der Schaar -
2021 Poster: SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data »
Alicia Curth · Changhee Lee · Mihaela van der Schaar -
2021 Poster: On Inductive Biases for Heterogeneous Treatment Effect Estimation »
Alicia Curth · Mihaela van der Schaar -
2021 Poster: SyncTwin: Treatment Effect Estimation with Longitudinal Outcomes »
Zhaozhi Qian · Yao Zhang · Ioana Bica · Angela Wood · Mihaela van der Schaar -
2021 Poster: Estimating Multi-cause Treatment Effects via Single-cause Perturbation »
Zhaozhi Qian · Alicia Curth · Mihaela van der Schaar -
2020 Workshop: Machine Learning for Health (ML4H): Advancing Healthcare for All »
Stephanie Hyland · Allen Schmaltz · Charles Onu · Ehi Nosakhare · Emily Alsentzer · Irene Y Chen · Matthew McDermott · Subhrajit Roy · Benjamin Akera · Dani Kiyasseh · Fabian Falck · Griffin Adams · Ioana Bica · Oliver J Bear Don't Walk IV · Suproteem Sarkar · Stephen Pfohl · Andrew Beam · Brett Beaulieu-Jones · Danielle Belgrave · Tristan Naumann -
2020 Poster: Strictly Batch Imitation Learning by Energy-based Distribution Matching »
Daniel Jarrett · Ioana Bica · Mihaela van der Schaar -
2020 Poster: Estimating the Effects of Continuous-valued Interventions using Generative Adversarial Networks »
Ioana Bica · James Jordon · Mihaela van der Schaar -
2020 Poster: OrganITE: Optimal transplant donor organ offering using an individual treatment effect »
Jeroen Berrevoets · James Jordon · Ioana Bica · alexander gimson · Mihaela van der Schaar -
2019 : Poster Session I »
Shuangjia Zheng · Arnav Kapur · Umar Asif · Eyal Rozenberg · Cyprien Gilet · Oleksii Sidorov · Yogesh Kumar · Tom Van Steenkiste · William Boag · David Ouyang · Paul Jaeger · Sheng Liu · Aparna Balagopalan · Deepta Rajan · Marta Skreta · Nikhil Pattisapu · Jann Goschenhofer · Viraj Prabhu · Di Jin · Laura-Jayne Gardiner · Irene Li · sriram kumar · Qiyuan Hu · Mehul Motani · Justin Lovelace · Usman Roshan · Lucy Lu Wang · Ilya Valmianski · Hyeonwoo Lee · Sunil Mallya · Elias Chaibub Neto · Jonas Kemp · Marie Charpignon · Amber Nigam · Wei-Hung Weng · Sabri Boughorbel · Alexis Bellot · Lovedeep Gondara · Haoran Zhang · Taha Bahadori · John Zech · Rulin Shao · Edward Choi · Laleh Seyyed-Kalantari · Emily Aiken · Ioana Bica · Yiqiu Shen · Kieran Chin-Cheong · Subhrajit Roy · Ioana Baldini · So Yeon Min · Dirk Deschrijver · Pekka Marttinen · Damian Pascual Ortiz · Supriya Nagesh · Niklas Rindtorff · Andriy Mulyar · Katharina Hoebel · Martha Shaka · Pierre Machart · Leon Gatys · Nathan Ng · Matthias Hüser · Devin Taylor · Dennis Barbour · Natalia Martinez · Clara McCreery · Benjamin Eyre · Vivek Natarajan · Ren Yi · Ruibin Ma · Chirag Nagpal · Nan Du · Chufan Gao · Anup Tuladhar · Sam Shleifer · Jason Ren · Pouria Mashouri · Ming Yang Lu · Farideh Bagherzadeh-Khiabani · Olivia Choudhury · Maithra Raghu · Scott Fleming · Mika Jain · GUO YANG · Alena Harley · Stephen Pfohl · Elisabeth Rumetshofer · Alex Fedorov · Saloni Dash · Jacob Pfau · Sabina Tomkins · Colin Targonski · Michael Brudno · Xinyu Li · Yiyang Yu · Nisarg Patel -
2019 Poster: Time-series Generative Adversarial Networks »
Jinsung Yoon · Daniel Jarrett · Mihaela van der Schaar -
2018 : Poster Session I »
Aniruddh Raghu · Daniel Jarrett · Kathleen Lewis · Elias Chaibub Neto · Nicholas Mastronarde · Shazia Akbar · Chun-Hung Chao · Henghui Zhu · Seth Stafford · Luna Zhang · Jen-Tang Lu · Changhee Lee · Adityanarayanan Radhakrishnan · Fabian Falck · Liyue Shen · Daniel Neil · Yusuf Roohani · Aparna Balagopalan · Brett Marinelli · Hagai Rossman · Sven Giesselbach · Jose Javier Gonzalez Ortiz · Edward De Brouwer · Byung-Hoon Kim · Rafid Mahmood · Tzu Ming Hsu · Antonio Ribeiro · Rumi Chunara · Agni Orfanoudaki · Kristen Severson · Mingjie Mai · Sonali Parbhoo · Albert Haque · Viraj Prabhu · Di Jin · Alena Harley · Geoffroy Dubourg-Felonneau · Xiaodan Hu · Maithra Raghu · Jonathan Warrell · Nelson Johansen · Wenyuan Li · Marko Järvenpää · Satya Narayan Shukla · Sarah Tan · Vincent Fortuin · Beau Norgeot · Yi-Te Hsu · Joel H Saltz · Veronica Tozzo · Andrew Miller · Guillaume Ausset · Azin Asgarian · Francesco Paolo Casale · Antoine Neuraz · Bhanu Pratap Singh Rawat · Turgay Ayer · Xinyu Li · Mehul Motani · Nathaniel Braman · Laetitia M Shao · Adrian Dalca · Hyunkwang Lee · Emma Pierson · Sandesh Ghimire · Yuji Kawai · Owen Lahav · Anna Goldenberg · Denny Wu · Pavitra Krishnaswamy · Colin Pawlowski · Arijit Ukil · Yuhui Zhang -
2016 Poster: Balancing Suspense and Surprise: Timely Decision Making with Endogenous Information Acquisition »
Ahmed Alaa · Mihaela van der Schaar -
2016 Poster: A Non-parametric Learning Method for Confidently Estimating Patient's Clinical State and Dynamics »
William Hoiles · Mihaela van der Schaar -
2014 Poster: Discovering, Learning and Exploiting Relevance »
Cem Tekin · Mihaela van der Schaar