Timezone: »
Imitation Learning has been shown to be successful in solving many challenging real-world problems. Some recent approaches give strong performance guarantees by training the policy iteratively. However, it is important to note that these guarantees depend on how well the policy we found can imitate the oracle on the training data. When there is a substantial difference between the oracle's ability and the learner's policy space, we may fail to find a policy that has low error on the training set. In such cases, we propose to use a coach that demonstrates easy-to-learn actions for the learner and gradually approaches the oracle. By a reduction of learning by demonstration to online learning, we prove that coaching can yield a lower regret bound than using the oracle. We apply our algorithm to a novel cost-sensitive dynamic feature selection problem, a hard decision problem that considers a user-specified accuracy-cost trade-off. Experimental results on UCI datasets show that our method outperforms state-of-the-art imitation learning methods in dynamic features selection and two static feature selection methods.
Author Information
He He (NYU)
Hal Daumé III (University of Maryland - College Park)
Jason Eisner (Johns Hopkins + Microsoft)
Jason Eisner is Professor of Computer Science at Johns Hopkins University, as well as Director of Research at Microsoft Semantic Machines. He is a Fellow of the Association for Computational Linguistics. At Johns Hopkins, he is also affiliated with the Center for Language and Speech Processing, the Machine Learning Group, the Cognitive Science Department, and the national Center of Excellence in Human Language Technology. His goal is to develop the probabilistic modeling, inference, and learning techniques needed for a unified model of all kinds of linguistic structure. His 135+ papers have presented various algorithms for parsing, machine translation, and weighted finite-state machines; formalizations, algorithms, theorems, and empirical results in computational phonology; and unsupervised or semi-supervised learning methods for syntax, morphology, and word-sense disambiguation. He is also the lead designer of Dyna, a new declarative programming language that provides an infrastructure for AI research. He has received two school-wide awards for excellence in teaching, as well as recent Best Paper Awards at ACL 2017 and EMNLP 2019.
More from the Same Authors
-
2021 : Poster: The Many Roles that Causal Reasoning Plays in Reasoning about Fairness in Machine Learning »
Irene Y Chen · Hal Daumé III · Solon Barocas -
2023 : Structure-Aware Path Inference for Neural Finite State Transducers »
Weiting Tan · Chu-Cheng Lin · Jason Eisner -
2023 Poster: BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing »
Subhro Roy · Samuel Thomson · Tongfei Chen · Richard Shin · Adam Pauls · Jason Eisner · Benjamin Van Durme -
2021 : The Many Roles that Causal Reasoning Plays in Reasoning about Fairness in Machine Learning »
Irene Y Chen · Hal Daumé III · Solon Barocas -
2020 Poster: Noise-Contrastive Estimation for Multivariate Point Processes »
Hongyuan Mei · Tom Wan · Jason Eisner -
2019 : Panel Discussion »
Jacob Andreas · Edward Gibson · Stefan Lee · Noga Zaslavsky · Jason Eisner · Jürgen Schmidhuber -
2019 : Invited Talk - 3 »
Jason Eisner -
2018 : Panel Discussion »
Rich Caruana · Mike Schuster · Ralf Schlüter · Hynek Hermansky · Renato De Mori · Samy Bengio · Michiel Bacchiani · Jason Eisner -
2018 : Jason Eisner, "BiLSTM-FSTs and Neural FSTs" »
Jason Eisner -
2018 Workshop: Wordplay: Reinforcement and Language Learning in Text-based Games »
Adam Trischler · Angeliki Lazaridou · Yonatan Bisk · Wendy Tay · Nate Kushman · Marc-Alexandre Côté · Alessandro Sordoni · Daniel Ricks · Tom Zahavy · Hal Daumé III -
2017 : Competition V: Human-Computer Question Answering »
Jordan Boyd-Graber · Hal Daumé III · He He · Mohit Iyyer · Pedro Rodriguez -
2017 Poster: The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process »
Hongyuan Mei · Jason Eisner -
2016 Poster: A Credit Assignment Compiler for Joint Prediction »
Kai-Wei Chang · He He · Stephane Ross · Hal Daumé III · John Langford -
2014 Workshop: Representation and Learning Methods for Complex Outputs »
Richard Zemel · Dale Schuurmans · Kilian Q Weinberger · Yuhong Guo · Jia Deng · Francesco Dinuzzo · Hal Daumé III · Honglak Lee · Noah A Smith · Richard Sutton · Jiaqian YU · Vitaly Kuznetsov · Luke Vilnis · Hanchen Xiong · Calvin Murdock · Thomas Unterthiner · Jean-Francis Roy · Martin Renqiang Min · Hichem SAHBI · Fabio Massimo Zanzotto -
2014 Poster: Learning to Search in Branch and Bound Algorithms »
He He · Hal Daumé III · Jason Eisner -
2012 Poster: Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression »
Piyush Rai · Abhishek Kumar · Hal Daumé III -
2012 Poster: Learned Prioritization for Trading Off Accuracy and Speed »
Jiarong Jiang · Adam Teichert · Hal Daumé III · Jason Eisner -
2011 Poster: Message-Passing for Approximate MAP Inference with Latent Variables »
Jiarong Jiang · Piyush Rai · Hal Daumé III -
2011 Poster: Co-regularized Multi-view Spectral Clustering »
Abhishek Kumar · Piyush Rai · Hal Daumé III -
2010 Poster: Learning Multiple Tasks using Manifold Regularization »
Arvind Agarwal · Hal Daumé III · Samuel Gerber -
2010 Poster: Co-regularization Based Semi-supervised Domain Adaptation »
Hal Daumé III · Abhishek Kumar · Avishek Saha -
2009 Poster: Multi-Label Prediction via Sparse Infinite CCA »
Piyush Rai · Hal Daumé III -
2008 Poster: Nonparametric Bayesian Sparse Hierarchical Factor Modeling and Regression »
Piyush Rai · Hal Daumé III -
2007 Poster: Bayesian Agglomerative Clustering with Coalescents »
Yee Whye Teh · Hal Daumé III · Daniel Roy -
2007 Oral: Bayesian Agglomerative Clustering with Coalescents »
Yee Whye Teh · Hal Daumé III · Daniel Roy