Timezone: »

 
Poster
Interaction-Grounded Learning with Action-Inclusive Feedback
Tengyang Xie · Akanksha Saran · Dylan J Foster · Lekan Molu · Ida Momennejad · Nan Jiang · Paul Mineiro · John Langford

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #110
in Poster Session 4 »

Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its policies. The agent observes a context vector, takes an action, and receives a feedback vector, using this information to effectively optimize a policy with respect to a latent reward function. Prior analyzed approaches fail when the feedback vector contains the action, which significantly limits IGL’s success in many potential scenarios such as Brain-computer interface (BCI) or Human-computer interface (HCI) applications. We address this by creating an algorithm and analysis which allows IGL to work even when the feedback vector contains the action, encoded in any fashion. We provide theoretical guarantees and large-scale experiments based on supervised datasets to demonstrate the effectiveness of the new approach.

Keywords: Reinforcement Learning · contextual bandits · human-AI interaction · interactive machine learning · interaction-grounded learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Tengyang Xie (University of Illinois at Urbana-Champaign)
Akanksha Saran (Microsoft Research)
Dylan J Foster (Microsoft Research)
Lekan Molu (Microsoft)
Ida Momennejad (Microsoft Research)
Nan Jiang (University of Illinois at Urbana-Champaign)
Paul Mineiro (Microsoft)
John Langford (Microsoft Research)

John Langford is a machine learning research scientist, a field which he says "is shifting from an academic discipline to an industrial tool". He is the author of the weblog hunch.net and the principal developer of Vowpal Wabbit. John works at Microsoft Research New York, of which he was one of the founding members, and was previously affiliated with Yahoo! Research, Toyota Technological Institute, and IBM's Watson Research Center. He studied Physics and Computer Science at the California Institute of Technology, earning a double bachelor's degree in 1997, and received his Ph.D. in Computer Science from Carnegie Mellon University in 2002. He was the program co-chair for the 2012 International Conference on Machine Learning.

More from the Same Authors