Timezone: »

 
Demonstration
Babble Labble: Learning from Natural Language Explanations
Braden Hancock · Paroma Varma · Percy Liang · Christopher Ré · Stephanie Wang

Wed Dec 06 07:00 PM -- 10:30 PM (PST) @ Pacific Ballroom Concourse #D8
in Demos Wed »

We introduce Babble Labble, a system for converting natural language explanations into massive training sets with probabilistic labels. In this demo, users will be shown unlabeled examples for a simple relation extraction task (identifying mentions of spouses in the news). For each example, instead of providing a label, users provide a sentence describing one reason why the given example should receive a certain label. These explanations are parsed into executable functions in real-time and applied to the unlabeled dataset. We use data programming to resolve conflicts between the functions and combine their weak labels into a single probabilistic label per example. This large weakly labeled training set is then used to train a discriminative model that improves generalization as it includes features never mentioned in the small set of explanations. Using the explanations the user wrote, we calculate the final quality of the complete system, finding in most cases that one to two dozen explanations achieve the same quality as hundreds or thousands of labels.

Author Information

Braden Hancock (Stanford Univeresity)

www.bradenhancock.com

Paroma Varma (Stanford University)
Percy Liang (Stanford University)
Percy Liang

Percy Liang is an Assistant Professor of Computer Science at Stanford University (B.S. from MIT, 2004; Ph.D. from UC Berkeley, 2011). His research spans machine learning and natural language processing, with the goal of developing trustworthy agents that can communicate effectively with people and improve over time through interaction. Specific topics include question answering, dialogue, program induction, interactive learning, and reliable machine learning. His awards include the IJCAI Computers and Thought Award (2016), an NSF CAREER Award (2016), a Sloan Research Fellowship (2015), and a Microsoft Research Faculty Fellowship (2014).

Christopher Ré (Stanford)
Stephanie Wang (Stanford University)

More from the Same Authors