Timezone: »

AI Assisted Data Labeling
Michael Desmond · Evelyn Duesterwald · Kristina Brimijoin · Michael Muller · Aabhas Sharma · Narendra Nath Joshi · Qian Pan · Casey Dugan · Zahra Ashktorab · Michelle Brachman

Tue Dec 08 06:20 PM -- 06:40 PM & Wed Dec 09 06:20 PM -- 06:40 PM (PST) @
Event URL: https://neurips-assistance.mybluemix.net/ »

Human-in-the-loop data labeling is generally considered a tedious, error-prone and expensive activity. Automation of the labeling task is desirable, but current approaches can conflict with principles of trust and human agency. We are developing a data labeling experience where the human labeler transparently interacts with an AI assistant to reach automation readiness, at which point the remainder of the labeling task can be delegated to a virtual assistant. Our approach combines semi-supervised learning, active learning, and human-machine decision tracking to reduce labeling effort and support reliable automation. The demo takes participant through an online end-to-end AI assisted data labeling experience, starting with manual labeling, then assisted labeling and ultimately transitioning to automated labeling via a system of readiness checkpoints.

Author Information

Michael Desmond (IBM Research AI)
Evelyn Duesterwald (IBM Research)
Kristina Brimijoin (IBM Research)
Michael Muller (IBM Research)

Michael Muller works in the AI Interactions group of IBM Research AI, where his work focuses on the human aspects of data science; ethics and values in applications of AI to human issues; metrics and analytics for enterprise social software applications, with particular application to employee engagement emergent social phenomena in social software. Recognitions include: ACM Distinguished Scientist; SIGCHI Academy; IBM Master Inventor. Steering Committees: EUSSET (European Society for the study of Socially Embedded Technologies); ACM GROUP conference series. Papers co-chair for ECSCW 2019 (European Computer Supported Cooperative Work conference).

Aabhas Sharma (IBM Research)
Narendra Nath Joshi (IBM Research)
Qian Pan (IBM Research AI)
Casey Dugan (IBM Research AI)
Zahra Ashktorab (IBM Research AI)
Michelle Brachman (IBM Research AI)

More from the Same Authors