Timezone: »

Active Learning for Probabilistic Hypotheses Using the Maximum Gibbs Error Criterion
Nguyen Viet Cuong · Wee Sun Lee · Nan Ye · Kian Ming Adam Chai · Hai Leong Chieu

Thu Dec 05 07:00 PM -- 11:59 PM (PST) @ Harrah's Special Events Center, 2nd Floor #None

We introduce a new objective function for pool-based Bayesian active learning with probabilistic hypotheses. This objective function, called the policy Gibbs error, is the expected error rate of a random classifier drawn from the prior distribution on the examples adaptively selected by the active learning policy. Exact maximization of the policy Gibbs error is hard, so we propose a greedy strategy that maximizes the Gibbs error at each iteration, where the Gibbs error on an instance is the expected error of a random classifier selected from the posterior label distribution on that instance. We apply this maximum Gibbs error criterion to three active learning scenarios: non-adaptive, adaptive, and batch active learning. In each scenario, we prove that the criterion achieves near-maximal policy Gibbs error when constrained to a fixed budget. For practical implementations, we provide approximations to the maximum Gibbs error criterion for Bayesian conditional random fields and transductive Naive Bayes. Our experimental results on a named entity recognition task and a text classification task show that the maximum Gibbs error criterion is an effective active learning criterion for noisy models.

Author Information

Nguyen Viet Cuong (National University of Singapore)
Wee Sun Lee (National University of Singapore)
Nan Ye (National University of Singapore)
Kian Ming Adam Chai (DSO National Laboratories)
Hai Leong Chieu (DSO National Laboratories)

More from the Same Authors