Timezone: »
A fundamental task in active learning involves performing a sequence of tests to identify an unknown hypothesis that is drawn from a known distribution. This problem, known as optimal decision tree induction, has been widely studied for decades and the asymptotically best-possible approximation algorithm has been devised for it. We study a generalization where certain test outcomes are noisy, even in the more general case when the noise is persistent, i.e., repeating the test on the scenario gives the same noisy output, disallowing simple repetition as a way to gain confidence. We design new approximation algorithms for both the non-adaptive setting, where the test sequence must be fixed a-priori, and the adaptive setting where the test sequence depends on the outcomes of prior tests. Previous work in the area assumed at most a constant number of noisy outcomes per test and per scenario and provided approximation ratios that were problem dependent (such as the minimum probability of a hypothesis). Our new approximation algorithms provide guarantees that are nearly best-possible and work for the general case of a large number of noisy outcomes per test or per hypothesis where the performance degrades smoothly with this number. Our results adapt and generalize methods used for submodular ranking and stochastic set cover. We evaluate the performance of our algorithms on two natural applications with noise: toxic chemical identification and active learning of linear classifiers. Despite our logarithmic theoretical approximation guarantees, our methods give solutions with cost very close to the information theoretic minimum, demonstrating the effectiveness of our methods.
Author Information
Su Jia (CMU)
viswanath nagarajan (Univ Michigan, Ann Arbor)
Fatemeh Navidi (University of Michigan)
R Ravi (CMU)
More from the Same Authors
-
2022 Spotlight: Dynamic Pricing with Monotonicity Constraint under Unknown Parametric Demand Model »
Su Jia · Andrew Li · R Ravi -
2022 Spotlight: Lightning Talks 4A-1 »
Jiawei Huang · Su Jia · Abdurakhmon Sadiev · Ruomin Huang · Yuanyu Wan · Denizalp Goktas · Jiechao Guan · Andrew Li · Wei-Wei Tu · Li Zhao · Amy Greenwald · Jiawei Huang · Dmitry Kovalev · Yong Liu · Wenjie Liu · Peter Richtarik · Lijun Zhang · Zhiwu Lu · R Ravi · Tao Qin · Wei Chen · Hu Ding · Nan Jiang · Tie-Yan Liu -
2022 Poster: Dynamic Pricing with Monotonicity Constraint under Unknown Parametric Demand Model »
Su Jia · Andrew Li · R Ravi -
2022 Poster: An Asymptotically Optimal Batched Algorithm for the Dueling Bandit Problem »
Arpit Agarwal · Rohan Ghuge · viswanath nagarajan -
2021 Poster: Greedy Approximation Algorithms for Active Sequential Hypothesis Testing »
Kyra Gan · Su Jia · Andrew Li