Timezone: »
We investigate a topic at the interface of machine learning and cognitive science. Human active learning, where learners can actively query the world for information, is contrasted with passive learning from random examples. Furthermore, we compare human active learning performance with predictions from statistical learning theory. We conduct a series of human category learning experiments inspired by a machine learning task for which active and passive learning error bounds are well understood, and dramatically distinct. Our results indicate that humans are capable of actively selecting informative queries, and in doing so learn better and faster than if they are given random training data, as predicted by learning theory. However, the improvement over passive learning is not as dramatic as that achieved by machine active learning algorithms. To the best of our knowledge, this is the first quantitative study comparing human category learning in active versus passive settings.
Author Information
Jerry Zhu (University of Wisconsin-Madison)
Rui M Castro (Columbia University)
Timothy T Rogers (University of Wisconsin-Madison)
Rob Nowak (Wisconsin)
Ruichen Qian
Chuck Kalish (University of Wisconsin-Madison)
More from the Same Authors
-
2021 : Game Redesign in No-regret Game Playing »
Yuzhe Ma · Young Wu · Jerry Zhu -
2021 : Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments »
Amin Rakhsha · Xuezhou Zhang · Jerry Zhu · Adish Singla -
2021 : Game Redesign in No-regret Game Playing »
Yuzhe Ma · Young Wu · Jerry Zhu -
2021 : Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environments »
Amin Rakhsha · Xuezhou Zhang · Jerry Zhu · Adish Singla -
2022 Poster: Provable Defense against Backdoor Policies in Reinforcement Learning »
Shubham Bharti · Xuezhou Zhang · Adish Singla · Jerry Zhu -
2019 Poster: Policy Poisoning in Batch Reinforcement Learning and Control »
Yuzhe Ma · Xuezhou Zhang · Wen Sun · Jerry Zhu -
2019 Poster: Preference-Based Batch and Sequential Teaching: Towards a Unified View of Models »
Farnam Mansouri · Yuxin Chen · Ara Vartanian · Jerry Zhu · Adish Singla -
2019 Poster: A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning »
Xuanqing Liu · Si Si · Jerry Zhu · Yang Li · Cho-Jui Hsieh -
2018 Poster: Adversarial Attacks on Stochastic Bandits »
Kwang-Sung Jun · Lihong Li · Yuzhe Ma · Jerry Zhu -
2017 Workshop: Teaching Machines, Robots, and Humans »
Maya Cakmak · Anna Rafferty · Adish Singla · Jerry Zhu · Sandra Zilles -
2016 : Optimal Teaching for Online Perceptrons »
Xuezhou Zhang · Jerry Zhu -
2016 Workshop: The Future of Interactive Machine Learning »
Kory Mathewson @korymath · Kaushik Subramanian · Mark Ho · Robert Loftin · Joseph L Austerweil · Anna Harutyunyan · Doina Precup · Layla El Asri · Matthew Gombolay · Jerry Zhu · Sonia Chernova · Charles Isbell · Patrick M Pilarski · Weng-Keen Wong · Manuela Veloso · Julie A Shah · Matthew Taylor · Brenna Argall · Michael Littman -
2016 Poster: Finite Sample Prediction and Recovery Bounds for Ordinal Embedding »
Lalit Jain · Kevin Jamieson · Rob Nowak -
2016 Poster: Active Learning with Oracle Epiphany »
Tzu-Kuo Huang · Lihong Li · Ara Vartanian · Saleema Amershi · Jerry Zhu -
2015 Workshop: Multiresolution methods for large-scale learning »
Inderjit Dhillon · Risi Kondor · Rob Nowak · Michael O'Neil · Nedelina Teneva -
2015 Poster: Human Memory Search as Initial-Visit Emitting Random Walk »
Kwang-Sung Jun · Jerry Zhu · Timothy T Rogers · Zhuoran Yang · Ming Yuan -
2015 Poster: NEXT: A System for Real-World Development, Evaluation, and Application of Active Learning »
Kevin G Jamieson · Lalit Jain · Chris Fernandez · Nicholas J. Glattard · Rob Nowak -
2015 Spotlight: NEXT: A System for Real-World Development, Evaluation, and Application of Active Learning »
Kevin G Jamieson · Lalit Jain · Chris Fernandez · Nicholas J. Glattard · Rob Nowak -
2014 Poster: Optimal Teaching for Limited-Capacity Human Learners »
Kaustubh R Patil · Jerry Zhu · Łukasz Kopeć · Bradley C Love -
2014 Spotlight: Optimal Teaching for Limited-Capacity Human Learners »
Kaustubh R Patil · Jerry Zhu · Łukasz Kopeć · Bradley C Love -
2013 Workshop: Modern Nonparametric Methods in Machine Learning »
Arthur Gretton · Mladen Kolar · Samory Kpotufe · John Lafferty · Han Liu · Bernhard Schölkopf · Alexander Smola · Rob Nowak · Mikhail Belkin · Lorenzo Rosasco · peter bickel · Yue Zhao -
2013 Poster: Sparse Overlapping Sets Lasso for Multitask Learning and its Application to fMRI Analysis »
Nikhil Rao · Christopher R Cox · Rob Nowak · Timothy T Rogers -
2013 Spotlight: Sparse Overlapping Sets Lasso for Multitask Learning and its Application to fMRI Analysis »
Nikhil Rao · Christopher R Cox · Rob Nowak · Timothy T Rogers -
2013 Poster: Machine Teaching for Bayesian Learners in the Exponential Family »
Jerry Zhu -
2012 Poster: Query Complexity of Derivative-Free Optimization »
Kevin G Jamieson · Rob Nowak · Benjamin Recht -
2011 Poster: How Do Humans Teach: On Curriculum Learning and Teaching Dimension »
Faisal Khan · Jerry Zhu · Bilge Mutlu -
2011 Poster: Learning Higher-Order Graph Structure with Features by Structure Penalty »
Shilin Ding · Grace Wahba · Jerry Zhu -
2011 Poster: Active Ranking using Pairwise Comparisons »
Kevin G Jamieson · Rob Nowak -
2010 Oral: Humans Learn Using Manifolds, Reluctantly »
Bryan R Gibson · Jerry Zhu · Timothy T Rogers · Chuck Kalish · Joseph Harrison -
2010 Poster: Humans Learn Using Manifolds, Reluctantly »
Bryan R Gibson · Jerry Zhu · Timothy T Rogers · Chuck Kalish · Joseph Harrison -
2010 Poster: Transduction with Matrix Completion: Three Birds with One Stone »
Andrew B Goldberg · Jerry Zhu · Benjamin Recht · Junming Sui · Rob Nowak -
2010 Session: Spotlights Session 1 »
Jerry Zhu -
2009 Workshop: Adaptive Sensing, Active Learning, and Experimental Design »
Rui M Castro · Nando de Freitas · Ruben Martinez-Cantin -
2009 Poster: Human Rademacher Complexity »
Jerry Zhu · Timothy T Rogers · Bryan R Gibson -
2009 Poster: Noisy Generalized Binary Search »
Rob Nowak -
2009 Session: Oral Session 1: Information Theory and Estimation »
Rob Nowak -
2008 Workshop: Machine learning meets human learning »
Nathaniel D Daw · Tom Griffiths · Josh Tenenbaum · Jerry Zhu -
2008 Poster: Unlabeled data: Now it helps, now it doesn't »
Aarti Singh · Rob Nowak · Jerry Zhu -
2008 Oral: Unlabeled data: Now it helps, now it doesn't »
Aarti Singh · Rob Nowak · Jerry Zhu -
2006 Poster: Inferring Network Structure from Co-Occurrences »
Michael Rabbat · Mario T Figueiredo · Rob Nowak