Timezone: »
When learning features for complex visual recognition problems, labeled image exemplars alone can be insufficient. While an \emph{object taxonomy} specifying the categories' semantic relationships could bolster the learning process, not all relationships are relevant to a given visual classification task, nor does a single taxonomy capture all ties that \emph{are} relevant. In light of these issues, we propose a discriminative feature learning approach that leverages \emph{multiple} hierarchical taxonomies representing different semantic views of the object categories (e.g., for animal classes, one taxonomy could reflect their phylogenic ties, while another could reflect their habitats). For each taxonomy, we first learn a tree of semantic kernels, where each node has a Mahalanobis kernel optimized to distinguish between the classes in its children nodes. Then, using the resulting \emph{semantic kernel forest}, we learn class-specific kernel combinations to select only those relationships relevant to recognize each object class. To learn the weights, we introduce a novel hierarchical regularization term that further exploits the taxonomies' structure. We demonstrate our method on challenging object recognition datasets, and show that interleaving multiple taxonomic views yields significant accuracy improvements.
Author Information
Sung Ju Hwang (University of Texas at Austin)
Kristen Grauman (University of Texas at Austin)
Fei Sha (University of Southern California (USC))
More from the Same Authors
-
2021 Spotlight: Shaping embodied agent behavior with activity-context priors from egocentric video »
Tushar Nagarajan · Kristen Grauman -
2023 Poster: EgoEnv: Human-centric environment representations from egocentric video »
Tushar Nagarajan · Santhosh Kumar Ramakrishnan · Ruta Desai · James Hillis · Kristen Grauman -
2023 Poster: Self-Supervised Visual Acoustic Matching »
Arjun Somayazulu · Changan Chen · Kristen Grauman -
2023 Poster: Video-Mined Task Graphs for Keystep Recognition in Instructional Videos »
Kumar Ashutosh · Santhosh Kumar Ramakrishnan · Triantafyllos Afouras · Kristen Grauman -
2023 Poster: Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment »
Zihui Xue · Kristen Grauman -
2023 Poster: EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding »
Shuhan Tan · Tushar Nagarajan · Kristen Grauman -
2023 Poster: Single-Stage Visual Query Localization in Egocentric Videos »
Hanwen Jiang · Santhosh Kumar Ramakrishnan · Kristen Grauman -
2022 Poster: SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning »
Changan Chen · Carl Schissler · Sanchit Garg · Philip Kobernik · Alexander Clegg · Paul Calamia · Dhruv Batra · Philip Robinson · Kristen Grauman -
2022 Poster: Few-Shot Audio-Visual Learning of Environment Acoustics »
Sagnik Majumder · Changan Chen · Ziad Al-Halah · Kristen Grauman -
2021 Poster: Shaping embodied agent behavior with activity-context priors from egocentric video »
Tushar Nagarajan · Kristen Grauman -
2020 : Panel Discussion & Closing »
Yejin Choi · Alexei Efros · Chelsea Finn · Kristen Grauman · Quoc V Le · Yann LeCun · Ruslan Salakhutdinov · Eric Xing -
2020 : Q & A and Panel Session with Dan Weld, Kristen Grauman, Scott Yih, Emma Brunskill, and Alex Ratner »
Kristen Grauman · Wen-tau Yih · Alexander Ratner · Emma Brunskill · Douwe Kiela · Daniel S. Weld -
2020 : QA: Kristen Grauman »
Kristen Grauman -
2020 : Invited Talk: Kristen Grauman »
Kristen Grauman -
2020 Poster: Learning Affordance Landscapes for Interaction Exploration in 3D Environments »
Tushar Nagarajan · Kristen Grauman -
2020 Spotlight: Learning Affordance Landscapes for Interaction Exploration in 3D Environments »
Tushar Nagarajan · Kristen Grauman -
2020 Session: Orals & Spotlights Track 01: Representation/Relational »
Laurens van der Maaten · Fei Sha -
2018 Poster: Synthesize Policies for Transfer and Adaptation across Tasks and Environments »
Hexiang Hu · Liyu Chen · Boqing Gong · Fei Sha -
2018 Spotlight: Synthesize Policies for Transfer and Adaptation across Tasks and Environments »
Hexiang Hu · Liyu Chen · Boqing Gong · Fei Sha -
2017 Workshop: Optimal Transport and Machine Learning »
Olivier Bousquet · Marco Cuturi · Gabriel Peyré · Fei Sha · Justin Solomon -
2017 Poster: Learning Spherical Convolution for Fast Features from 360° Imagery »
Yu-Chuan Su · Kristen Grauman -
2017 Poster: An Empirical Study on The Properties of Random Bases for Kernel Methods »
Maximilian Alber · Pieter-Jan Kindermans · Kristof Schütt · Klaus-Robert Müller · Fei Sha -
2015 : Do Shallow Kernel Methods Match Deep Neural Networks »
Fei Sha -
2015 : Do Shallow Kernel Methods Match Deep Neural Networks? »
Fei Sha -
2014 Poster: Diverse Sequential Subset Selection for Supervised Video Summarization »
Boqing Gong · Wei-Lun Chao · Kristen Grauman · Fei Sha -
2014 Poster: Predicting Useful Neighborhoods for Lazy Local Learning »
Aron Yu · Kristen Grauman -
2014 Poster: Zero-shot recognition with unreliable attributes »
Dinesh Jayaraman · Kristen Grauman -
2013 Workshop: New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks »
Urun Dogan · Marius Kloft · Tatiana Tommasi · Francesco Orabona · Massimiliano Pontil · Sinno Jialin Pan · Shai Ben-David · Arthur Gretton · Fei Sha · Marco Signoretto · Rajhans Samdani · Yun-Qian Miao · Mohammad Gheshlaghi azar · Ruth Urner · Christoph Lampert · Jonathan How -
2013 Poster: Reshaping Visual Datasets for Domain Adaptation »
Boqing Gong · Kristen Grauman · Fei Sha -
2013 Poster: Similarity Component Analysis »
Soravit Changpinyo · Kuan Liu · Fei Sha -
2012 Poster: Non-linear Metric Learning »
Dor Kedem · Stephen Tyree · Kilian Q Weinberger · Fei Sha · Gert Lanckriet -
2012 Session: Oral Session 5 »
Fei Sha -
2011 Poster: Learning a Tree of Metrics with Disjoint Visual Features »
Sung Ju Hwang · Kristen Grauman · Fei Sha -
2010 Workshop: Challenges of Data Visualization »
Barbara Hammer · Laurens van der Maaten · Fei Sha · Alexander Smola -
2010 Poster: Hashing Hyperplane Queries to Near Points with Applications to Large-Scale Active Learning »
Prateek Jain · Sudheendra Vijayanarasimhan · Kristen Grauman -
2010 Poster: Unsupervised Kernel Dimension Reduction »
Meihong Wang · Fei Sha · Michael Jordan -
2009 Workshop: Statistical Machine Learning for Visual Analytics »
Guy Lebanon · Fei Sha -
2008 Oral: Multi-Level Active Prediction of Useful Image Annotations for Recognition »
Sudheendra N Vijayanarasimhan · Kristen Grauman -
2008 Poster: Multi-Level Active Prediction of Useful Image Annotations for Recognition »
Sudheendra N Vijayanarasimhan · Kristen Grauman -
2008 Poster: Online Metric Learning and Fast Similarity Search »
Prateek Jain · Brian Kulis · Inderjit Dhillon · Kristen Grauman -
2008 Poster: DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification »
Simon Lacoste-Julien · Fei Sha · Michael Jordan -
2008 Oral: Online Metric Learning and Fast Similarity Search »
Prateek Jain · Brian Kulis · Inderjit Dhillon · Kristen Grauman -
2008 Session: Oral session 1: Clustering »
Fei Sha -
2007 Workshop: Machine Learning for Systems Problems (Part 2) »
Archana Ganapathi · Sumit Basu · Fei Sha · Emre Kiciman -
2007 Workshop: Machine Learning for Systems Problems (Part 1) »
Archana Ganapathi · Sumit Basu · Fei Sha · Emre Kiciman -
2007 Session: Session 7: Systems and Applications »
Fei Sha -
2006 Poster: Large Margin Gaussian Mixture Models for Automatic Speech Recognition »
Fei Sha · Lawrence Saul -
2006 Poster: Approximate Correspondences in High Dimensions »
Kristen Grauman · Trevor Darrell -
2006 Spotlight: Approximate Correspondences in High Dimensions »
Kristen Grauman · Trevor Darrell -
2006 Talk: Large Margin Gaussian Mixture Models for Automatic Speech Recognition »
Fei Sha · Lawrence Saul -
2006 Poster: Graph Regularization for Maximum Variance Unfolding with an Application to Sensor Localization »
Kilian Q Weinberger · Fei Sha · Qihui Zhu · Lawrence Saul