Timezone: »
Real-world machine-learning applications require robust models that generalize well to distribution shift settings, which is typical in real-world situations. Domain adaptation techniques aim to address this issue of distribution shift by minimizing the disparities between domains to ensure that the model trained on the source domain performs well on the target domain. Nevertheless, the existing domain adaptation methods are computationally very expensive. In this work, we aim to improve the efficiency of existing supervised domain adaptation (SDA) methods by using a subset of source data that is similar to target data for faster model training. Specifically, we propose ORIENT, a subset selection framework that uses the submodular mutual information (SMI) functions to select a source data subset similar to the target data for faster training. Additionally, we demonstrate how existing robust subset selection strategies, such as GLISTER, GRADMATCH, and CRAIG, when used with a held-out query set, fit within our proposed framework and demonstrate the connections with them. Finally, we empirically demonstrate that SDA approaches like d-SNE, CCSA, and standard Cross-entropy training, when employed together with ORIENT, achieve a) faster training and b) better performance on the target data.
Author Information
Athresh Karanam (University of Texas, Dallas)
Krishnateja Killamsetty (University of Texas, Dallas)
Harsha Kokel (University of Texas, Dallas)
Rishabh Iyer (University of Texas, Dallas)
Bio: Prof. Rishabh Iyer is currently an Assistant Professor at the University of Texas, Dallas, where he leads the CARAML Lab. He is also a Visiting Assistant Professor at the Indian Institute of Technology, Bombay. He completed his Ph.D. in 2015 from the University of Washington, Seattle. He is excited in making ML more efficient (both computational and labeling efficiency), robust, and fair. He has received the best paper award at Neural Information Processing Systems (NeurIPS/NIPS) in 2013, the International Conference of Machine Learning (ICML) in 2013, and an Honorable Mention at CODS-COMAD in 2021. He has also won a Microsoft Research Ph.D. Fellowship, a Facebook Ph.D. Fellowship, and the Yang Award for Outstanding Graduate Student from the University of Washington.
More from the Same Authors
-
2021 : Deep RePReL--Combining Planning and Deep RL for acting in relational domains »
Harsha Kokel · Arjun Manoharan · Sriraam Natarajan · Balaraman Ravindran · Prasad Tadepalli -
2021 : A Nested Bi-level Optimization Framework for Robust Few Shot Learning »
Krishnateja Killamsetty · Changbin Li · Chen Zhao · Rishabh Iyer · Feng Chen -
2021 : Targeted Active Learning using Submodular Mutual Information for Imbalanced Medical Image Classification »
Suraj Kothawade · Lakshman Tamil · Rishabh Iyer -
2022 : Using Informative Data Subsets for Efficient Training of Large Language Models: An Initial Study »
H S V N S Kowndinya Renduchintala · Krishnateja Killamsetty · Sumit Bhatia · Milan Aggarwal · Ganesh Ramakrishnan · Rishabh Iyer -
2022 : TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information »
Suraj Kothawade · Saikat Ghosh · Sumit Shekhar · Yu Xiang · Rishabh Iyer -
2023 Poster: Learning to Select a Subset of Training Examples to Generalize Efficient Model Training »
Eeshaan Jain · Tushar Nandy · Gaurav Aggarwal · Ashish Tendulkar · Rishabh Iyer · Abir De -
2022 Poster: AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning »
Krishnateja Killamsetty · Guttu Sai Abhishek · Aakriti Lnu · Ganesh Ramakrishnan · Alexandre Evfimievski · Lucian Popa · Rishabh Iyer -
2021 Poster: SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios »
Suraj Kothawade · Nathan Beck · Krishnateja Killamsetty · Rishabh Iyer -
2021 Poster: Learning to Select Exogenous Events for Marked Temporal Point Process »
Ping Zhang · Rishabh Iyer · Ashish Tendulkar · Gaurav Aggarwal · Abir De -
2021 Poster: Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models »
Matej Zečević · Devendra Dhami · Athresh Karanam · Sriraam Natarajan · Kristian Kersting -
2021 Poster: RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning »
Krishnateja Killamsetty · Xujiang Zhao · Feng Chen · Rishabh Iyer -
2015 Poster: Submodular Hamming Metrics »
Jennifer Gillenwater · Rishabh K Iyer · Bethany Lusch · Rahul Kidambi · Jeffrey A Bilmes -
2015 Spotlight: Submodular Hamming Metrics »
Jennifer Gillenwater · Rishabh K Iyer · Bethany Lusch · Rahul Kidambi · Jeffrey A Bilmes -
2015 Poster: Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications »
Kai Wei · Rishabh K Iyer · Shengjie Wang · Wenruo Bai · Jeffrey A Bilmes -
2014 Poster: Learning Mixtures of Submodular Functions for Image Collection Summarization »
Sebastian Tschiatschek · Rishabh K Iyer · Haochen Wei · Jeffrey A Bilmes -
2013 Poster: Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints »
Rishabh K Iyer · Jeffrey A Bilmes -
2013 Oral: Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints »
Rishabh K Iyer · Jeffrey A Bilmes -
2013 Poster: Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions »
Rishabh K Iyer · Stefanie Jegelka · Jeffrey A Bilmes -
2012 Poster: Submodular Bregman Divergences with Applications »
Rishabh K Iyer · Jeffrey A Bilmes