Timezone: »
Learning the common structure shared by a set of supervised tasks is an important practical and theoretical problem. Knowledge of this structure may lead to better generalization performance on the tasks and may also facilitate learning new tasks. We propose a framework for solving this problem, which is based on regularization with spectral functions of matrices. This class of regularization problems exhibits appealing computational properties and can be optimized efficiently by an alternating minimization algorithm. In addition, we provide a necessary and sufficient condition for convexity of the regularizer. We analyze concrete examples of the framework, which are equivalent to regularization with L_p matrix norms. Experiments on two real data sets indicate that the algorithm scales well with the number of tasks and improves on state of the art statistical performance.
Author Information
Andreas Argyriou (Ecole Centrale de Paris)
Charles A. Micchelli
Massimiliano Pontil (IIT & UCL)
Yiming Ying (State University of New York at Albany)
Related Events (a corresponding poster, oral, or spotlight)
-
2007 Poster: A Spectral Regularization Framework for Multi-Task Structure Learning »
Wed. Dec 5th 06:30 -- 06:40 PM Room
More from the Same Authors
-
2022 Poster: Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks »
Yunwen Lei · Rong Jin · Yiming Ying -
2022 Poster: Stability and Generalization for Markov Chain Stochastic Gradient Methods »
Puyu Wang · Yunwen Lei · Yiming Ying · Ding-Xuan Zhou -
2021 Poster: Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning »
ZHENHUAN YANG · Yunwen Lei · Puyu Wang · Tianbao Yang · Yiming Ying -
2021 Poster: Generalization Guarantee of SGD for Pairwise Learning »
Yunwen Lei · Mingrui Liu · Yiming Ying -
2020 Poster: Exploiting Higher Order Smoothness in Derivative-free Optimization and Continuous Bandits »
Arya Akhavan · Massimiliano Pontil · Alexandre Tsybakov -
2020 Poster: The Advantage of Conditional Meta-Learning for Biased Regularization and Fine Tuning »
Giulia Denevi · Massimiliano Pontil · Carlo Ciliberto -
2020 Poster: Estimating weighted areas under the ROC curve »
Andreas Maurer · Massimiliano Pontil -
2019 Poster: Online-Within-Online Meta-Learning »
Giulia Denevi · Dimitris Stamos · Carlo Ciliberto · Massimiliano Pontil -
2019 Poster: Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm »
Giulia Luise · Saverio Salzo · Massimiliano Pontil · Carlo Ciliberto -
2019 Spotlight: Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm »
Giulia Luise · Saverio Salzo · Massimiliano Pontil · Carlo Ciliberto -
2018 Poster: Bilevel learning of the Group Lasso structure »
Jordan Frecon · Saverio Salzo · Massimiliano Pontil -
2018 Poster: Learning To Learn Around A Common Mean »
Giulia Denevi · Carlo Ciliberto · Dimitris Stamos · Massimiliano Pontil -
2018 Spotlight: Bilevel learning of the Group Lasso structure »
Jordan Frecon · Saverio Salzo · Massimiliano Pontil -
2017 : An Efficient Method to Impose Fairness in Linear Models »
Massimiliano Pontil · John Shawe-Taylor -
2017 Workshop: Workshop on Prioritising Online Content »
John Shawe-Taylor · Massimiliano Pontil · Nicolò Cesa-Bianchi · Emine Yilmaz · Chris Watkins · Sebastian Riedel · Marko Grobelnik -
2017 Poster: Consistent Multitask Learning with Nonlinear Output Relations »
Carlo Ciliberto · Alessandro Rudi · Lorenzo Rosasco · Massimiliano Pontil -
2016 Poster: Mistake Bounds for Binary Matrix Completion »
Mark Herbster · Stephen Pasteris · Massimiliano Pontil -
2015 : The Benefit of Multitask Representation Learning »
Massimiliano Pontil -
2014 Poster: Spectral k-Support Norm Regularization »
Andrew McDonald · Massimiliano Pontil · Dimitris Stamos -
2013 Workshop: New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks »
Urun Dogan · Marius Kloft · Tatiana Tommasi · Francesco Orabona · Massimiliano Pontil · Sinno Jialin Pan · Shai Ben-David · Arthur Gretton · Fei Sha · Marco Signoretto · Rajhans Samdani · Yun-Qian Miao · Mohammad Gheshlaghi azar · Ruth Urner · Christoph Lampert · Jonathan How -
2013 Poster: A New Convex Relaxation for Tensor Completion »
Bernardino Romera-Paredes · Massimiliano Pontil -
2012 Poster: Sparse Prediction with the $k$-Support Norm »
Andreas Argyriou · Rina Foygel · Nati Srebro -
2012 Spotlight: Sparse Prediction with the $k$-Support Norm »
Andreas Argyriou · Rina Foygel · Nati Srebro -
2012 Poster: Optimal kernel choice for large-scale two-sample tests »
Arthur Gretton · Bharath Sriperumbudur · Dino Sejdinovic · Heiko Strathmann · Sivaraman Balakrishnan · Massimiliano Pontil · Kenji Fukumizu -
2010 Workshop: Tensors, Kernels, and Machine Learning »
Tamara G Kolda · Vicente Malave · David F Gleich · Johan Suykens · Marco Signoretto · Andreas Argyriou -
2010 Spotlight: A Family of Penalty Functions for Structured Sparsity »
Charles A Micchelli · Jean M Morales · Massimiliano Pontil -
2010 Poster: A Family of Penalty Functions for Structured Sparsity »
Charles A Micchelli · Jean M Morales · Massimiliano Pontil -
2009 Poster: Sparse Metric Learning via Smooth Optimization »
Yiming Ying · Kaizhu Huang · Colin I Campbell -
2009 Poster: Analysis of SVM with Indefinite Kernels »
Yiming Ying · Colin I Campbell · Mark A Girolami -
2009 Spotlight: Analysis of SVM with Indefinite Kernels »
Yiming Ying · Colin I Campbell · Mark A Girolami -
2008 Poster: Fast Prediction on a Tree »
Mark Herbster · Massimiliano Pontil · Sergio Rojas Galeano -
2008 Oral: Fast Prediction on a Tree »
Mark Herbster · Massimiliano Pontil · Sergio Rojas Galeano -
2008 Poster: On-Line Prediction on Large Diameter Graphs »
Mark Herbster · Massimiliano Pontil · Guy Lever -
2006 Poster: Prediction on a Graph with a Perceptron »
Mark Herbster · Massimiliano Pontil -
2006 Spotlight: Prediction on a Graph with a Perceptron »
Mark Herbster · Massimiliano Pontil -
2006 Poster: Multi-Task Feature Learning »
Andreas Argyriou · Theos Evgeniou · Massimiliano Pontil