Timezone: »
In forming learning objectives, one oftentimes needs to aggregate a set of individual values to a single output. Such cases occur in the aggregate loss, which combines individual losses of a learning model over each training sample, and in the individual loss for multi-label learning, which combines prediction scores over all class labels. In this work, we introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. The minimization of SoRR is solved with the difference of convex algorithm (DCA). We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary classification and the TKML individual loss for multi-label/multi-class classification. Our empirical results highlight the effectiveness of the proposed optimization framework and demonstrate the applicability of proposed losses using synthetic and real datasets.
Author Information
Shu Hu (University at Buffalo, State University of New York)
Yiming Ying (State University of New York at Albany)
xin wang (CuraCloud)
Siwei Lyu (University at Albany)
More from the Same Authors
-
2023 Poster: Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance »
Lisha Chen · Heshan Fernando · Yiming Ying · Tianyi Chen -
2023 Poster: DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection »
Zhiyuan Yan · Yong Zhang · Xinhang Yuan · Siwei Lyu · Baoyuan Wu -
2022 Poster: Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks »
Yunwen Lei · Rong Jin · Yiming Ying -
2022 Poster: Stability and Generalization for Markov Chain Stochastic Gradient Methods »
Puyu Wang · Yunwen Lei · Yiming Ying · Ding-Xuan Zhou -
2021 Poster: Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning »
ZHENHUAN YANG · Yunwen Lei · Puyu Wang · Tianbao Yang · Yiming Ying -
2021 Poster: Generalization Guarantee of SGD for Pairwise Learning »
Yunwen Lei · Mingrui Liu · Yiming Ying -
2020 Poster: Uncertainty Aware Semi-Supervised Learning on Graph Data »
Xujiang Zhao · Feng Chen · Shu Hu · Jin-Hee Cho -
2020 Spotlight: Uncertainty Aware Semi-Supervised Learning on Graph Data »
Xujiang Zhao · Feng Chen · Shu Hu · Jin-Hee Cho -
2017 Poster: Learning with Average Top-k Loss »
Yanbo Fan · Siwei Lyu · Yiming Ying · Baogang Hu -
2016 Poster: Stochastic Online AUC Maximization »
Yiming Ying · Longyin Wen · Siwei Lyu -
2016 Oral: Stochastic Online AUC Maximization »
Yiming Ying · Longyin Wen · Siwei Lyu -
2009 Poster: Sparse Metric Learning via Smooth Optimization »
Yiming Ying · Kaizhu Huang · Colin I Campbell -
2009 Poster: Analysis of SVM with Indefinite Kernels »
Yiming Ying · Colin I Campbell · Mark A Girolami -
2009 Spotlight: Analysis of SVM with Indefinite Kernels »
Yiming Ying · Colin I Campbell · Mark A Girolami -
2007 Spotlight: A Spectral Regularization Framework for Multi-Task Structure Learning »
Andreas Argyriou · Charles A. Micchelli · Massimiliano Pontil · Yiming Ying -
2007 Poster: A Spectral Regularization Framework for Multi-Task Structure Learning »
Andreas Argyriou · Charles A. Micchelli · Massimiliano Pontil · Yiming Ying