Timezone: »
A crucial assumption in most statistical learning theory is that samples are independently and identically distributed (i.i.d.). However, for many real applications, the i.i.d. assumption does not hold. We consider learning problems in which examples are dependent and their dependency relation is characterized by a graph. To establish algorithm-dependent generalization theory for learning with non-i.i.d. data, we first prove novel McDiarmid-type concentration inequalities for Lipschitz functions of graph-dependent random variables. We show that concentration relies on the forest complexity of the graph, which characterizes the strength of the dependency. We demonstrate that for many types of dependent data, the forest complexity is small and thus implies good concentration. Based on our new inequalities we are able to build stability bounds for learning from graph-dependent data.
Author Information
Rui (Ray) Zhang (School of Mathematics, Monash University)
Xingwu Liu (University of Chinese Academy of Sciences)
Yuyi Wang (ETH Zurich)
Liwei Wang (Peking University)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Poster: McDiarmid-Type Inequalities for Graph-Dependent Variables and Stability Bounds »
Thu Dec 12th 01:00 -- 03:00 AM Room East Exhibition Hall B + C
More from the Same Authors
-
2020 Poster: Improved Analysis of Clipping Algorithms for Non-convex Optimization »
Bohang Zhang · Jikai Jin · Cong Fang · Liwei Wang -
2020 Poster: Locally Differentially Private (Contextual) Bandits Learning »
Kai Zheng · Tianle Cai · Weiran Huang · Zhenguo Li · Liwei Wang -
2020 Poster: Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot »
Jingtong Su · Yihang Chen · Tianle Cai · Tianhao Wu · Ruiqi Gao · Liwei Wang · Jason Lee -
2020 Poster: RepPoints v2: Verification Meets Regression for Object Detection »
Yihong Chen · Zheng Zhang · Yue Cao · Liwei Wang · Stephen Lin · Han Hu -
2019 Poster: Convergence of Adversarial Training in Overparametrized Neural Networks »
Ruiqi Gao · Tianle Cai · Haochuan Li · Cho-Jui Hsieh · Liwei Wang · Jason Lee -
2019 Spotlight: Convergence of Adversarial Training in Overparametrized Neural Networks »
Ruiqi Gao · Tianle Cai · Haochuan Li · Cho-Jui Hsieh · Liwei Wang · Jason Lee -
2019 Poster: Equipping Experts/Bandits with Long-term Memory »
Kai Zheng · Haipeng Luo · Ilias Diakonikolas · Liwei Wang -
2018 Poster: Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation »
Liwei Wang · Lunjia Hu · Jiayuan Gu · Zhiqiang Hu · Yue Wu · Kun He · John Hopcroft -
2018 Spotlight: Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation »
Liwei Wang · Lunjia Hu · Jiayuan Gu · Zhiqiang Hu · Yue Wu · Kun He · John Hopcroft -
2018 Poster: FRAGE: Frequency-Agnostic Word Representation »
Chengyue Gong · Di He · Xu Tan · Tao Qin · Liwei Wang · Tie-Yan Liu -
2017 Poster: Decoding with Value Networks for Neural Machine Translation »
Di He · Hanqing Lu · Yingce Xia · Tao Qin · Liwei Wang · Tie-Yan Liu -
2017 Poster: The Expressive Power of Neural Networks: A View from the Width »
Zhou Lu · Hongming Pu · Feicheng Wang · Zhiqiang Hu · Liwei Wang -
2016 Poster: Dual Learning for Machine Translation »
Di He · Yingce Xia · Tao Qin · Liwei Wang · Nenghai Yu · Tie-Yan Liu · Wei-Ying Ma -
2013 Poster: Efficient Algorithm for Privately Releasing Smooth Queries »
Ziteng Wang · Kai Fan · Jiaqi Zhang · Liwei Wang -
2012 Poster: Dimensionality Dependent PAC-Bayes Margin Bound »
Chi Jin · Liwei Wang -
2009 Poster: Sufficient Conditions for Agnostic Active Learnable »
Liwei Wang