Timezone: »
Spotlight
Information-theoretic analysis of generalization capability of learning algorithms
Aolin Xu · Maxim Raginsky
We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The upper bounds provide theoretical guidelines for striking the right balance between data fit and generalization by controlling the input-output mutual information of a learning algorithm. The results can also be used to analyze the generalization capability of learning algorithms under adaptive composition, and the bias-accuracy tradeoffs in adaptive data analytics. Our work extends and leads to nontrivial improvements on the recent results of Russo and Zou.
Author Information
Aolin Xu (University of Illinois at Urbana-Champaign)
Maxim Raginsky (University of Illinois at Urbana-Champaign)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Poster: Information-theoretic analysis of generalization capability of learning algorithms »
Wed. Dec 6th 02:30 -- 06:30 AM Room Pacific Ballroom #217
More from the Same Authors
-
2021 Poster: Information-theoretic generalization bounds for black-box learning algorithms »
Hrayr Harutyunyan · Maxim Raginsky · Greg Ver Steeg · Aram Galstyan -
2019 Poster: Universal Approximation of Input-Output Maps by Temporal Convolutional Nets »
Joshua Hanson · Maxim Raginsky -
2018 Poster: Minimax Statistical Learning with Wasserstein distances »
Jaeho Lee · Maxim Raginsky -
2018 Spotlight: Minimax Statistical Learning with Wasserstein distances »
Jaeho Lee · Maxim Raginsky -
2011 Poster: Lower Bounds for Passive and Active Learning »
Maxim Raginsky · Sasha Rakhlin -
2011 Spotlight: Lower Bounds for Passive and Active Learning »
Maxim Raginsky · Sasha Rakhlin -
2009 Poster: Locality-sensitive binary codes from shift-invariant kernels »
Maxim Raginsky · Svetlana Lazebnik -
2009 Oral: Locality-Sensitive Binary Codes from Shift-Invariant Kernels »
Maxim Raginsky · Svetlana Lazebnik -
2008 Poster: Near-minimax recursive density estimation on the binary hypercube »
Maxim Raginsky · Svetlana Lazebnik · Rebecca Willett · Jorge G Silva -
2008 Spotlight: Near-minimax recursive density estimation on the binary hypercube »
Maxim Raginsky · Svetlana Lazebnik · Rebecca Willett · Jorge G Silva