Perturbations, Optimization, and Statistics

In nearly all machine learning tasks, decisions must be made given current knowledge (e.g., choose which label to predict). Perhaps surprisingly, always making the best decision is not always the best strategy, particularly while learning. Recently, there is an emerging body of work on learning under different rules that apply perturbations to the decision procedure. These works provide simple and efficient learning rules with improved theoretical guarantees. This workshop will bring together the growing community of researchers interested in different aspects of this area, and it will broaden our understanding of why and how perturbation methods can be useful.

Last year, at the highly successful NIPS workshop on Perturbations, Optimization, and Statistics, we looked at how injecting perturbations (whether it be random or adversarial “noise”) into learning and inference procedures can be beneficial. The focus was on two angles: first, on how stochastic perturbations can be used to construct new types of probability models for structured data; and second, how deterministic perturbations affect the regularization and the generalization properties of learning algorithms.

The goal of this workshop is to expand the scope of last year and also explore different ways to apply perturbations within optimization and statistics to enhance and improve machine learning approaches. This year, we would like to: (a) Look at exciting new developments related to the above core themes. (b) Emphasize their implications on topics that received less coverage last year, specifically highlighting connections to decision theory, risk analysis, game theory, and economics.

More generally, we shall specifically be interested in understanding the following issues:

* Repeated games and online learning: How to use random perturbations to explore unseens options in repeated games? How to exploit connections to Bayesian risk?
* Adversarial Uncertainty: How to play complex games with adversarial uncertainty? What are the computational qualities of such solutions, and do Nash-equilibria exists in these cases?

* Stochastic risk: How to average predictions with random perturbations to get improved generalization guarantees? How stochastic perturbations imply approximated Bayesian risk and regularization?
* Dropout: How stochastic dropout regularizes learning of complex models and what is its generalization power? What are the relationships between stochastic and adversarial dropouts?
* Robust optimization: In what ways can learning be improved by perturbing the input measurements?
* Choice theory: What is the best way to use perturbations to compensate lack of knowledge? What lessons in modeling can machine learning take from random utility theory?
* Theory: How does the maximum of a random process relate to its complexity? How can the maximum of random perturbations be used to measure the uncertainty of a system?