Timezone: »

 
Poster
Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
Antoine Gautier · Quynh Nguyen · Matthias Hein

Tue Dec 06 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #98
in Tuesday Posters »

The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guarantee to achieve the global optimum. In contrast we show under quite weak assumptions on the data that a particular class of feedforward neural networks can be trained globally optimal with a linear convergence rate. Up to our knowledge this is the first practically feasible method which achieves such a guarantee. While the method can in principle be applied to deep networks, we restrict ourselves for simplicity in this paper to one- and two hidden layer networks. Our experiments confirms that these models are already rich enough to achieve good performance on a series of real-world datasets.

Keywords: Deep Learning or Neural Networks · Spectral Methods · (Other) Optimization
[ PDF] 

Author Information

Antoine Gautier (Saarland University)
Quynh Nguyen (Saarland University)
Matthias Hein (Saarland University)

More from the Same Authors