Timezone: »

 
Poster
Porcupine Neural Networks: Approximating Neural Network Landscapes
Soheil Feizi · Hamid Javadi · Jesse Zhang · David Tse

Wed Dec 05 07:45 AM -- 09:45 AM (PST) @ Room 210 #21
in Wed Poster Session A »

Neural networks have been used prominently in several machine learning and statistics applications. In general, the underlying optimization of neural networks is non-convex which makes analyzing their performance challenging. In this paper, we take another approach to this problem by constraining the network such that the corresponding optimization landscape has good theoretical properties without significantly compromising performance. In particular, for two-layer neural networks we introduce Porcupine Neural Networks (PNNs) whose weight vectors are constrained to lie over a finite set of lines. We show that most local optima of PNN optimizations are global while we have a characterization of regions where bad local optimizers may exist. Moreover, our theoretical and empirical results suggest that an unconstrained neural network can be approximated using a polynomially-large PNN.

Keywords: Non-Convex Optimization · Spaces of Functions and Kernels
[ Paper]  [ Poster] 

Author Information

Soheil Feizi (University of Maryland, College Park)
Hamid Javadi (Rice University)
Jesse Zhang (Stanford University)
David Tse (Stanford University)

More from the Same Authors