Timezone: »

 
Poster
Understanding Dropout
Pierre Baldi · Peter Sadowski

Fri Dec 06 07:00 PM -- 11:59 PM (PST) @ Harrah's Special Events Center, 2nd Floor
in Poster Session »

Dropout is a relatively new algorithm for training neural networks which relies on stochastically "dropping out'' neurons during training in order to avoid the co-adaptation of feature detectors. We introduce a general formalism for studying dropout on either units or connections, with arbitrary probability values, and use it to analyze the averaging and regularizing properties of dropout in both linear and non-linear networks. For deep neural networks, the averaging properties of dropout are characterized by three recursive equations, including the approximation of expectations by normalized weighted geometric means. We provide estimates and bounds for these approximations and corroborate the results with simulations. We also show in simple cases how dropout performs stochastic gradient descent on a regularized error function.

[ PDF] 

Author Information

Pierre Baldi (UC Irvine)
Peter Sadowski (University of Hawai‘i)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors