Timezone: »

Optimal Architectures in a Solvable Model of Deep Networks
Jonathan Kadmon · Haim Sompolinsky

Wed Dec 07 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #18 #None

Deep neural networks have received a considerable attention due to the success of their training for real world machine learning applications. They are also of great interest to the understanding of sensory processing in cortical sensory hierarchies. The purpose of this work is to advance our theoretical understanding of the computational benefits of these architectures. Using a simple model of clustered noisy inputs and a simple learning rule, we provide analytically derived recursion relations describing the propagation of the signals along the deep network. By analysis of these equations, and defining performance measures, we show that these model networks have optimal depths. We further explore the dependence of the optimal architecture on the system parameters.

Author Information

Jonathan Kadmon (Hebrew University)
Haim Sompolinsky (Hebrew University and Harvard University)

More from the Same Authors