Timezone: »
We study the application of a strongly non-linear generative model to image patches. As in standard approaches such as Sparse Coding or Independent Component Analysis, the model assumes a sparse prior with independent hidden variables. However, in the place where standard approaches use the sum to combine basis functions we use the maximum. To derive tractable approximations for parameter estimation we apply a novel approach based on variational Expectation Maximization. The derived learning algorithm can be applied to large-scale problems with hundreds of observed and hidden variables. Furthermore, we can infer all model parameters including observation noise and the degree of sparseness. In applications to image patches we find that Gabor-like basis functions are obtained. Gabor-like functions are thus not a feature exclusive to approaches assuming linear superposition. Quantitatively, the inferred basis functions show a large diversity of shapes with many strongly elongated and many circular symmetric functions. The distribution of basis function shapes reflects properties of simple cell receptive fields that are not reproduced by standard linear approaches. In the study of natural image statistics, the implications of using different superposition assumptions have so far not been investigated systematically because models with strong non-linearities have been found analytically and computationally challenging. The presented algorithm represents the first large-scale application of such an approach.
Author Information
Jose G Puertas (FIAS)
Jörg Bornschein (University of Montreal)
Jörg Lücke (TU Berlin)
More from the Same Authors
-
2013 Poster: What Are the Invariant Occlusive Components of Image Patches? A Probabilistic Generative Approach »
Zhenwen Dai · Georgios Exarchakis · Jörg Lücke -
2012 Poster: Why MCA? Nonlinear Spike-and-slab Sparse Coding for Neurally Plausible Image Encoding »
Jacquelyn A Shelton · Philip Sterne · Jörg Bornschein · Abdul Saboor Sheikh · Jörg Lücke -
2011 Poster: Select and Sample: A Model of Efficient Neural Inference and Learning »
Jacquelyn A Shelton · Jörg Bornschein · Abdul Saboor Sheikh · Pietro Berkes · Jörg Lücke -
2009 Poster: Occlusive Components Analysis »
Jörg Lücke · Richard Turner · Maneesh Sahani · Marc Henniges