Timezone: »

 
Poster
Recovery and Generalization in Over-Realized Dictionary Learning
Jeremias Sulam · Chong You · Zhihui Zhu

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #1003
in Poster Session 6 »

In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization is carried out in the same model class as that of the underlying dictionary. This work characterizes the surprising phenomenon that dictionary recovery can be facilitated by searching over the space of larger over-realized models. This observation is general and independent of the specific dictionary learning algorithm used. We thoroughly demonstrate this observation in practice and provide an analysis of this phenomenon by tying recovery measures to generalization bounds. In particular, we show that model recovery can be upper-bounded by the empirical risk, a model-dependent quantity and the generalization gap, reflecting our empirical findings. We further show that an efficient and provably correct distillation approach can be employed to recover the correct atoms from the over-realized model. As a result, our meta-algorithm provides dictionary estimates with consistently better recovery of the ground-truth model.

Keywords: Journal Track · JMLR
[ Poster] 

Author Information

Jeremias Sulam (Johns Hopkins University)
Chong You
Zhihui Zhu

More from the Same Authors