Timezone: »

 
Poster
Understanding Deflation Process in Over-parametrized Tensor Decomposition
Rong Ge · Yunwei Ren · Xiang Wang · Mo Zhou

Thu Dec 09 04:30 PM -- 06:00 PM (PST) @
in Poster Session 7 »

In this paper we study the training dynamics for gradient flow on over-parametrized tensor decomposition problems. Empirically, such training process often first fits larger components and then discovers smaller components, which is similar to a tensor deflation process that is commonly used in tensor decomposition algorithms. We prove that for orthogonally decomposable tensor, a slightly modified version of gradient flow would follow a tensor deflation process and recover all the tensor components. Our proof suggests that for orthogonal tensors, gradient flow dynamics works similarly as greedy low-rank learning in the matrix setting, which is a first step towards understanding the implicit regularization effect of over-parametrized models for low-rank tensors.

Keywords:

Author Information

Rong Ge (Duke University)
Yunwei Ren (Shanghai Jiao Tong University)
Xiang Wang (Duke University)
Mo Zhou (Duke University)

More from the Same Authors