Timezone: »

 
Cubic Regularized Quasi-Newton Methods
Dmitry Kamzolov · Klea Ziu · Artem Agafonov · Martin Takac

@
in Order up! The Benefits of Higher-Order Optimization in Machine Learning »

In this paper, we propose a Cubic Regularized L-BFGS. Cubic Regularized Newton outperforms the classical Newton method in terms of global performance. In classics, L-BFGS approximation is applied for the Newton method. We propose a new variant of inexact Cubic Regularized Newton. Then, we use L-BFGS approximation as an inexact Hessian for Cubic Regularized Newton. It allows us to get better theoretical convergence rates and good practical performance, especially from the points where classical Newton is diverging.

[ Poster] 

Author Information

Dmitry Kamzolov (Mohamed bin Zayed University of Artificial Intelligence)
Klea Ziu (MBZUAI)
Artem Agafonov (Mohamed bin Zayed University of Artificial Intelligence)
Martin Takac (Mohamed bin Zayed University of Artificial Intelligence (MBZUAI))

More from the Same Authors