Timezone: »

 
Poster
Computing Higher Order Derivatives of Matrix and Tensor Expressions
Sören Laue · Matthias Mitterreiter · Joachim Giesen

Wed Dec 05 02:00 PM -- 04:00 PM (PST) @ Room 210 #28
in Wed Poster Session B »

Optimization is an integral part of most machine learning systems and most numerical optimization schemes rely on the computation of derivatives. Therefore, frameworks for computing derivatives are an active area of machine learning research. Surprisingly, as of yet, no existing framework is capable of computing higher order matrix and tensor derivatives directly. Here, we close this fundamental gap and present an algorithmic framework for computing matrix and tensor derivatives that extends seamlessly to higher order derivatives. The framework can be used for symbolic as well as for forward and reverse mode automatic differentiation. Experiments show a speedup between one and four orders of magnitude over state-of-the-art frameworks when evaluating higher order derivatives.

Keywords: Convex Optimization · Optimization · Non-Convex Optimization · AutoML
[ Paper] 

Author Information

Sören Laue (Universitaet Jena)
Matthias Mitterreiter (Friedrich Schiller University Jena)
Joachim Giesen (Friedrich-Schiller-Universitat Jena)

More from the Same Authors