Timezone: »

 
Backpropagation through Back substitution with a Backslash
Ekin Akyürek · Alan Edelman · Bernie Wang

Mon Dec 13 09:25 AM -- 10:40 AM (PST) @
in Differentiable Programming Workshop »
Event URL: https://openreview.net/forum?id=K8MP-aDkZTM »

We present a linear algebra formulation of backpropogation that serves as an alternative to the traditional approach.Using matrices allows the calculation of gradients given the availability of a generically written Gaussian elimination which is representedby the ``backslash" symbol. Backpropogation is often connected to the chain rule for multivariate calculus, but we propose that this may be seen as a distraction from the underlying algebraic structure.The implementation shows how generic linear algebra can allow operators as elements of matrices, and without rewriting of any code, the software carries through to completion giving the correct answer. We demonstrate in a suitable programming language consisting of generic linear algebra operators such as Julia \cite{bezanson2017julia}, it is possibleto realize this abstraction in code.

Author Information

Ekin Akyürek (Massachusetts Institute of Technology)
Alan Edelman (Massachusetts Institute of Technology)
Bernie Wang (AWS AI Labs)

More from the Same Authors