A cookbook for hardware-friendly implicit learning on static data

The following aims to be a pragmatic introduction to hardware-friendly learning of implicit models, which encompass a broad class of models from feedforward nets to physical systems, taking static data as inputs. Starting from first principles, we present a minimal hierarchy of independent concepts to circumvent some problems inherent to the hardware implementation of standard differentiation. This way, we avoid entangling essential ingredients with arbitrary design choices by naively listing existing algorithms and instead propose the draft of a “cookbook” to help the exploration of many possible combinations of these independent mechanisms.