Timezone: »

 
Desiderata for next generation of ML model serving
Sherif Akoush · Andrei Paleyes · Arnaud Van Looveren · Clive Cox

Fri Dec 09 03:50 AM -- 04:30 AM (PST) @
in Challenges in Deploying and Monitoring Machine Learning Systems »

Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice.

[ Poster] 

Author Information

Sherif Akoush (Seldon Technologies)
Andrei Paleyes (Universtiy of Cambridge)
Arnaud Van Looveren (Seldon Technologies)
Clive Cox (Seldon)

More from the Same Authors