NIPS 2018 Expo Demo
Nov. 28, 2022
FPGA based CNN system for video analytics
Sponsor: Alibaba Group
Xi Xu (Alibaba)
Xi Xu (Alibaba)
In this demonstration, we will show an FPGA based CNN system which has been deployed in real-world video analytics scenarios. According to our experience, real-world video analytics applications require properties on FPGA based system as follows. (1) Supporting the latest CNN networks with high efficiency, which usually contain new operators such as Deconv, Dilated Conv, Leaky ReLu, etc. (2) Running multiple highly-diverse CNN networks simultaneously without FPGA reconfiguration. (3) Comparable inference performance with the latest GPU and ASIC. In order to fulfill the requirements above, we applied approaches as follows, respectively. (1) We combine network compilation/optimization and hardware architecture design to support various operators and network structures efficiently. (2) We design the FPGA based CNN processor in a uniform way that highly-diverse CNN networks can be mapped to the same processor architecture. (3) We apply novel reduced precision method in model inference to exploit the rich configurable logic resource on FPGA. Combining with the deeply optimized processing element for FPGA, our implementation on last generation Xilinx Ultrascale FPGA can achieve comparable performance with the latest Nvidia P4 GPU.
PAI is the machine learning training/serving platform for various Alibaba internal workloads and also machine learning service on AliCloud. Deeply integrated with the MaxCompute big data processing platform in Alibaba, PAI is able to provide one-stop service for various machine learning needs. During the serving for many workloads, PAI has accumulated a set of techniques including large scale distributed training, joint optimization between algorithm, engine and hardware, etc. Moreover, though focusing on deep learning workloads such as CNN/RNN/DNN, PAI also support a richer set of machine learning algorithms like reinforcement learning, trasfer learning and tradition algorithms like linear regression, decision tree, kmean, etc.