Timezone: »

 
Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs
Benjamin Fuhrer · Yuval Shpigelman · Chen Tessler · Shie Mannor · Gal Chechik · Eitan Zahavi · Gal Dalal

@
in Reinforcement Learning for Real Life (RL4RealLife) Workshop »
Cloud datacenters are exponentially growing both in numbers and size. This increase results in a network activity surge that warrants better congestion avoidance. The resulting challenge is two-fold: (i) designing algorithms that can be custom-tuned to the complex traffic patterns of a given datacenter; but, at the same time (ii) run on low-level hardware with the required low latency of effective Congestion Control (CC). In this work, we present a Reinforcement Learning (RL) based CC solution that learns from certain traffic scenarios and successfully generalizes to others. We then distill the RL neural network policy into binary decision trees to achieve the desired $\mu$sec decision latency required for real-time inference. We deploy the distilled policy on NVIDIA NICs in a real network and demonstrate state-of-the-art performance, balancing all tested metrics simultaneously: bandwidth, latency, fairness, and drops.
[ Poster] 

Author Information

Benjamin Fuhrer (NVIDIA)
Yuval Shpigelman (NVIDIA)
Chen Tessler (NVIDIA)
Shie Mannor (Nvidia Research, Technion Institute of Technology)
Gal Chechik (Nvidia)
Eitan Zahavi (Nvidia Networking)
Gal Dalal (NVIDIA)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors