Timezone: »

 
Spotlight
MCUNet: Tiny Deep Learning on IoT Devices
Ji Lin · Wei-Ming Chen · Yujun Lin · john cohn · Chuang Gan · Song Han

Thu Dec 10 08:00 PM -- 08:10 PM (PST) @ Orals & Spotlights: Health/AutoML/(Soft|Hard)ware
in Orals & Spotlights Track 33: Health/AutoML/(Soft|Hard)ware »

Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude smaller even than mobile phones. We propose MCUNet, a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine), enabling ImageNet-scale inference on microcontrollers. TinyNAS adopts a two-stage neural architecture search approach that first optimizes the search space to fit the resource constraints, then specializes the network architecture in the optimized search space. TinyNAS can automatically handle diverse constraints (i.e. device, latency, energy, memory) under low search costs. TinyNAS is co-designed with TinyEngine, a memory-efficient inference library to expand the search space and fit a larger model. TinyEngine adapts the memory scheduling according to the overall network topology rather than layer-wise optimization, reducing the memory usage by 3.4×, and accelerating the inference by 1.7-3.3× compared to TF-Lite Micro [3] and CMSIS-NN [28]. MCUNet is the first to achieves >70% ImageNet top1 accuracy on an off-the-shelf commercial microcontroller, using 3.5× less SRAM and 5.7× less Flash compared to quantized MobileNetV2 and ResNet-18. On visual&audio wake words tasks, MCUNet achieves state-of-the-art accuracy and runs 2.4-3.4× faster than Mo- bileNetV2 and ProxylessNAS-based solutions with 3.7-4.1× smaller peak SRAM. Our study suggests that the era of always-on tiny machine learning on IoT devices has arrived.

Author Information

Ji Lin (MIT)
Wei-Ming Chen (National Taiwan University)
Yujun Lin (MIT)
john cohn (MIT-IBM Watson AI Lab, IBM Research)

John Cohn is an IBM Fellow in the MIT-IBM Watson AI Research Group based in Cambridge, MA. John earned a BSEE MIT, and a Ph.D in Computer Engineering from Carnegie Mellon University He has authored more than 30 technical papers, contributed to four books and has more than 100 worldwide patents. In 2005 John was elected a Fellow of the IEEE. John is active in education issues at a local, state and national level. He is so passionate about promoting STEM careers that he spent 59 days living and inventing in an abandoned steel mill as part of Discovery Channel’s technical survival show “The Colony”. John lives with his family in a restored 19th century schoolhouse in Jonesville Vermont and is eager to share his love of science and technology with anyone who will listen.

Chuang Gan (MIT-IBM Watson AI Lab)
Song Han (MIT)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors