Timezone: »

 
Poster
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONAS
Han Shi · Renjie Pi · Hang Xu · Zhenguo Li · James Kwok · Tong Zhang

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #1036
in Poster Session 4 »

Neural Architecture Search (NAS) has shown great potentials in finding better neural network designs. Sample-based NAS is the most reliable approach which aims at exploring the search space and evaluating the most promising architectures. However, it is computationally very costly. As a remedy, the one-shot approach has emerged as a popular technique for accelerating NAS using weight-sharing. However, due to the weight-sharing of vastly different networks, the one-shot approach is less reliable than the sample-based approach. In this work, we propose BONAS (Bayesian Optimized Neural Architecture Search), a sample-based NAS framework which is accelerated using weight-sharing to evaluate multiple related architectures simultaneously. Specifically, we apply Graph Convolutional Network predictor as a surrogate model for Bayesian Optimization to select multiple related candidate models in each iteration. We then apply weight-sharing to train multiple candidate models simultaneously. This approach not only accelerates the traditional sample-based approach significantly, but also keeps its reliability. This is because weight-sharing among related architectures are more reliable than those in the one-shot approach. Extensive experiments are conducted to verify the effectiveness of our method over many competing algorithms.

Keywords: Sparsity and Compressed Sensing · Algorithms

Author Information

Han Shi (Hong Kong University of Science and Technology)
Renjie Pi (Huawei Noah’s Ark Lab)
Hang Xu (Huawei Noah's Ark Lab)
Zhenguo Li (Noah's Ark Lab, Huawei Tech Investment Co Ltd)
James Kwok (Hong Kong University of Science and Technology)
Tong Zhang (The Hong Kong University of Science and Technology)

More from the Same Authors