Timezone: »

 
Spotlight
TA-GATES: An Encoding Scheme for Neural Network Architectures
Xuefei Ning · Zixuan Zhou · Junbo Zhao · Tianchen Zhao · Yiping Deng · Changcheng Tang · Shuang Liang · Huazhong Yang · Yu Wang

Wed Dec 07 05:00 PM -- 07:00 PM (PST) @
in Lightning Talks 4B-1 »

Neural architecture search tries to shift the manual design of neural network (NN) architectures to algorithmic design. In these cases, the NN architecture itself can be viewed as data and needs to be modeled. A better modeling could help explore novel architectures automatically and open the black box of automated architecture design. To this end, this work proposes a new encoding scheme for neural architectures, the Training-Analogous Graph-based ArchiTecture Encoding Scheme (TA-GATES). TA-GATES encodes an NN architecture in a way that is analogous to its training. Extensive experiments demonstrate that the flexibility and discriminative power of TA-GATES lead to better modeling of NN architectures. We expect our methodology of explicitly modeling the NN training process to benefit broader automated deep learning systems. The code is available at https://github.com/walkerning/aw_nas.

Keywords: AutoML · Predictor-based NAS · Architecture Encoding Scheme · Neural architecture search (NAS)

Author Information

Xuefei Ning (Tsinghua University)
Zixuan Zhou (Tsinghua University, Tsinghua University)
Junbo Zhao (Tsinghua University, Tsinghua University)
Tianchen Zhao (Beihang University)
Yiping Deng (Tsinghua University, Tsinghua University)
Changcheng Tang (Beijing Novauto Co. Ltd)
Shuang Liang
Huazhong Yang (Tsinghua University, Tsinghua University)
Yu Wang (Tsinghua University)

Yu Wang received his B.S. degree in 2002 and Ph.D. degree (with honor) in 2007 from Tsinghua University, Beijing. He is currently a Tenured Associate Professor with the Department of Electronic Engineering, Tsinghua University. His research interests include brain inspired computing, application specific hardware computing, parallel circuit analysis, and power/reliability aware system design methodology. Dr. Wang has authored and coauthored over 150 papers in refereed journals and conferences. He has received Best Paper Award in FPGA 2017, ISVLSI 2012, and Best Poster Award in HEART 2012 with 8 Best Paper Nominations. He is a recipient of IBM X10 Faculty Award in 2010. He served as TPC chair for ICFPT 2011 and Finance Chair of ISLPED 2012-2016, and served as program committee member for leading conferences in these areas, including top EDA conferences such as DAC, DATE, ICCAD, ASP-DAC, and top FPGA conferences such as FPGA and FPT. Currently he serves as Co-EIC for SIGDA E-Newsletter, Associate Editor for IEEE Transactions on CAD and Journal of Circuits, Systems, and Computers. He also serves as guest editor for Integration, the VLSI Journal and IEEE Transactions on Multi-Scale Computing Systems. He is a recipient of NSFC Excellent Young Scholar，and is now serving as ACM distinguished speaker. He is an IEEE/ACM senior member.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors