Timezone: »

 
Poster
TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets
Chengrun Yang · Gabriel Bender · Hanxiao Liu · Pieter-Jan Kindermans · Madeleine Udell · Yifeng Lu · Quoc V Le · Da Huang

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #336
in Poster Session 2 »

The best neural architecture for a given machine learning problem depends on many factors: not only the complexity and structure of the dataset, but also on resource constraints including latency, compute, energy consumption, etc. Neural architecture search (NAS) for tabular datasets is an important but under-explored problem. Previous NAS algorithms designed for image search spaces incorporate resource constraints directly into the reinforcement learning (RL) rewards. However, for NAS on tabular datasets, this protocol often discovers suboptimal architectures. This paper develops TabNAS, a new and more effective approach to handle resource constraints in tabular NAS using an RL controller motivated by the idea of rejection sampling. TabNAS immediately discards any architecture that violates the resource constraints without training or learning from that architecture. TabNAS uses a Monte-Carlo-based correction to the RL policy gradient update to account for this extra filtering step. Results on several tabular datasets demonstrate the superiority of TabNAS over previous reward-shaping methods: it finds better models that obey the constraints.

Keywords: Reinforcement Learning · Neural Architecture Search · rejection sampling · tabular dataset · Monte-Carlo sampling
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Chengrun Yang (Google Research)
Gabriel Bender (Google Brain)
Hanxiao Liu (Google Brain)
Pieter-Jan Kindermans (Google Brain)
Madeleine Udell (Cornell)
Yifeng Lu
Quoc V Le (Google)
Da Huang (Google)

More from the Same Authors