Timezone: »

 
Poster
Reward Augmented Maximum Likelihood for Neural Structured Prediction
Mohammad Norouzi · Samy Bengio · zhifeng Chen · Navdeep Jaitly · Mike Schuster · Yonghui Wu · Dale Schuurmans

Mon Dec 05 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #41 #None
in Monday Posters »

A key problem in structured output prediction is enabling direct optimization of the task reward function that matters for test evaluation. This paper presents a simple and computationally efficient method that incorporates task reward into maximum likelihood training. We establish a connection between maximum likelihood and regularized expected reward, showing that they are approximately equivalent in the vicinity of the optimal solution. Then we show how maximum likelihood can be generalized by optimizing the conditional probability of auxiliary outputs that are sampled proportional to their exponentiated scaled rewards. We apply this framework to optimize edit distance in the output space, by sampling from edited targets. Experiments on speech recognition and machine translation for neural sequence to sequence models show notable improvements over maximum likelihood baseline by simply sampling from target output augmentations.

Keywords: Deep Learning or Neural Networks · Structured Prediction

Author Information

Mohammad Norouzi (Google Brain)
Samy Bengio (Google Research, Brain Team)
ZF Chen (Google Brain)
Navdeep Jaitly (Google Brain)
Mike Schuster (Google)
Yonghui Wu (Google)
Dale Schuurmans (Google Brain & University of Alberta)

More from the Same Authors