Timezone: »

 
Poster
Efficient Gradient Computation for Structured Output Learning with Rational and Tropical Losses
Corinna Cortes · Vitaly Kuznetsov · Mehryar Mohri · Dmitry Storcheus · Scott Yang

Thu Dec 06 07:45 AM -- 09:45 AM (PST) @ Room 517 AB #138
in Thu Poster Session A »
Many structured prediction problems admit a natural loss function for evaluation such as the edit-distance or $n$-gram loss. However, existing learning algorithms are typically designed to optimize alternative objectives such as the cross-entropy. This is because a na\"{i}ve implementation of the natural loss functions often results in intractable gradient computations. In this paper, we design efficient gradient computation algorithms for two broad families of structured prediction loss functions: rational and tropical losses. These families include as special cases the $n$-gram loss, the edit-distance loss, and many other loss functions commonly used in natural language processing and computational biology tasks that are based on sequence similarity measures. Our algorithms make use of weighted automata and graph operations over appropriate semirings to design efficient solutions. They facilitate efficient gradient computation and hence enable one to train learning models such as neural networks with complex structured losses.
Keywords: Structured Prediction · Natural Language Processing · Efficient Training Methods

Author Information

Corinna Cortes (Google Research)
Vitaly Kuznetsov (Google)
Mehryar Mohri (Courant Inst. of Math. Sciences & Google Research)
Dmitry Storcheus (Google Research)
Scott Yang (D. E. Shaw & Co.)

More from the Same Authors