Timezone: »

 
Poster
Efficient multiple hyperparameter learning for log-linear models
Chuong B Do · Chuan-Sheng Foo · Andrew Y Ng

Mon Dec 03 10:30 AM -- 10:40 AM (PST) @
in Poster Session »

Using multiple regularization hyperparameters is an effective method for managing model complexity in problems where input features have varying amounts of noise. While algorithms for choosing multiple hyperparameters are often used in neural networks and support vector machines, they are not common in structured prediction tasks, such as sequence labeling or parsing. In this paper, we consider the problem of learning regularization hyperparameters for log-linear models, a class of probabilistic models for structured prediction tasks which includes conditional random fields (CRFs). Using an implicit differentiation trick, we derive an efficient gradient-based method for learning Gaussian regularization priors with multiple hyperparameters. In both simulations and the real-world task of computational RNA secondary structure prediction, we find that multiple hyperparameter learning provides a significant boost in accuracy compared to models learned using only a single regularization hyperparameter.

[ PDF] 

Author Information

Chuong B Do (Stanford University)
Chuan-Sheng Foo (Computer Science Department, Stanford University)
Andrew Y Ng (DeepLearning.AI)

More from the Same Authors