Timezone: »

 
Workshop
Confluence between Kernel Methods and Graphical Models
Le Song · Arthur Gretton · Alexander Smola

Sat Dec 07:30 AM -- 06:30 PM PST @ Emerald Bay 2, Harveys Convention Center Floor (CC)
Event URL: https://sites.google.com/site/kernelgraphical/ »

Website: https://sites.google.com/site/kernelgraphical/

Kernel methods and graphical models are two important families of techniques for machine learning. Our community has witnessed many major but separate advances in the theory and applications of both subfields. For kernel methods, the advances include kernels on structured data, Hilbert-space embeddings of distributions, and applications of kernel methods to multiple kernel learning, transfer learning, and multi-task learning. For graphical models, the advances include variational inference, nonparametric Bayes techniques, and applications of graphical models to topic modeling, computational biology and social network problems.

This workshop addresses two main research questions: first, how may kernel methods be used to address difficult learning problems for graphical models, such as inference for multi-modal continuous distributions on many variables, and dealing with non-conjugate priors? And second, how might kernel methods be advanced by bringing in concepts from graphical models, for instance by incorporating sophisticated conditional independence structures, latent variables, and prior information?


Kernel algorithms have traditionally had the advantage of being solved via convex optimization or eigenproblems, and having strong statistical guarantees on convergence. The graphical model literature has focused on modelling complex dependence structures in a flexible way, although approximations may be required to make inference tractable. Can we develop a new set of methods which blend these strengths?

There has recently been a number of publications combining kernel and graphical model techniques, including kernel hidden Markov models [SBSGS08], kernel belief propagation [SGBLG11], kernel Bayes rule [FSG11], kernel topic models [HSHG12], kernel variational inference [GHB12], kernel herding as Bayesian quadrature [HD12], kernel beta processes [RWDC08], a connection between kernel k-means and Bayesian nonparametrics [KJ12] and kernel determinantal point processes for recommendations [KT12]. Each of these results deals with different inference tasks, and makes use of a range of RKHS properties. We propose this workshop so as to "connect the dots" and develop a unified toolkit to address a broad range of learning problems, to the mutual benefit of researchers in kernels and graphical models. The goals of the workshop are thus twofold: first, to provide an accessible review and synthesis of recent results combining graphical models and kernels. Second, to provide a discussion forum for open problems and technical challenges.

Selected bibliography:
%%%%%%%%%%%%%

[SBSGS08] Song, L. and Boots, B. and Siddiqi, S. and Gordon, G. and Smola, A., Hilbert space embeddings of hidden Markov models, ICML'10.

[SGBLG11] Song, L. and Gretton, A. and Bickson, D. and Low, Y. and Guestrin, C., Kernel belief propagation, AISTATS'11.

[FSG11] Fukumizu, K. and Song, L. and Gretton, A., Kernel Bayes rules, NIPS'11.

[HSHG12] Hennig, P. and Stern, D. and Herbrich, R. and Graepel, T., Kernel topic models, AISTATS'12.

[HD12] Huszar, F. and Duvenaud, D., Optimally-weighted herding is Bayesian quadrature, UAI'12.

[RWDC08] Ren, L. and Wang, Y. and Dunson, D.B. and Carin, L., Kernel beta processes, NIPS'11.

[GHB12] Gershman, S. and Hoffman, M. and Blei, D., Nonparametric variational inference, ICML'12.

[KJ12] Kulis, B. and Jordan, M., Revisiting k-means: new algorithms via Bayesian nonparametrics, ICML'12.

[KT12] Kulesza, A. and Taskar, B., Determinantal point processes for machine learning, arXiv:1207.6083

Author Information

Le Song (Ant Financial & Georgia Institute of Technology)
Arthur Gretton (Gatsby Unit, UCL)

Arthur Gretton is a Professor with the Gatsby Computational Neuroscience Unit at UCL. He received degrees in Physics and Systems Engineering from the Australian National University, and a PhD with Microsoft Research and the Signal Processing and Communications Laboratory at the University of Cambridge. He previously worked at the MPI for Biological Cybernetics, and at the Machine Learning Department, Carnegie Mellon University. Arthur's recent research interests in machine learning include the design and training of generative models, both implicit (e.g. GANs) and explicit (high/infinite dimensional exponential family models), nonparametric hypothesis testing, and kernel methods. He has been an associate editor at IEEE Transactions on Pattern Analysis and Machine Intelligence from 2009 to 2013, an Action Editor for JMLR since April 2013, an Area Chair for NeurIPS in 2008 and 2009, a Senior Area Chair for NeurIPS in 2018, an Area Chair for ICML in 2011 and 2012, and a member of the COLT Program Committee in 2013. Arthur was program chair for AISTATS in 2016 (with Christian Robert), tutorials chair for ICML 2018 (with Ruslan Salakhutdinov), workshops chair for ICML 2019 (with Honglak Lee), program chair for the Dali workshop in 2019 (with Krikamol Muandet and Shakir Mohammed), and co-organsier of the Machine Learning Summer School 2019 in London (with Marc Deisenroth).

Alex Smola (Amazon - We are hiring!)

**Amazon AWS Machine Learning** We are hiring!

More from the Same Authors