Timezone: »

 
Workshop
Probabilistic Programming: Foundations and Applications (2 day)
Vikash Mansinghka · Daniel Roy · Noah Goodman

Sat Dec 08 07:30 AM -- 06:30 PM (PST) @ Tahoe A, Harrah’s Special Events Center 2nd Floor
Event URL: http://probabilistic-programming.org/wiki/NIPS*2012_Workshop »

Probabilistic models and algorithmic techniques for inference have become standard tools for interpreting data and building systems that learn from their experience. Growing out of an extensive body of work in machine learning, statistics, robotics, vision, artificial intelligence, neuroscience and cognitive science, rich probabilistic models and inference techniques have more recently spread to other branches of science and engineering, from astrophysics to climate science to marketing to web site personalization. This explosion is largely due to the development of probabilistic graphical models, which provide a formal lingua franca for modeling, and a common target for efficient inference algorithms.

However, even simple probabilistic models can require significant effort and specialized expertise to develop and use, frequently involving custom mathematics, algorithm design and software development. More innovative and useful models far outstrip the representational capacity of graphical models and their associated inference techniques. They are communicated using a mix of natural language, pseudo code, and formulas, often eliding crucial aspects such as fine-grained independence, abstraction and recursion, and are fit to data via special purpose, one-off inference algorithms.

PROBABILISTIC PROGRAMMING LANGUAGES aim to close this gap, going beyond graphical models in representational capacity while providing automatic probabilistic inference. Rather than marry statistics with graph theory, probabilistic programming marries Bayesian probability with universal computation. Instead of modeling joint distributions over a set of random variables, probabilistic programs model distributions over the execution histories of programs, including programs that analyze, transform and write other programs. Users specify a probabilistic model in its entirety (e.g., by writing code that generates a sample from the joint distribution) and inference follows automatically given the specification. These languages provide the full power of modern programming languages for describing complex distributions, and can enable reuse of libraries of models, support interactive modeling and formal verification, and provide a much-needed abstraction barrier to foster generic, efficient inference in universal model classes.

We believe that the probabilistic programming language approach within AI has the potential to fundamentally change the way we understand, design, build, test and deploy probabilistic systems. This approach has seen growing interest within AI over the last 10 years, and builds on over 40 years of work in range of diverse fields including mathematical logic, theoretical computer science, programming languages, as well as machine learning, computational statistics, systems biology, probabilistic AI. However, considerable engineering challenges need to be solved before these techniques can be broadly adopted --- such as making inference efficient and producing robust, general-purpose software for probabilistic programming --- as do foundational questions about the complexity of inference and learning.

Our 2008 NIPS workshop helped to create the probabilistic programming community. For our 2012 workshop, we propose to:

  • Assess and synthesize progress in probabilistic programming since 2008, including progress in languages like BLOG, Church, Figaro, ICL, Markov Logic, CSoft/Infer.NET and ProbLog

  • Organize the growing community around key technical problems and benchmarks, to spur and systematize progress in the development and analysis of probabilistic programming systems

  • Expose interested funding agencies to probabilistic programming research

Author Information

Vikash Mansinghka (Massachusetts Institute of Technology)

Vikash Mansinghka is a research scientist at MIT, where he leads the Probabilistic Computing Project. Vikash holds S.B. degrees in Mathematics and in Computer Science from MIT, as well as an M.Eng. in Computer Science and a PhD in Computation. He also held graduate fellowships from the National Science Foundation and MIT’s Lincoln Laboratory. His PhD dissertation on natively probabilistic computation won the MIT George M. Sprowls dissertation award in computer science, and his research on the Picture probabilistic programming language won an award at CVPR. He served on DARPA’s Information Science and Technology advisory board from 2010-2012, and currently serves on the editorial boards for the Journal of Machine Learning Research and the journal Statistics and Computation. He was an advisor to Google DeepMind and has co-founded two AI-related startups, one acquired and one currently operational.

Daniel Roy (U of Toronto; Vector)
Noah Goodman (Stanford University)

More from the Same Authors