Probabilistic Programming: Foundations and Applications (2 day)

Probabilistic models and algorithmic techniques for inference have become standard tools for interpreting data and building systems that learn from their experience. Growing out of an extensive body of work in machine learning, statistics, robotics, vision, artificial intelligence, neuroscience and cognitive science, rich probabilistic models and inference techniques have more recently spread to other branches of science and engineering, from astrophysics to climate science to marketing to web site personalization. This explosion is largely due to the development of probabilistic graphical models, which provide a formal lingua franca for modeling, and a common target for efficient inference algorithms.

However, even simple probabilistic models can require significant effort and specialized expertise to develop and use, frequently involving custom mathematics, algorithm design and software development. More innovative and useful models far outstrip the representational capacity of graphical models and their associated inference techniques. They are communicated using a mix of natural language, pseudo code, and formulas, often eliding crucial aspects such as fine-grained independence, abstraction and recursion, and are fit to data via special purpose, one-off inference algorithms.

PROBABILISTIC PROGRAMMING LANGUAGES aim to close this gap, going beyond graphical models in representational capacity while providing automatic probabilistic inference. Rather than marry statistics with graph theory, probabilistic programming marries Bayesian probability with universal computation. Instead of modeling joint distributions over a set of random variables, probabilistic programs model distributions over the execution histories of programs, including programs that analyze, transform and write other programs. Users specify a probabilistic model in its entirety (e.g., by writing code that generates a sample from the joint distribution) and inference follows automatically given the specification. These languages provide the full power of modern programming languages for describing complex distributions, and can enable reuse of libraries of models, support interactive modeling and formal verification, and provide a much-needed abstraction barrier to foster generic, efficient inference in universal model classes.

We believe that the probabilistic programming language approach within AI has the potential to fundamentally change the way we understand, design, build, test and deploy probabilistic systems. This approach has seen growing interest within AI over the last 10 years, and builds on over 40 years of work in range of diverse fields including mathematical logic, theoretical computer science, programming languages, as well as machine learning, computational statistics, systems biology, probabilistic AI. However, considerable engineering challenges need to be solved before these techniques can be broadly adopted --- such as making inference efficient and producing robust, general-purpose software for probabilistic programming --- as do foundational questions about the complexity of inference and learning.

Our 2008 NIPS workshop helped to create the probabilistic programming community. For our 2012 workshop, we propose to:

• Assess and synthesize progress in probabilistic programming since 2008, including progress in languages like BLOG, Church, Figaro, ICL, Markov Logic, CSoft/Infer.NET and ProbLog

• Organize the growing community around key technical problems and benchmarks, to spur and systematize progress in the development and analysis of probabilistic programming systems

• Expose interested funding agencies to probabilistic programming research