Recent successes in computer vision, natural language processing and other areas of artificial intelligence have been largely driven by methods for sophisticated pattern recognition — most prominently deep neural networks. But human intelligence is more than just pattern recognition. In particular, it depends on a suite of cognitive capacities for modeling the world: for making judgment calls in ambiguous situations, explaining and understanding what we see, imagining things we could see but haven’t yet, solving problems and planning actions to make these things real, and building new models as we learn more about the world. We will talk about prospects for reverse-engineering these capacities at the heart of human intelligence, and using what we learn to make machines smarter in more human-like ways. We introduce basic concepts and techniques of probabilistic programs, inference programming and program induction, which together with tools from deep learning and modern video game engines provide an approach to capturing many aspects of everyday intelligence.

Specific units in our tutorial will show how:

(1) Defining probabilistic programs over algorithms and representations drawn from modern video game engines — graphics engines, physics engines, and planning engines — allows us to capture how people can perceive rich three-dimensional structure in visual scenes and objects, perceive and predict objects' motion based on their physical characteristics, and infer the mental states of other people from observing their actions.

(2) By formulating model learning as higher-order inference in these systems, we can construct ``program-learning programs’’. These programs can learn new concepts from just one or a few examples.

(3) It is possible to build probabilistic programming systems in which scalable, general-purpose, efficient inference and model discovery algorithms can be easily and flexibly programmed by end users. These languages provide powerful tools for robotics, interactive data analysis, and scientific discovery.