Tutorial Speakers
William Bialek [tutorial slides pdf]
Optimization Principles in Neural Coding and Computation
Abstract: We walk through the
world largely unaware of the computations done by our brains. Although we
tend to focus on our errors, most of the time these computations seem to get
the right answer or we probably wouldn't be here to argue about it. But
is this process one in which a complicated evolutionary history has led to
messy solutions that are nonetheless "good enough" for survival, or
is it possible that the brain in fact finds optimal or near optimal solutions
to its problems? Optimal solutions would be attractive, not least because
this would open the way to having a real theory of neural computation-based
optimization principles, in effect a theory of why the nervous system works the
way it does rather than just a model of how it works; this is an ambitious
goal. In this tutorial I will review the evidence for optimal performance
at a wide variety of tasks in neural coding and computation, ranging from
photon counting in vision to echolocation in bats and pitch perception in human
hearing. Recent work has focused attention on the crucial prediction that
-- given the statistical structure of our sensory world -- optimal strategies
are almost always adaptive, and hence one can do experiments to observe the
process of optimization in action; this has led to some dramatic results,
providing strong support for the applicability of optimization principles.
I will try to emphasize the generality of this approach, looking at many
different sensory systems and also making connections to work on the motor
system described in Wolpert's tutorial at this meeting. I will address
some classic examples of superficially sub-optimal behavior, and argue that
optimization principles for neural computation can generate counterintuitive
predictions that may illuminate these paradoxes. Finally, we'll step back
and ask if the many different optimization ideas can be unified, and if so
whether this provides a glimpse of a theory that is strong enough to carry us
from low level sensory processing all the way to cognition.
Bio:
William Bialek is the John Archibald
Wheeler/Battelle Professor in Physics at Princeton University. He
received his PhD in Biophysics from the University of California,
Berkeley. After postdoctoral appointments at the Rijksuniversiteit
Groningen in the Netherlands and at the Institute for Theoretical Physics in
Santa Barbara, he returned to Berkeley to join the faculty in 1986. In
late 1990, he moved to the newly formed NEC Research Institute, where he
eventually became an Institute Fellow. He joined the Princeton faculty in
2001. Best known for contributions to our understanding of neural coding,
Bialek and his collaborators have worked on a wide variety of theoretical
problems at the interface of physics and biology, from the dynamics of
individual biological molecules to learning and cognition. He served as
co-director of the computational neuroscience course at the Marine Biological
Laboratory in Woods Hole, Massachusetts from 1998 to 2002, and currently is
involved in a major educational experiment at Princeton to create an integrated
and mathematically sophisticated introduction to the natural sciences for first
year college students.
Shahar Mendelson [ tutorial slides pdf ]
Tutorial: A Geometric Approach to Statistical Learning Theory
Abstract: We
will present a survey of the recent developments in Statistical Learning Theory
from a geometric point of view. It turns out that understanding the
"hardness" of a prediction problem is governed by the geometry of the
class of functions one uses; in particular, what is most important is the
richness of finite dimensional coordinate projection the class (that is, the
restriction of the class to the sample points) as the dimension of the
projection (the size of the sample) increases. This observation has
resulted in a fruitful interplay between statistical learning theory and
asymptotic geometry. We will focus on two aspects which were particularly
influenced by Asymptotic Geometry. The first is prediction bounds. We will show
how the complexity of the class, as captured by the geometry of its coordinate
projections, determines the error rate, and the way classical parameters, such
as the combinatorial dimension (a real-valued version of the VC dimension) or
the covering numbers are simply a sufficient condition which ensures the required
control on the geometry of the class. The second topic we will explore is
the limitation of kernel methods. For example, we will present recent results
which show that kernel methods, or even any conceivable extension of these
methods (embedding into other spaces, other choices of a loss functional, etc),
are bound to fail in a very strong sense.
Bio: Shahar Mendelson
is a Fellow at the Institute of Advanced Studies at the Australian National
University. He received his BA in Mathematics in 1992, MSc in Mathematics in
1994 and PhD in Applied Mathematics in 1998, all from the Technion, I.I.T
(Haifa, Israel). His research interest is high dimensional
phenomena, and in particular, the connections between Asymptotic Geometric
Analysis and Learning Theory.
Radford Neal [ tutorial slides pdf ]
Tutorial:
Bayesian Methods for Machine Learning
Abstract:
The Bayesian approach to machine learning is
based on the idea that all forms of uncertainty should be represented using
probability. This begins with a formulation of the problem in terms
of a probabilistic model along with a "prior'' probability distribution
for the model's parameters, and continues through to the production of
predictions in probabilistic form. The Bayesian approach differs
fundamentally from "learning theory'' approaches, which focus on concepts
such as VC dimension and the bias-variance tradeoff, in which probability plays
a more restricted role. This difference mirrors the long-standing
contention in statistics between Bayesian and "frequentist'' approaches to
inference.
In this tutorial, I will introduce the Bayesian
idea with some simple examples, and show how it leads to some fundamental
differences in how one approaches problems in machine learning. I
will then discuss the two big challenges in applying Bayesian methods in
practice - how to deal intellectually with the complexity of realistic models,
and how to deal computationally with the high-dimensional integrals that are
needed to produce Bayesian predictions. As examples of complex models
that can nevertheless be understood well enough to be useful, I will briefly
discuss Bayesian neural network models, Gaussian process models, and Bayesian
mixture models with an infinite number of components. I will also briefly
discuss Markov chain Monte Carlo and variational methods for performing
Bayesian computations. I will then present some illustrations of how the
Bayesian approach can be used to tackle complex problems that are difficult to
handle by other methods, and to achieve performance on simpler problems that is
superior to that of non-Bayesian methods. Finally, I will discuss the
current limitations of Bayesian methods, as well as some currently popular Bayesian
techniques that need to be used with caution.
Bio:
Radford Neal is a professor in the departments
of Statistics and Computer Science at the University of Toronto, where he holds
the Canada Research Chair in Statistics and Machine Learning. He
received his BSc and MSc degrees in Computer Science from the University of
Calgary in 1977 and 1980, after which he spent some years working on software
development and research projects in industry and academia. He received
his PhD in Computer Science from the University of Toronto in 1994, and then
joined the University of Toronto faculty. His primary research interests
are flexible Bayesian models for complex problems, computation for such models
using Markov chain Monte Carlo methods, and applications of such models,
especially to problems in bioinformatics. He is the first recipient of
the Lindley Prize, awarded for innovative work in Bayesian statistics appearing
as a contributed paper in the proceedings of the Valencia and ISBA conferences.
David Parkes [ tutorial slides pdf ]
Computational Mechanism Design and Auctions
Abstract: Computational Mechanism Design (CMD) aims to
develop protocols to implement desirable outcomes in a distributed system of
self-interested agents. An outcome might define a joint plan of action, or an
allocation of resources to agents. CMD brings together economic concepts of
game-theoretic equilibrium and incentive-compatibility constraints with the
traditional computer science concerns of computational tractability and
communication complexity.
This tutorial is designed for a computer
science audience, and it should have interest to both novices and experts in
computational mechanism design and computational game theory. As motivation, we
will begin with a brief survey of some of the problems in multi-agent systems
to which CMD can be applied. The introduction will also provide a brief review
of the most relevant solution concepts from game theory. The tutorial will
close with some extended comments about future directions in CMD. The bulk of
the tutorial will be devoted to the following four topics: i)
Existing characterizations of strategyproof mechanisms; ii) The
Vickrey-Clarke-Groves mechanism: A case-study in centralized implementation and
the tension between approximation and strategyproofness. iii) Iterative
mechanisms: Towards decentralized implementations (addressing communication and
valuation complexity). iv) Future directions: Sequential implementation;
Distributed implementation; and Adaptive implementation.
Bio:
David Parkes is a Gordon McKay Assistant
Professor of Computer Science at Harvard University. He received his Ph.D.
degree in Computer and Information Science from the University of Pennsylvania
in 2001, and an M.Eng. (First class) in Engineering and Computing Science from
Oxford University in 1995. He has published technical papers on electronic
commerce, auction design, computational mechanism design, multi-agent systems,
and bounded-rationality. Dr. Parkes was awarded the prestigious NSF CAREER
Award in 2002, and the IBM Faculty Partnership Award in 2002 and 2003. He
serves on the editorial board of the Journal of Artificial Intelligence
Research, and on the Program Committee of a number of leading conferences in
artificial intelligence, multiagent systems, and electronic commerce.
Richard Szeliski [ tutorial slides pdf PowerPoint ]
Acquiring Detailed 3D Models From Images and Video
Abstract: Creating photo-realistic 3D models from images
and video has been one of the long-standing goals of computer vision. More
recently, image-based rendering techniques have been developed in computer
graphics for manipulating and rendering such models while maintaining visual
fidelity, often by going back to the original data and using various forms of
sampling. This parallels a similar evolution in many other fields,
including speech recognition and AI, where data-driven and learning approaches
now predominate over what used to be hand-crafted models and algorithms.
In this tutorial, I will overview how 3D and
temporal models are constructed from images and video. This
includes feature matching, camera pose estimation, structure (shape) recovery,
and appearance (albedo and BRDF) estimation. In the temporal domain, I
will discuss how Video Textures and other video-based rendering techniques can
maintain visual realism using data-driven approaches. I will also discuss
open research problems where machine learning and statistical approaches show
promise for future improvements.
Bio:
Richard Szeliski is a Senior Researcher in the
Vision Technology Group at Microsoft Research, where he is pursuing research in
3-D computer vision, video scene analysis, and image-based rendering. His
current focus is on constructing photorealistic 3D scene models from multiple
images and video. He received a Ph.D. degree in Computer Science from
Carnegie Mellon University, Pittsburgh, in 1988. He joined Microsoft
Research in 1995. Prior to Microsoft, he worked at Bell-Northern Research,
Schlumberger Palo Alto Research, the Artificial Intelligence Center of SRI International,
and the Cambridge Research Lab of Digital Equipment Corporation. Dr.
Szeliski has published over 100 research papers in computer vision, computer
graphics, medical imaging, and neural nets, as well as the book Bayesian
Modeling of Uncertainty in Low-Level Vision. He was a Program Committee
Chair for ICCV'2001, and has served on the Editorial Board of the International
Journal of Computer Vision and as an Associate Editor of the IEEE Transactions
on Pattern Analysis and Machine Intelligence.
Daniel Wolpert [ tutorial slides pdf ]
Probabilistic Computations in Human Sensorimotor Control
Abstract: Although computers can now beat grandmasters at
chess, no robot can manipulate a chess piece with the dexterity of a
six-year-old child. In this tutorial I will explore techniques for reverse
engineering the human sensorimotor control system. Over recent years there has
been an explosion of interest in using machine learning algorithms to
understand human performance. This tutorial will first provide a brief overview
of the new experimental methodologies available to study human sensorimotor
control, such as robotic and virtual reality interfaces. In the second part I
will review how machine learning concepts can be tested experimentally in
humans. This part will cover approaches such as Bayesian
decision theory, reinforcement learning, representation and generalization,
optimal control and learning, state estimation and prediction. This tutorial is
well-suited to theorists who would like to learn how to test their models or
develop algorithms that are inspired by human performance, as well as to
experimentalist who wish for a stronger link to theory.
Bio: Daniel Wolpert is Professor of Motor Neuroscience and co-director of
the Institute of Movement Neuroscience at the Institute of Neurology,
University College London. He received his bachelor's degree in medical
sciences at Cambridge University in 1985 and a clinical medical degree from
Oxford University in 1988. He received a PhD in Physiology at Oxford University
in 1992 and subsequently worked as a postdoctoral fellow in Michael Jordan's
group in the Department of Brain and Cognitive Sciences at MIT. He joined the
Sobell Department of Motor Neuroscience & Movement Disorders, Institute of
Neurology in 1995. His research interests are computational and experimental
approaches to human sensorimotor control.