Workshop Abstracts
Organizers: Terry Sejnowski, The Salk Institute and
the University of California, San Diego;
Steve Zucker, Yale University
The problem of finding sparse representations in overcomplete
bases has many applications in signal processing and is an area of intense
research. Overcomplete
representations can be more sparse than complete representations and have
greater flexibility in matching the structure in the data. In some circumstances the
overcomplete codes can have great
robustness to noise. Overcomplete
codes have also been proposed as a model of some of the response properties
of neurons in primary visual
cortex, which is x100 overcomplete compared to the number of input fibers from the retina. A wide range of topics will be covered, including
inference, learning, analysis, optimality criteria, natural statistics, and
applications.
For a complete list of topics and invited speakers see:
http://www.cnl.salk.edu/OvercompleteRepresentations.php
Learning in Graphics and
Vision
Organizers: Radek Grzeszczuk, Intel Research; Aaron
Hertzmann, University of Toronto
The goal of the workshop is bring together researchers using
data-driven methods in vision and graphics, who are often using similar tools
to solve complementary problems.
Recent years have seen a growing overlap between computer graphics and
computer vision; indeed, there has been much discussion about the
"convergence" of these two fields. In particular, computer graphics researchers are
increasingly drawn to "data-driven" methods for creating models of
shape, appearance, motion, and style, due to the enormous difficulty of
hand-crafting such models.
Moreover, machine learning methods are beginning to appear in computer
graphics research, while there is a long history of interaction between
learning and vision. Each of these
fields have a lot to offer to the others: computer graphics provides
high-quality and efficient visual models, computer vision provides effective visual
analysis, and machine learning provides principled techniques for analyzing
data in general. This workshop
aims to explore and strengthen these connections.
For further information see:
http://www.dgp.toronto.edu/~radek/nips2004
Structured Data and
Representations in Probabilistic Models for Categorization
Organizers: Josh
Tenenbaum, MIT; David Danks, Carnegie Mellon University; Chris Williams,
University of Edinburgh; Charles Kemp, MIT; Amy Perfors, MIT
While probabilistic models have historically been widespread in
both human and machine categorization, they have typically exploited only
minimal structure in either the data or the model. In both cognitive science
and machine learning, researchers have focused on categorization of minimally
structured data (usually flat vectors of feature/attribute values) using
relatively flat models (such as mixture models or sets of conditional
probability tables in machine learning, and various measures of distance to
prototypes or exemplars in computational psychological models). There has been
a recent surge of interest in computational cognitive science and machine
learning in developing models and
techniques that exploit additional
structure in either (i) the relations among the categories (e.g., hierarchical models), or (ii) the data
being categorized (e.g.,
relational data). This workshop will focus both on the recent advances in each individual field, and
on connections between them.
Example topics include: learning and inference with hierarchical or causal models of inter-category
relationships; category learning
and inference based on relationships between objects; learning and inference of hierarchical
probabilistic models for single
categories; and strongly structured models for image and text classification.
More information, as well as a call for posters, can be found at:
http://web.mit.edu/cocosci/nips04.html
Learning With Structured
Outputs
Organizers: Gškhan
Bakir, MPI; Arthur Gretton, MPI;
Thomas Hoffmann, Brown University; Bernhard Schšlkopf, MPI
Learning general functional dependencies between arbitrary input
and output spaces is one of the main goals in machine learning. Recent
progress, in particular in the field of kernel-based methods, has focused on
designing flexible and powerful input representations. This workshop aims to
investigate the complementary
issue of problems involving complex and structured outputs. The latter
covers numerous problems that cannot be modeled well in the standard settings
of regression or supervised classification and that are linked to highly
relevant applications. This includes problems concerning mappings from -
strings to strings (e.g. machine
translation, pronunciation models, protein secondary structure prediction), -
strings to trees (e.g. grammar learning, document markup), - matrices to
matrices (e.g. image restoration, super-resolution), - observation sequences to
label sequences (e.g. optical character recognition, information extraction),
The workshop intends to bring together an interdisciplinary group of
international researchers from machine learning, computational biology, natural
language processing, information retrieval, computer vision and other fields for
discussing results and dissemination of ideas.
For further information see:
http://nips2004.kyb.tuebingen.mpg.de/
Towards Brain Computer
Interfacing
Organizers: Guido
Dornhege, Fraunhofer FIRST.IDA; Thilo Hinterberger, University of Tuebingen;
Dennis McFarland, NYS Department of Health; Jose del R Millan, IDIAP Research
Institute; Klaus-Robert MŸller, Fraunhofer FIRST.IDA
During the last years research interest is growing to develop a so
called 'Brain-Computer Interface' which allows one-sided communication of
humans with machines like computers, wheelchairs or prostheses only by use of
brain signals. On the one hand, such an interface provides a new (and possibly
only) communication channel for people who suffer from severe physical
disabilities while having intact cognitive functions (e.g. ALS). For healthy
subjects on the other hand, this interface can enhance and facilitate
man-machine interaction by providing additional control options. Several results
during the last decades have proven that it is possible to implement a BCI
system and even some locked-in patients could use BCI systems to express their
thoughts and wishes to the outside world. But so far the applicability of BCI
systems is limited by factors, like high error rates, long training and
preparation times, high subject variability etc. During the workshop different issues (e.g. invasive vs.
non-invasive measurements of brain signals, subject training (operand
conditioning) vs. machine training, evoked potentials vs. spontaneous brain
responses, signal processing and machine learning techniques for BCI) and their
success, applicability in BCI and possible enhancements of existing BCI systems
will be discussed.
Details can be found on
http://www.bbci.de/nips04_workshop/index.html
Organizers: Nando de
Freitas, University of British Columbia; Daniel Huttenlocher, Cornell
Many algorithms for statistical inference have a O(N^2) time
dependence on the size of the state space (N). The N^2 cost arises whenever one
computes a kernel density estimate.
The list of learning problems, where this operation arises, includes
Gaussian processes, radial basis networks, SVMs, belief propagation, particle
smoothing, population Monte Carlo methods, multidimensional scaling and kernel
methods for dimensionality reduction, classification, regression and
semi-supervised learning. In recent years, researchers in different communities
(physics, numerical computation, machine learning and computer vision) have
proposed techniques to reduce this cost to O(NlogN) and even O(N) in some
cases. Examples of these techniques include fast multipole methods, the fast
Gauss transform, distance transforms, kd-trees, ball trees, dual tree
recursions and several other more specialized methods. The workshop will
provide tutorials on these various techniques as well as new developments and
comparisons. The target audience is anyone who would like to see their favorite
large N learning algorithm running in seconds as opposed to hours or days.
More information can be found at:
http://www.cs.ubc.ca/~nando/nipsfast/
Reinforcement Learning and
the Brain: Beyond the Dopamine
System
Organizers: Nathaniel
D. Daw, University College London; Samuel McClure, Princeton University
Essentially from its conception, the computational study of
reinforcement learning has been applied to understanding learning and decision
making in biological organisms. The most prominent success of this program has
been the suggestion that the release of the neuromodulator dopamine functions
as an error signal for the prediction and optimization of future reward, using
an algorithm like temporal-difference learning. The parallels between the
temporal-difference signal and the dopamine response have now been worked out
in great detail; what is still much less clear is how the rest of the brain
(particularly the dopamine neurons' inputs and targets) might participate in
reinforcement learning. Variations on the RL framework are now being used to
inform and analyze experiments focusing on many disparate brain systems and
using many different methodologies. This workshop will bring together
theoreticians and experimentalists involved in much of this work in order to
try to fit it into a larger understanding of the neural substrates of
reinforcement learning. In particular, we are focusing on research with broader
targets than the dopamine system itself.
More information is at:
http://www.gatsby.ucl.ac.uk/~daw/nips2004workshop/
Organizers: Richard
Shiffrin, Department of Psychology; Jason Gold, Department of Psychology;
Andrew Cohen, Department of Psychology; Florin Cutzu, Department
of Computer Science
The psychophysics and neuroscience communities have been looking
at correlations of noisy stimulus inputs with behavioral decisions and neural
responses in order to infer the stimulus features that mediate sensory and
perceptual processes. The Cognitive Science community has been developing
related techniques to identify stimulus features extracted by human observers
and testing the rules by which these features are combined to produce
categorical decisions. The computational/statistical/machine learning community
has been developing optimal and other techniques for classifying target
categories in noisy input. This workshop will bring together researchers from
each of these fields to learn about and discuss each other's approaches to
solving the feature induction problem.
Information on speakers and the workshop format will be posted as
they become available at: http://vislab.psych.indiana.edu/~jgold/jgold/jmg/nips2004/.
Organizers: Tue
Lehn-Schioler, The Technical University of Denmark; Samy Bengio, IDIAP; Lars
Kai Hansen, The Technical University of Denmark; Stephane Canu, l'INSA de Rouen
Natural signal processing systems have the ability to combine
impressions from different senses. An example of this can be seen in most human
to human interactions where speech, facial expression, smell, gestures,
haptics, etc play a role. While each of these modalities has been modeled at
high levels of sophistication, multi-media modeling is still in its infancy.
Focusing on sound-image combination and wearable computing the aim of the
workshop is to:
* Introduce multimodal problems to machine
learning researchers.
* Identify and compare different
integration strategies.
* Determine the major challenges in using
more than one modality.
* Propose novel methods to improve the
state of the art.
You will find more information at:
http://www.imm.dtu.dk/multimodal/
Organizers: Alex J.
Smola, National ICT Australia; S V N Vishwanathan, National ICT Australia; Ben
Taskar, Stanford University
Graphical models provide a natural method to model variables with
structured conditional independence properties. Kernel methods excel at
modeling data which need not be structured at all, by using mappings into
high-dimensional spaces (also popularly called the kernel trick). The
popularity of kernel methods is primarily due to their strong theoretical
foundations and the relatively simple convex optimization problems. Recent
progress towards a unification of the two areas has seen work on Maximum Margin
Markov Networks, structured output spaces, and kernelized Conditional Random
Fields. Some work has also been done on using fundamental properties of the
exponential family of probability distributions to establish links. The aim of
this workshop is to bring together researchers from both the communities together
in order to facilitate interactions.
More details, including a call for papers can be found at:
http://mlg.rsise.anu.edu.au/~smola/workshops/nips04/
MIPS -- Music Information
Processing Systems
Organizers: Douglas
Eck, University of Montreal; Daniel P. W. Ellis, Columbia University; Ali
Taylan Cemgil, University of Amsterdam; Jean-François Paiement, IDIAP Research
Institute;
MIPS (Music Information Processing Systems) is a Workshop focusing
on machine learning in the domain of music. Musical signals are dynamic and exhibit long term temporal
dependencies. Methods for discovering and representing these dependencies will
enable the development of algorithms for music analysis and generation. Of
particular interest are algorithms that use learned musical structure to
compose music and to interact with musicians. Additionally, creating and
navigating large databases of music are difficult information retrieval and
classification challenge. The MIPS workshop will explore state-of-the-art
machine learning methods in the domain of music. Areas of interest include, but
are not limited to: discovery of musical structure, generative models of music
(at signal level and above), interactive systems, music similarity, and
classification & retrieval. Workshop format includes a brief introduction
by workshop organizers, short morning and afternoon talks, two panel
discussions and a brief wrap-up by organizers. In addition to a lunch break, long
coffee breaks will be offered both in the morning and afternoon.
For more information please visit the MIPS website at:
http://www.iro.umontreal.ca/~eckdoug/mips
Organizers: Shai
Ben-David, University of Waterloo; John Langford, TTI-Chicago; John
Shawe-Taylor, Southampton; Bob Williamson, NICTA
The sample complexity bounds community has internal disagreements about
what is (and is not) a useful bound, what is (and is not) a tight bound, how
(and where) bounds might reasonably be used, and which bounds-related questions
should be answered. One goal of this workshop is to debate the merits of these
different issues in order to foster better understanding internally as well as
externally. This workshop is of
interest to the broader machine learning community, since notions of bounds
play a major role in many aspects of this area. To the best of our knowledge,
there has never been a workshop focused on the evaluation of this aspect of
learning theory, so the novelty factor will be quite high.
A URL for the workshop can be found at: http://www.hunch.net/~jl/conferences/abuse_of_bounds/abuse_of_bounds.html
The Neurobiology of Planning
and Deciding: Studies from Many Levels of Brain Organization
Organizers: Patricia
S. Churchland, University of California, San Diego; Terry Sejnowski, The Salk
Institute and the University of California, San Diego
In evolutionary terms, the fundamental business of nervous systems
is to enable the organism to move appropriately, so as to succeed in: feeding, fleeing, fighting, and
reproducing. Moving appropriately
requires integration of signals: incoming signals, stored information, and
signals reporting on the inner milieu of the organism. Using these signals, the nervous system
makes a decision about what the animal needs to do next, and perhaps next after
that. Until very recently, it has
not been possible to begin to piece together the story of how that integration
is achieved or the nature of the decision-making process: Are there distinct decision-stages,
from very general to highly specific?
How is relevant background information tapped? How are relevant options selected? How are options evaluated and used in planning? Some of the most exciting work in
neuroscience now is addressing these questions. From the level of behavior and the whole brain, to the level
of the single cell, research is beginning to reveal the structure and
organization of decisions, choices and plans. From the human perspective, choice and responsibility have
been at the heart of discussions about ethics and the nature of humankind. Consequently the new research invites
us to rethink longstanding assumptions about free will.
For information see: http://www.cnl.salk.edu/OvercompleteRepresentations.php
Calibration and
Probabilistic Prediction in Supervised Learning
Organizers: Rich
Caruana, Cornell University; Greg Grudic, University of Colorado
Calibration refers to how accurately the probabilities predicted
by a model correspond to empirical observations. For example, if a model predicts that 1000 cases have
predicted probability p = 0.2 of being positive class, the model is said to be
well calibrated if about 200 of the 1000 cases are in the positive class. A model is well calibrated if this is
true for all predicted values of p. Calibrated probabilistic models have
important applications in fields ranging from medicine to finance to particle
physics to robotics. Despite the
recent explosion of interest in probabilistic models in machine learning, there
has been little work in assessing the quality of the probabilistic predictions
models make. Measuring model
calibration is challenging, and classification models can have high accuracy or
ROC area, but be poorly calibrated.
In fact, learning methods such as boosting and other max margin methods
that perform well on many other measures reduce calibration as a consequence of
maximizing the margin. This workshop will focus on questions such as: When is
calibration important? How should
we measure calibration? What
learning methods yield good (or bad) calibration? Is it true that graphical models often yield poor
calibration? Can models that have excellent performance on other metrics, but
poor calibration, be calibrated so that they predict good probabilities?
For further information see:
http://www.cs.colorado.edu/~grudic/NIPS2004_Prob_Calib
Activity Recognition and
Discovery
Organizers: Irfan
Essa, Georgia Tech; Dieter Fox, University of Washington
Recognition of high-level events from data streams is a much
needed resource towards building intelligent machines that provide automatic
and autonomous support in our every day lives. Recently, there has been significant progress towards such
activity recognition and discovery from data. Research progress is evident in (a) building real systems
that extract information from a variety to sensors, to (b) developing and
extending statistical machine learning approaches to model data for recognition
and matching, to (c) the study of specific domains to extract relevant
higher-level context which in turn can be leveraged to support recognition.
However, many significant questions in this area remain and require discussion
among researchers from various fields.
In this workshop, we will bring together experts from machine learning,
sensing and perception, and ubiquitous computing to discuss important issues in
this area, with discussions focusing on what has been achieved to date and what
are the upcoming unsolved problems worth addressing.
More information at:
http://www.cc.gatech.edu/conferences/nips04-ARD/.
Interneurons and Cortical
Function: A Fair and Balanced Workshop
Organizers: Bartlett
W. Mel, University of Southern California; Judith A. Hirsch, University of
Southern California; Kenneth D. Miller, University of California
How many genuinely different roles do inhibitory neurons
play? An extensive body of
knowledge now exists, arising from several scientific and engineering
disciplines, that bears on this question. At an intellectual level, the area is
rich in possibilities: inhibitory interneurons are extremely diverse, ranging
dramatically in their sizes, dendritic and axonal morphologies, lamina of
origin, categories of synaptic input, post-synaptic targets, input and output
synaptic dynamics, firing patterns, receptive field properties, etc. At the
same time, inhibition has been proposed to subserve a large and diverse set of
functional roles in cortical processing, including map formation, control of
neural plasticity during critical periods elimination of statistical
dependencies between neurons, contrast gain control, contrast invariant tuning,
sensitivity to input synchrony, sensory adaptation, logical "veto"
operations, direction selectivity, light-dark opponency, figure-ground
segregation, bistability effects such as binocular rivalry, visual
segmentation, focal visual attention, saccadic suppression, predictive sensory
cancellation, synchronization of neuronal populations, generation of a number
of cortical rhythms, generation of sustained activity during working memory,
and the prevention of runaway excitation underlying epilepsy. Though inhibition
in cortex has been much studied, relatively little crosstalk has occurred among
investigators working in the diverse fields, particularly between
neurobiologically vs. computationally-oriented investigators, so that major
opportunities for conceptual progress remain to be exploited. This workshop will bring together
investigators using anatomical, physiological, computational, and technical
approaches to understand the roles of inhibitory interneurons in cortical information
processing. The goal of the
workshop, to be achieved through short talks and moderated discussion, will be
to look for structure in the data and distill out common principles, to look
for mappings between classes of cortical computations and classes of inhibitory
interneurons, and to identify key unanswered questions regarding the
multifarious roles of feedforward, lateral, and feedback inhibitory neurons in
cortical function.
For more information see:
http://lnc.usc.edu/NIPS04-workshop/index.html
Verification, Validation,
and Testing of Learning Systems
Organizers: Dragos
Margineantu, The Boeing Company; Pramod Gupta, NASA Ames Research Center;
Johann Schumann, NASA Ames Research Center; Michael Drumheller, The Boeing
Company; Roman Fresnedo, The Boeing Company
Learning has the potential to provide key capabilities to systems
that involve complex decision making (such as adaptive or autonomous systems).
In the meantime, most of these systems require a reliable deployment and operation,
and for most applications, in order to be deployed, learning components need to
be proven as "trustable" to the users (engineers, designers, quality
control specialists). Failures of these systems can occur and will occur,
regardless of whether they contain learned models or learning components, but
only very little research effort has been dedicated to developing principled
approaches and tools for assessing the goodness of complex systems that contain
learned components, estimating the quality of learned models in the context of
the actual problem that is addressed, assessing online and anytime learning
methods, evaluating learning methods employed in safety-critical tasks, or for
understanding the trade-offs between robustness and risk in making complex
decisions. The purpose of the workshop is to bring together researchers and
users of learning and adaptive systems and to create a forum for discussing
recent advances in verification, validation, and testing of learning systems,
to understand better the practical requirements for developing and deploying
learning systems, and to inspire research on new methods and techniques for
verification, validation, and testing.
Details about the topics of interest for the workshop and about
attending and submitting position papers can be found at:
http://www.dmargineantu.net/nips2004
New Problems and Methods in
Computational Biology
Organizers: Gal
Chechik, Department of Computer Science; Christina Leslie, Center for
Computational Learning Systems, Columbia University; Gunnar Ratsch, Max Planck
Institute for Biological Cybernetics (Tuebingen) and Fraunhofer FIRST (Berlin);
Koji Tsuda, Max Planck Institute
for Biological Cybernetics (Tuebingen) and AIST Computational Biology Research
Center (Tokyo)
The field of computational biology has seen a dramatic growth over
the past few years, both in terms of new available data, new scientific
questions and new challenges and for learning and inference. In particular,
biological data is often relationally structured and highly diverse, thus
requires to combine multiple weak evidence from heterogeneous sources.
The goal of this workshop is to present emerging problems and
machine learning techniques in computational biology. Speakers from the
biology/bioinformatics community will present current research problems in
bioinformatics, and we invite contributed talks on novel learning approaches in
computational biology. We encourage contributions describing either progress on
new bioinformatics problems or work on established problems using methods that
are substantially different from standard approaches. Kernel methods, graphical
models, feature selection and other techniques applied to relevant
bioinformatics problems would all be appropriate for the workshop.
For further information see:
http://www.stanford.edu/~gal/nips-compbio.html
Reinforcement Learning
Benchmarks and Bake-offs
Organizers: Richard
S. Sutton, University of Alberta; Michael L. Littman, Rutgers University
It has often been suggested that the field of reinforcement
learning would benefit from the establishment of standard benchmark problems
and regular competitive events (bake-offs). Competitions can greatly increase the interest and focus in
an area by clarifying its objectives and challenges, publicly acknowledging the
best algorithms, and generally making the area more exciting and
enjoyable. Standard benchmarks can
make it much easier to apply new algorithms to existing problems and thus
provide clear first steps toward their evaluation. This workshop will explore the establishment of standard
benchmarks and competitive events to enhance reinforcement-learning
research. The workshop will
ideally produce the following outputs: 1) a proposed specification for
implementing benchmark problems; 2) identification of a list of initial
benchmarks, with assignment of responsibility for their implementation; 3)
policies for extending the benchmark set to address new issues; 4) specific
proposals for a series of competitive events comparing different
reinforcement-learning methods on various kinds of problems; and 5) the
formation of a policy committee to guide the construction and evolution of the
benchmarks and competitions. For
the purposes of this workshop, "reinforcement learning" is meant to
include a broad range of interactive learning problems, including POMDPs,
navigation, control problems, probabilistic planning, and sequential prediction
problems with and without actions.
For further information see:
nEUro-IT.net Workshop on the Neural Mechanisms of Imitation
Organizers: Aude Billard, EPFL - Swiss Federal Institute of Technology; Stefan Schaal, University of Southern California
Imitation learning is a powerful mechanism for dimensionality reduction. When observing either good or bad examples, one can improve its search for a possible solution, by either starting the search from the observed good solution (local optima), or conversely, by eliminating from the search space what is known as a bad solution. Imitation learning appears thus to be a powerful tool for learning in both animals and artifacts. Imitation learning has for long been a key topic of Psychology and Robotics. Recent neurosciences evidence of specific brain pathways underlying primates’ imitation capability have set the ground for computational studies of these neural correlates. This workshop aims at assessing recent progresses in thus area. Because imitation learning has at core motor learning, the workshop will gather experts in both motor learning and imitation learning. Key questions that will be discussed in the workshop include: Can imitation use known motor learning techniques or does it require the development of new learning and control policies? How does imitation contribute and complement motor learning? Does imitation speed up skill learning? What are the costs of imitation learning? How could the metric of imitation learning drive the choice of learning technique? How could we define a general metric of imitation performance? What is the role of visual attention and gesture recognition in imitation? Do models of human kinematics, used in gesture recognition, drive the reproduction of the task? Can one find a level of representation of movement common to both gesture recognition and motor control?
For further information see: http://asl.epfl.ch/events/nips04Workshop/index.php