NIPS : Conferences : 2003 : Workshops

Workshops

NIPS Workshops provide multi-track intensive sessions on a wide range of topics. The venue and schedule facilitate informality and depth.

The workshops will be held at the Westin Resort and Spa in Whistler, British Columbia, Canada, December 11 through 13, 2003. The Workshop sessions will end with a banquet on Saturday evening, December 13, 2003.

Workshop Schedule

Thursday, December 11, 2003
6:30pm - 8:30pm	Reception and Registration
Friday, December 12, 2003
7:00am - 11:00am	Registration
7:30am - 10:30am	Workshop Sessions
4:00pm - 7:00pm	Workshop Sessions
Saturday, December 13, 2003
7:30am - 10:30am	Workshop Sessions
4:00pm - 7:00pm	Workshop Sessions
7:30pm - 10:30pm	Closing Banquet

Workshops

Note: Please review the Workshop Preferences page and let us know which Workshops you are likely to attend.

Workshops – December 12-13, 2003

Two-day Workshops – Friday and Saturday:

Neural-Inspired Architectures for Nanoelectronics

Valeriu Beiu, Washington State University; Ulrich Ruckert, Heinz Nixdorf Institute/Paderborn University, Germany

Robust Communication Dynamics in Complex Networks

Rajarshi Das, IBM T. J. Watson Research Laboratory, New York; Irina RIsh, IBM T. J. Watson Research Laboratory, New York; Gerry Tesauro, IBM T. J. Watson Research Laboratory, New York; Cris Moore, University of New Mexico and Santa Fe Institute

Friday Workshops:

Estimation of Entropy and Information of Undersampled Probability Distributions: Theory, Algorithms, and Applications to the Neural Code

William Bialek, Princeton University; Ilya Nemenman, University of California at Santa Barbara

Feature Extraction Challenge

Isabelle Guyon, Clopinet Enterprises, California; Masoud Nikravesh, University of California, Berkeley; Kristin Bennett, Rensselaer Polytechnic Institute, New York; Richard Caruana, Carnegie Mellon University; Asa Ben-Hur, Stanford University; André Elisseeff, IBM, Zurich, Switzerland; Gideon Dror, Academic College of Tel-Aviv-Yaffo, Tel Aviv, Israel; Steve Gunn, University of Southampton, UK

Hyperspectral Remote Sensing and Machine Learning

J. Anthony Gualtieri, Applied Informations System and Global Science and Technology, NASA/GSFC, Maryland, USA

Machine Learning Meets the User Interface

John Shawe-Taylor, Department of Electronics and Computer Science, University of Southampton; John Platt, Microsoft Research

Neural Representation of Uncertainty

Sophie Deneve, Gatsby Computational Neuroscience Unit, UCL, London; Angela J Yu, Gatsby Computational Neuroscience Unit, UCL, London

New Problems and Methods in Bioinformatics

Christina Leslie, Columbia University; William Stafford Noble, University of Washington

The RNNaissance Workshop (Recurrent Neural Networks)

Juergen Schmidhuber, IDSIA, Manno-Lugano, Switzerland; Alex Graves, IDSIA, Manno-Lugano, Switzerland; Bram Bakker, IDSIA, Manno-Lugano, Switzerland, and University of Amsterdam, The Netherlands

Syntax, Semantics, and Statistics

Richard M. Shiffrin, Indiana University, Bloomington; Mark Steyvers, University of California Irvine; David Blei, University of California, Berkeley; Tom Griffiths, Stanford University

Saturday, December 13:

Approximate Nearest Neighbor Techniques for Local Learning and Perception

Trevor Darrell, MIT Computer Science and Artificial Intelligence Lab; Piotr Indyk, MIT Computer Science and Artificial Intelligence Lab; Greg Shakhnarovich, MIT Computer Science and Artificial Intelligence Lab; Paul Viola, Microsoft Research

Computing With Spikes: Implementation of Biology and Theory

Ralph Etienne-Cummings, Johns Hopkins University and University of Maryland; Timothy Horiuchi, University of Maryland; Giacomo Indiveri, Institute of Neuroinformatics, ETHZ, Zurich

ICA: Sparse Representations in Signal Processing

Barak A. Pearlmutter, Hamilton Institute, NUI Maynooth; Scott T. Rickard, University College Dublin, Ireland; Justinian Rosca, Siemens Corporate Research; Stefan Harmeling, Fraunhofer FIRST, Berlin

Information Theory and Learning: The Bottleneck and Information Distortion Approach

Naftali Tishby, Hebrew University, Israel; Tomas Gedeon, Montana State University

Neural Processing of Complex Acoustic Signals

Melissa Dominguez, McMaster University, Ontario; Ian C. Bruce, McMaster University, Ontario; Sue Becker, McMaster University, Ontario

Nonparametric Bayesian Methods and Infinite Models

Matthew J. Beal, University of Toronto, Ontario; Yee-Whye Teh, University of California at Berkeley

Open Challenges in Cognitive Vision

Barbara Caputo, NADA --CVAP/CAS, KTH, Stockholm; Henrik Christensen, NADA --CVAP/CAS, KTH, Stockholm; Christian Wallraven, MPIK, Tuebingen

Planning for the Real-World: The Promises and Challenges of Dealing with Uncertainty

Joelle Pineau, Carnegie Mellon University; Nicholas Roy, Massachusetts Institute of Technology; Drew Bagnell, Carnegie Mellon University

****************************************************

Workshop Abstracts

Approximate Nearest Neighbors Methods for Learning and Vision

Recent research efforts in the theoretical and algorithmic community have produced algorithms for very fast approximate similarity search, which have the potential of making example-based learning feasible again. There has been growing interest in the learning and vision community in this topic, and in the last two years some published results have started to appear. The main goal of this workshop is to try to formulate and analyze research agenda in this new area, and to identify important directions in which related research will move in the following years. By bringing together researchers from the communities working on machine learning, vision and theory we aim at developing common understanding of the potential and limitations of the newly available tools, what problems can benefit from these tools, and share the experience and insights that have been accumulating in the past few years.

Further details can be found at: http://www.ai.mit.edu/projects/vip/nips03ann

Computing With Spikes: Implementation of Biology and Theory

Ralph Etienne-Cummings, Johns Hopkins University and University of Maryland; Timothy Horiuchi, University of Maryland; Giacomo Indiveri, Institute of Neuroinformatics, ETHZ, Zurich

Most neurons of the brain communicate using trains of voltage spikes. These spike trains carry complex spatiotemporal information, without degradation, over long distances. Spikes are not, however, only used for communications. They are also used for computation. For example, within the cortex, spike are used by neural microcircuits that span multiple layers and also recurrently within layers. The rates, patterns or oscillations of single spike trains, as well as correlations and synchrony across different spike trains, microcircuits and brain regions, have been studied by many investigators, however, many open questions still remain on the roles of spiking in biology and the computational efficacy of spike based computation. More recently, there has been a push in the implementation community to also mimic the spiking nature of neural computation. This workshop will attempt to lay biological and theoretic foundation of spike-based computation, and discuss some of the open questions in the field, as it pertains to implementation.

Further details can be found at: http://www.ini.unizh.ch/~giacomo/nips03_workshop

Estimation of Entropy and Information of Undersampled Probability Distributions: Theory, Algorithms, and Applications to the Neural Code

William Bialek, Princeton University; Ilya Nemenman, University of California at Santa Barbara

For many biological and engineering applications it is often important to estimate entropies and other information theoretic quantities from small samples. No general way of doing so exists, but many ideas have surfaced recently indicating that some progress is possible with surprisingly few additional assumptions. This workshop will bring together scientists developing such methods and experimentalists, notably neuroscientists, who may benefit from using them. We expect to discuss a broad range of approaches, including (a) traditional compression-based (using repetitions in the spirit of Lempel-Ziv), (b) frequentist statistics, (c) various Bayesian approaches, (d) methods based on empirically observed properties of various probability distributions. We may also discuss (e) negative results for entropy and information estimation, (f) PAC-type bounds on the estimates, (g) computational complexity of entropy approximation. Finally, we expect that some of the speakers will present preliminary results of experimental applications of their methods.

The workshop's home page is: http://www.menem.com/~ilya/pages/NIPS03/index.html

Feature Extraction Challenge

Isabelle Guyon, Clopinet Enterprises, California; Masoud Nikravesh, University of California, Berkeley; Kristin Bennett, Rensselaer Polytechnic Institute, New York; Richard Caruana, Carnegie Mellon University; Asa Ben-Hur, Stanford University; André Elisseeff, IBM, Zurich, Switzerland; Fernando Perez-Cruz, University Carlos III, Madrid, Spain; Steve Gunn, University of Southampton, UK

Our NIPS 2003 workshop and its associated **benchmark** will push the frontiers of comparison and unification of feature extraction methods, including feature construction and feature selection. We welcome workshop contributions to experimental design, algorithms, theoretical analysis, and applications. Part of the workshop will be devoted to presentations and discussions of the result of a challenge on feature selection. We formatted five datasets for the purpose of benchmarking feature selection algorithms in a controlled manner.

To participate and/or compete see: http://clopinet.com/isabelle/Projects/NIPS2003/. Deadline December 1st, 2003. Good luck!

Hyperspectral Remote Sensing and Machine Learning

J. Anthony Gualtieri, Applied Informations System and Global Science and Technology, NASA/GSFC, Maryland, USA

The rapid growth of high-dimensional data sets from hyperspectral remote sensors has put a premium on finding new analysis techniques capable of extracting the full information content in such data. For example NASA's AVIRIS and Hyperion instruments can in a few minutes generate hyperspectral data cubes of order 600 X 5000 pixels X 220 contiguous spectral measurements or about 1.3 GB of data. The data cube can be envisioned as a stack of 220 registered gray scale images, each one representing the radiance in a narrow (typically 10 nm) channel around a wavelength from 400 nm through the visible out to the near infrared at 2400 nm. Thus at each pixel there is radiance spectra. These sensors ability to generate data has outstripped our ability to extract the information in the data. Existing analysis of hyperspectral data has demonstrated many applications including vegetation mapping down to the species level, assessment of plant health, environmental monitoring for remediation, and mineral detection, to name a few. However, it is also said that only 10-20% of the information in the data is currently being extracted. It is the high-dimensionality of the data, the large number of features attached to a pixel - also known as Bellman's curse of dimensionality -- that has caused the failure of analysis techniques that are successful when used for multi-resolution data sets with ten or fewer spectral bands, such as Landsat. This often leads researchers to perform spectral sub-setting of the data to reach manageable dimensionality -- but at the loss of using all the data. At the same time our NIPS community has seen dramatic success in bringing machine learning methods to bear on large high-dimensional data sets for the purposes of exploratory data analysis, and unsupervised and supervised learning.

The intent of this workshop, described at: http://backserv.gsfc.nasa.gov/nips2003hyperspectral.html is to bring practitioners from hyperspectral remote sensing who are keenly aware of the shortcomings of conventional techniques together with NIPS researchers who know methods that can improve on conventional techniques.

ICA: Sparse Representations in Signal Processing

Barak A. Pearlmutter, Hamilton Institute, NUI Maynooth; Scott T. Rickard, University College Dublin, Ireland; Justinian Rosca, Siemens Corporate Research; Stefan Harmeling, Fraunhofer FIRST, Berlin

We continue the tradition of annual NIPS Workshops on ICA, with a topic of concentration on Sparse Representations. Signal processing methods relying on "sparse representations" have proven useful in compression, de-noising, classification, and inverse problems in imaging, acoustics/speech, and communications. Sparseness in the context of Blind Source Separation (BSS) or Independent Component Analysis (ICA) have led to powerful techniques capable of, for example, separating many speech signals from two (or in some cases just one!) mixtures. De-mixing more sources than sensors is of particular interest because the classical mathematical formulation of that problem is ill-posed, and only through the exploitation of sparse representations of the signals of interest have efficient solutions become possible. Sparse representations have also led to significant improvements in the performance of the BSS/ICA techniques in the standard case, when the number of sources is equal to or smaller than the number of sensors, as in the analysis of various sorts of functional brain imaging data. We will focus on: definitions and measures of sparsity; methods of efficiently finding sparse representations; application of sparse representations to BSS/ICA problems.

For more information, consult the workshop webpage: http://www.first.fraunhofer.de/~harmeli/nips_workshop03/

Information Theory and Learning: The Bottleneck and Information Distortion Approach

Naftali Tishby, Hebrew University, Israel; Tomas Gedeon, Montana State University

Information Bottleneck is an unsupervised non-parametric data organization technique, which has drawn much attention in recent years. It has been successfully used in clustering documents, data mining and neural coding problems. Related approaches include: information distortion method used in neural coding problem and deterministic annealing used in clustering, compression, regression and other optimization problems. The method is also related to IMAX (Becker & Hinton) and Infomax (Linsker). The workshop will provide an overview of the method and its recent extensions and as a forum to exchange ideas between various groups which use these techniques. By bringing together theorists and practitioners in we hope to expose and discuss the theoretical developments and improvements in the various algorithms.

The web address is: http://www.cs.huji.ac.il/~tishby/NIPS-Workshop

Machine Learning Meets the User Interface

John Shawe-Taylor, Department of Electronics and Computer Science, University of Southampton; John Platt, Microsoft Research

How can we make computers interact more intelligently with us? Does the field of Human/Computer Interface (HCI) suggest challenging new problems for machine learning? This workshop will address these and other related questions. We will focus discussion on four topics in HCI which have the greatest connection to machine learning: 1) user modeling & personalization, 2) multimodal & perceptual user interfaces, 3) autonomous agents, and 4) intelligent dialog systems. The goal of the workshop is to cross-fertilize HCI with machine learning by fostering discussion between researchers in the two fields.

Further details can be found at: http://research.microsoft.com/workshops/MLUI03

Neural-Inspired Architectures for Nanoelectronics

Valeriu Beiu, Washington State University; Ulrich Ruckert, Heinz Nixdorf Institute/Paderborn University, Germany

This workshop aims to bring together experts from neuroscience, computing, and nanoelectronis for discussing novel brain-inspired nano-architectures, and how these will be able to satisfy future stringent requirements on: (i) low-power; (ii) high reliability;(iii) on-line reconfigurability (also in relation to reliability); (iv) asynchronous communication (also in relation to low-power); and (v) manageable design complexity.

Further details can be found at: http://www.eecs.wsu.edu/~vbeiu/workshop_nips03/

Neural Processing of Complex Acoustic Signals

Melissa Dominguez, McMaster University, Ontario; Ian C. Bruce, McMaster University, Ontario; Sue Becker, McMaster University, Ontario

The human auditory system is capable of processing complex acoustic stimuli such as speech, music, and rich environmental "soundscapes" in a way that is still unrivaled by human-designed algorithms. Therefore, auditory physiology research is increasingly focused on the neural mechanisms underlying the processing of such complex stimuli. Some modeling efforts are likewise concerned with capturing these mechanisms in order to aid in the development of devices and treatments for auditory dysfunction. The aim of this workshop is to discuss the latest experimental and modeling studies of neural representations of complex stimuli at various levels of the auditory pathways.

The workshop webpage can be accessed at: http://psycserv.mcmaster.ca/~domingm/workshop/acoustic.html

Neural Representation of Uncertainty

Sophie Deneve, Gatsby Computational Neuroscience Unit, UCL, London; Angela J Yu, Gatsby Computational Neuroscience Unit, UCL, London

Perception, action, learning, and decision making are thought to require sophisticated internal models that represent regularity as well as stochasticity of the external world. Among other things, probabilistic uncertainty can arise from neuronal noise, nonstationarity of the environment, and incompleteness of sensory knowledge. An increasing number of studies indicates that humans and other animals use uncertainty information to perform optimal probabilistic computations, such as Bayesian sensory cue integration, reward prediction, and Kalman filtering for motor control. However, relatively little is known about the neural basis of this capacity. In this workshop, we will approach this important issue from various vantage points: attention and neuromodulation, multisensory integration, neuronal integration and decision making, population coding, belief propagation, and optimal inference in recurrent networks.

Further details can be found at: http://www.gatsby.ucl.ac.uk/~sdeneve/workshop2003.html

New Problems and Methods in Bioinformatics

Christina Leslie, Columbia Universit; William Stafford Noble, University of Washington; Koji Tsuda, Max Planck Institute for Biological Cybernetics, Tuebingen

Over the past few years, the field of computational biology has seen a dramatic growth in diverse kinds of data: sequenced genomes for an increasing number of organisms, gene expression data from multiple technologies, protein sequence and 3D structural data, protein-protein interaction data, gene ontology and pathway databases, annotated single nucleotide polymorphisms (SNPs) and other types of genetic variations in humans, and an enormous amount of text data in the form of the biological and medical scientific literature. Clearly, the field presents both opportunities and challenges to the machine learning community, as new learning problems arise both as a result of new sources of high throughput data and recent developments in biology. The goal of this workshop is to present emerging problems and machine learning techniques in computational biology. We plan to have several speakers from the biology/bioinformatics community who will present current research problems in bioinformatics, and we invite contributed talks on novel learning approaches in computational biology. We encourage contributions describing either progress on new bioinformatics problems or work on established problems using methods that are substantially different from standard approaches.

Deadline for submissions is November 15. More information will be posted on the workshop website at: http://www.cs.columbia.edu/compbio/nips-bioinfo.html

Nonparametric Bayesian Methods and Infinite Models

Matthew J. Beal, University of Toronto, Ontario; Yee-Whye Teh, University of California at Berkeley

A long standing issue with learning in graphical models has been determining appropriate model sizes and structure. In many real world applications, traditional models with a small number of latent variables seem inadequate. In the quest for creating more flexible modelling tools, recent research has turned to the limit of such models with infinitely many latent variables and parameters, and often involving the use of Dirichlet processes. This limit corresponds to the field traditionally covered by nonparametric Bayesian statistics, which assumes a priori that the data was generated from a nonparametric model, with a possibly infinite number of parameters, experts, or hidden states, etc. The workshop will bring together researchers and practitioners of nonparametric Bayesian methods, both statisticians and machine learners, to share their methodologies and expertise.

Further details can be found at: http://www.cs.toronto.edu/~beal/npbayes

Open Challenges in Cognitive Vision

Barbara Caputo, NADA --CVAP/CAS, KTH, Stockholm; Henrik Christensen, NADA --CVAP/CAS, KTH, Stockholm; Christian Wallraven, MPIK, Tuebingen

Basic visual operations such as categorization and complex tasks such as scene interpretation have long been major challenges for computational vision. At least some of the these issues call for integration of methods into systems. Construction of systems for operation in realistic environments requires integration of methods from signal processing, geometry, statistics and reasoning. Naturally a number of both component methods and system level behaviors can be acquired from studies of biological systems. Traditionally vision has been studied using a reductionistic approach. Given the complexity of cognitive tasks, however, it is not obvious that such an approach is the most efficient way to address the core problems. Some issues such as multi-cue figure ground segmentation, embodied categorization, and behavior / skill acquisition can only be studied in the context of systems. Recent progress in studies of categorization, statistical learning theory, active perception, software engineering and computational neuroscience is paving the way for improved understanding of cognitive functionalities in biological and artificial systems. This workshop will focus on discussion of components methods such as memory, learning, categorization and the integration of these into systems. The emphasis will be on assessment of state of the art and identification of key challenges.

More informations to be found at: http://www.nada.kth.se/~caputo/cognitive-nipsw03.html

Planning for the Real-World: The Promises and Challenges of Dealing with Uncertainty

Joelle Pineau, Carnegie Mellon University; Nicholas Roy, Massachusetts Institute of Technology; Drew Bagnell, Carnegie Mellon University

As autonomous systems move towards increasingly complex real-world domains, new challenges are arising with respect to integrated planning and control. Foremost amongst these is the problem of planning and acting under uncertainty, however this has long been dominated by overwhelming computational costs. In recent years, more scalable approaches have been proposed, some of which have been applied to real-world problems. In light of these novel approaches, the goal of this workshop is to discuss three important questions. 1) What are currently the best techniques for planning under uncertainty? 2) What are the remaining frontiers in terms of algorithmic challenges, for planning under uncertainty? 3) What are the open real-world problems where planning under uncertainty can make a difference?

Workshop URL: http://www.cs.cmu.edu/~nickr/nips_workshop

The RNNaissance Workshop (Recurrent Neural Networks)

Juergen Schmidhuber, IDSIA, Manno-Lugano, Switzerland; Alex Graves, IDSIA, Manno-Lugano, Switzerland; Bram Bakker, IDSIA, Manno-Lugano, Switzerland, and University of Amsterdam, The Netherlands

Recurrent neural networks (RNNs) are currently experiencing a second wave of attention. The enthusiasm of the 1980s and early 90s was fuelled by the obvious theoretical advantages of RNNs: unlike feedforward neural networks (FNNs) and SVMs, RNNs have an internal state which is essential for many temporal processing tasks. And unlike in HMMs those internal states can take on both discrete and continuous values. Practitioners, however, had sobering experiences when they tried to apply RNNs to speech recognition, robot control, and other important problems that require sequential processing of information. The first RNNs simply did not work very well, and their functioning was poorly understood, since it is inherently more complex than the one of FNNs. Recent progress, however, has overcome major drawbacks of traditional RNNs. This progress has come in the form of new architectures, learning algorithms (including reinforcement learning and evolutionary algorithms), and also in a better understanding of RNN behavior, which is necessary to improve and apply RNNs. The new RNNs can learn to solve many previously unlearnable tasks, including control in partially observable environments, processing of symbolic data, music improvisation and composition, and aspects of speech recognition. RNN optimists are claiming that we are at the beginning of an RNNaissance, and that soon we will see more and more applications of the new RNNs. The pessimists are claiming otherwise. We expect a lively discussion between optimists and pessimists.

The website http://www.idsia.ch/~juergen/rnnaissance.html contains regularly updated information about the workshop (we will have a poster session as well).

Robust Communication Dynamics in Complex Networks

Large-scale distributed systems with complex patterns of communication between elements abound in both nature (e.g., genetic pathways) and in man-made systems (e.g., Internet, email networks, and the World Wide Web). The main objective of this workshop is to explore how various local communication schemes in distributed systems (e.g., Belief Propagation, Gossip-style protocols, Survey propagation) may robustly achieve global objectives, such as accurate global computation, in the presence of various forms of noise, errors and attacks, and how their performance is affected by network dynamics and topology. The workshop aims at cross-fertilization among several research areas that attracted an immense current interest, including: belief-propagation schemes for probabilistic inference and their close relationship to free energy approximations; distributed machine learning which was touched upon in last year's NIPS; workshop on Multi-Agent Learning; statistical dynamics of complex network phenomena, a rapidly growing multi-disciplinary research topic that combines methods from computer science, statistical physics, nonlinear dynamics, econometrics and social network theory to study common problems in many systems exhibiting complex network structure. This topic has attracted much recent attention in the scientific literature as well as in popular publications (Barabasi, 2002; Watts, 2003), but so far has not been presented at NIPS.

See http://www.research.ibm.com/nips03workshop for more information about the workshop.

Syntax, Semantics, and Statistics

Richard M. Shiffrin, Indiana University, Bloomington; Mark Steyvers, University of California Irvine; David Blei, University of California, Berkeley; Tom Griffiths, Stanford University

Statistical approaches to the extraction of syntactic and semantic information have generated increasing interest in several communities, including machine learning, information retrieval, computational linguistics, and cognitive science. Recent years have seen the development of sophisticated probabilistic methods for identifying structure in large text databases, image collections, patterns of authorship, and sequence data, as well as advances in our understanding of how statistical regularities are exploited in human learning and cognition. Leading researchers from different fields will present their explorations of the interaction between syntax, semantics, and statistics, and will promote greater awareness of the breadth of this work. The workshop will cover methods, extensions, pitfalls, and important open problems.

Details, additional information, and presentations will be posted on: http://psiexp.ss.uci.edu/research/nips2003