Workshops

In the past few years, the field of molecular biology of the brain has been transformed from hypothesis-based experiments to high-throughput experiments. The massive growth of data, including measures of the brain transcriptome, methylome and proteome, now raises new questions in neurobiology and new challenges in analysis of these complex and vast datasets. While many of these challenges are shared with other computational biology studies, the complexity of the brain poses special challenges. Brain genomics data includes high-resolution molecular imagery, developmental time courses and most importantly, underlies complex behavioral phenotypes and psychiatric diseases. New methods are needed to address questions about the brain-wide, genome-wide and life-long genomic patterns in the brain and their relation to brain functions like plasticity and information processing.

The goal of the workshop is to bring together people from the neuroscience, cognitive science and the machine learning community. It aims to ease the path for scientists to connect the wealth of genomic data to the issues of cognition and learning that are central to NIPS, with an eye to the emerging high-throughput behavioral data which many are gathering. We invite contributed talks on novel methods of analysis to brain genomics, as well as techniques to make meaningful statistical relationships to phenotypes.

The target audience includes two main groups: people interested in developing machine learning approaches to neuroscience, and people from neuroscience and cognitive science interested in connecting their work to brain genomics.

The technical term “robust” was coined in 1953 by G. E. P. Box and exemplifies his adage, “all models are wrong, but some are useful”. Over the past decade, a broad range of new paradigms have appeared that allow useful inference when standard modeling assumptions are violated. Classic examples include heavy tailed formulations that mitigate the effect of outliers which would otherwise degrade the performance of Gaussian-based methods.

High-dimensional data are becoming ubiquitous in diverse domains such as genomics, neuroimaging, economics, and finance. Such data exacerbate the relevance of robustness as errors and model misspecification are prevalent in such modern applications. To extract pertinent information from large scale data, robust formulations require a comprehensive understanding of machine learning, optimization, and statistical signal processing, thereby integrating recovery guarantees, statistical and computational efficiency, algorithm design and scaling issues. For example, robust Principal Component Analysis (RPCA) can be approached using both convex and nonconvex formulations, giving rise to tradeoffs between computational efficiency and theoretical guarantees.

The goal of this workshop is to bring together machine learning, high-dimensional statistics, optimization and select large-scale applications, in order to investigate the interplay between robust modeling and computation in the large-scale setting. We incorporate several important examples that are strongly linked by this theme:

(a) Low rank matrix recovery, robust PCA, and robust dictionary learning: High-dimensional data problems where the number of variables may greatly exceed the number of observations can be accurately solved by leveraging low-dimensional structural constraints upon the parameters to be estimated. For matrix-structured parameters, low-rank recovery is a prime example of such low-dimensional assumption. To efficiently recover the low-rank structure characterizing the data, Robust PCA extends classical PCA in order to accommodate grossly corrupted observations that have become ubiquitous in modern applications. Sparse coding and dictionary learning build upon the fact that many real-world data can be represented as a sparse linear combination of basis vectors over an over-complete dictionary and aims at learning such an efficient representation of the data. Sparse coding and dictionary learning are being used in a variety of tasks including image denoising and inpainting, texture synthesis, image classification and unusual event detection.


(b) Robust inference for large scale inverse problems and machine learning: Many data commonly encountered are heavy-tailed where the Gaussian assumption does not apply. The issue of robustness has been largely overlooked in the high-dimensional learning literature, yet this aspect is critical when dealing with high dimensional noisy data. Traditional likelihood-based estimators (including Lasso and Group Lasso) are known to lack resilience to outliers and model misspecification. Despite this fact, there has been limited focus on robust learning methods in high-dimensional modeling.

(c) Non-convex formulations: heavy tails, factorized matrix inversion, nonlinear forward models. Combining robustness with statistical efficiency requires non-convexity of the loss function. Surprisingly, it is often possible to show that either certain non-convex problems have exact convex relaxations, or that algorithms directly solving non-convex problems may produce points that are statistically indistinguishable from the global optimum.

(d) Robust optimization: avoiding overfitting on precise but unreliable parameters. This classic topic has become increasingly relevant as researchers purposefully perturb problems. This perturbation comes in many forms: from “sketching” functions with Johnson-Lindenstrauss-like transformations, using randomized algorithms to speed up linear algebra, using randomized coordinate descent, and/or stochastic gradient algorithms. Recently the techniques of robust optimization have been applied to these situations.


It is the aim of this workshop to bring together researchers from statistics, machine learning, optimization, and applications, in order to focus on a comprehensive understanding of robust modeling and computation. In particular, we will see challenges of implementing robust formulations in the large-scale and nonconvex setting, as well as examples of success in these areas.

The workshop follows in the footsteps if the “Robust ML” workshop at NIPS in 2010. The field is very active and there have been significant advances in the past 4 years. We also expect to have new topics, such as new applications of robust optimization to user-perturbed problems and Markov Decision Processes.

Understanding the semantic structure of unstructured data -- text, dialogs, images -- is a critical challenge given their central role in many applications, including question answering, dialog systems, information retrieval... In recent years, there has been much interest in designing models and algorithms to automatically extract and manipulate these semantic representations from raw data.

Semantics is a diverse field. It encompases extracting structured data from text and dialog data (knowledge base extraction, logical form extraction, information extraction), linguistic approaches to extract and compose representation of meaning, inference and reasoning over meaning representation based on logic or algebra. It also includes approaches that aims at grounding language by learning relations between language and visual observations, linking language to the physical world (e.g. through robotics, machine commands). Despite spanning different disciplines with seemingly incompatible views, these approaches to semantics all aims at enabling computers to evolve and interact with humans and the physical world in general.

The goal of the workshop is dual. First, we aim at gathering experts from the different fields of semantics to favor cross-fertilization, discussions and constructive debates. Second, we encourage invited speakers and participants to expose their future research directions, take position and highlight the key challenges the community need to face. The workshop devotes most of the program to panel sessions about future directions.

* Contributions *

We will welcome contributions (up to 4 pages abstract) in the following areas and related topics:
- Word similarities and sense disambiguation
- Information and relation extraction
- Lexical and compositional semantics
- Learning semantic frames and semantic role labelling
- Grounded language learning
- Semantic representation for dialog understanding
- Visual scene understanding
- Multi-modal semantic representation and reasoning

* Relevant Literature *

Beltagy, I., Chau, C., Boleda, G., Garrette, D., Erk, K., Mooney, R.: Montague Meets Markov: Deep Semantics with Probabilistic Logical Form. Proceedings of the 2nd Joint Conference on Lexical and Computational Semantics (*SEM), June 13-14, 2013
Bordes, A., Glorot, X., Weston, J., Bengio., Y.: Joint learning of words and meaning representations for open-text semantic parsing. Proc. of the 17th Intern. Conf. on Artif. Intel. and Stat (2012)
L. Bottou: From machine learning to machine reasoning: an essay,Machine Learning, 94:133-149 (2014)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. Journal of Machine Learning Research 12, 2493–2537 (2011)
Krishnamurthy, J., Mitchell, T.: Vector Space Semantic Parsing: A Framework for Compositional Vector Space Models. Proceedings of the ACL 2013 Workshop on Continuous Vector Space Models and their Compositionality (2013).
Lewis, D.: General semantics. Synthese 22, 18–67 (1970). DOI 10.1007/BF00413598. URL http://dx.doi.org/10.1007/BF00413598
Liang, P., Jordan, M.I., Klein, D.: Learning dependency-based compositional semantics. Association for Computational Linguistics (ACL), pp. 590–599 (2011)
Mitchell, J., Lapata, M.: Vector-based models of semantic composition. Proceedings of ACL-08: HLT pp. 236–244 (2008)
Poon, H., Domingos, P.: Unsupervised ontology induction from text. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 296–305. (2010)
Riedel, S., Yao, L., McCallum, A., Marlin, B.: Relation Extraction with Matrix Factorization and Universal Schemas. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2013).
Socher, R., Lin, C.C., Ng, A.Y., Manning, C.D.: Parsing Natural Scenes and Natural Language with Recursive Neural Networks. Proceedings of the 26th International Conference on Machine Learning (ICML) (2011)
Turney, P., Pantel, P.: From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141–188 (2010)
Zelle, J., Mooney, R.: Learning to parse database queries using inductive logic programming. Proceedings of the National Conference on Artificial Intelligence (1996)
Zettlemoyer, L., Collins, M.: Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. Proceedings of the Conference on Uncertainty in Artificial Intelligence (2005)
C. L. Zitnick, D. Parikh and L. Vanderwende: Learning the Visual Interpretation of Sentences, International Conference on Computer Vision (2013)

The structure, complexity, and sheer diversity and variety of human language makes Natural Language Processing (NLP) distinct from other areas of AI. Certain core NLP problems have traditionally been an inspiration for machine learning (ML) solutions e.g., sequence tagging, syntactic parsing, and language modeling, primarily because these tasks can be easily abstracted into machine learning formulations (e.g., structured prediction, dimensionality reduction, or simpler regression and classification techniques). In turn, these formulations have facil-
itated the transfer of ideas such as (but not limited to) discriminative methods, Bayesian nonparametrics, neural networks, and low rank / spectral techniques into NLP. Problems in NLP are particularly appealing to those doing core ML research
due to the high-dimensional nature of the spaces involved (both the data and the label spaces) and the need to handle noise robustly, while principled, well-understood ML techniques are attractive to those in NLP since they potentially offer a solution to ill-behaved heuristics and training-test domain mismatch due
to the lack of generalization ability these heuristics possess.

But there are many other areas within NLP where the ML community is less involved, such as semantics, discourse and pragmatics analysis, summarization, and parts of machine translation, and that continue to rely on linguistically-
motivated but imprecise heuristics which may benefit from new machine learning approaches. Similarly, there are paradigms in ML, statistics, and optimization ranging from sub-modularity to bandit theory to Hilbert space embeddings that have not been well explored in the context of NLP.

The goal of this workshop is to bring together both applied and theoretical researchers in natural language processing and machine learning to facilitate the discussion of new frameworks that can help advance modern NLP. Some key questions we will address include (but not limited to):

- How can ML help provide novel representations and models to capture the structure of natural language?

- What NLP problems could benefit from new inference/optimization techniques?

- How can we design new ML paradigms to address the lack of annotated data in complex structured prediction problems such as knowledge extraction and semantics?

- What technical challenges posed by multilinguality, lexical
variation in social media, and nonstandard dialects are under-researched in ML?

- Does ML offer more principled ways for dealing with "overfitting" resulting from repeated evaluation on the same benchmark datasets?

- How can we tackle "scalability bottlenecks" unique to natural language?

Interest amongst both communities is high, as evidenced by a previous joint ACL-ICML symposium (2011) and a joint NAACL-ICML symposium (2013), and we hope to continue the exploration of topics beneficial to both fields that these symposiums initiated.

The emergence of large distributed matrices in many applications has brought with it a slew of new algorithms and tools. Over the past few years, machine learning and numerical linear algebra on distributed matrices has become a thriving field. Manipulating such large matrices makes it necessary to think about distributed systems issues such as communication cost.

This workshop aims to bring closer researchers in distributed systems and large scale numerical linear algebra to foster cross-talk between the two fields. The goal is to encourage distributed systems researchers to work on machine learning and numerical linear algebra problems, to inform machine learning researchers about new developments on large scale matrix analysis, and to identify unique challenges and opportunities. The workshop will conclude with a session of contributed posters.

Challenges in Machine Learning have proven to be efficient and cost-effective ways to quickly bring to industry solutions that may have been confined to research. In addition, the playful nature of challenges naturally attracts students, making challenge a great teaching resource. Challenge participants range from undergraduate students to retirees, joining forces in a rewarding environment allowing them to learn, perform research, and demonstrate excellence. Therefore challenges can be used as a means of directing research, advancing the state-of-the-art or venturing in completely new domains.

Yet, despite initial successes and efforts made to facilitate challenge organization with the availability of competition platforms, little effort has been put into the theoretical foundations of challenge design and the optimization of challenge protocols. This workshop will bring together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. The themes to be discussed will include new paradigms of challenge organization to tackle complex problems (e.g. tasks involving multiple data modalities and/or multiple levels of processing).

Bayesian optimization has emerged as an exciting subfield of machine learning that is concerned with the global optimization of noisy, black-box functions using probabilistic methods. Systems implementing Bayesian optimization techniques have been successfully used to solve difficult problems in a diverse set of applications. There have been many recent advances in the methodologies and theory underpinning Bayesian optimization that have extended the framework to new applications as well as provided greater insights into the behaviour of these algorithms. Bayesian optimization is now increasingly being used in industrial settings, providing new and interesting challenges that require new algorithms and theoretical insights.

At last year’s NIPS workshop on Bayesian optimization the focus was on the intersection of “Theory and Practice”. The workshop this year will follow this trend by again looking at theoretical contributions, but also by focusing on the practical side of Bayesian optimization in industry. The goal of this workshop is not only to bring together both practical and theoretical research knowledge from academia, but also to facilitate cross-fertilization with industry. Specifically, we would like to carefully examine the types of problems where Bayesian optimization works well in industrial settings, but also the types of situations where additional performance is needed. The key questions we will discuss are: how to scale Bayesian optimization to long time-horizons and many observations? How to tackle high-dimensional data? How to make Bayesian optimization work in massive, distributed systems? What kind of structural assumptions are we able to make? And finally, what can we say about these questions both empirically and theoretically?

The target audience for this workshop consists of both industrial and academic practitioners of Bayesian optimization as well as researchers working on theoretical advances in probabilistic global optimization. To this end we have invited many industrial users of Bayesian optimization to attend and speak at the workshop. We expect this exchange of industrial and academic knowledge will lead to a significant interchange of ideas and a clearer understanding of the challenges and successes of Bayesian optimization as a whole.

A further goal is to encourage collaboration between the diverse set of researchers involved in Bayesian optimization. This includes not only interchange between industrial and academic researchers, but also between the many different sub-fields of machine learning which make use of Bayesian optimization. We are also reaching out to the wider global optimization and Bayesian inference communities for involvement.

A detailed understanding of brain function is a still-elusive grand challenge. Major advances in recording technologies (e.g. 2-photon and light-sheet microscopic imaging of calcium signals) are now beginning to provide measurements of neural activity at an unprecedented size and quality. Computational tools will be of critical importance both for the high-throughput acquisition and analysis of large-scale datasets. Reliable and robust tools for automated high-throughput analysis of such data that works have not been available so far. As a consequence, experimental reality is still characterized by semi-manual analysis or makeshift scripts that are specialized to a single setting. Similarly, many analysis still focus on the response properties of single neurons or on pairwise correlations across neurons, thereby potentially missing information which is only available at the population level.

The goal of this workshop is to discuss challenges and opportunities for
computational neuroscience and machine learning which arise from large-scale recording techniques:


* What kind of data will be generated by large-scale functional measurements in the next decade? How will it be quantitatively or qualitatively different to the kind of data we have had previously? What are the computational bottlenecks for their analysis?

* What kind of computational tools play an important role on high-throughput data acquisition, e. g. visualization/dimensionality reduction/information quantification? How can we figure out which algorithms work best, and which are the important challenges that are not met by existing techniques?


* What have we really learned from high-dimensional recordings that is new? What theories could we test, if only we had access to recordings from more neurons at the same time? What kind of statistics will be powerful enough to verify/falsify population coding theories? What can we infer about the network structure and dynamics?

We have invited scientists whose research addresses these questions, including researchers developing recording technologies, experimental and computational neuroscientists. We foresee active discussions amongst this multidisciplinary group of scientists to create a chance to discuss priorities and perspective, debate about the currently most relevant problems in the field, and emphasize the most promising future research directions. The target audience of this workshop includes industry and academic researchers interested in machine learning, neuroscience, big data and statistical inference.

Abstract:

Advances in medical information technology have resulted in enormous warehouses of data that are at once overwhelming and sparse. A single patient visit may result in tens to thousands of measurements and structured information, including clinical factors, diagnostic imaging, lab tests, genomic and proteomic tests. Hospitals may see thousands of patients each year. However, each patient may have relatively few visits to any particular medical provider. The resulting data are a heterogeneous amalgam of patient demographics, vital signs, diagnoses, records of treatment and medication receipt and annotations made by nurses or doctors, each with its own idiosyncrasies.
The objective of this workshop is to discuss how advanced machine learning techniques can derive clinical and scientific impact from these messy, incomplete, and partial data. We will bring together machine learning researchers and experts in medical informatics who are involved in the development of algorithms or intelligent systems designed to improve quality of healthcare. Relevant areas include health monitoring systems, clinical data labelling and clustering, clinical outcome prediction, efficient and scalable processing of medical records, feature selection or dimensionality reduction in clinical data, tools for personalized medicine, time-series analysis with medical applications and clinical genomics.



Detailed Description:

An important issue in clinical applications is the peculiarity of the available data – an amalgam of patient demographics, collected vital signs, diagnostics, records of administered treatment and medication and, potentially, annotations made by nurses or doctors. Vital signs are available typically as moving averages over varying time horizons [1], and occasionally in their original form (sampled at high frequency). The extensive data collection usually results in an overall abundance of data, which might lead to the falsely optimistic conclusion that its sheer magnitude will make training of any system trivial. The insidious issue with clinical data, which not even the best put-together repositories [2] manage to overcome, is its lack of continuity/consistency. The data comes from a vast number of patients, each with very specific clinical conditions. Data on individual patients may however be quite sparse and/or incomplete and often contain significant gaps due to circumstance or equipment malfunction. Not only are the samples limited for a given patient, but the health status of a single person can vary due to difference in external factors such as medication. These circumstances make short work of typical assumptions made by learning techniques. Thus, IID samples, coming from the same distribution, satisfying some tidy noise condition are virtually impossible to encounter in longitudinal physiologic data, on which medical diagnoses and decisions are based. To further complicate matters, records can be missing or outright incorrect, adding to the inevitable noise in vital sign readings, diagnostics and treatment records. Moreover, a patient can be attributed several diagnostics given out of a list of thousands of ICD9 codes. All things considered, the so-called ‘big data’ present in clinical applications is surprisingly sparse if the entire feature space is taken into account.

Despite the existence of algorithms that address some of these problems, a number of important research topics still remain open, including but not limited to:
(i) What individual-level predictions can be made from such partial, incomplete data?
(ii) How can partial, incomplete time series from multiple patients be combined to create a population and sub-population levels of understanding about treatment and disease? What are the best ways to stratify or cluster the data - using patient demographics, diagnostics and/or treatment - to ensure a plausible trade-off between model specialization and sample sufficiency? What is the best way to deal with outliers and how to detect incorrect data?
(iii) How can machine learning methods circumvent some of the inherent problems of large-scale clinical data? Can machine learning techniques and clinical tools (e.g. clinical review, expert ontologies, inter-institutional data) be used to adapt to the sparsity and biases in the data?
(iv) How can these data be used to assess standards of care and investigate the efficacy of various treatment programs? Generally, how can these data be used to help us better understand many of the complex causal relationships in medicine?
(v) Training classification models requires accurate labeling, and this in turn requires considerable effort on the part of human experts – can we reduce the amount of labeling needed through active learning? Can we use yet unlabeled data and combine semi-supervised approach with active learning to obtain higher accuracy?
(vi) What are the robust ways of modeling cross-signal correlations? How can we incorporate diagnostics and sparse, high-dimensional treatment information in clinical models? Can we characterize the effect of treatment on vital signs?

As just one example application where such research questions would be highly relevant, consider a vital sign monitoring system. Monitoring patient status is a crucial aspect of health care, with the task of anticipating and preventing critical episodes being traditionally entrusted to the nursing staff. However, there is increasing interest and demand for automated tools used to detect any abrupt worsening of health status in critically ill patients [3,4]. Most of the initial efforts were focused towards processing only one signal, a notable example being the detection of arrhythmias from electrocardiograms. However, it became increasingly clear that considering correlations across signals and deriving features over varying time windows holds great promise for the prognosis of adverse events [5,6]. Additionally, with the emergence of personalized care [7] and wearable technology for health monitoring [8], there is an increasing need for real-time online processing of vital signs and for adaptive models suitable to the ever-changing parameters specific to these applications. Given the heterogeneous data available, how can we develop flexible models that can gradually adapt to the characteristics of a patient as more data is obtained? Can this update be efficiently performed?

The workshop will approach the identified challenges from two perspectives. On one hand, healthcare experts will describe their requirements and describe the main issues in processing medical data. On the other hand, machine learning researchers will present algorithms and tools they have developed for clinical applications, describing their relevance. Most importantly, the discussions are meant to establish the limitations of current approaches, the feasibility of extending them to deal with the aforementioned data issues and to brainstorm on promising ML techniques that have been insufficiently exploited for these tasks.

References:
[1] Lawhern V., Hairston W.D., Robbins K., "Optimal Feature Selection for Artifact Classification in EEG Time Series", Foundations of Augmented Cognition Lecture Notes in Computer Science Volume 8027, 2013, pp 326-334
[2] MIMIC + other repositories
[3] Gopalakrishnan V., Lustgarten J., Visweswaran S., and Cooper G, “Bayesian rule learning for biomedical data mining”. Journal of Bioinformatics, 26, 2010.
[4] Randall Moorman J., Delos J. B., Flower A., Cao H., Kovatchev B.P., Richman J. S., and Lake D.E., “Cardiovascular oscillations at the bedside: early diagnosis of neonatal sepsis using heart rate characteristics monitoring”. Physiol Meas, 32 (11):1821-32, Nov 2011.
[5] Fiterau M., Dubrawski A., and Ye C., “Real-time adaptive monitoring of vital signs for clinical alarm preemption”, In Proceedings of the 2010 International Society for Disease Surveillance Annual Conference, 2011.
[6] Seely A.J. E., “Complexity at the bedside”, Journal of Critical Care”, Jun 2011.
[7] Narimatsu H., Kitanaka C., Kubota I., Sato S., Ueno Y., Kato T., Fukao A., Yamashita H,
Kayama T., “New developments in medical education for the realization of next-generation personalized medicine: concept and design of a medical education and training program through the genomic cohort study”, Journal of Human Genetics 2013 June 2013
[8] Pantelopoulos, A.,Bourbakis, N.G., "A Survey on Wearable Sensor-Based Systems for Health Monitoring and Prognosis," Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on , vol.40, no.1, pp.1,12, Jan. 2010

In nearly all machine learning tasks, decisions must be made given current knowledge (e.g., choose which label to predict). Perhaps surprisingly, always making the best decision is not always the best strategy, particularly while learning. Recently, there is an emerging body of work on learning under different rules that apply perturbations to the decision procedure. These works provide simple and efficient learning rules with improved theoretical guarantees. This workshop will bring together the growing community of researchers interested in different aspects of this area, and it will broaden our understanding of why and how perturbation methods can be useful.

In the last couple of years, at the highly successful NIPS workshops on Perturbations, Optimization, and Statistics, we looked at how injecting perturbations (whether it be random or adversarial “noise”) into learning and inference procedures can be beneficial. The focus was on two angles: first, on how stochastic perturbations can be used to construct new types of probability models for structured data; and second, how deterministic perturbations affect the regularization and the generalization properties of learning algorithms.

The goal of this workshop is to expand the scope of previous workshops and also explore different ways to apply perturbations within optimization and statistics to enhance and improve machine learning approaches. This year, we would like to look at exciting new developments related to the above core themes.

More generally, we shall specifically be interested in understanding the following issues:

Modeling: which models lend efficient learning by perturbations?

Regularization: whether randomness can be replaced by other mathematical object while keeping the computational and statistical guarantees?
* Robust optimization: how stochastic and adversarial perturbations affect the learning outcome?

* Dropout: How stochastic dropout regularizes online learning tasks?
* Sampling: how perturbation can be applied to sample from continuous spaces?

This interdisciplinary workshop will consider issues of fairness, accountability, and transparency in machine learning. It will address growing anxieties about the role that machine learning plays in consequential decision-making in such areas as commerce, employment, healthcare, education, and policing.

Reflecting these concerns, President Obama at the start of 2014 called for a 90-day review of Big Data. The resulting report, "Big Data: Seizing Opportunities, Preserving Values", concluded that "big data technologies can cause societal harms beyond damages to privacy". It voiced particular concern about the possibility that decisions informed by big data could have discriminatory effects, even in the absence of discriminatory intent, and could further subject already disadvantaged groups to less favorable treatment. It also expressed alarm about the threat that an "opaque decision-making environment" and an "impenetrable set of algorithms" pose to autonomy. In its recommendations to the President, the report called for additional "technical expertise to stop discrimination", and for further research into the dangers of "encoding discrimination in automated decisions".

Our workshop takes up this call. It will focus on these issues both as challenging constraints on the practical application of machine learning, as well as problems that can lend themselves to novel computational solutions.

Questions to the machine learning community include:

• How can we achieve high classification accuracy while eliminating discriminatory biases? What are meaningful formal fairness properties?

• How can we design expressive yet easily interpretable classifiers?

• Can we ensure that a classifier remains accurate even if the statistical signal it relies on is exposed to public scrutiny?
Are there practical methods to test existing classifiers for compliance with a policy?

Participants will work together to understand the key normative and legal issues at stake, map the relevant computer science scholarship, evaluate the state of the solutions thus far proposed, and explore opportunities for new research and thinking within machine learning itself.

In the context of building a machine learning framework that scales, the current modus operandi is a monolithic, centralised model building approach. These large scale models have different components, which have to be designed and specified in order to fit in with the model as a whole. The result is a machine learning process that needs a grand designer. It is analogous to a planned economy.

There is an alternative. Instead of a centralised planner being in charge of each and every component in the model, we can design incentive mechanisms for independent component designers to build components that contribute to the overall model design. Once those incentive mechanisms are in place, the overall planner need no longer have control over each individual component. This is analogous to a market economy.

The result is a transactional machine learning. The problem is transformed to one of setting up good incentive mechanisms that enable the large scale machine learning models to build themselves. Approaches of this form have been discussed in a number of different areas of research, including machine learning markets, collectives, agent-directed learning, ad-hoc sensor networks, crowdsourcing and distributed machine learning.

It turns out that many of the issues in incentivised transactional machine learning are also common to the issues that turn up in modern e-commerce setting. These issues include issues of mechanism design, encouraging idealised behaviour while modelling for real behaviour, issues surrounding prediction markets, questions of improving market efficiencies, and handling arbitrage, issue on matching both human and machine market interfaces and much more. On the theoretical side, there is a direct relationships between scoring rules, market scoring rules, and exponential family via Bregman Divergences. On the practical side, the issues that turn up in auction design relate to issues regarding efficient probabilistic inference.

The chances for each community to make big strides from understanding the developments in the others is significant. This workshop will bring together those involved in transactional and agent-based methods for machine learning, those involved in the development of methods and theory in e-commerce, those considering practical working algorithms for e-commerce or distributed machine learning and those working on financially incentivised crowdsourcing. The workshop will explore issues around incentivisation, handling combinatorial markets, and developing distributed machine learning. However the primary benefit will be the interaction and informal discussion that will occur throughout the workshop.

This topic is of particular interest because of the increasing importance of machine learning in the e-commerce setting, and the increasing interest in a distributed large scale machine learning. The workshop has some flavour of “multidisciplinary design optimization”: perhaps the optimum of the simultaneous problem of machine learning and e-commerce design is superior to the design found by optimizing each discipline sequentially, since it can exploit the interactions between the disciplines.

The expected outcomes are long lasting interactions between the communities and novel ideas in each individual community gained from learning from the others. The target group of participants are those working in machine learning markets, collectives, agent-directed learning, ad-hoc sensor networks, economic mechanisms in crowdsourcing and distributed machine learning, those working in areas of economics and markets, along with those looking at theory or practice in e-commerce, ad auctions, prediction markets and market design.

Deep Learning algorithms attempt to discover good representations, at multiple levels of abstraction. There has been rapid progress in this area in recent years, both in terms of algorithms and in terms of applications, but many challenges remain. The workshop aims at bringing together researchers in that field and discussing these challenges, brainstorming about new solutions.

Approximate Bayesian computation (ABC) or likelihood-free (LF) methods have developed mostly beyond the radar of the machine learning community, but are important tools for a large segment of the scientific community. This is particularly true for systems and population biology, computational psychology, computational chemistry, computational finance, etc. Recent work has applied both machine learning models and algorithms to general ABC inference (e.g., NN, forests, GPs, LDA) and ABC inference to machine learning (e.g. using computer graphics to solve computer vision using ABC). In general, however, there is significant room for more intense collaboration between both communities. Submissions on the following topics are encouraged (but not limited to):

Examples of topics of interest in the workshop include (but are not limited to):
* Applications of ABC to machine learning, e.g., computer vision, other inverse problems (RL)…
* ABC Reinforcement Learning (other inverse problems)
* Machine learning models of simulations, e.g., NN models of simulation responses, GPs etc.
* Selection of sufficient statistics and massive dimension reduction methods
* Online and post-hoc error
* ABC with very expensive simulations and acceleration methods (surrogate modeling, choice of design/simulation points)
* Relation between ABC and probabilistic programming
* Posterior evaluation of scientific problems/interaction with scientists
* Post-computational error assessment
* Impact on resulting ABC inference

Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops. We aim to foster discussion, discovery, and dissemination of the state-of-the-art in optimization relevant to ML.

This year, as the seventh in its series, the workshop's special topic will be the challenges in non-convex optimization, with contributions spanning both the challenges (hardness results) and the opportunities (modeling flexibility) of non-convex optimization. Irrespective of the special topic, the workshop will again warmly welcome contributed talks and posters on all topics in optimization for machine learning.

The confirmed invited speakers for this year are:

* Amir Beck (Technion, Israel)
* Jean Bernard Lasserre (CNRS, France)
* Yoshua Bengio (University of Montreal, Canada)

From online news to online shopping to scholarly research, we are inundated with a torrent of information on a daily basis. With our limited time, money and attention, we often struggle to extract actionable knowledge from this deluge of data. A common approach for addressing this challenge is personalization, where results are automatically filtered to match the tastes and preferences of individual users.

This workshop aims to bring together researchers from industry and academia in order to describe recent advances and discuss future research directions pertaining to the personalization of digital systems, broadly construed. We aim to highlight new and emerging research opportunities for the machine learning community that arise from the evolving needs for personalization.

The driving factor for new opportunities in personalization is the rapid growth and sophistication of online digital systems that users can interact with (and the resulting interaction data). Personalization first gained significant traction as a way to improve the quality of information retrieval and recommender systems. As the diversity of online content has grown, the development of more effective personalized retrieval and recommender systems remains an important goal. In addition, the emergence of new types of digital systems has expanded the opportunities for personalization to be applied to a wider range of interaction paradigms. Examples of new paradigms include data organization services such as CiteULike and Pinterest, online tutoring systems, and question & answer services such as Quora.

Because the primary asset that enables personalization is the wealth of interaction data, machine learning will play a central role in virtually all future research directions. As a premier machine learning conference, NIPS is an ideal venue for hosting this workshop. Interaction data can pose many interesting machine learning challenges, such as the sheer scale, the multi-task nature of personalizing to populations of users, the exploration/exploitation trade-off when personalizing “on-the-fly”, structured prediction such as formulating a lesson plan in tutoring systems, how to interpret implicit feedback for unbiased learning from interaction data, and how to infer complex sensemaking goals from observing fine-grained interaction sequences.


In summary, our technical topics of interest include (but are not limited to):
- Learning fine-grained representations of user preferences
- Large-scale personalization
- Interpreting observable human behavior
- Interactive algorithms for “on-the-fly” personalization
- Learning to personalize using rich user interactions
- Modeling complex sensemaking goals
- Applications beyond conventional recommender systems

To autonomously assist human beings, future robots have to autonomously learn a rich set of complex behaviors. So far, the role of machine learning in robotics has been limited to solve pre-specified sub-problems that occur in robotics and, in many cases, off-the-shelf machine learning methods. The approached problems are mostly homogeneous, e.g., learning a single type of movement is sufficient to solve the task, and do not reflect the complexities that are involved in solving real-world tasks.

In a real-world environment, learning is much more challenging than solving such homogeneous problems. The agent has to autonomously explore its environment and discover versatile behaviours that can be used to solve a multitude of different tasks throughout the future learning progress. It needs to determine when to reuse already known skills by adapting, sequencing or combining the learned behaviour and when to learn new behaviours. To do so, it needs to autonomously decompose complex real-world tasks into simpler sub-tasks such that the learned solutions for these sub-tasks can be re-used in a new situation. It needs to form internal representations of its environment, which is possibly containing a large variety of different objects or also different agents, such as other robots or humans. Such internal representations also need to shape the structure of the used policy and/or the used value function of the algorithm, which need to be flexible enough such to capture the huge variability of tasks that can be encountered in the real world. Due to the multitude of possible tasks, it also cannot rely on a manually tuned reward function for each task, and, hence, it needs to find a more general representations for the reward function. Yet, an autonomous robot is likely to interact with one or more human operators that are typically experts in a certain task, but not necessarily experts in robotics. Hence, an autonomously learning robot also should make effective use of feedback that can be acquired from a human operator.
Typically, different types of instructions from the human are available, such as demonstrations and evaluative feedback in form of a continuous quality rating, a ranking between solutions or a set of preferences. In order to facilitate the learning problem, such additional human instructions should be used autonomously whenever available. Yet, the robot also needs to be able to reason about its competence to solve a task. If the robot thinks it has poor competence or the uncertainty of the competence is high, the robot should request more instructions from the human expert.

Most machine learning algorithms are missing these types of autonomy. They still rely on a large amount of engineering and fine-tuning from a human expert. The human typically needs to specify the representation of the reward-function, of the state, of the policy or of other internal representations used by the learning algorithms. Typically, the decomposition of complex tasks into sub-tasks is performed by the human expert and the parameters of such algorithms are fine tuned by hand. The algorithms typically learn from a pre-specified source of feedback and can not autonomously request more instructions such as demonstrations, evaluative feedback or corrective actions. We belief that this lack of autonomy is one of the key reasons why robot learning could not be scaled to
more complex, real world tasks. Learning such tasks would require a huge amount of fine tuning which is very costly on real robot systems.

Goal:

In this workshop, we want to bring together people from the fields of robotics, reinforcement learning, active learning, representation learning and motor control. The goal in this multi-disciplinary workshop is to develop new ideas to increase the autonomy of current robot learning algorithms and to make their usage more practical for real world applications. In this context, among the questions which we intend to tackle are

More Autonomous Reinforcement Learning
- How can we automatically tune hyper-parameters of reinforcement learning algorithms such as learning and exploration rates?
- Can we find reinforcement learning algorithms that are less sensitive to the settings of their hyper-parameters and therefore, can be used for a multitude of tasks with the same parameter values?
- How can we efficiently generalize learned skills to new situations?
- Can we transfer the success of deep learning methods to robot learning?
- How do learn on several levels of abstractions and also identify useful abstractions?
- How can we identify useful elemental behaviours that can be used for a multitude of tasks?
- How do use RL on the raw sensory input without a hand-coded representation of the state?
- Can we learn forward models of the robot and its environment from high dimensional sensory data? How can these forward models be used effectively for model-based reinforcement learning?
- Can we autonomously decide when to learn value functions and when to use direct policy search?

Autonomous Exploration and Active Learning
- How can we autonomously explore the state space of the robot without the risk of breaking the robot?
- Can we use strategies for intrinsic motivation, such as artificial curiosity or empowerment, to autonomously acquire a rich set of behaviours that can be re-used in the future learning progress?
- How can we measure the competence of the agent as well as our certainty in this competence?
- Can we use active learning to acquire improve the quality of learned forward models as well as to probe the environment to gain more information about the state of the environment?

Autonomous Learning from Instructions
- Can we combine learning from demonstrations, inverse reinforcement learning and preference learning to make more effective use of human instructions?
- How can we decide when to request new instructions from a human experts?
- How can we scale inverse reinforcement learning and preference learning to high dimensional continuous spaces?
- Can we use demonstrations and human preferences to identify relevant features from the high dimensional sensory input of the robot?

Autonomous Feature Extraction
- Can we use feature extraction techniques such as deep learning to find a general purpose feature representation that can be used for a multitude of tasks.
- Can recent advances for kernel based methods be scaled to reinforcement learning and policy search in high dimensional spaces?
- What are good priors to simplify the feature extraction problem?
- What are good features to represent the policy, the value function or the reward function? Can we find algorithms that extract features specialized for these representations?

OVERVIEW This workshop aims to gather researchers in the area of sequential decision making to discuss recent findings and new challenges around the concept of model misspecification. A misspecified model is a model that either (1) cannot be tractably solved, (2) solving the model does not produce an acceptable solution for the target problem, or (3) the model clearly does not describe the available data perfectly. However, even though the model has its issues, we are interested in finding a good policy. The question is thus: How can misspecified models be made to lead to good policies?

We refer to the following (non exhaustive) types of misspecification.
1. States and Context. A misspecified state representation relates to research problems such as Hidden Markov Models, Predictive State Representations, Feature Reinforcement Learning, Partially Observable Markov Decision Problems, etc. The related question of misspecified context in contextual bandits is also relevant.
2. Dynamics. Consider learning a policy for a class of several MDPs rather than a single MDP, or optimizing a risk averse (as opposed to expected) objective. These approaches could be used to derive a reasonable policy for the target MDP even if the model we solved to obtain it is misspecified. Thus, robustness, safety, and risk-aversion are examples of relevant approaches to this question.
3. Actions. The underlying insight of working with high-level actions built on top of lower-level actions is that if we had the right high-level actions, we would have faster learning/planning. However, finding an appropriate set of high-level actions can be difficult. One form of model misspecification occurs when the given high-level actions cannot be combined to derive an acceptable policy.

More generally, since misspecification may slow learning or prevent an algorithm from finding any acceptable solution, improving the efficiency of planning and learning methods under misspecification is of primary importance. At another level, all these challenges can benefit greatly from the identification of finer properties of MDPs (local recoverability, etc.) and better notions of complexity. These questions are deeply rooted in theory and in recent applications in fields diverse as air-traffic control, marketing, and robotics. We thus also want to encourage presentations of challenges that provide a red-line and agenda for future research, or a survey of the current achievements and difficulties. This includes concrete problems like Energy management, Smart grids, Computational sustainability and Recommender systems.

We welcome contributions on these exciting questions, with the goals of (1) helping close the gap between strong theoretical guarantees and challenging application requirements, (2) identifying promising directions of near future research, for both applications and theory of sequential decision making, and (3) triggering collaborations amongst researchers on learning good policies despite being given misspecified models.

MOTIVATION, OBJECTIVES Despite the success of sequential decision making theory at providing solutions to challenging settings, the field faces a limitation. Often strong theoretical guarantees depend on the assumption that a solution to the class of models considered is a good solution to the target problem. A popular example is that of finite-state MDP learning for which the model of the state-space is assumed known. Such an assumption is however rarely met in practice. Similarly, in recommender systems and contextual bandits, the context may not capture an accurate summary of the users. Developing a methodology for finding, estimating, and dealing with the limitations of the model is paramount to the success of sequential decision processes. Another example of model misspecification occurs in Hierarchical Reinforcement Learning: In many real-world applications, we could solve the problem easily if we had the right set of high-level actions. Instead, we need to find a way to build those from a cruder set of primitive actions or existing high-level actions that do not suit the current task.
Yet another applicative challenge is when we face a process that can only be modeled as an MDP evolving in some class of MDPs, instead of a fixed MDP. leading to robust reinforcement learning, or when we call for safety or risk-averse guarantees.

These problems are important bottlenecks standing in the way of applying sequential decision making to challenging application, and motivate the triple goal of this workshop.

RELEVANCE TO THE COMMUNITY Misspecification of models (in the senses we consider here) is an important problem that is faced in many – if not all – real-world applications of sequential decision making under uncertainty. While theoretical results have primarily focused on the case when models of the environment are well-specified, little work has been done on extending the theory to the case of misspecification. Attempting at understanding why and when incorrectly specified models lead to good empirical performance beyond what the current theory explains is also an important goal. We believe that this workshop will be of great interest for both theoreticians and applied researchers in the field.

PAPER SUBMISSIONS The workshop aims to spark vibrant discussion with talks from invited speakers, presentations from authors of accepted papers, and a poster session. We are soliciting two types of contributions:
• Papers (4-6 pages) for oral or interactive poster presentations
• Extended abstracts (2 pages) for interactive poster presentation
We encourage submissions from different fields of sequential decision making (e.g., reinforcement learning, online learning, active learning), as well as application-domain experts (from e.g., digital marketing, recommender systems, personalized medicine, etc.) addressing the following (non-
exhaustive) list of questions and topics:
• Misspecification in model selection.
• State-representations in Reinforcement learning: Hidden Markov Models, Predictive State Representations, Feature Reinforcement Learning, Partially Observable Markov Decision Processes.
• Latent variables in sequential decision making and techniques to handle them.
• Robustness, Safety and Risk-aversion in Reinforcement Learning.
• Curiosity and Autonomous learning (reward misspecification).
• Reinforcement Learning with Options.
• Application for the Reinforcement Learning community (Computational Sustainability, Smart Cities, Smart grids, etc.).
• Other topics whose relevance to the workshop is well supported.
Solutions to such challenges will benefit the machine learning community at large, since they also appear in many real-world applications.

Learning problems that involve complex outputs are becoming increasingly prevalent in machine learning research. For example, work on image and document tagging now considers thousands of labels chosen from an open vocabulary, with only partially labeled instances available for training. Given limited labeled data, these settings also create zero-shot learning problems with respect to omitted tags, leading to the challenge of inducing semantic label representations. Furthermore, prediction targets are often abstractions that are difficult to predict from raw input data, but can be better predicted from learned latent representations. Finally, when labels exhibit complex inter-relationships it is imperative to capture latent label relatedness to improve generalization.

This workshop will bring together separate communities that have been working on novel representation and learning methods for problems with complex outputs. Although representation learning has already achieved state of the art results in standard settings, recent research has begun to explore the use of learned representations in more complex scenarios, such as structured output prediction, multiple modality co-embedding, multi-label prediction, and zero shot learning. Unfortunately, these emerging research topics have been conducted in separate sub-areas, without proper connections drawn to similar ideas in other areas, hence general methods and understanding have not yet emerged from the disconnected pursuits. The aim of this workshop is to identify fundamental strategies, highlight differences, and identify the prospects for developing a set of systematic theory and methods for learning problems with complex outputs. The target communities include researchers working on image tagging, document categorization, natural language processing, large vocabulary speech recognition, deep learning, latent variable modeling, and large scale multi-label learning.

Relevant topics include:
- Multi-label learning with large and/or incomplete output spaces
- Zero-shot learning
- Label embedding and Co-embedding
- Learning output kernels
- Output structure learning

Problems involving networks and massive network datasets motivate some of the most difficult and exciting inferential challenges today, from social, economic, and biological domains. Modern network data are often more than just vertices and edges, containing rich information on vertex attributes, edge weights, and characteristics that change over time. Enormous in size, detail, and heterogeneity, these networks are often best represented as highly annotated sequences of graphs. Although much progress has been made on developing rigorous tools for analyzing and modeling some types of large, complex, real-world networks, much work still remains, and a principled, coherent framework remains elusive, in part because network analysis is a young and highly cross-disciplinary field.

This workshop aims to bring together a diverse and cross-disciplinary set of researchers to discuss recent advances and future directions for developing new network methods in statistics and machine learning. By network methods, we broadly include those models and algorithms whose goal is to learn the patterns of interaction, flow of information, or propagation of effects in social, biological, and informational systems. We also welcome empirical studies in applied domains such as the social sciences, biology, medicine, neuroscience, physics, finance, social media, and economics. And, we are particularly interested in research that unifies the study of both structure and content in rich network datasets.

While this research field is already broad and diverse, there are emerging signs of convergence, maturation, and increased methodological awareness. For example, in the study of information diffusion, social media and social network researchers are beginning to use rigorous tools to distinguish effects driven by social influence, homophily, or external processes — subjects historically of intense interest amongst statisticians and social scientists. Similarly, there is a growing statistics literature developing learning approaches to study topics popularized earlier within the physics community, including clustering in graphs, network evolution, and random-graph models. Finally, learning methods are increasingly used in highly complex application domains, such as brain networks, and massive social networks like Facebook, and these applications are stimulating new scientific and practical questions that sometimes cut across disciplinary boundaries.

The workshop's primary goal is to facilitate the technical maturation of network analysis, promote greater technical sophistication and practical relevance, and identify future directions of research. This workshop will thus bring together researchers from disciplines like computer science, statistics, physics, informatics, economics, sociology, with an emphasis on theoretical discussions of fundamental questions.

The technical focus of the workshop is the statistical, methodological and computational issues that arise when modeling and analyzing large collections of heterogeneous and potentially dynamic network data. We seek to foster cross-disciplinary collaborations and intellectual exchange between the different communities and their respective ideas and tools. The communities identified above have long-standing interest in network modeling, and we aim to explore the similarities and differences both in methods and in goals.

The NIPS community is well positioned as a middle ground for effective dialog between applied and methodological concerns. We aim to further leverage this position to facilitate an open, cross-disciplinary discussion among researchers to assess progress and stimulate further debate on networks. We believe these efforts will ultimately yield novel modeling approaches and the identification of new applications or open problems that will guide future research in networks.

Reinforcement learning (RL) and MDPs have been topics of intense research since the middle of the last century. It was shown that Dynamic Programming (DP) [B, H, SB] gives the optimal policy and its computational cost is polynomial in the number of states and actions. This polynomial dependence on the size of the state space limits exact DP to small state problems. Modern applications of RL need to deal with large state problems that arise in many areas ranging from robotics to medical trials to finance.

Solving a large state MDP problem can be computationally intractable in the worst case [PT, CT]. Despite these negative results, several algorithms are shown to perform remarkably well in certain large state problems. Examples are UCT algorithm of Kocsis and Szepesvari [KS] applied in heuristic search and games, Rapidly exploring Random Trees (RRT) of LaValle and Kuffner [LK] in motion planning, policy gradient methods applied in robotics [KP, GE], approximate linear programming (ALP) applied in queuing networks [FV, DFM], and approximate dynamic programming applied in very large scale industrial applications [P]. These algorithms are developed mostly independently in different communities. Despite some similarities, the relation between them and what makes them effective is not very clear.

The computational problem that we discussed above is the focus of optimization and ADP communities. The challenge is to find a computationally efficient way to achieve something that would be easy if we had infinite computational resources. In reinforcement learning, we encounter additional statistical challenges; even if we have infinite computational power, it is not clear how we should best make inferences from observations and select actions that balance between exploration and exploitation.

This workshop will bring researchers from different communities together to discuss and exchange ideas about effective approaches and open problems in large scale MDP problems.

References:

[B] R. Bellman, “Dynamic Programming”, Princeton University Press, 1957.
[H] R. A. Howard, “Dynamic Programming and Markov Processes”, MIT, 1960.
[SB] R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction . Bradford Book. MIT Press, 1998.
[PT] C. H. Papadimitriou and J. N. Tsitsiklis. "The Complexity of Markov Decision Processes", Mathematics of Operations Research, Vol. 12, No. 3, 1987, pp. 441-450.
[CT] C.-S. Chow and J. N. Tsitsiklis, "The Complexity of Dynamic Programming", Journal of Complexity, Vol. 5, No. 4, 1989, pp. 466-488.
[GE] M. Ghavamzadeh and Y. Engel, "Bayesian Policy Gradient Algorithms". Neural Information Processing Systems (NIPS), 2006.
[LK] S. M. LaValle, J. J. Kuffner, “Randomized kinodynamic planning”, The International Journal of Robotics Research 20 (5), 378-400, 2001
[KS] L. Kocsis and C. Szepesvari, "Bandit based monte-carlo planning", ECML, 2006.
[KP] J. Kober and J. Peters, "Policy Search for Motor Primitives in Robotics", Machine Learning, 84, 1-2, pp.171-203, 2011.
[FV] D. P. de Farias and B. Van Roy, "The linear programming approach to approximate dynamic programming", Operations Research, 51, 2003.
[DFM] V. V. Desai, V. F. Farias, and C. C. Moallemi, "Approximate dynamic programming via a smoothed linear program", Operations Research , 60(3):655–674, 2012.
[P] W. B. Powell, “Approximate Dynamic Programming: Solving the curses of dimensionality”, John Wiley & Sons, 2007.

In typical applications of machine learning (ML), humans typically enter the process at an early stage, in determining an initial representation of the problem and in preparing the data, and at a late stage, in interpreting and making decisions based on the results. Consequently, the bulk of the ML literature deals with such situations. Much less research has been devoted to ML involving “humans-in-the-loop,” where humans play a more intrinsic role in the process, interacting with the ML system to iterate towards a solution to which both humans and machines have contributed. In these situations, the goal is to optimize some quantity that can be obtained only by evaluating human responses and judgments. Examples of this hybrid, “human-in-the-loop” ML approach include:

-- ML-based education, where a scheduling system acquires information about learners with the goal of selecting and recommending optimal lessons;
-- Adaptive testing in psychological surveys, educational assessments, and recommender systems, where the system acquires testees’ responses and selects the next item in an adaptive and automated manner;
-- Interactive topic modeling, where human interpretations of the topics are used to iteratively refine an estimated model;
-- Image classification, where human judgments can be leveraged to improve the quality and information content of image features or classifiers.

The key difference between typical ML problems and problems involving “humans-in-the-loop” and is that in the latter case we aim to fit a model of human behavior as we collect data from subjects and adapt the experiments we conduct based on our model fit. This difference demands flexible and robust algorithms and systems, since the resulting adaptive experimental design depends on potentially unreliable human feedback (e.g., humans might game the system, make mistakes, or act lazily or adversarially). Moreover, the “humans-in-the-loop” paradigm requires a statistical model for human interactions with the environment, which controls how the experimental design adapts to human feedback; such designs are, in general, difficult to construct due to the complex nature of human behavior. Suitable algorithms also need to be very accurate and reliable, since humans prefer a minimal amount of interaction with ML systems; this aspect also prevents the use of computationally intensive parameter selection methods (e.g., a simple grid search over the parameter space). These requirements and real-world constraints render “humans-in-the-loop” ML problems much more challenging than more standard ML problems.

In this workshop, we will focus on the emerging new theories, algorithms, and applications of human-in-the-loop ML algorithms. Creating and estimating statistical models of human behavior and developing computationally efficient and accurate methods will be a focal point of the workshop. This human-behavior aspect of ML has not been well studied in other fields that rely on human inputs such as active learning and experimental design. We will also explore other potential interesting applications involving humans in the loop in different fields, including, for example, education, crowdsourcing, mobile health, pain management, security, defense, psychology, game theory, and economics.

The goal of this workshop is to bring together experts from different fields of ML, cognitive and behavioral sciences, and human-computer interaction (HCI) to explore the interdisciplinary nature of research on this topic. In particular, we aim to elicit new connections among these diverse fields, identify novel tools and models that can be transferred from one to the other, and explore novel ML applications that will benefit from the human-in-the-loop of ML algorithms paradigm. We believe that a successful workshop will lead to new research directions in a variety of areas and will also inspire the development of novel theories and tools.

1. Aim
   
   MLINI workshop focuses on machine learning approaches in neuroscience, neuroimaging, with a specific extension to behavioral experiments and psychology. The special topic of this year is "Going Beyond the Scanner", which includes making inference about the subject's mental states from ''cheap'' data such as subject's speech and/or text, audio, video, EEG and other wearable devices.
   
   We believe that machine learning has a prominent role in shaping how questions in neuroscience are framed, and that the machine-learning mind set is now entering modern psychology and behavioral studies. It is also equally important that practical applications in these fields motivate a rapidly evolving line or research in the machine learning community. In parallel, there is an intense interest in learning more about brain function in the context of rich naturalistic environments and scenes. Efforts to go beyond highly specific paradigms that pinpoint a single function, towards schemes for measuring the interaction with natural and more varied scene are made. In this context, many controversies and open questions exist.
   
   The goal of the workshop is to pinpoint the most pressing issues and common challenges across the fields, and to sketch future directions and open questions in the light of novel methodology. The proposed workshop is aimed at offering a forum that joins machine learning, neuroscience, and psychology community, and should facilitate formulating and discussing the issues at their interface.
   
   Motivated by the previous workshops in this series, MLINI ‘11, MLINI’12, and MLINI’13, we will center this workshop around invited talks, and two panel discussions. Triggered by these discussions, this year we plan to adapt the workshop topics to a less traditional scope neuroimaging scope and investigate the role of behavioral models and psychology, including topics such as psycholinguistics.
   
   Besides interpretation, and the shift of paradigms, many open questions remain at the intersection of machine learning, neuroimaging and psychology. Among them:
   
   
   - How can we move towards more naturalistic stimuli, tasks and paradigms in neuroimaging and neuro-signal analysis?
   
   - What kind of mental states can be inferred from cheaper and easier to collect data sources (as an alternative to fMRI scanner) such as text, speech, audio, video, EEG, and wearable devices?
   
   - How can we leave the lab when acquiring neuroimaging data, towards exploiting mobile acquisition (EEG and NIRS)?
   
   - What type of features should be extracted from naturalistic stimuli such as text, voice, etc., to detect specific mental states and/or mental disorders?
   
   - How can we combine traditional neuroimaging with naturalistic data collected from a subject or group of subjects?
   
   - In general, can we characterize situations when multivariate predictive analysis (MVPA) and inference methods are better suited for brain imaging analysis than more traditional techniques?
   
   - Given recent advances of deep learning in image analysis and other applications, a natural question to ask is whether neuroimaging analysis can benefit from such approaches?
   
   - How well can functional networks and dynamical models capture the brain activity, and when using network and dynamics information is superior to standard task-based brain activations?
   
   
   2. Overview
   
   Modern multivariate statistical methods have been increasingly applied to various problems in neuroimaging, including “mind reading”, “brain mapping”, clinical diagnosis and prognosis. Multivariate pattern analysis (MVPA) methods are designed to examine complex relationships between large-dimensional signals, such as brain MRI images, and an outcome of interest, such as the category of a stimulus, with a limited amount of data. The MVPA approach is in contrast with the classical mass-univariate (MUV) approach that treats each individual imaging measurement in isolation.
   
   Recent multivariate methods give researchers more latitude in their choice of intricate models of behaviour and psychological state, beyond traditional cognitive and clinical neuroscience studies often limited to binary classification (e.g., healthy vs schizophrenic, etc), and traditionally driven by staitisical tools such as SPM oriented towards contrastive analysis. For example ‘zero-shot-learning’ methods (Mitchell 2008) managed to generalize predictions of brain activity beyond training data, by using a modelled descriptive latent space (in this case a vector space of word meaning). Work by John Anderson predicts variations in local processing load with a general model of cognitive function, instantiated with very specific operations, such as mental arithmetic.
   
   
   Finally, an important and rapidly growing area of brain imaging is the study of brain’s functional connectivity, i.e. focusing on brain as a network of functionally dependent areas, as well as brain’s dynamical models (Granger causality, etc). It was demonstrated that functional networks can be very informative about particular mental states and/or diseases even when standard activation-based MUV approaches fail. Modern machine-learning approaches to network analysis, including large-scale (sparse) probabilistic graphical models, such as Gaussian MRFs, that go beyond standard correlation-based functional network, can advance our understanding of brain activity even further (e.g., see Honorio et al, and other work). Finally, dynamical models (from differential equations to dynamic graphical models) should provide even more accurate tools for capturing the activity of the brain, perhaps the most complicated dynamical system, and relating it to mental states and behavior.
   
   In this workshop, we intend to investigate the implications that follow from adopting multivariate machine-learning methods for studying brain function. In particular, this concerns the question how these methods may be used to represent cognitive states, and what ramifications this has for consequent theories of cognition. Besides providing a rationale for the use of machine-learning methods in studying brain function, a further goal of this workshop is to identify shortcomings of state-of-the-art approaches and initiate research efforts that increase the impact of machine learning on cognitive neuroscience. Open questions and possible topics for contribution will be structured around the
   following 4 main topics: I) machine learning and pattern recognition methodology in brain research, II) functional connectivity and dynamical models of brain activity, III) multi-modal analysis including mental state inference from behavioral data, and IV) linking machine learning, neuroimaging and neuroscience.
   
   
   3. New topic this year: Beyond the scanner - capturing behavior, cognition and psychology
   
   This year, we propose to shift the focus of the workshop towards mental state detection and prediction ‘’beyond the scanner”, and focus on making inferences not only from imaging data, but also from relatively cheap data sources such as text of interviews with the patients, as well as voice and other behavioral data. Recent results on applying multivariate statistical techniques to behavioral data, such as text/voice data from interviews with the psychiatric patients, open new exciting opportunities on objectively quantifying mental states from subject’s behavior, i.e. extending the traditional, and rather subjective, diagnostic approaches to the ones based on objective measures computed from behavioral data (i.e., ``computational psychiatry’’). For example, recent exciting directions along these lines include mental state classification using behavioral data such as voice and/or text from interviews with subjects; e.g., several recent papers that demonstrated the possibility of accurate classification of various mental conditions, including schizophrenia, mania, as well as drug influence (ecstasy, meth), based on syntactic graph features as well as semantic features extracted from interviews with subject; moreover, acoustic features of speech were shown to allow for an accurate discrimination of Altzheimer’s patients from MCI and from controls.
   
   Our aim this year is to explore scientific and practical applications of computational linguistics and other approaches to mental state inference from relatively cheap behavioral data that can be collected on everyday basis, from multiple sources such as smartphones, emails, blogs etc, as well as the data collectible from wearable devices such as inexpensive EEG devices (e.g., NeuroSky), and other wearables (e.g., smart watches etc), in order to analyze, track and potentially improve people's mental state via ''smart personal assistants'' (less seriously, we refer to this approach as to "Freud in a box").

Goal:

Extracting knowledge from Web pages, and integrating it into a coherent knowledge base (KB) is a task that spans the areas of natural language processing, information extraction, information integration, databases, search, and machine learning. Recent years have seen significant advances here, both in academia and in the industry. Most prominently, all major search engine providers (Yahoo!, Microsoft Bing, and Google) nowadays experiment with semantic KBs. Our workshop serves as a forum for researchers on knowledge base construction in both academia and industry.

Unlike many other workshops, our workshop puts less emphasis on conventional paper submissions and presentations, but more on visionary papers and discussions. In addition, one of its unique characteristics is that it is centered on keynotes by high-profile speakers. AKBC 2010, AKBC 2012, and AKBC 2013 each had a dozen invited talks from leaders in this area from academia, industry, and government agencies. We had senior invited speakers from Google, Microsoft, Yahoo, several leading universities (MIT, University of Washington, CMU, University of Massachusetts, and more), and DARPA. With this year’s proposal, we would like to resume this positive experience. By inviting established researchers for keynotes, and by focusing particularly on vision paper submissions, we aim to provide a vivid forum of discussion about the field of automated knowledge base construction.

Topics of interest:

* machine learning on text; unsupervised, lightly-supervised and distantly-supervised learning; learning from naturally-available data.
* human-computer collaboration in knowledge base construction; automated population of wikis.
* inference for graphical models and structured prediction; scalable approximate inference.
* information extraction; open information extraction, named entity extraction; ontology construction;
* entity resolution, relation extraction, information integration; schema alignment; ontology alignment; monolingual alignment, alignment between knowledge bases and text;
* pattern analysis, semantic analysis of natural language, reading the web, learning by reading
databases; distributed information systems; probabilistic databases;
* scalable computation; distributed computation.
* queries on mixtures of structured and unstructured data; querying under uncertainty;
* dynamic data, online/on-the-fly adaptation of knowledge.
* languages, toolkits and systems for automated knowledge base construction.
* demonstrations of existing automatically-built knowledge bases.

Audience:

AKBC 2012 and AKBC 2013 attracted 70-100 participants. These were researchers from academia, industry, and government, as well as students. The workshop brought together people from the areas of natural language processing, machine learning, and information extraction. We would expect a similar composition of the audience also for AKBC 2014. Since our keynote talks are given by very senior researchers (usually the coordinators of entire scientific projects), the talks are usually high-level and easily understandable. Therefore, we are confident that the workshop will be of interest also to novices in the area or first year students, who wish to get an overview of the automated KB construction. At the same time, the high calibre of our speakers is almost certain to attract established researchers who wish to get a survey of the latest developments in the field. The vision papers, too, play their role in attracting the audience, as these papers are deliberately designed to provoke thought and discussion from domain experts and novices to the field alike.

We are organizing a one day NIPS 2014 workshop that will cover topics at the intersection of machine learning and software architecture/engineering. This intersection is a critical area for deploying machine learning methods in practice, but is often overlooked in the literature. As a result, much of the publicly available code for download is disorganized, undocumented, and buggy. Therefore, it cannot serve as an example of how actual deployed machine-learning-heavy software should be written. Those looking to implement actual software could greatly benefit from a workshop that can provide guidance on software practices.

There are several topics this workshop will cover through contributed and invited talks:
1. Scaling machine learning: Solutions to practical issues involving taking single machine algorithms and making them ready for “big data” by distributing them with Spark or Hadoop/MapReduce are welcome here.
2. Accelerating machine learning prototypes: Methods and tips for moving single machine Matlab/R/Python code to C/C++ code, as well as GPU acceleration.
3. Software paradigms: When is it best to work in an object oriented, procedural, or functional framework when developing machine learning software?
4. When to use probabilistic programming environments? If so, which tool (e.g. Infer.NET, Stan, Church, etc.) is most appropriate for your project requirements?
5. Systematic testing: This is often overlooked but important area for the workshop to cover. Can we develop better methods for systematically testing our methods to make sure they are implemented correctly? This includes unit testing and regression testing.
(a) There is a perception among some practitioners that systematic methods like unit tests are not applicable to machine learning because “The whole reason we are doing the computation in the first place is that we do not know the answer.” One goal of this workshop is to try and change that perception with guidance and examples.
(b) What are some of the common ways to break down a machine learning project into units where unit testing is possible? Monte Carlo unit tests: Unlike most projects many unit tests in machine learning are Monte Carlo tests.
(c) Different inference methods will have their own methods that can be used to test their implementation correctness: VB, EP, MCMC [3; 1], etc.
6. Documentation: How should people in machine learning be doing a better job at documenting their code? Do the usual guidelines for software documentation need to be augmented or modified for machine learning software? Could tools such as literate programming [4] be more useful than the typical documentation tools (e.g. Doxygen or Javadoc)? We could examine issues involving requirements documents [6] for machine learning algorithms.
7. Advice for machine learning people in interfacing with traditional software designers. What are common misunderstandings and things we should be ready to explain?
8. Collaboration on machine learning projects. For instance, platforms that make it easy for engineers to reuse features and code from other teams make every feature engineer much more impactful.
9. Issues with regard to open source in machine learning. Talks involving intellectual property issues in machine learning would also be welcome.
10. Getting data into the software development process is also a possible talk. Handling organization restrictions with regard to security and privacy issues is an important area.
11. Building automatic benchmarking systems. A critical part of machine learning project is to first setup an independent evaluation system to benchmark the current version of the software. This system can ensure that software is not accidentally “peaking” at the test data. Other subtle issues include excessive benchmarking against a test set which could result in overfitting, or not placing any confidence intervals on the benchmarks used. Machine learning competitions can provide some guidance here.
12. Methods for testing and ensuring numerical stability. How do we deal with numerical stability in deployed, or real-time, software systems?
13. Differences between using machine learning in client side vs. server side software. Processing the training set client side, and outside of the designers control, poses many more challenges than only processing the test set on a client machine.
14. Reproducibility: What advice do we have for making machine learning experiments completely reproducible? This is largely an extension of using revision control systems and procedures for logging results.
15. Design patterns: What advice is there for utilizing ideas, for example from Gamma et al. [2], in machine learning projects?

Many of the above items are about utilizing and adapting advice from tradition software development, such as from McConnell [5].

PDF formatted copy of proposal available at: http://goo.gl/1v51aS

References
[1] Cook, S. R., Gelman, A., and Rubin, D. B. (2006). Validation of software for Bayesian models using
posterior quantiles. Journal of Computational and Graphical Statistics, 15(3):675–692.
[2] Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1994). Design patterns: elements of reusable objectoriented
software. Pearson Education.
[3] Geweke, J. (2004). Getting it right: Joint distribution tests of posterior simulators. Journal of the American
Statistical Association, 99(467):799–804.
[4] Knuth, D. E. (1984). Literate programming. The Computer Journal, 27(2):97–111.
[5] McConnell, S. (2004). Code Complete: A Practical Handbook Of Software Construction. Microsoft Press.
[6] Tripp, L. L. (1998). IEEE recommended practice for software requirements specifications. IEEE Std
830-1998, pages 1–40.

Nonparametric methods (kernel methods, kNN, classification trees, etc) are designed to handle complex pattern recognition problems. Such complex problems arise in modern applications such as genomic experiments, climate analysis, robotic control, social network analysis, and so forth. In fact, contemporary statistical procedures are making inroads into a variety of modern application areas as part of solutions to larger problems. As such there is a growing need for statistical procedures that can be used "off-the-shelf", i.e. procedures with as few parameters as possible, or better yet, procedures which can "self-tune" to a particular application at hand.

The problem of devising 'parameter-free' procedures has been addressed in separate areas of the pattern-recognition literature under various names and different emphasis.

In traditional statistics, much effort has gone into so called "adaptive" procedures which can attain optimal risks over large sets of models of increasing complexity. Examples are model selection approaches based on penalized empirical risk minimization, approaches based on stability of estimates (e.g. Lepski’s methods), thresholding approaches under sparsity assumptions, and model averaging approaches. Most of these approaches rely on having tight bounds on the risk of learning procedures (under any parameter setting), hence other approaches concentrate on tight estimations of the actual risks, e.g., Stein’s risk estimators, bootstrapping methods, data dependent learning bounds.

In theoretical machine learning, much of the work has focused on proper tuning of the actual optimization procedures used to minimize (penalized) empirical risks. In particular, great effort has gone into the automatic setting of important tuning parameters such as 'learning rates' and 'step sizes'.

Another approach out of machine learning arises in the kernel literature for 'automatic representation learning'. The aim of the approach, similar to theoretical work on model selection, is to automatically learn an appropriate (kernel) transformation of the data for use with kernel methods such as SVMs or Gaussian processes.

In practice, the simplest self-tuning procedures take the form of cross-validation and variants. Cross-validation can however be expensive in practice, and impractical in various constrained settings -- e.g., streaming settings, in settings with large amounts of tuning parameters, and generally in unsupervised learning problems.

More generally, many existing self-tuning or parameter-free methods are unfortunately expensive given large modern data sizes and dimensionality, while the cheaper methods tend to self-tune only to small model classes. Ideally we would want self-tuning procedures that can adapt to easy or difficult (nonparametric) problems, while satisfying the practical constraints of modern applications.

A main aim of this workshop is to cover the various approaches proposed so far towards automating the learning pipeline, and the practicality of these approaches in light of modern constraints. We are particularly interested in understanding whether large datasizes and dimensionality might help the automation effort since such datasets in fact provide more information on the patterns being learned.

Through a number of invited and contributed talks and a focused panel discussion, we plan to bring together both theoretical and applied researchers to discuss these challenges in detail, share insight on existing solutions, and lay out some of the important future directions towards answering the demands of modern applications.

This workshop addresses questions at the intersection of discrete and combinatorial optimization and machine learning.


Solving optimization problems with ultimately discrete solutions is becoming increasingly important in machine learning. At the core of statistical machine learning is to make inferences from data, and when the variables underlying the data are discrete, both the tasks of inferring the model from data as well as performing predictions using the estimated model are inherently discrete optimization problems. Many of these optimization problems are notoriously hard. As a result, abundant and steadily increasing amounts of data -- despite being statistically beneficial -- quickly render standard off-the-shelf optimization procedures either impractical, intractable, or both.

While many problems are hard in the worst case, the problems of practical interest are often much more well-behaved, or are well modeled by assuming properties that make them so. Indeed, many discrete problems in machine learning can possess beneficial structure; such structure has been an important ingredient in many successful (approximate) solution strategies. Examples include the marginal polytope, which is determined by the graph structure of the model, or sparsity that makes it possible to handle high dimensions. Symmetry and exchangeability are further exploitable characteristics. In addition, functional properties such as submodularity, a discrete analog of convexity, are proving to be useful to an increasing number of machine learning problems. One of the primary goals of this workshop is to provide a platform for exchange of ideas on how to discover, exploit, and deploy such structure.

Machine learning, algorithms, discrete mathematics and combinatorics as well as applications in computer vision, speech, NLP, biology and network analysis are all active areas of research, each with an increasingly large body of foundational knowledge. The workshop aims to ask questions that enable communication across these fields.

Probabilistic models and approximate inference algorithms are powerful, widely-used tools, central to fields ranging from machine learning to robotics to genetics. However, even simple variations on models and algorithms from the standard machine learning toolkit can be difficult and time-consuming to design, specify, analyze, implement, optimize and debug. The emerging field of probabilistic programming aims to address these challenges by developing formal languages and software systems that integrate key ideas from probabilistic modeling and inference with programming languages and Turing-universal computation.

Over the past two years, the field has rapidly grown and begun to mature. Systems have developed enough that they are seeing significant adoption in real-world applications while also highlighting the need for research in profiling, testing, verification and debugging. New academic and industrial languages have been developed, yielding new applications as well as new technical problems. General-purpose probabilistic programming languages have emerged, complementing previous work focused on specific domains; some offer programmable inference so that experts can take over where automatic techniques fail. The field is has also begun to explore new AI architectures that perform probabilistic reasoning or hierarchical Bayesian approaches for inductive learning over rich data structures and software simulators.

This workshop will survey recent progress, emphasizing results from the ongoing DARPA PPAML program on probabilistic programming. A key theme will be articulating formal connections between probabilistic programming and other fields central to the NIPS community.

Optimal transport (OT) has emerged as a novel tool to solve problems in machine learning and related fields, e.g. graphics, statistics, data analysis, computer vision, economics and imaging.

In particular, the toolbox of OT (including for instance the Wasserstein/Earth Mover's Distances) offers robust mathematical techniques to study probability measures and compare complex objects described using bags-of-features representations.

Scaling OT algorithms to datasets of large dimension and sample size presents, however, a considerable computational challenge. Taking for granted that these challenges are partially solved, there remains many salient open research questions on how to integrate OT in statistical methodologies (dimensionality reduction, inference, modeling) beyond its classical use in retrieval. OTML 2014 will be the first international workshop to address state-of-the-art research in this exciting area.

Traditional machine learning and data analysis methods often assume that the input data can be represented by vectors in Euclidean space. While this assumption has worked well for many applications, researchers have increasingly realized that if the data is intrinsically non-Euclidean, ignoring this geometrical structure can lead to suboptimal results.

In the existing literature, there are two common approaches for exploiting data geometry when the data is assumed to lie on a Riemannian manifold.

In the first direction, often referred to as manifold learning, the data is assumed to lie on an unknown Riemannian manifold and the structure of this manifold is exploited through the training data, either labeled or unlabeled. Examples of manifold learning techniques include Manifold Regularization via the graph Laplacian, Locally Linear Embedding, and Isometric Mapping.

In the second direction, which is gaining increasing importance and success, the Riemannian manifold representing the input data is assumed to be known explicitly. Some manifolds that have been widely used for data representation are: the manifold of symmetric, positive definite matrices, the Grassmannian manifold of subspaces of a vector space, and the Kendall manifold of shapes. When the manifold is known, the full power of the mathematical theory of Riemannian geometry can be exploited in both the formulation of algorithms as well as their theoretical analysis.
Successful applications of these approaches are numerous and range from brain imaging and low rank matrix completion to computer vision tasks such as object detection and tracking.

This workshop focuses on the latter direction. We aim to bring together researchers in statistics, machine learning, computer vision, and other areas, to discuss and exchange current state of the art results , both theoretically and computationally, and identify potential future research directions

The mathematical analysis and understanding of rank data has been a fascinating topic for centuries, and has been investigated in disciplines as wide-ranging as social choice/voting theory, decision theory, probability, statistics, and combinatorics. In modern times, huge amounts of data are generated in the form of rankings on a daily basis: restaurant ratings, product ratings/comparisons, employer ratings, hospital rankings, doctor rankings, and an endless variety of rankings from committee deliberations (including, for example, deliberations of conference program committees such as NIPS!). These applications have led to several new trends and challenges: for example, one must frequently deal with very large numbers of candidates/alternatives to be ranked, with partial or missing ranking information, with noisy ranking information, with the need to ensure reliability and/or privacy of the rank data provided, and so on.

Given the increasing universality of settings involving large amounts of rank data and associated challenges as above, powerful computational frameworks and tools for addressing such challenges have emerged over the last few years in a variety of areas, including in particular in machine learning, operations research, and computational social choice. Despite the fact that many important practical problems in each area could benefit from the algorithmic solutions and analysis techniques developed in other areas, there has been limited interaction between these areas. Given both the increasing maturity of the research into ranking in these respective areas and the increasing range of practical ranking problems in need of better solutions, it is the aim of this workshop to bring together recent advances in analyzing rank data in machine learning, operations research, and computational social choice under one umbrella, to enable greater interaction and cross-fertilization of ideas.

A primary goal will be to discover connections between recent approaches developed for analyzing rank data in each of the three areas above. To this end, we will have invited talks by leading experts in the analysis of rank data in each area. In addition, we will include perspectives from practitioners who work with rank data in various applied domains on both the benefits and limitations of currently available solutions to the problems they encounter. In the end, we hope to both develop a shared language for the analysis and understanding of rank data in modern times, and identify important challenges that persist and could benefit from a shared understanding.

The topics of interest include:
- discrete choice modeling and revenue management
- voting and social decision making, preference elicitation
- social choice (rank aggregation) versus individual choice (recommendation systems)
- stochastic versus active sampling of preferences
- statistical/learning-theoretic guarantees
- effects of computational approximations

The last decade has witnessed a series of technological advances: social networks, cloud servers, personalized advertising, autonomous cars, personalized healthcare, robotics, security systems, just to name a few. These new technologies have in turn substantially reshaped our demands from adaptive reinforcement learning systems, defining novel yet urgent challenges. In response, a wealth of novel ideas and trends have emerged, tackling problems such as modelling rich and high-dimensional dynamics, life-long learning, resource-bounded planning, and multi-agent cooperation.

The objective of the workshop is to provide a platform for researchers from various areas (e.g., deep learning, game theory, robotics, computational neuroscience, information theory, Bayesian modelling) to disseminate and exchange ideas, evaluating their advantages and caveats. In particular, we will ask participants to address the following questions:

1) What is the future of reinforcement learning?
2) What are the most important challenges?
3) What tools do we need the most?

A final panel discussion will then review the provided answers and focus on elaborating a list of trends and future challenges. Recent advances will be presented in short talks and a poster session based on contributed material.

The field of computational biology has seen dramatic growth over the past few years. A wide range of high-throughput technologies developed in the last decade now enable us to measure parts of a biological system at various resolutions—at the genome, epigenome, transcriptome, and proteome levels. These data are high-dimensional, heterogeneous, and are impacted by a range of confounding factors, presenting new challenges for standard learning and inference approaches. Therefore, fully realizing the scientific and clinical potential of these data requires development of novel supervised and unsupervised learning methods that are scalable, can accommodate heterogeneity, are robust to systematic noise and confounding factors, and provide mechanistic insights.

The goal of this workshop is to present emerging problems and innovative machine learning techniques in computational biology. We will invite several speakers from the biology/bioinformatics community who will present current research problems in computational biology. We will also have the usual rigorous screening of contributed talks on novel learning approaches in computational biology. We encourage contributions describing either progress on new bioinformatics problems or work on established problems using methods that are substantially different from established alternatives. Kernel methods, graphical models, feature selection, non-parametric models and other techniques applied to relevant bioinformatics problems would all be appropriate for the workshop. We are particularly keen on considering contributions related to the prediction of functions from genotypes and to applications in personalized medicine, as illustrated by our invited speakers. The targeted audience are people with interest in learning and applications to relevant problems from the life sciences, including NIPS participants without any existing research link to computational biology.

The ever-increasing size of data sets has resulted in an immense effort in machine learning and statistics to develop more powerful and scalable probabilistic models. Efficient inference remains a challenge and limits the use of these models in large-scale scientific and industrial applications. Traditional unbiased inference schemes such as Markov chain Monte Carlo (MCMC) are often slow to run and difficult to evaluate in finite time. In contrast, variational inference allows for competitive run times and more reliable convergence diagnostics on large-scale and streaming data—while continuing to allow for complex, hierarchical modelling. This workshop aims to bring together researchers and practitioners addressing problems of scalable approximate inference to discuss recent advances in variational inference, and to debate the roadmap towards further improvements and wider adoption of variational methods.

The recent resurgence of interest in variational methods includes new methods for scalability using stochastic gradient methods, extensions to the streaming variational setting, improved local variational methods, inference in non-linear dynamical systems, principled regularisation in deep neural networks, and inference-based decision making in reinforcement learning, amongst others. Variational methods have clearly emerged as a preferred way to allow for tractable Bayesian inference. Despite this interest, there remain significant trade-offs in speed, accuracy, simplicity, applicability, and learned model complexity between variational inference and other approximative schemes such as MCMC and point estimation. In this workshop, we will discuss how to rigorously characterise these tradeoffs, as well as how they might be made more favourable. Moreover, we will address other issues of adoption in scientific communities that could benefit from the use of variational inference including, but not limited to, the development of relevant software packages.

Motivation
Crowdsourcing aims to combine human knowledge and expertise with computing to help solve problems and scientific challenges that neither machines nor humans can solve alone. In addition to a number of human-powered scientific projects, including GalaxyZoo, eBird, and Foldit, crowdsourcing is impacting the ability of academic researchers to build new systems and run new experiments involving people, and is also gaining a lot of use within industry for collecting training data for the purpose of machine learning. There are a number of online marketplaces for crowdsourcing, including Amazon’s Mechanical Turk, ODesk and MobileWorks. The fundamental question that we plan to explore in this workshop is:

How can we build systems that combine the intelligence of humans and the computing power of machines for solving challenging scientific and engineering problems?

The goal is to improve the performance of complex human-powered systems by making them more efficient, robust, and scalable.

Current research in crowdsourcing often focuses on micro-tasking (for example, labeling a set of images), designing algorithms for solving optimization problems from the job requester’s perspective and with simple models of worker behavior. However, the participants are people with rich capabilities including learning, collaboration and so forth, suggesting the need for more nuanced approaches that place special emphasis on the participants. Such human-powered systems could involve large numbers of people with varying expertise, skills, interests, and incentives. This poses many interesting research questions and exciting opportunities for the machine learning community. The goal of this workshop is to foster these ideas and work towards this goal by bringing together experts from the field of machine learning, cognitive science, economics, game theory, and human-computer interaction.


Topics of Interest
Topics of interests in the workshop include:

* Social aspects and collaboration: How can systems exploit the social ties of the underlying participants or users to create incentives for users to collaborate? How can online social networks be used to create tasks with a gamification component and engage users in useful activities? With ever-increasing time on the Internet being spent on online social networks, there is a huge opportunity to elicit useful contributions from users at scale, by carefully designing tasks.

* Incentives, pricing mechanisms and budget allocation: How to design the right incentive structure and pricing policies for participants that maximize the satisfaction of participants as well as utility of the job requester for a given budget? How can techniques from machine learning, economics and game theory be used to learn optimal pricing policies and to infer optimal incentive designs?

* Learning by participants: How can we use insights from machine learning to build tools for training and teaching the participants for carrying out complex or difficult tasks? How can this training be actively adapted based on the skills or expertise of the participants and by tracking the learning process?

* Peer prediction and knowledge aggregation: How can complex crowdsourcing tasks be decomposed into simpler micro-tasks? How can techniques of peer prediction be used to elicit informative responses from participants and incentivize effort? Can we design models and algorithms to effectively aggregate responses and knowledge, especially for complex tasks?

* Privacy aspects: The question of privacy in human-powered systems has often been ignored and we seek to understand the privacy aspects both from job requester as well as privacy of the participants. How can a job requester (such as firm interested in translating legal documents) carry out crowdsourcing tasks without revealing private information to the crowd? How can systems negotiate the access to private information of participants (such as the GPS location in community sensing applications) in return of appropriate incentives?

* Open theoretical questions and novel applications: What are the open research questions, emerging trends and novel applications related to design of incentives in human computation and crowdsourcing systems?


Participants
We expect diverse participation from researchers with a wide variety of scientific interests spanning economics, game theory, cognitive science, and human-computer interaction. Given the widespread use of crowdsourcing in the industry, such Amazon, Google and Bing, we expect active participation from industry.

Transfer, domain adaptation and multi-task learning methods have been developed to better exploit the available data at training time, originally moved by the need to deal with a reduced amount of information. Nowadays, gathering data is much easier than in the past thanks to the low price of different acquisition devices (e.g. cameras) and to the World Wide Web that connects million of devices users. Existing methods must embrace new challenges to manage large scale data that do not lack anymore in size but may lack in quality or may continuously change over time. All this comes with several open questions, for instance:

- what are the limits of existing multi-task learning methods when the number of tasks grows while each task is described by only a small bunch of samples (“big T, small n”)?
- theory vs. practice: can multi-task learning for very big data (n>10^7) be performed with extremely randomized trees?
- what is the right way to leverage over noisy data gathered from the Internet as reference for a new task?

- can we get an advantage by overcoming the dataset bias and aligning multiple existing but partially related data collections before using them as source knowledge for a new target problem?

- how can an automatic system process a continuous stream of information in time and progressively adapt for life-long learning?
- since deep learning has demonstrated high performance on large scale data, is it possible to combine it with transfer and multiple kernel learning in a principled manner?
- can deep learning help to learn the right representation (e.g., task similarity matrix) in kernel-based transfer and multi-task learning?
- How can notions from reinforcement learning such as source task selection be connected to notions from convex multi-task learning such as the task similarity matrix?
- How can similarities across languages help us adapt to different domains in natural language processing tasks?

After the first workshop edition where we investigated new directions for learning across domains, we want now to call the attention of the machine learning community on the emerging problem of big data and its particular challenges regarding multi-task and transfer learning and its practical effects in many application areas like computer vision, robotics, medicine, bioinformatics etc. where transfer, domain adaptation and multi-task learning have been previously used with success. We will encourage applied researchers to contribute to the workshop in order to create a synergy with theoreticians and lead to a global advancement of the field.

A selection of the papers accepted to the workshop and voted by the reviewers will be re-evaluated also as invited contributions to the planned JMLR special topic on Domain Adaptation, Multi-task and Transfer Learning. The proposal for this special topic is currently under evaluation.

References:
[1] I. Kuzborskij, F. Orabona. Stability and Hypothesis Transfer Learning. ICML 2013
[2] T. Tommasi, F. Orabona, B. Caputo. Learning Categories from few Examples with Multi Model Knowledge Transfer. PAMI 36(5), 2014.
[3] U. Rückert, M. Kloft. Transfer Learning with Adaptive Regularizers. ECML 2011.
[4] A. Pentina, C. H. Lampert. A PAC-Bayesian bound for Lifelong Learning. ICML 2014.
[5] X. Glorot , A. Bordes , Y. Bengio. Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach. ICML 2011.
[6] A. Kumar, A. Saha, H. Daumé III. A Co-regularization Based Semi-supervised Domain Adaptation. NIPS 2010.
[7] Cortes, Corinna, and Mehryar Mohri. Domain adaptation and sample bias correction theory and algorithm for regression. In Theoretical Computer Science 519 (2014): 103-126.
[8] C. Widmer, M. Kloft, G. Rätsch. Multi-task Learning for Computational Biology: Overview and Outlook. In Schölkopf et al: Festschrift in Honor of Vladimir Vapnik, Spinger 2013.

See link.