( events)   Timezone: »  
Program Highlights »
Sat Dec 09 08:00 AM -- 06:30 PM (PST) @ S1
Machine Learning Challenges as a Research Tool
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy

Workshop Home Page

Challenges in machine learning and data science are competitions running over several weeks or months to resolve problems using provided datasets or simulated environments. The playful nature of challenges naturally attracts students, making challenge a great teaching resource. For this fourth edition of the CiML workshop at NIPS we want to explore the impact of machine learning challenges as a research tool. The workshop will give a large part to discussions around several axes: (1) benefits and limitations of challenges as a research tool; (2) methods to induce and train young researchers; (3) experimental design to foster contributions that will push the state of the art.
CiML is a forum that brings together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. Following the success of last year's workshop, in which a fruitful exchange led to many innovations, we propose to reconvene and discuss new opportunities for challenges as a research tool, one of the hottest topics identified in last year's discussions. We have invited prominent speakers in this field.
We will also reserve time to an open discussion to dig into other topic including open innovation, collaborative challenges (coopetitions), platform interoperability, and tool mutualisation.
The audience of this workshop is targeted to workshop organizers, participants, and anyone with scientific problem involving machine learning, which may be formulated as a challenge. The emphasis of the workshop is on challenge design. Hence it complements nicely the workshop on the NIPS 2017 competition track and will help paving the way toward next year's competition program.

Introduction - Isabelle Guyon and Evelyne Viegas (Announcement)
Baázs Kégl, RAMP platform (Invited talk)
Automatic evaluation of chatbots - Varvara Logacheva (Talk)
TrackML - David Rousseau (Talk)
Data Science Bowl - Andre Farris (Talk)
CrowdAI - Mohanty Sarada (Invited talk)
Break, poster viewing 1 (Poster)
Ben Hamner, Kaggle platform (Invited talk)
Incentivizing productivity in data science - Jacob Abernethy (Discussion)
Establishing Uniform Image Segmentation - Maximilian Chen and Michael C. Darling (Poster)
Lunch Break, poster viewing 2 (Poster)
Welcome - Isabelle Guyon and Evelyne Viegas (Announcement)
Project Malmo, Minecraft - Katja Hofmann (Invited talk)
Project Alloy - Laura Seaman and Israel Ridgley (Talk)
Education and public service - Jonathan C. Stroud (Talk)
Break, poster viewing 3 (Poster)
AutoDL (Google challenge) - Olivier Bousquet (Talk)
Scoring rule markets - Rafael Frongillo and Bo Waggoner (Talk)
ENCODE-DREAM challenge - Akshay Balsubramani (Talk)
Codalab platform, Xavier Baro (Invited talk)
New opportunities for challenges, looking to the future - Sergio Escalera (Discussion)
Wrap up -Evelyne Viegas and Isabelle Guyon (Announcement)