Timezone: »
Challenges in machine learning and data science are competitions running over several weeks or months to resolve problems using provided datasets or simulated environments. The playful nature of challenges naturally attracts students, making challenge a great teaching resource. For this third edition of the CiML workshop at NIPS we want to explore more in depth the opportunities that challenges offer as teaching tools. The workshop will give a large part to discussions around several axes: (1) benefits and limitations of challenges to give students problem-solving skills and teach them best practices in machine learning; (2) challenges and continuous education and up-skilling in the enterprise; (3) design issues to make challenges more effective teaching aids; (4) curricula involving students in challenge design as a means of educating them about rigorous experimental design, reproducible research, and project leadership.
CiML is a forum that brings together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. Following the success of last year's workshop (http://ciml.chalearn.org/), in which a fruitful exchange led to many innovations, we propose to reconvene and discuss new opportunities for challenges in education, one of the hottest topics identified in last year's discussions. We have invited prominent speakers in this field.
We will also reserve time to an open discussion to dig into other topic including open innovation, coopetitions, platform interoperability, and tool mutualisation.
Thu 11:00 p.m. - 11:30 p.m.
|
Welcome
(
Presentation
)
|
Evelyne Viegas 🔗 |
Thu 11:30 p.m. - 12:10 a.m.
|
Gathering common sense knowledge: how to game it? ( Invited talk ) link » | Larry Zitnick 🔗 |
Fri 12:10 a.m. - 12:30 a.m.
|
The Michigan Data Science Team: A Student Organization for Machine Learning Challenges ( Contributed talk ) link » | Jonathan C Stroud 🔗 |
Fri 12:30 a.m. - 12:50 a.m.
|
Energy generation prediction: Lessons learned from the use of Kaggle in Machine Learning Course ( Contributed talk ) link » | Jesus Fernandez-Bes 🔗 |
Fri 12:50 a.m. - 1:30 a.m.
|
Learning to improve learning: ML in the classroom ( Invited talk ) link » | Emma Brunskill 🔗 |
Fri 2:00 a.m. - 3:00 a.m.
|
Challenges in education
(
Discussion
)
|
Balázs Kégl · Ben Hamner 🔗 |
Fri 3:00 a.m. - 5:00 a.m.
|
Lunch, posters and discussions
(
Posters
)
|
🔗 |
Fri 5:00 a.m. - 5:40 a.m.
|
OpenML in research and education ( Invited talk ) link » | Joaquin Vanschoren 🔗 |
Fri 5:40 a.m. - 6:00 a.m.
|
ImageCLEF 2017 LifeLog task ( Contributed talk ) link » | Duc Tien Dang Nguyen 🔗 |
Fri 6:30 a.m. - 7:10 a.m.
|
Evaluation-as-a-Service: a serious game ( Invited talk ) link » | Henning Mueller 🔗 |
Fri 7:10 a.m. - 7:50 a.m.
|
Reproducible Research: moving to the BEAT ( Invited talk ) link » | Sébastien Marcel 🔗 |
Fri 7:50 a.m. - 8:10 a.m.
|
CAFA: a Challenge Dedicated to Understanding the Function of Biological Macromolecules ( Contributed talk ) link » | Predrag Radivojac 🔗 |
Fri 8:10 a.m. - 8:30 a.m.
|
Interactive Machine Learning (iML): a challenge for Game-based approaches ( Contributed talk ) link » | Andreas Holzinger 🔗 |
Fri 8:30 a.m. - 9:30 a.m.
|
Gaming challenges and encouraging collaborations
(
Discussion
)
|
Sergio Escalera · Isabelle Guyon 🔗 |
Author Information
Isabelle Guyon (U. Paris-Saclay & ChaLearn)
Isabelle Guyon recently joined Google Brain as a research scientist. She is also professor of artificial intelligence at Université Paris-Saclay (Orsay). Her areas of expertise include computer vision, bioinformatics, and power systems. She is best known for being a co-inventor of Support Vector Machines. Her recent interests are in automated machine learning, meta-learning, and data-centric AI. She has been a strong promoter of challenges and benchmarks, and is president of ChaLearn, a non-profit dedicated to organizing machine learning challenges. She is community lead of Codalab competitions, a challenge platform used both in academia and industry. She co-organized the “Challenges in Machine Learning Workshop” @ NeurIPS between 2014 and 2019, launched the "NeurIPS challenge track" in 2017 while she was general chair, and pushed the creation of the "NeurIPS datasets and benchmark track" in 2021, as a NeurIPS board member.
Evelyne Viegas (Microsoft Research)
Balázs Kégl (Université Paris Saclay/CNRS)
Ben Hamner (Kaggle)
Sergio Escalera (Computer Vision Center and University of Barcelona)
More from the Same Authors
-
2022 : Fifteen-minute Competition Overview Video »
Dustin Carrión-Ojeda · Ihsan Ullah · Sergio Escalera · Isabelle Guyon · Felix Mohr · Manh Hung Nguyen · Joaquin Vanschoren -
2022 Competition: Cross-Domain MetaDL: Any-Way Any-Shot Learning Competition with Novel Datasets from Practical Domains »
Dustin Carrión-Ojeda · Ihsan Ullah · Sergio Escalera · Isabelle Guyon · Felix Mohr · Manh Hung Nguyen · Joaquin Vanschoren -
2022 Poster: Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classification »
Ihsan Ullah · Dustin Carrión-Ojeda · Sergio Escalera · Isabelle Guyon · Mike Huisman · Felix Mohr · Jan N. van Rijn · Haozhe Sun · Joaquin Vanschoren · Phan Anh Vu -
2022 : Isabelle Guyon »
Isabelle Guyon -
2022 Invited Talk: The Data-Centric Era: How ML is Becoming an Experimental Science »
Isabelle Guyon -
2022 : NeurIPS Competitions – Evolution and Opportunities »
Isabelle Guyon · Evelyne Viegas -
2021 Panel: The Role of Benchmarks in the Scientific Progress of Machine Learning »
Lora Aroyo · Samuel Bowman · Isabelle Guyon · Joaquin Vanschoren -
2020 : Keynote talk by Isabelle Guyon and Evelyne Viegas - "AI Competitions and the Science Behind Contests" »
Isabelle Guyon · Evelyne Viegas -
2020 Poster: Deep Statistical Solvers »
Balthazar Donon · Zhengying Liu · Wenzhuo LIU · Isabelle Guyon · Antoine Marot · Marc Schoenauer -
2019 : Open Space Topic “The Organization of Challenges for the Benefit of More Diverse Communities” »
Adrienne Mendrik · Isabelle Guyon · Wei-Wei Tu · Evelyne Viegas · Ming LI -
2019 Workshop: CiML 2019: Machine Learning Competitions for All »
Adrienne Mendrik · Wei-Wei Tu · Wei-Wei Tu · Isabelle Guyon · Evelyne Viegas · Ming LI -
2019 : Welcome and Opening Remarks »
Adrienne Mendrik · Wei-Wei Tu · Isabelle Guyon · Evelyne Viegas · Ming LI -
2018 : Afternoon Welcome - Isabelle Guyon and Evelyne Viegas »
Isabelle Guyon -
2018 Workshop: CiML 2018 - Machine Learning competitions "in the wild": Playing in the real world or in real time »
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy -
2018 : Morning Welcome - - Isabelle Guyon and Evelyne Viegas »
Evelyne Viegas -
2018 : Datasets and Benchmarks for Causal Learning »
Csaba Szepesvari · Isabelle Guyon · Nicolai Meinshausen · David Blei · Elias Bareinboim · Bernhard Schölkopf · Pietro Perona -
2018 : AutoML3 - LifeLong ML with concept drift Challenge: Overview and award ceremony »
Hugo Jair Escalante · Isabelle Guyon · Daniel Silver · Evelyne Viegas · Wei-Wei Tu -
2018 : Evaluating Causation Coefficients »
Isabelle Guyon -
2017 : Ben Hamner, Kaggle platform »
Ben Hamner -
2017 : Baázs Kégl, RAMP platform »
Balázs Kégl -
2017 Workshop: Machine Learning Challenges as a Research Tool »
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy -
2017 : Introduction - Isabelle Guyon and Evelyne Viegas »
Isabelle Guyon -
2016 : Out-of-class novelty generation: an experimental foundation »
Balázs Kégl -
2016 Workshop: Machine Learning for Spatiotemporal Forecasting »
Florin Popescu · Sergio Escalera · Xavier Baró · Stephane Ayache · Isabelle Guyon -
2016 : Gaming challenges and encouraging collaborations »
Sergio Escalera · Isabelle Guyon -
2016 : Challenges in education »
Balázs Kégl · Ben Hamner -
2016 : Welcome »
Evelyne Viegas -
2016 Demonstration: Biometric applications of CNNs: get a job at "Impending Technologies"! »
Sergio Escalera · Isabelle Guyon · Baiyu Chen · Marc Quintana · Umut Güçlü · Yağmur Güçlütürk · Xavier Baró · Rob van Lier · Carlos Andujar · Marcel A. J. van Gerven · Bernhard E Boser · Luke Wang -
2016 Demonstration: Project Malmo - Minecraft for AI Research »
Katja Hofmann · Matthew A Johnson · Fernando Diaz · Alekh Agarwal · Tim Hutton · David Bignell · Evelyne Viegas -
2015 Workshop: Challenges in Machine Learning (CiML 2015): "Open Innovation" and "Coopetitions" »
Isabelle Guyon · Evelyne Viegas · Ben Hamner · Balázs Kégl -
2015 : The HiggsML Story »
Balázs Kégl -
2015 Demonstration: CodaLab Worksheets for Reproducible, Executable Papers »
Percy Liang · Evelyne Viegas -
2014 Workshop: High-energy particle physics, machine learning, and the HiggsML data challenge (HEPML) »
Glen Cowan · Balázs Kégl · Kyle Cranmer · Gábor Melis · Tim Salimans · Vladimir Vava Gligorov · Daniel Whiteson · Lester Mackey · Wojciech Kotlowski · Roberto Díaz Morales · Pierre Baldi · Cecile Germain · David Rousseau · Isabelle Guyon · Tianqi Chen -
2014 Workshop: Challenges in Machine Learning workshop (CiML 2014) »
Isabelle Guyon · Evelyne Viegas · Percy Liang · Olga Russakovsky · Rinat Sergeev · Gábor Melis · Michele Sebag · Gustavo Stolovitzky · Jaume Bacardit · Michael S Kim · Ben Hamner -
2013 Workshop: NIPS 2013 Workshop on Causality: Large-scale Experiment Design and Inference of Causal Mechanisms »
Isabelle Guyon · Leon Bottou · Bernhard Schölkopf · Alexander Statnikov · Evelyne Viegas · james m robins -
2012 Demonstration: Gesture recognition with Kinect »
Isabelle Guyon -
2011 Poster: Algorithms for Hyper-Parameter Optimization »
James Bergstra · Rémi Bardenet · Yoshua Bengio · Balázs Kégl -
2009 Workshop: Clustering: Science or art? Towards principled approaches »
Margareta Ackerman · Shai Ben-David · Avrim Blum · Isabelle Guyon · Ulrike von Luxburg · Robert Williamson · Reza Zadeh -
2009 Mini Symposium: Causality and Time Series Analysis »
Florin Popescu · Isabelle Guyon · Guido Nolte -
2009 Demonstration: Causality Workbench »
Isabelle Guyon -
2008 Workshop: Causality: objectives and assessment »
Isabelle Guyon · Dominik Janzing · Bernhard Schölkopf -
2007 Poster: Learning the 2-D Topology of Images »
Nicolas Le Roux · Yoshua Bengio · Pascal Lamblin · Marc Joliveau · Balázs Kégl -
2007 Demonstration: CLOP: a Matlab Learning Object Package »
Amir Reza Saffari Azar Alamdari · Isabelle Guyon · Hugo Jair Escalante · Gökhan H Bakir · Gavin Cawley -
2006 Workshop: Multi-level Inference Workshop and Model Selection Game »
Isabelle Guyon