Challenges in Machine Learning have proven to be efficient and cost-effective ways to quickly bring to industry solutions that may have been confined to research. In addition, the playful nature of challenges naturally attracts students, making challenge a great teaching resource. Challenge participants range from undergraduate students to retirees, joining forces in a rewarding environment allowing them to learn, perform research, and demonstrate excellence. Therefore challenges can be used as a means of directing research, advancing the state-of-the-art or venturing in completely new domains.

Yet, despite initial successes and efforts made to facilitate challenge organization with the availability of competition platforms, little effort has been put into the theoretical foundations of challenge design and the optimization of challenge protocols. This workshop will bring together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. The themes to be discussed will include new paradigms of challenge organization to tackle complex problems (e.g. tasks involving multiple data modalities and/or multiple levels of processing).