The last few years have seen the emergence of billion parameter models trained on 'infinite' data that achieve impressive performance on many tasks, suggesting that big data and big models may be all we need. But how far can this approach take us, in particular on domains where data is more limited? In many situations adding structured architectural priors to models may be key to achieving faster learning, better generalisation and learning from less data. Structure can be added at the level of perception and at the level of reasoning - the goal of GOFAI research. In this tutorial we will use the idea of symmetries and symbolic reasoning as an overarching theoretical framework to describe many of the common structural priors that have been successful in the past for building more data efficient and generalisable perceptual models, and models that support better reasoning in neuro-symbolic approaches.

Control systems are mechanisms that enable realization of desirable behaviors from dynamical systems, such as automobiles, robots, and manufacturing processes; although invisible, they are often essential for our daily lives. Control engineering involves the analysis and design of control systems, and optimal control is one of the important problems in control engineering. In an optimal control problem, the control input is determined to minimize a cost function given certain constraints. Even if a mathematical model of the control system is known, it is generally difficult to find its optimal control input owing to heavy computations or data storage, and the development of efficient algorithms for optimal control problems has been an active area of research for several decades. Realization of optimal control for dynamical systems by adaptation or learning is challenging when their mathematical models are unknown; moreover, developing practical optimal control methods for unknown dynamical systems is a challenge both in control engineering and machine learning. Therefore, control systems provide ample motivation and opportunity for machine learning. This tutorial aims to help researchers and engineers in the field of machine learning tackle problems in control systems. An overview of the problems and concepts in control engineering is provided first, …

Low Resourced languages pose an interesting challenge for Machine Learning algorithms, representation, data collection and accessibility of machine learning in general. In this tutorial, we work to provide a journey through machine learning in low resourced languages that covers a breadth of sub topics and depth in some of the areas of focus. We will do this through the lens of Natural Language processing for African languages. We present some historical context, recent advances and current opportunities that researchers can take advantage of to do impactful research in this area. We hope for this tutorial to not only shed light on the subject area, but to expand the number of practitioners who interact in a thoughtful and considerate way with the wider ML community working in these areas. We hope this to be as interactive as possible and to provide resources for researchers to tackle the challenges.

Quantum computing, a discipline that investigates how computing changes if we take into account quantum effects, has turned into an emerging technology that produced the first generation of hardware prototypes. In search of applications for these new devices, researchers turned to machine learning and found a wealth of exciting questions: Do machine learning algorithms gain a better computational complexity if we outsource parts of them to quantum computers? How does the problem of empirical risk minimisation change if our model class is made up of quantum algorithms? How does quantum hardware fit into AI pipelines? And, vice versa, can machine learning help us to study the behaviour of quantum systems?

In this tutorial we want to unpack these questions and sketch the landscape of preliminary answers found so far. For example, we will look at carefully constructed learning problems for which quantum computers have a provable complexity advantage, and motivate why it is so hard to make conclusive statements about more natural problem settings. We will explore how data can be represented as physical states of quantum systems, and how manipulating these systems leads to algorithms that are just kernel methods with a special kind of Hilbert space. We will …

Gaussian processes (GP) are Bayesian nonparametric models for continuous functions which allow for uncertainty quantification, interpretability, and the incorporation of expert knowledge. The theory and practice of GPs have flourished in the last decade, where researchers have looked into the expressiveness and efficiency of GP-based models and practitioners have applied them to a plethora of disciplines. This tutorial presents both the foundational theory and modern developments of data modelling using GPs, following step by step intuitions, illustrations and real-world examples. The tutorial will start with emphasis on the building blocks of the GP model, to then move onto the choice of the kernel function, cost-effective training strategies and non-Gaussian extensions. The second part of the tutorial will showcase more recent advances, such as latent variable models, deep GPs, current trends on kernel design and connections between GPs and deep neural networks. We hope that this exhibition, featuring classic and contemporary GP works, inspires attendees to incorporate GPs into their applications and motivates them to continue learning and contributing to the current developments in the field.

Physics research and deep learning have a symbiotic relationship, and this bond has become stronger over the past several years. In this tutorial, we will present both sides of this story. How has deep learning benefited from concepts in physics and other sciences? How have different subfields of physics research capitalized on deep learning? What are some yet-unexplored applications of deep learning to physics which could strongly benefit from machine learning? We will discuss the past and present of this intersection, and then theorize possible directions for the future of this connection. In the second part of this talk, we will outline some existing deep learning techniques which have exploited ideas from physics, and point out some intriguing new directions in this area.

The machine learning community is seeing an increased focus on fairness-oriented methods of model and dataset development. However, much of this work is constrained by a purely technical understanding of fairness -- an understanding that has come to mean parity of model performance across sociodemographic groups -- that offers a narrow way of understanding how machine learning technologies intersect with systems of oppression that structure their development and use in the real world. In contrast to this approach, we believe it is essential to approach machine learning technologies from a sociotechnical lens, examining how marginalized communities are excluded from their development and impacted by their deployment. Our tutorial will center the perspectives and stories of communities who have been harmed by machine learning technologies and the dominant logics operative within this field. We believe it is important to host these conversations from within the NeurIPS venue so that researchers and practitioners within the machine learning field can engage with these perspectives and understand the lived realities of marginalized communities impacted by the outputs of the field. In doing so, we hope to shift the focus away from singular technical understandings of fairness and towards justice, equity, and accountability. We believe …

The assessment of climate variability and change is enriched by novel applications of statistics and machine learning methodologies. This tutorial will be an introduction to some of the common statistical and machine learning problems that arise in climate science. The goal is to give attendees a sense of the intersections between the fields and to help promote future interdisciplinary collaborations. We will introduce you to different climate data sources (e.g., in situ measurements, satellite data, climate model data, etc.) and discuss problems including: characterizing changes in extreme events like heatwaves or extreme precipitation, summarizing high-dimensional spatiotemporal climate data, and using statistical methods to predict climate variability and potentially improve future projections. The focus will be on methodological applications; we will discuss both core methodologies and recent innovations. Prior knowledge of climate science is not assumed and we will emphasize the value of engaging substantively with domain experts.

Self-supervised learning is a great way to extract training signals from massive amounts of unlabelled data and to learn good representation to facilitate downstream tasks where it is expensive to collect task-specific labels. This tutorial will focus on two major approaches for self-supervised learning, self-prediction and contrastive learning. Self-prediction refers to self-supervised training tasks where the model learns to predict a portion of the available data from the rest. Contrastive learning is to learn a representation space in which similar data samples stay close to each other while dissimilar ones are far apart, by constructing similar and dissimilar pairs from the dataset. This tutorial will cover methods on both topics and across various applications including vision, language, video, multimodal, and reinforcement learning.

Message passing algorithms are distributed algorithms that operate on graphs, where each node uses only information present locally at the node and incident edges, and send information only to its neighbouring nodes. They are often highly effective in machine learning and are relatively easy to parallelise. Examples include approximate inference algorithms on probabilistic graphical models, the value iteration algorithm for Markov decision process, graph neural networks and attention networks.

This tutorial presents commonly used approximate inference algorithms for probabilistic graphical models and the value iteration algorithm for Markov decision process, focusing on understanding the objectives that the algorithms are optimising for. We then consider more flexible but less interpretable message passing algorithms including graph neural networks and attention networks. We discuss how these more flexible networks can simulate the more interpretable algorithms, providing some understanding of the inductive biases of these networks through algorithmic alignment and allowing the understanding to be used for network design.