Timezone: »
Progress toward the United Nations Sustainable Development Goals (SDGs) has been hindered by a lack of data on key environmental and socioeconomic indicators, which historically have come from ground surveys with sparse temporal and spatial coverage. Recent advances in machine learning have made it possible to utilize abundant, frequently-updated, and globally available data, such as from satellites or social media, to provide insights into progress toward SDGs. Despite promising early results, approaches to using such data for SDG measurement thus far have largely evaluated on different datasets or used inconsistent evaluation metrics, making it hard to understand whether performance is improving and where additional research would be most fruitful. Furthermore, processing satellite and ground survey data requires domain knowledge that many in the machine learning community lack. In this paper, we introduce SustainBench, a collection of 15 benchmark tasks across 7 SDGs, including tasks related to economic development, agriculture, health, education, water and sanitation, climate action, and life on land. Datasets for 11 of the 15 tasks are released publicly for the first time. Our goals for SustainBench are to (1) lower the barriers to entry for the machine learning community to contribute to measuring and achieving the SDGs; (2) provide standard benchmarks for evaluating machine learning models on tasks across a variety of SDGs; and (3) encourage the development of novel machine learning methods where improved model performance facilitates progress towards the SDGs.
Author Information
Christopher Yeh (Caltech)
Christopher Yeh is a PhD student at Caltech studying computing and mathematical sciences, with a specialization in machine learning. His research focuses on computational sustainability and active learning.
Chenlin Meng (Stanford University)
Sherrie Wang (Stanford University)
Anne Driscoll (Stanford University)
Erik Rozi (Stanford University)
Patrick Liu (Stanford University)
Jihyeon Lee (Google Research, Stanford University)
Jihyeon is an incoming software engineer at Google Research. She graduated with a BS and MS in Computer Science from Stanford University, where she was a researcher in the Sustain Lab advised by Professor Stefano Ermon and in the Vision & Learning Lab advised by Professor Fei-Fei Li. She is interested in semi-supervised learning and AI methods to solve problems at the intersection of environment, policy, and people.
Marshall Burke (Stanford University)
David Lobell (Stanford University)
Stefano Ermon (Stanford)
More from the Same Authors
-
2020 : Paper 46: Disagreement-Regularized Imitation of Complex Multi-Agent Interactions »
Jiaming Song · Stefano Ermon -
2021 Spotlight: IQ-Learn: Inverse soft-Q Learning for Imitation »
Divyansh Garg · Shuvam Chakraborty · Chris Cundy · Jiaming Song · Stefano Ermon -
2021 Spotlight: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2021 : Scalable Variational Approaches for Bayesian Causal Discovery »
Chris Cundy · Aditya Grover · Stefano Ermon -
2021 : Likelihood-free Density Ratio Acquisition Functions are not Equivalent to Expected Improvements »
Jiaming Song · Stefano Ermon -
2022 : LMPriors: Pre-Trained Language Models as Task-Specific Priors »
Kristy Choi · Chris Cundy · Sanjari Srivastava · Stefano Ermon -
2022 : Pushing the Accuracy-Fairness Tradeoff Frontier with Introspective Self-play »
Jeremiah Liu · Krishnamurthy Dvijotham · Jihyeon Lee · Quan Yuan · Martin Strobel · Balaji Lakshminarayanan · Deepak Ramachandran -
2022 : Relaxing the Kolmogorov Structure Function for Realistic Computational Constraints »
Yoonho Lee · Chelsea Finn · Stefano Ermon -
2022 : Regularizing Score-based Models with Score Fokker-Planck Equations »
Chieh-Hsin Lai · Yuhta Takida · Naoki Murata · Toshimitsu Uesaka · Yuki Mitsufuji · Stefano Ermon -
2022 : On Distillation of Guided Diffusion Models »
Chenlin Meng · Ruiqi Gao · Diederik Kingma · Stefano Ermon · Jonathan Ho · Tim Salimans -
2022 : JPEG Artifact Correction using Denoising Diffusion Restoration Models »
Bahjat Kawar · Jiaming Song · Stefano Ermon · Michael Elad -
2022 : But Are You Sure? Quantifying Uncertainty in Model Explanations »
Charles Marx · Youngsuk Park · Hilaf Hasson · Yuyang (Bernie) Wang · Stefano Ermon · Chaitanya Baru -
2023 Workshop: Adaptive Experimental Design and Active Learning in the Real World »
Willie Neiswanger · Mojmir Mutny · Ilija Bogunovic · Ava Soleimany · Zi Wang · Stefano Ermon · Andreas Krause -
2023 Workshop: Generative AI and Biology (GenBio@NeurIPS2023) »
Minkai Xu · Regina Barzilay · Jure Leskovec · Wenxian Shi · Menghua Wu · Zhenqiao Song · Lei Li · Fan Yang · Stefano Ermon -
2022 : Invited Talk: Chenlin Meng »
Chenlin Meng -
2022 Poster: Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models »
Muyang Li · Ji Lin · Chenlin Meng · Stefano Ermon · Song Han · Jun-Yan Zhu -
2022 Poster: Concrete Score Matching: Generalized Score Matching for Discrete Data »
Chenlin Meng · Kristy Choi · Jiaming Song · Stefano Ermon -
2022 Poster: LISA: Learning Interpretable Skill Abstractions from Language »
Divyansh Garg · Skanda Vaidyanath · Kuno Kim · Jiaming Song · Stefano Ermon -
2022 Poster: Training and Inference on Any-Order Autoregressive Models the Right Way »
Andy Shih · Dorsa Sadigh · Stefano Ermon -
2022 Poster: SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery »
Yezhen Cong · Samar Khanna · Chenlin Meng · Patrick Liu · Erik Rozi · Yutong He · Marshall Burke · David Lobell · Stefano Ermon -
2022 Poster: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness »
Tri Dao · Dan Fu · Stefano Ermon · Atri Rudra · Christopher Ré -
2022 Poster: Denoising Diffusion Restoration Models »
Bahjat Kawar · Michael Elad · Stefano Ermon · Jiaming Song -
2022 Poster: Generalizing Bayesian Optimization with Decision-theoretic Entropies »
Willie Neiswanger · Lantao Yu · Shengjia Zhao · Chenlin Meng · Stefano Ermon -
2022 Poster: Transform Once: Efficient Operator Learning in Frequency Domain »
Michael Poli · Stefano Massaroli · Federico Berto · Jinkyoo Park · Tri Dao · Christopher Ré · Stefano Ermon -
2022 Poster: Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations »
Michael Poli · Winnie Xu · Stefano Massaroli · Chenlin Meng · Kuno Kim · Stefano Ermon -
2022 Poster: Improving Self-Supervised Learning by Characterizing Idealized Representations »
Yann Dubois · Stefano Ermon · Tatsunori Hashimoto · Percy Liang -
2022 Poster: Exploration via Planning for Information about the Optimal Trajectory »
Viraj Mehta · Ian Char · Joseph Abbate · Rory Conlin · Mark Boyer · Stefano Ermon · Jeff Schneider · Willie Neiswanger -
2021 : TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch »
Michael Poli · Stefano Massaroli · Atsushi Yamashita · Hajime Asama · Jinkyoo Park · Stefano Ermon -
2021 : Cundy, Grover, Ermon - BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery »
Chris Cundy · Aditya Grover · Stefano Ermon -
2021 Poster: HyperSPNs: Compact and Expressive Probabilistic Circuits »
Andy Shih · Dorsa Sadigh · Stefano Ermon -
2021 Poster: Imitation with Neural Density Models »
Kuno Kim · Akshat Jindal · Yang Song · Jiaming Song · Yanan Sui · Stefano Ermon -
2021 Poster: Reliable Decisions with Threshold Calibration »
Roshni Sahoo · Shengjia Zhao · Alyssa Chen · Stefano Ermon -
2021 Poster: D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation »
Abhishek Sinha · Jiaming Song · Chenlin Meng · Stefano Ermon -
2021 Poster: Improving Compositionality of Neural Networks by Decoding Representations to Inputs »
Mike Wu · Noah Goodman · Stefano Ermon -
2021 Poster: Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis »
Yutong He · Dingjie Wang · Nicholas Lai · William Zhang · Chenlin Meng · Marshall Burke · David Lobell · Stefano Ermon -
2021 Poster: Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration »
Shengjia Zhao · Michael Kim · Roshni Sahoo · Tengyu Ma · Stefano Ermon -
2021 Poster: Estimating High Order Gradients of the Data Distribution by Denoising »
Chenlin Meng · Yang Song · Wenzhe Li · Stefano Ermon -
2021 Poster: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2021 Poster: Pseudo-Spherical Contrastive Divergence »
Lantao Yu · Jiaming Song · Yang Song · Stefano Ermon -
2021 Poster: IQ-Learn: Inverse soft-Q Learning for Imitation »
Divyansh Garg · Shuvam Chakraborty · Chris Cundy · Jiaming Song · Stefano Ermon -
2021 Poster: CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation »
Yusuke Tashiro · Jiaming Song · Yang Song · Stefano Ermon -
2021 Poster: PiRank: Scalable Learning To Rank via Differentiable Sorting »
Robin Swezey · Aditya Grover · Bruno Charron · Stefano Ermon -
2021 Poster: BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery »
Chris Cundy · Aditya Grover · Stefano Ermon -
2020 : Spotlight Talk: Lee »
Jihyeon Lee -
2020 : Stefano Emron - Generative Modeling via Denoising »
Stefano Ermon -
2020 Poster: Improved Techniques for Training Score-Based Generative Models »
Yang Song · Stefano Ermon -
2020 Poster: Probabilistic Circuits for Variational Inference in Discrete Graphical Models »
Andy Shih · Stefano Ermon -
2020 Poster: Efficient Learning of Generative Models via Finite-Difference Score Matching »
Tianyu Pang · Kun Xu · Chongxuan LI · Yang Song · Stefano Ermon · Jun Zhu -
2020 Poster: Belief Propagation Neural Networks »
Jonathan Kuck · Shuvam Chakraborty · Hao Tang · Rachel Luo · Jiaming Song · Ashish Sabharwal · Stefano Ermon -
2020 Poster: HiPPO: Recurrent Memory with Optimal Polynomial Projections »
Albert Gu · Tri Dao · Stefano Ermon · Atri Rudra · Christopher Ré -
2020 Spotlight: HiPPO: Recurrent Memory with Optimal Polynomial Projections »
Albert Gu · Tri Dao · Stefano Ermon · Atri Rudra · Christopher Ré -
2020 Poster: Autoregressive Score Matching »
Chenlin Meng · Lantao Yu · Yang Song · Jiaming Song · Stefano Ermon -
2020 Poster: Diversity can be Transferred: Output Diversification for White- and Black-box Attacks »
Yusuke Tashiro · Yang Song · Stefano Ermon -
2020 Poster: MOPO: Model-based Offline Policy Optimization »
Tianhe Yu · Garrett Thomas · Lantao Yu · Stefano Ermon · James Zou · Sergey Levine · Chelsea Finn · Tengyu Ma -
2020 Poster: Multi-label Contrastive Predictive Coding »
Jiaming Song · Stefano Ermon -
2020 Oral: Multi-label Contrastive Predictive Coding »
Jiaming Song · Stefano Ermon -
2019 : Poster Session »
Ethan Harris · Tom White · Oh Hyeon Choung · Takashi Shinozaki · Dipan Pal · Katherine L. Hermann · Judy Borowski · Camilo Fosco · Chaz Firestone · Vijay Veerabadran · Benjamin Lahner · Chaitanya Ryali · Fenil Doshi · Pulkit Singh · Sharon Zhou · Michel Besserve · Michael Chang · Anelise Newman · Mahesan Niranjan · Jonathon Hare · Daniela Mihai · Marios Savvides · Simon Kornblith · Christina M Funke · Aude Oliva · Virginia de Sa · Dmitry Krotov · Colin Conwell · George Alvarez · Alex Kolchinski · Shengjia Zhao · Mitchell Gordon · Michael Bernstein · Stefano Ermon · Arash Mehrjou · Bernhard Schölkopf · John Co-Reyes · Michael Janner · Jiajun Wu · Josh Tenenbaum · Sergey Levine · Yalda Mohsenzadeh · Zhenglong Zhou -
2019 Workshop: Information Theory and Machine Learning »
Shengjia Zhao · Jiaming Song · Yanjun Han · Kristy Choi · Pratyusha Kalluri · Ben Poole · Alex Dimakis · Jiantao Jiao · Tsachy Weissman · Stefano Ermon -
2019 Poster: Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations. »
Sawyer Birnbaum · Volodymyr Kuleshov · Zayd Enam · Pang Wei Koh · Stefano Ermon -
2019 Poster: MintNet: Building Invertible Neural Networks with Masked Convolutions »
Yang Song · Chenlin Meng · Stefano Ermon -
2019 Poster: Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting »
Aditya Grover · Jiaming Song · Ashish Kapoor · Kenneth Tran · Alekh Agarwal · Eric Horvitz · Stefano Ermon -
2019 Poster: Meta-Inverse Reinforcement Learning with Probabilistic Context Variables »
Lantao Yu · Tianhe Yu · Chelsea Finn · Stefano Ermon -
2019 Poster: Approximating the Permanent by Sampling from Adaptive Partitions »
Jonathan Kuck · Tri Dao · Hamid Rezatofighi · Ashish Sabharwal · Stefano Ermon -
2019 Poster: Generative Modeling by Estimating Gradients of the Data Distribution »
Yang Song · Stefano Ermon -
2019 Oral: Generative Modeling by Estimating Gradients of the Data Distribution »
Yang Song · Stefano Ermon -
2018 Workshop: Relational Representation Learning »
Aditya Grover · Paroma Varma · Frederic Sala · Christopher Ré · Jennifer Neville · Stefano Ermon · Steven Holtzen -
2018 : Stefano Ermon (Stanford University): Weakly Supervised Spatio-temporal Regression »
Stefano Ermon -
2018 Poster: Streamlining Variational Inference for Constraint Satisfaction Problems »
Aditya Grover · Tudor Achim · Stefano Ermon -
2018 Poster: Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance »
Neal Jean · Sang Michael Xie · Stefano Ermon -
2018 Poster: Multi-Agent Generative Adversarial Imitation Learning »
Jiaming Song · Hongyu Ren · Dorsa Sadigh · Stefano Ermon -
2018 Poster: Constructing Unrestricted Adversarial Examples with Generative Models »
Yang Song · Rui Shu · Nate Kushman · Stefano Ermon -
2018 Poster: Bias and Generalization in Deep Generative Models: An Empirical Study »
Shengjia Zhao · Hongyu Ren · Arianna Yuan · Jiaming Song · Noah Goodman · Stefano Ermon -
2018 Spotlight: Bias and Generalization in Deep Generative Models: An Empirical Study »
Shengjia Zhao · Hongyu Ren · Arianna Yuan · Jiaming Song · Noah Goodman · Stefano Ermon -
2018 Poster: Amortized Inference Regularization »
Rui Shu · Hung Bui · Shengjia Zhao · Mykel J Kochenderfer · Stefano Ermon -
2017 : Generative Adversarial Imitation Learning, Stefano Ermon, Stanford »
Stefano Ermon -
2017 : Stefano Ermon (Stanford): Measuring Progress Towards Sustainable Development Goals with Machine Learning »
Stefano Ermon -
2017 Poster: A-NICE-MC: Adversarial Training for MCMC »
Jiaming Song · Shengjia Zhao · Stefano Ermon -
2017 Poster: InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations »
Yunzhu Li · Jiaming Song · Stefano Ermon -
2017 Poster: Neural Variational Inference and Learning in Undirected Graphical Models »
Volodymyr Kuleshov · Stefano Ermon -
2016 Poster: Solving Marginal MAP Problems with NP Oracles and Parity Constraints »
Yexiang Xue · zhiyuan li · Stefano Ermon · Carla Gomes · Bart Selman -
2016 Poster: Generative Adversarial Imitation Learning »
Jonathan Ho · Stefano Ermon -
2016 Poster: Variational Bayes on Monte Carlo Steroids »
Aditya Grover · Stefano Ermon -
2016 Poster: Adaptive Concentration Inequalities for Sequential Decision Problems »
Shengjia Zhao · Enze Zhou · Ashish Sabharwal · Stefano Ermon -
2013 Poster: Embed and Project: Discrete Sampling with Universal Hashing »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2012 Poster: Density Propagation and Improved Bounds on the Partition Function »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2011 Poster: Accelerated Adaptive Markov Chain for Partition Function Computation »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2011 Spotlight: Accelerated Adaptive Markov Chain for Partition Function Computation »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman