Timezone: »
Spotlight
Earthformer: Exploring Space-Time Transformers for Earth System Forecasting
Zhihan Gao · Xingjian Shi · Hao Wang · Yi Zhu · Yuyang (Bernie) Wang · Mu Li · Dit-Yan Yeung
Conventionally, Earth system (e.g., weather and climate) forecasting relies on numerical simulation with complex physical models and hence is both expensive in computation and demanding on domain expertise. With the explosive growth of spatiotemporal Earth observation data in the past decade, data-driven models that apply Deep Learning (DL) are demonstrating impressive potential for various Earth system forecasting tasks. The Transformer as an emerging DL architecture, despite its broad success in other domains, has limited adoption in this area. In this paper, we propose Earthformer, a space-time Transformer for Earth system forecasting. Earthformer is based on a generic, flexible and efficient space-time attention block, named Cuboid Attention. The idea is to decompose the data into cuboids and apply cuboid-level self-attention in parallel. These cuboids are further connected with a collection of global vectors. We conduct experiments on the MovingMNIST dataset and a newly proposed chaotic $N$-body MNIST dataset to verify the effectiveness of cuboid attention and figure out the best design of Earthformer. Experiments on two real-world benchmarks about precipitation nowcasting and El Niño/Southern Oscillation (ENSO) forecasting show that Earthformer achieves state-of-the-art performance.
Author Information
Zhihan Gao (HKUST)
Xingjian Shi (HKUST)
Hao Wang (Rutgers University)
Yi Zhu (Amazon)
Yuyang (Bernie) Wang (AWS AI Labs)
Mu Li (Amazon)
Dit-Yan Yeung (Hong Kong University of Science and Technology)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: Earthformer: Exploring Space-Time Transformers for Earth System Forecasting »
Tue. Nov 29th through Wed the 30th Room Hall J #218
More from the Same Authors
-
2021 : Benchmarking Multimodal AutoML for Tabular Data with Text Fields »
Xingjian Shi · Jonas Mueller · Nick Erickson · Mu Li · Alexander Smola -
2022 : First De-Trend then Attend: Rethinking Attention for Time-Series Forecasting »
Xiyuan Zhang · Xiaoyong Jin · Karthick Gopalswamy · Gaurav Gupta · Youngsuk Park · Xingjian Shi · Hao Wang · Danielle Maddix · Yuyang (Bernie) Wang -
2022 : Towards Reverse Causal Inference on Panel Data: Precise Formulation and Challenges »
Jiayao Zhang · Youngsuk Park · Danielle Maddix · Dan Roth · Yuyang (Bernie) Wang -
2022 : Benchmarking Robustness under Distribution Shift of Multimodal Image-Text Models »
Jielin Qiu · Yi Zhu · Xingjian Shi · Zhiqiang Tang · DING ZHAO · Bo Li · Mu Li -
2022 : But Are You Sure? Quantifying Uncertainty in Model Explanations »
Charles Marx · Youngsuk Park · Hilaf Hasson · Yuyang (Bernie) Wang · Stefano Ermon · Chaitanya Baru -
2023 Poster: Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting »
Marcel Kollovieh · Abdul Fatir Ansari · Michael Bohlke-Schneider · Jasper Zschiegner · Hao Wang · Yuyang (Bernie) Wang -
2023 Poster: Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition »
Shuhuai Ren · Aston Zhang · Yi Zhu · Shuai Zhang · Shuai Zheng · Mu Li · Alexander Smola · Xu Sun -
2023 Poster: Adaptive Online Replanning with Diffusion Models »
Siyuan Zhou · Yilun Du · Shun Zhang · Mengdi Xu · Yikang Shen · Wei Xiao · Dit-Yan Yeung · Chuang Gan -
2023 Poster: Variational Imbalanced Regression »
Ziyan Wang · Hao Wang -
2023 Poster: PreDiff: Precipitation Nowcasting with Latent Diffusion Models »
Zhihan Gao · Xingjian Shi · Boran Han · Hao Wang · Xiaoyong Jin · Danielle Maddix · Yi Zhu · Yuyang (Bernie) Wang · Mu Li · Dit-Yan Yeung -
2023 Poster: A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm »
Haizhou Shi · Hao Wang -
2022 Spotlight: Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator »
Zifan Shi · Yinghao Xu · Yujun Shen · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Spotlight: Lightning Talks 5B-1 »
Devansh Arpit · Xiaojun Xu · Zifan Shi · Ivan Skorokhodov · Shayan Shekarforoush · Zhan Tong · Yiqun Wang · Shichong Peng · Linyi Li · Ivan Skorokhodov · Huan Wang · Yibing Song · David Lindell · Yinghao Xu · Seyed Alireza Moazenipourasil · Sergey Tulyakov · Peter Wonka · Yiqun Wang · Ke Li · David Fleet · Yujun Shen · Yingbo Zhou · Bo Li · Jue Wang · Peter Wonka · Marcus Brubaker · Caiming Xiong · Limin Wang · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Spotlight: Lightning Talks 4A-3 »
Zhihan Gao · Yabin Wang · Xingyu Qu · Luziwei Leng · Mingqing Xiao · Bohan Wang · Yu Shen · Zhiwu Huang · Xingjian Shi · Qi Meng · Yupeng Lu · Diyang Li · Qingyan Meng · Kaiwei Che · Yang Li · Hao Wang · Huishuai Zhang · Zongpeng Zhang · Kaixuan Zhang · Xiaopeng Hong · Xiaohan Zhao · Di He · Jianguo Zhang · Yaofeng Tu · Bin Gu · Yi Zhu · Ruoyu Sun · Yuyang (Bernie) Wang · Zhouchen Lin · Qinghu Meng · Wei Chen · Wentao Zhang · Bin CUI · Jie Cheng · Zhi-Ming Ma · Mu Li · Qinghai Guo · Dit-Yan Yeung · Tie-Yan Liu · Jianxing Liao -
2022 Workshop: A causal view on dynamical systems »
Sören Becker · Alexis Bellot · Cecilia Casolo · Niki Kilbertus · Sara Magliacane · Yuyang (Bernie) Wang -
2022 Poster: Extrapolative Continuous-time Bayesian Neural Network for Fast Training-free Test-time Adaptation »
Hengguan Huang · Xiangming Gu · Hao Wang · Chang Xiao · Hongfu Liu · Ye Wang -
2022 Poster: Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator »
Zifan Shi · Yinghao Xu · Yujun Shen · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Poster: On the detrimental effect of invariances in the likelihood for variational inference »
Richard Kurle · Ralf Herbrich · Tim Januschowski · Yuyang (Bernie) Wang · Jan Gasthaus -
2021 Poster: Blending Anti-Aliasing into Vision Transformer »
Shengju Qian · Hao Shao · Yi Zhu · Mu Li · Jiaya Jia -
2021 Poster: Progressive Coordinate Transforms for Monocular 3D Object Detection »
Li Wang · Li Zhang · Yi Zhu · Zhi Zhang · Tong He · Mu Li · Xiangyang Xue -
2020 Poster: CSER: Communication-efficient SGD with Error Reset »
Cong Xie · Shuai Zheng · Sanmi Koyejo · Indranil Gupta · Mu Li · Haibin Lin -
2017 Poster: Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model »
Xingjian Shi · Zhihan Gao · Leonard Lausen · Hao Wang · Dit-Yan Yeung · Wai-kin Wong · Wang-chun WOO -
2017 Spotlight: Deep Learning for Precipitation Nowcasting: A Benchmark and A New Model »
Xingjian Shi · Zhihan Gao · Leonard Lausen · Hao Wang · Dit-Yan Yeung · Wai-kin Wong · Wang-chun WOO -
2016 Poster: Natural-Parameter Networks: A Class of Probabilistic Neural Networks »
Hao Wang · Xingjian SHI · Dit-Yan Yeung -
2016 Poster: Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks »
Hao Wang · Xingjian SHI · Dit-Yan Yeung -
2015 Poster: Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting »
Xingjian Shi · Zhourong Chen · Hao Wang · Dit-Yan Yeung · Wai-kin Wong · Wang-chun WOO -
2013 Poster: Learning a Deep Compact Image Representation for Visual Tracking »
Naiyan Wang · Dit-Yan Yeung -
2012 Poster: Co-Regularized Hashing for Multimodal Data »
Yi Zhen · Dit-Yan Yeung -
2010 Poster: Worst-Case Linear Discriminant Analysis »
Yu Zhang · Dit-Yan Yeung -
2010 Poster: Probabilistic Multi-Task Feature Selection »
Yu Zhang · Dit-Yan Yeung · Qian Xu -
2009 Poster: Probabilistic Relational PCA »
Wu-Jun Li · Dit-Yan Yeung · Zhihua Zhang -
2009 Spotlight: Probabilistic Relational PCA »
Wu-Jun Li · Dit-Yan Yeung · Zhihua Zhang -
2008 Poster: Posterior Consistency of the Silverman g-prior in Bayesian Model Choice »
Zhihua Zhang · Michael Jordan · Dit-Yan Yeung -
2008 Spotlight: Posterior Consistency of the Silverman g-prior in Bayesian Model Choice »
Zhihua Zhang · Michael Jordan · Dit-Yan Yeung