Timezone: »
We present a framework for video modeling based on denoising diffusion probabilistic models that produces long-duration video completions in a variety of realistic environments. We introduce a generative model that can at test-time sample any arbitrary subset of video frames conditioned on any other subset and present an architecture adapted for this purpose. Doing so allows us to efficiently compare and optimize a variety of schedules for the order in which frames in a long video are sampled and use selective sparse and long-range conditioning on previously sampled frames. We demonstrate improved video modeling over prior work on a number of datasets and sample temporally coherent videos over 25 minutes in length. We additionally release a new video modeling dataset and semantically meaningful metrics based on videos generated in the CARLA autonomous driving simulator.
Author Information
William Harvey (University of British Columbia)
Saeid Naderiparizi (University of British Columbia)
Vaden Masrani (University of British Columbia)
Christian Weilbach (University of British Columbia)
Frank Wood (University of British Columbia)
More from the Same Authors
-
2021 : TITRATED: Learned Human Driving Behavior without Infractions via Amortized Inference »
Vasileios Lioutas · Adam Scibior · Frank Wood -
2022 : Physics aware inference for the cryo-EM inverse problem: anisotropic network model heterogeneity, global 3D pose and microscope defocus »
Geoffrey Woollard · Shayan Shekarforoush · Frank Wood · Marcus Brubaker · Khanh Dao Duc -
2022 Poster: BayesPCN: A Continually Learnable Predictive Coding Associative Memory »
Jinsoo Yoo · Frank Wood -
2020 Poster: Gaussian Process Bandit Optimization of the Thermodynamic Variational Objective »
Vu Nguyen · Vaden Masrani · Rob Brekelmans · Michael A Osborne · Frank Wood -
2019 : Opening Remarks »
Atilim Gunes Baydin · Juan Carrasquilla · Shirley Ho · Karthik Kashinath · Michela Paganini · Savannah Thais · Anima Anandkumar · Kyle Cranmer · Roger Melko · Mr. Prabhat · Frank Wood -
2019 Workshop: Machine Learning and the Physical Sciences »
Atilim Gunes Baydin · Juan Carrasquilla · Shirley Ho · Karthik Kashinath · Michela Paganini · Savannah Thais · Anima Anandkumar · Kyle Cranmer · Roger Melko · Mr. Prabhat · Frank Wood -
2019 Poster: The Thermodynamic Variational Objective »
Vaden Masrani · Tuan Anh Le · Frank Wood -
2019 Poster: Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model »
Atilim Gunes Baydin · Lei Shao · Wahid Bhimji · Lukas Heinrich · Saeid Naderiparizi · Andreas Munk · Jialin Liu · Bradley Gram-Hansen · Gilles Louppe · Lawrence Meadows · Philip Torr · Victor Lee · Kyle Cranmer · Mr. Prabhat · Frank Wood -
2018 Poster: Faithful Inversion of Generative Models for Effective Amortized Inference »
Stefan Webb · Adam Golinski · Rob Zinkov · Siddharth N · Thomas Rainforth · Yee Whye Teh · Frank Wood -
2018 Poster: Bayesian Distributed Stochastic Gradient Descent »
Michael Teng · Frank Wood