Timezone: »
Video summarization is a challenging problem with great application potential. Whereas prior approaches, largely unsupervised in nature, focus on sampling useful frames and assembling them as summaries, we consider video summarization as a supervised subset selection problem. Our idea is to teach the system to learn from human-created summaries how to select informative and diverse subsets, so as to best meet evaluation metrics derived from human-perceived quality. To this end, we propose the sequential determinantal point process (seqDPP), a probabilistic model for diverse sequential subset selection. Our novel seqDPP heeds the inherent sequential structures in video data, thus overcoming the deficiency of the standard DPP, which treats video frames as randomly permutable items. Meanwhile, seqDPP retains the power of modeling diverse subsets, essential for summarization. Our extensive results of summarizing videos from 3 datasets demonstrate the superior performance of our method, compared to not only existing unsupervised methods but also naive applications of the standard DPP model.
Author Information
Boqing Gong (University of Southern California (USC))
Wei-Lun Chao (Ohio State University (OSU))
Kristen Grauman (University of Texas at Austin)
Fei Sha (University of Southern California (USC))
More from the Same Authors
-
2021 Spotlight: Shaping embodied agent behavior with activity-context priors from egocentric video »
Tushar Nagarajan · Kristen Grauman -
2022 : Understanding Federated Learning through Loss Landscape Visualizations: A Pilot Study »
Ziwei Li · Hong-You Chen · Han Wei Shen · Wei-Lun Chao -
2022 Poster: Unsupervised Adaptation from Repeated Traversals for Autonomous Driving »
Yurong You · Cheng Perng Phoo · Katie Luo · Travis Zhang · Wei-Lun Chao · Bharath Hariharan · Mark Campbell · Kilian Weinberger -
2022 Poster: SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning »
Changan Chen · Carl Schissler · Sanchit Garg · Philip Kobernik · Alexander Clegg · Paul Calamia · Dhruv Batra · Philip Robinson · Kristen Grauman -
2022 Poster: Few-Shot Audio-Visual Learning of Environment Acoustics »
Sagnik Majumder · Changan Chen · Ziad Al-Halah · Kristen Grauman -
2021 Poster: Shaping embodied agent behavior with activity-context priors from egocentric video »
Tushar Nagarajan · Kristen Grauman -
2021 Poster: Gradual Domain Adaptation without Indexed Intermediate Domains »
Hong-You Chen · Wei-Lun Chao -
2021 Poster: On Model Calibration for Long-Tailed Object Detection and Instance Segmentation »
Tai-Yu Pan · Cheng Zhang · Yandong Li · Hexiang Hu · Dong Xuan · Soravit Changpinyo · Boqing Gong · Wei-Lun Chao -
2020 : Panel Discussion & Closing »
Yejin Choi · Alexei Efros · Chelsea Finn · Kristen Grauman · Quoc V Le · Yann LeCun · Ruslan Salakhutdinov · Eric Xing -
2020 : Q & A and Panel Session with Dan Weld, Kristen Grauman, Scott Yih, Emma Brunskill, and Alex Ratner »
Kristen Grauman · Wen-tau Yih · Alexander Ratner · Emma Brunskill · Douwe Kiela · Daniel S. Weld -
2020 : QA: Kristen Grauman »
Kristen Grauman -
2020 : Invited Talk: Kristen Grauman »
Kristen Grauman -
2020 Poster: Learning Affordance Landscapes for Interaction Exploration in 3D Environments »
Tushar Nagarajan · Kristen Grauman -
2020 Spotlight: Learning Affordance Landscapes for Interaction Exploration in 3D Environments »
Tushar Nagarajan · Kristen Grauman -
2020 Poster: Wasserstein Distances for Stereo Disparity Estimation »
Divyansh Garg · Yan Wang · Bharath Hariharan · Mark Campbell · Kilian Weinberger · Wei-Lun Chao -
2020 Spotlight: Wasserstein Distances for Stereo Disparity Estimation »
Divyansh Garg · Yan Wang · Bharath Hariharan · Mark Campbell · Kilian Weinberger · Wei-Lun Chao -
2020 Session: Orals & Spotlights Track 01: Representation/Relational »
Laurens van der Maaten · Fei Sha -
2019 Poster: A New Defense Against Adversarial Images: Turning a Weakness into a Strength »
Shengyuan Hu · Tao Yu · Chuan Guo · Wei-Lun Chao · Kilian Weinberger -
2018 Poster: Synthesize Policies for Transfer and Adaptation across Tasks and Environments »
Hexiang Hu · Liyu Chen · Boqing Gong · Fei Sha -
2018 Spotlight: Synthesize Policies for Transfer and Adaptation across Tasks and Environments »
Hexiang Hu · Liyu Chen · Boqing Gong · Fei Sha -
2017 Workshop: Optimal Transport and Machine Learning »
Olivier Bousquet · Marco Cuturi · Gabriel Peyré · Fei Sha · Justin Solomon -
2017 Poster: Learning Spherical Convolution for Fast Features from 360° Imagery »
Yu-Chuan Su · Kristen Grauman -
2017 Poster: An Empirical Study on The Properties of Random Bases for Kernel Methods »
Maximilian Alber · Pieter-Jan Kindermans · Kristof Schütt · Klaus-Robert Müller · Fei Sha -
2016 Poster: Improved Dropout for Shallow and Deep Learning »
Zhe Li · Boqing Gong · Tianbao Yang -
2015 : Do Shallow Kernel Methods Match Deep Neural Networks »
Fei Sha -
2015 : Do Shallow Kernel Methods Match Deep Neural Networks? »
Fei Sha -
2014 Poster: Predicting Useful Neighborhoods for Lazy Local Learning »
Aron Yu · Kristen Grauman -
2014 Poster: Zero-shot recognition with unreliable attributes »
Dinesh Jayaraman · Kristen Grauman -
2013 Workshop: New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks »
Urun Dogan · Marius Kloft · Tatiana Tommasi · Francesco Orabona · Massimiliano Pontil · Sinno Jialin Pan · Shai Ben-David · Arthur Gretton · Fei Sha · Marco Signoretto · Rajhans Samdani · Yun-Qian Miao · Mohammad Gheshlaghi azar · Ruth Urner · Christoph Lampert · Jonathan How -
2013 Poster: Reshaping Visual Datasets for Domain Adaptation »
Boqing Gong · Kristen Grauman · Fei Sha -
2013 Poster: Similarity Component Analysis »
Soravit Changpinyo · Kuan Liu · Fei Sha -
2012 Poster: Non-linear Metric Learning »
Dor Kedem · Stephen Tyree · Kilian Q Weinberger · Fei Sha · Gert Lanckriet -
2012 Session: Oral Session 5 »
Fei Sha -
2012 Poster: Semantic Kernel Forests from Multiple Taxonomies »
Sung Ju Hwang · Kristen Grauman · Fei Sha -
2011 Poster: Learning a Tree of Metrics with Disjoint Visual Features »
Sung Ju Hwang · Kristen Grauman · Fei Sha -
2010 Workshop: Challenges of Data Visualization »
Barbara Hammer · Laurens van der Maaten · Fei Sha · Alexander Smola -
2010 Poster: Hashing Hyperplane Queries to Near Points with Applications to Large-Scale Active Learning »
Prateek Jain · Sudheendra Vijayanarasimhan · Kristen Grauman -
2010 Poster: Unsupervised Kernel Dimension Reduction »
Meihong Wang · Fei Sha · Michael Jordan -
2009 Workshop: Statistical Machine Learning for Visual Analytics »
Guy Lebanon · Fei Sha -
2008 Oral: Multi-Level Active Prediction of Useful Image Annotations for Recognition »
Sudheendra N Vijayanarasimhan · Kristen Grauman -
2008 Poster: Multi-Level Active Prediction of Useful Image Annotations for Recognition »
Sudheendra N Vijayanarasimhan · Kristen Grauman -
2008 Poster: Online Metric Learning and Fast Similarity Search »
Prateek Jain · Brian Kulis · Inderjit Dhillon · Kristen Grauman -
2008 Poster: DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification »
Simon Lacoste-Julien · Fei Sha · Michael Jordan -
2008 Oral: Online Metric Learning and Fast Similarity Search »
Prateek Jain · Brian Kulis · Inderjit Dhillon · Kristen Grauman -
2008 Session: Oral session 1: Clustering »
Fei Sha -
2007 Workshop: Machine Learning for Systems Problems (Part 2) »
Archana Ganapathi · Sumit Basu · Fei Sha · Emre Kiciman -
2007 Workshop: Machine Learning for Systems Problems (Part 1) »
Archana Ganapathi · Sumit Basu · Fei Sha · Emre Kiciman -
2007 Session: Session 7: Systems and Applications »
Fei Sha -
2006 Poster: Large Margin Gaussian Mixture Models for Automatic Speech Recognition »
Fei Sha · Lawrence Saul -
2006 Poster: Approximate Correspondences in High Dimensions »
Kristen Grauman · Trevor Darrell -
2006 Spotlight: Approximate Correspondences in High Dimensions »
Kristen Grauman · Trevor Darrell -
2006 Talk: Large Margin Gaussian Mixture Models for Automatic Speech Recognition »
Fei Sha · Lawrence Saul -
2006 Poster: Graph Regularization for Maximum Variance Unfolding with an Application to Sensor Localization »
Kilian Q Weinberger · Fei Sha · Qihui Zhu · Lawrence Saul