Timezone: »

Temporal Coherency based Criteria for Predicting Video Frames using Deep Multi-stage Generative Adversarial Networks
Prateep Bhattacharjee · S. Das

Tue Dec 05 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #89 #None

Predicting the future from a sequence of video frames has been recently a sought after yet challenging task in the field of computer vision and machine learning. Although there have been efforts for tracking using motion trajectories and flow features, the complex problem of generating unseen frames has not been studied extensively. In this paper, we deal with this problem using convolutional models within a multi-stage Generative Adversarial Networks (GAN) framework. The proposed method uses two stages of GANs to generate a crisp and clear set of future frames. Although GANs have been used in the past for predicting the future, none of the works consider the relation between subsequent frames in the temporal dimension. Our main contribution lies in formulating two objective functions based on the Normalized Cross Correlation (NCC) and the Pairwise Contrastive Divergence (PCD) for solving this problem. This method, coupled with the traditional L1 loss, has been experimented with three real-world video datasets, viz. Sports-1M, UCF-101 and the KITTI. Performance analysis reveals superior results over the recent state-of-the-art methods.

Author Information

Prateep Bhattacharjee (Indian Institute of Technology Madras)
S. Das (IIT Madras)

Dr. Sukhendu Das is currently employed as a Professor in the Deptt. Of Computer Science and Engg., IIT Madras, Chennai, India. He completed his B.Tech degree from IIT Kharagpur in the Deptt. Of Electrical Engg. in 1985 and M. Tech Degree in the area of Computer Technology from IIT Delhi in 1987. He then obtained his Ph.D degree from IIT Kharagpur in 1993. His current areas of research interests are: Visual Perception, Computer Vision: Digital Image Processing and Pattern Recognition, Computer Graphics, Artificial Neural Networks, Computational Science and engineering, Soft Computing, Deep Learning and Computational brain modeling. Dr. Sukhendu Das has been a faculty of the Deptt. of CS&E, IIT Madras, INDIA since 1989. He has worked as a visiting scientist in the University of Applied Sciences, Pforzheim, Germany, for post-doctoral research work, from Dec. 2001 till May 2003; and as a visiting fellow/scientist in the Univ. of UWA, Perth, Australia, during June-Aug. 2006, and July-Sept. 2008. He has guided Six (currently guiding 2) Ph. D students, 26 (currently guiding 7) M.S., 42 M. Tech. (+ Dual) and 8 B. Tech students. He had completed several international and national sponsored projects and consultancies, both as principle and co-investigators. He has published about 150 technical papers in international and national journals and conferences. He has reviewed several papers in international journals (IEEE, IET, Elsevier, Springer etc.) and chaired several sessions in conferences. He has received three (3) best papers and a best design contest award. Significant and novel technical contributions are: MST-CSS representation for CBVR tasks; ESLAR framework for smart CBIR; SUBBAND face, Eigen-domain transformation (EDT) and Eigen-scale space (ESS) representations for face-based biometry applications; Creation of an Outdoor Surveillance Face Database (support from MCIT, GOI) for biometry; and Manifold based alignment based optimization using Domain Adaptation, for applications in face, object and video categorization tasks.