Timezone: »

 
Poster
Trajectory Convolution for Action Recognition
Yue Zhao · Yuanjun Xiong · Dahua Lin

Tue Dec 04 02:00 PM -- 04:00 PM (PST) @ Room 210 #96
in Tue Poster Session B »

How to leverage the temporal dimension is a key question in video analysis. Recent works suggest an efficient approach to video feature learning, i.e., factorizing 3D convolutions into separate components respectively for spatial and temporal convolutions. The temporal convolution, however, comes with an implicit assumption – the feature maps across time steps are well aligned so that the features at the same locations can be aggregated. This assumption may be overly strong in practical applications, especially in action recognition where the motion serves as a crucial cue. In this work, we propose a new CNN architecture TrajectoryNet, which incorporates trajectory convolution, a new operation for integrating features along the temporal dimension, to replace the existing temporal convolution. This operation explicitly takes into account the changes in contents caused by deformation or motion, allowing the visual features to be aggregated along the the motion paths, trajectories. On two large-scale action recognition datasets, namely, Something-Something and Kinetics, the proposed network architecture achieves notable improvement over strong baselines.

Keywords: Computer Vision · Applications · Video Analysis · Activity and Event Recognition
[ Paper] 

Author Information

Yue Zhao (The Chinese University of Hong Kong)
Yuanjun Xiong (Amazon)
Dahua Lin (The Chinese University of Hong Kong)

More from the Same Authors