Timezone: »

 
Poster
Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers
Mikita Dvornik · Isma Hadji · Konstantinos Derpanis · Animesh Garg · Allan Jepson

Thu Dec 09 08:30 AM -- 10:00 AM (PST) @
in Poster Session 6 »

In this work, we consider the problem of sequence-to-sequence alignment for signals containing outliers. Assuming the absence of outliers, the standard Dynamic Time Warping (DTW) algorithm efficiently computes the optimal alignment between two (generally) variable-length sequences. While DTW is robust to temporal shifts and dilations of the signal, it fails to align sequences in a meaningful way in the presence of outliers that can be arbitrarily interspersed in the sequences. To address this problem, we introduce Drop-DTW, a novel algorithm that aligns the common signal between the sequences while automatically dropping the outlier elements from the matching. The entire procedure is implemented as a single dynamic program that is efficient and fully differentiable. In our experiments, we show that Drop-DTW is a robust similarity measure for sequence retrieval and demonstrate its effectiveness as a training loss on diverse applications. With Drop-DTW, we address temporal step localization on instructional videos, representation learning from noisy videos, and cross-modal representation learning for audio-visual retrieval and localization. In all applications, we take a weakly- or unsupervised approach and demonstrate state-of-the-art results under these settings.

Keywords: Representation Learning

Author Information

Mikita Dvornik (Samsung AI Centre)
Isma Hadji (York University)
Konstantinos Derpanis (Ryerson University)
Animesh Garg (University of Toronto, Nvidia, Vector Institute)

I am a Assistant Professor of Computer Science at University of Toronto and a Faculty Member at the Vector Institute. I work on machine learning for perception and control in robotics.

Allan Jepson (Samsung AIC-Toronto)

More from the Same Authors