Timezone: »
We propose Differentiable Temporal Logic (DTL), a model-agnostic framework that introduces temporal constraints to deep networks. DTL treats the outputs of a network as a truth assignment of a temporal logic formula, and computes a temporal logic loss reflecting the consistency between the output and the constraints. We propose a comprehensive set of constraints, which are implicit in data annotations, and incorporate them with deep networks via DTL. We evaluate the effectiveness of DTL on the temporal action segmentation task and observe improved performance and reduced logical errors in the output of different task models. Furthermore, we provide an extensive analysis to visualize the desirable effects of DTL.
Author Information
Ziwei Xu (National University of Singapore)
Yogesh Rawat (University of Central Florida)
Yongkang Wong (National University of Singapore)
Mohan Kankanhalli (National University of Singapore,)
Mubarak Shah (University of Central Florida)
More from the Same Authors
-
2022 : Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks »
Saeed Vahidian · Mahdi Morafah · Chen Chen · Mubarak Shah · Bill Lin -
2022 Workshop: Vision Transformers: Theory and applications »
Fahad Shahbaz Khan · Gul Varol · Salman Khan · Ping Luo · Rao Anwer · Ashish Vaswani · Hisham Cholakkal · Niki Parmar · Joost van de Weijer · Mubarak Shah -
2022 : Contrastive Learning on Synthetic Videos for GAN Latent Disentangling »
Kevin Duarte · Wei-An Lin · Ratheesh Kalarot · Jingwan (Cynthia) Lu · Eli Shechtman · Shabnam Ghadar · Mubarak Shah -
2022 Poster: Are all Frames Equal? Active Sparse Labeling for Video Action Detection »
Aayush Rana · Yogesh Rawat -
2022 Poster: Robustness Analysis of Video-Language Models Against Visual and Language Perturbations »
Madeline Chantry · Shruti Vyas · Hamid Palangi · Yogesh Rawat · Vibhav Vineet -
2021 Poster: Learning to Predict Trustworthiness with Steep Slope Loss »
Yan Luo · Yongkang Wong · Mohan Kankanhalli · Qi Zhao -
2021 Poster: Unsupervised Motion Representation Learning with Capsule Autoencoders »
Ziwei Xu · Xudong Shen · Yongkang Wong · Mohan Kankanhalli -
2021 Poster: Reformulating Zero-shot Action Recognition for Multi-label Actions »
Alec Kerrigan · Kevin Duarte · Yogesh Rawat · Mubarak Shah -
2019 Poster: Embedding Symbolic Knowledge into Deep Networks »
Yaqi Xie · Ziwei Xu · Kuldeep S Meel · Mohan Kankanhalli · Harold Soh -
2018 Poster: Unsupervised Learning of View-invariant Action Representations »
Junnan Li · Yongkang Wong · Qi Zhao · Mohan Kankanhalli -
2018 Poster: VideoCapsuleNet: A Simplified Network for Action Detection »
Kevin Duarte · Yogesh Rawat · Mubarak Shah