Timezone: »
We propose Differentiable Temporal Logic (DTL), a model-agnostic framework that introduces temporal constraints to deep networks. DTL treats the outputs of a network as a truth assignment of a temporal logic formula, and computes a temporal logic loss reflecting the consistency between the output and the constraints. We propose a comprehensive set of constraints, which are implicit in data annotations, and incorporate them with deep networks via DTL. We evaluate the effectiveness of DTL on the temporal action segmentation task and observe improved performance and reduced logical errors in the output of different task models. Furthermore, we provide an extensive analysis to visualize the desirable effects of DTL.
Author Information
Ziwei Xu (National University of Singapore)
Yogesh Rawat (University of Central Florida)
Yongkang Wong (National University of Singapore)
Mohan Kankanhalli (National University of Singapore,)
Mubarak Shah (University of Central Florida)
More from the Same Authors
-
2022 : Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks »
Saeed Vahidian · Mahdi Morafah · Chen Chen · Mubarak Shah · Bill Lin -
2023 Poster: Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization »
Xilie Xu · Jingfeng ZHANG · Feng Liu · Masashi Sugiyama · Mohan Kankanhalli -
2023 Poster: Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection »
Xilie Xu · Jingfeng ZHANG · Feng Liu · Masashi Sugiyama · Mohan Kankanhalli -
2023 Poster: GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization »
Vicente Vivanco Cepeda · Gaurav Kumar Nayak · Mubarak Shah -
2023 Poster: Revealing the unseen: Benchmarking video action recognition under occlusion »
Shresth Grover · Vibhav Vineet · Yogesh Rawat -
2023 Poster: On Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes »
Rajat Modi · Vibhav Vineet · Yogesh Rawat -
2022 Workshop: Vision Transformers: Theory and applications »
Fahad Shahbaz Khan · Gul Varol · Salman Khan · Ping Luo · Rao Anwer · Ashish Vaswani · Hisham Cholakkal · Niki Parmar · Joost van de Weijer · Mubarak Shah -
2022 : Contrastive Learning on Synthetic Videos for GAN Latent Disentangling »
Kevin Duarte · Wei-An Lin · Ratheesh Kalarot · Jingwan (Cynthia) Lu · Eli Shechtman · Shabnam Ghadar · Mubarak Shah -
2022 Poster: Are all Frames Equal? Active Sparse Labeling for Video Action Detection »
Aayush Rana · Yogesh Rawat -
2022 Poster: Robustness Analysis of Video-Language Models Against Visual and Language Perturbations »
Madeline Chantry · Shruti Vyas · Hamid Palangi · Yogesh Rawat · Vibhav Vineet -
2021 Poster: Learning to Predict Trustworthiness with Steep Slope Loss »
Yan Luo · Yongkang Wong · Mohan Kankanhalli · Qi Zhao -
2021 Poster: Unsupervised Motion Representation Learning with Capsule Autoencoders »
Ziwei Xu · Xudong Shen · Yongkang Wong · Mohan Kankanhalli -
2021 Poster: Reformulating Zero-shot Action Recognition for Multi-label Actions »
Alec Kerrigan · Kevin Duarte · Yogesh Rawat · Mubarak Shah -
2019 Poster: Embedding Symbolic Knowledge into Deep Networks »
Yaqi Xie · Ziwei Xu · Kuldeep S Meel · Mohan Kankanhalli · Harold Soh -
2018 Poster: Unsupervised Learning of View-invariant Action Representations »
Junnan Li · Yongkang Wong · Qi Zhao · Mohan Kankanhalli -
2018 Poster: VideoCapsuleNet: A Simplified Network for Action Detection »
Kevin Duarte · Yogesh Rawat · Mubarak Shah