Timezone: »

 
Poster
Don't Pour Cereal into Coffee: Differentiable Temporal Logic for Temporal Action Segmentation
Ziwei Xu · Yogesh Rawat · Yongkang Wong · Mohan Kankanhalli · Mubarak Shah

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #800
in Poster Session 1 »

We propose Differentiable Temporal Logic (DTL), a model-agnostic framework that introduces temporal constraints to deep networks. DTL treats the outputs of a network as a truth assignment of a temporal logic formula, and computes a temporal logic loss reflecting the consistency between the output and the constraints. We propose a comprehensive set of constraints, which are implicit in data annotations, and incorporate them with deep networks via DTL. We evaluate the effectiveness of DTL on the temporal action segmentation task and observe improved performance and reduced logical errors in the output of different task models. Furthermore, we provide an extensive analysis to visualize the desirable effects of DTL.

Keywords: linear temporal logic · Temporal action segmentation · neurosymbolic methods
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Ziwei Xu (National University of Singapore)
Yogesh Rawat (University of Central Florida)
Yongkang Wong (National University of Singapore)
Mohan Kankanhalli (National University of Singapore,)
Mubarak Shah (University of Central Florida)

More from the Same Authors