Timezone: »

Out-of-Distribution Generalization Challenge in Dialog State Tracking
Jiasheng Ye · Yawen Ouyang · Zhen Wu · Xinyu Dai
Event URL: https://openreview.net/forum?id=Z-k91NB8Eh »

Dialog State Tracking (DST) is a core component for multi-turn Task-Oriented Dialog (TOD) systems to understand the dialogs. DST models need to generalize to Out-of-Distribution (OOD) utterances due to the open environments dialog systems face. Unfortunately, utterances in TOD are multi-labeled, and most of them appear in specific contexts (i.e., the dialog histories). Both characteristics make them different from the conventional focus of OOD generalization research and remain unexplored. In this paper, we formally define OOD utterances in TOD and evaluate the generalizability of existing competitive DST models on the OOD utterances. Our experimental result shows that the performance of all models drops considerably in dialogs with OOD utterances, indicating an OOD generalization challenge in DST.

Author Information

Jiasheng Ye (Nanjing University)
Yawen Ouyang (Nanjing University)
Zhen Wu (Nanjing University)
Xinyu Dai (Nanjing University)

More from the Same Authors