Learning to be Proactive from Missed User-Signals in Multi-turn Dialogues

Task-oriented dialogue (TOD) systems have traditionally emphasized goal completion with fixed slot ontologies and database-backed execution. Recent research emphasizes the need for proactive agents that can take initiative to elicit missing task information. Prior approaches learn policies for proactive actions but assume a fixed action space defined by a static slot ontology, while other works on slot schema induction identifies what task ontology should be captured yet without operationalizing it into proactive agent behaviors. We introduce a method that learns proactive agent behaviors directly from dialogue interactions by mining missed opportunities---instances where users voluntarily provide unrequested information. Our approach uses large language models (LLMs) to (i) detect such opportunities, (ii) reverse-generate candidate proactive questions, and (iii) incrementally cluster them into a hierarchical slot ontology with priorities and examples. This evolving structure is then integrated into the agent’s action space, enabling domain-adaptive, information-seeking proactivity. Experiments on MultiWOZ 2.4 show that adding our proactive framework on top of a base LLM leads to consistent improvements in recall, precision, and early slot coverage.