Task-oriented dialog is inherently about contextual action: users address the system from a specific context and the system must decide what to do in response. This talk will present some of the core principles of the Semantic Machines team's approach to conversational AI: program synthesis for action prediction, compositionality for handling complex tasks, metacomputation for reference and revision, error handling for dialog management, and dynamic generation for truthful output. I will also mention ways in which real-world constraints can help to inform the design of conversational systems.