Graph2step: A System for Knowledge Driven Procedural Step Generation

Procedural step generation (i.e. instruction generation) is an important task, particularly because of its applicability on fields such as usability of technology and education. An effective procedural step generation system should be capable of generating steps to guide a person to accomplish a procedure, and in the case of a mishap in the procedure, generate steps to correct the mishap and continue and complete the original procedure. Procedural step generation is complicated for modern natural language systems, particularly because of knowledge that may be implicit in the steps, and essential for its completion. We present a work-in-progress of a multi-step system that is capable of, given a goal and facts related to a goal, to generate procedural steps to accomplish a procedure. The system is also capable of reusing past knowledge through a neural memory to handle goal changes (i.e., mishaps) while performing procedures. To accomplish this, we leverage a contextual commonsense inference model which can generate contextually relevant facts (i.e., assertions) about a procedure, and train a model which selects facts that are necessary to accomplish a goal and translate these into a procedural step.