Syntactically guided text generation

Abstract

In recent years, as researchers have achieved breakthrough in the generic text generation, increased works turned their attention to controllable text generation to imply external knowledge to constrain the semantics or syntax of the generated text. In this work, we propose a guided neural text generation framework which incorporate syntactic guidance to pilot the syntax structure of the output. Two models are included in this framework. The first is a syntax expansion model that expands a high-level constituency parse template to a full-fledged parse using the additional information from the input source text. The second model is for guided text generation. It collects the semantics from the input text and extract the syntactic information from the full-fledged constituency parse, integrating them together to generate sentences that not only have the desired semantics but comply with the target syntax as well. The framework is evaluated on the paraphrasing task with automatic metrics. The results indicate that it outperforms state-of-the-art syntactically controlled text generation models both semantically and syntactically by a large margin.