Adversarial Text Generation Via Feature-Mover's Distance

Authors:
Liqun Chen Duke University
Shuyang Dai Duke University
Chenyang Tao Duke University
Haichao Zhang Baidu Research
Zhe Gan Microsoft
Dinghan Shen Duke University
Yizhe Zhang Microsoft Research
Guoyin Wang Duke University
Ruiyi Zhang Duke University
Lawrence Carin Duke University

Introduction:

Generative adversarial networks (GANs) have achieved significant success in generating real-valued data.Instead of using the standard GAN objective, the authors propose to improve text-generation GAN via a novel approach inspired by optimal transport.

Abstract:

Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text-generation tasks. Instead of using the standard GAN objective, we propose to improve text-generation GAN via a novel approach inspired by optimal transport. Specifically, we consider matching the latent feature distributions of real and synthetic sentences using a novel metric, termed the feature-mover's distance (FMD). This formulation leads to a highly discriminative critic and easy-to-optimize objective, overcoming the mode-collapsing and brittle-training problems in existing methods. Extensive experiments are conducted on a variety of tasks to evaluate the proposed model empirically, including unconditional text generation, style transfer from non-parallel text, and unsupervised cipher cracking. The proposed model yields superior performance, demonstrating wide applicability and effectiveness.

You may want to know: