DE-GAN: Domain Embedded GAN for High Quality Face Image Inpainting

Abstract Domain knowledge of face shapes and structures plays an important role in face inpainting. However, general inpainting methods focus mainly on the resolution of generated images without considering the particular structure of human faces and generally produce inharmonious facial parts. Existing face-inpainting methods incorporate only one type of facial feature for face completion, and their results are still undesirable. To improve face inpainting quality, we propose a Domain Embedded Generative Adversarial Network (DE-GAN) for face inpainting. DE-GAN embeds three types of face domain knowledge (i.e., face mask, face part, and landmark image) via a hierarchical variational auto-encoder (HVAE) into a latent variable space to guide face completion. Two adversarial discriminators, a global discriminator and a patch discriminator, are used to judge whether the generated distribution is close to the real distribution or not. Experiments on two public face datasets demonstrate that our proposed method generates higher quality inpainting results with consistent and harmonious facial structures and appearance than existing methods and achieves the state-of-the-art performance, esp. for inpainting under-pose variations.