Adversarial Scene Editing: Automatic Object Removal From Weak Supervision

Authors:
Rakshith R Shetty Max Planck Institute for Informatics
Mario Fritz CISPA Helmholtz Center i.G.
Bernt Schiele Max Planck Institute for Informatics

Introduction:

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.In this work, the authors take a step towards general scene-level image editing by developing an automatic interaction-free object removal model.

Abstract:

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.In this work, we take a step towards general scene-level image editing by developing an automatic interaction-free object removal model. Our model learns to find and remove objects from general scene images using image-level labels and unpaired data in a generative adversarial network (GAN) framework. We achieve this with two key contributions: a two-stage editor architecture consisting of a mask generator and image in-painter that co-operate to remove objects, and a novel GAN based prior for the mask generator that allows us to flexibly incorporate knowledge about object shapes. We experimentally show on two datasets that our method effectively removes a wide variety of objects using weak supervision only.

You may want to know: