GL-GAN: Adaptive global and local bilevel optimization for generative adversarial network

Abstract Although Generative Adversarial Networks (GAN) have shown remarkable performance in image generation, there exist some challenges in instability and convergence speed. During the training, the results of some models display the imbalances of quality within a generated image, in which some defective parts appear compared with other regions. Different from general single global optimization methods, we introduce an adaptive global and local bilevel optimization model (GL-GAN). The model achieves the generation of high-resolution images in a complementary and promoting way, where global optimization is to optimize the whole images and local is only to optimize the low-quality areas. Based on DCGAN, GL-GAN is able to effectively avoid the nature of imbalance by local bilevel optimization, which is accomplished by first locating low-quality areas and then optimizing them. Moreover, through feature map cues from discriminator output, we propose the adaptive local and global optimization method (Ada-OP) for interactive optimization and observe that it boosts the convergence speed. Compared with the current GAN methods, our model has shown impressive performance on CelebA, Oxford Flowers, CelebA-HQ and LSUN datasets.