Unsupervised Real-World Super-resolution Using Variational Auto-encoder and Generative Adversarial Network

Convolutional Neural Networks (CNNs) have shown promising results on Single Image Super-Resolution (SISR) task. A pair of Low-Resolution (LR) and High-Resolution (HR) images are typically used in the CNN models to train them to super-resolve LR images in a fully supervised manner. Owing to non-availability of true LR-HR pairs, the LR images are generally synthesized from HR data by applying synthetic degradation such as bicubic downsampling. Such networks under-perform when used on real-world data where degradation is different from the synthetically generated LR image. As obtaining true LR-HR pair is a tedious and resource (time and effort) consuming task, we propose a new approach and architecture to super-resolve the real-world LR images in an unsupervised manner by using a Generative Adversarial Network (GAN) framework with Variational Auto-Encoder (VAE). Along with a new network architecture, we also introduce a novel loss metric based on no-reference quality scores of SR images to improve the perceptual fidelity of the SR images. Through the experiments on NTIRE-2020 Real-World SR Challenge dataset, we demonstrate the superiority of the proposed approach over the other competing state-of-the-art methods.