Super-Resolution by Latent Space Exploration: Training with Poorly-Aligned Clinical and Micro CT Image Dataset

This paper proposes a super-resolution (SR) method, for performing SR on a poorly-aligned dataset. Super-resolution methods commonly needs aligned low-resolution (LR) and high-resolution (HR) images for training. For obtaining paired LR and HR images in medical imaging, we need to align low and high-resolution data using image registration technology. However, since the hardness of aligning LR and HR images, the aligned LR-HR dataset is always low quality. Conventional SR methods always fail to train using poorly-aligned datasets since these methods need high-quality LR-HR datasets. To tackle this problem, we propose a two-step framework for SR using poorly-aligned datasets. In the first step, we decompose image representation into two parts: one is a content code that captures the image content; the other is a style code that captures the image style and anatomy difference between LR / HR images. To perform SR of a given LR image, we input the content code and a latent variable simultaneously into the SR network to obtain an SR result. In the second step, using the trained SR network and an LR image, we search for a content code, and a style code for generating the most proper SR image. This is conducted by searching for the best content code and the best style code by latent space exploration. We conducted experiments using a poorly-aligned clinical-micro CT lung specimen dataset. Experimental results illustrated the proposed method outperformed conventional SR methods by increasing SSIM from 0.309 to 0.312, and have much more convincing perceptual quality than conventional SR methods.