Self‐supervised Learning for Accelerated 3D High‐resolution Ultrasound Imaging

Purpose Ultrasound (US) imaging has been widely used in diagnosis, image-guided intervention and therapy, where high-quality three-dimensional (3D) images are highly desired from sparsely acquired two-dimensional (2D) images. This study aims to develop a deep learning-based algorithm to reconstruct high-resolution 3D US images only reliant on the acquired sparsely distributed 2D images. Methods We propose a self-supervised learning framework using cycle consistent generative adversarial network (cycleGAN), where two independent cycleGAN models are respectively trained with paired original US images and two sets of low-resolution US images. The two sets of low-resolution US images are respectively obtained through down sampling the original US images along the two axes. In US imaging, in-plane spatial resolution is generally much higher than through-plane resolution. By learning the mapping from down-sampled in-plane low-resolution images to original high-resolution US images, cycleGAN can generate through-plane high-resolution images from original sparely distributed 2D images. Finally, high-resolution 3D US images are reconstructed by combining the generated 2D images from the two cycleGAN models. Results The proposed method was assessed on two different datasets. One is automatic breast ultrasound (ABUS) images from 70 breast cancer patients, the other is collected from 45 prostate cancer patients. By applying a spatial resolution enhancement factor of 3 to the breast cases, our proposed method achieved the mean absolute error (MAE) value of 0.90±0.15, the peak signal-to-noise ratio (PSNR) value of 37.88±0.88 dB, and the visual information fidelity (VIF) value of 0.69±0.01, which significantly outperforms bicubic interpolation. Similar performances have been achieved using the enhancement factor of 5 in these breast cases and using the enhancement factor of 5 and 10 in the prostate cases. Conclusions We have proposed and investigated a new deep learning-based algorithm for reconstructing high-resolution 3D US images from sparely acquired 2D images. Significant improvement on through-plane resolution has been achieved by only using the acquired 2D images without any external atlas images. Its self-supervision capability could accelerate high-resolution US imaging.