Leveraging Other Datasets for Medical Imaging Classification: Evaluation of Transfer, Multi-task and Semi-supervised Learning

To address the data scarcity challenge in developing deep learning based medical imaging classification, a widely-used strategy is to leverage other available datasets in training. Three machine learning algorithms belong to this concept, namely, transfer learning (TL), multi-task learning (MTL) and semi-supervised learning (SSL). TL and MTL bring another labeled dataset usually from different categories, while SSL utilizes an unlabeled dataset from the same category. Each has proven useful for medical imaging tasks. In this work, we unified these three algorithms into one framework, to directly compare individual contribution and combine them to extract extra performance. For SSL, state-of-the-art consistency based methods were evaluated, including \(\varPi \)-Model and virtual adversarial training. Experiments were done on classifying gastric diseases given endoscopic images trained with various amount of data. It was observed that individually TL has the most while SSL has the least performance gain. When used together, their contribution build up constructively leading to further improved performance especially with larger capacity network. This work helps guide applying each or combination of TL/MTL/SSL for other medical applications.