NDDR-LCS: A Multi-Task Learning Method for Classification of Carotid Plaques

Carotid plaque classification plays a critical role in the identification of vulnerable plaques, so it is crucial for early risk estimation of cardiovascular and cerebrovascular events. Carotid ultrasound examination with ultrasound images and reports produced by professional doctors is the most common way to assess atherosclerotic plaques in clinical practice. However, existing deep learning methods for carotid ultrasound image analysis ignore the information in the ultrasound report. In this paper, we propose a multi-task learning (MTL) method named NDDR-LCS based on convolutional neural network (CNN) that leverages auxiliary information from ultrasound reports to assist the carotid plaque classification task. NDDR-LCS utilizes dense blocks as feature descriptors and organically combines three novel MTL mechanisms that are Neural Discriminative Dimensionality Reduction (NDDR), Learning Mixtures, and Cross-Stitch, to learn dependencies between ultrasound images and ultrasound reports. Based on carotid ultrasound images and their corresponding diagnostic reports, we conduct sufficient experiments to prove that NDDR-LCS outperforms state-of-the-art CNN methods for carotid plaque classification.