Learning Structure from Visual Semantic Features and Radiology Ontology for Lymph Node Classification on MRI.

Medical image classification (for example, lesions on MRI scans) is a very challenging task due to the complicated relationships between different lesion sub-types and expensive cost to collect high quality labelled training datasets. Graph model has been used to model the complicated relationship for medical imaging classification successfully in many previous work. However, most existing graph based models assumed the structure is known or pre-defined, and the classification performance severely depends on the pre-defined structure. To address all the problems of current graph learning models, we proposed to jointly learn the graph structure and use it for classification task in one framework. Besides imaging features, we also use the disease semantic features (learned from clinical reports), and predefined lymph node ontology graph to construct the graph structure. We evaluated our model on a T2 MRI image dataset with 821 samples and 14 types of lymph nodes. Although this dataset is very unbalanced on different types of lymph nodes, our model shows promising classification results on this challenging datasets compared to several state of art methods.