Multi-criterion decision making-based multi-channel hierarchical fusion of digital breast tomosynthesis and digital mammography for breast mass discrimination

Abstract Multifaceted features decoded from mammographic images may describe various perspectives of the breast mass heterogeneity, in this study, we aimed to explore a methodology to effectively integrate multifaceted mass representations extracted from the digital breast tomosynthesis (DBT) and full-field digital mammography (FFDM) to enhance breast cancer discrimination. A novel multi-criterion decision making-based multi-channel fusion (MDMF) framework was proposed to fuse different breast mass representations processed in multi-channels built on the deep convolutional neural network (DCNN) and the multilayer perceptron (MLP) at the decision level. A hierarchical framework (HFMM) was also developed for multi-modality images and multi-channel fusion to integrate multimodality information from DBT and FFDM. We retrospectively collected 441 patients with both DBT and FFDM, and the regions of interest (ROIs) covering the malignant, benign, and normal tissues were extracted for validation. The MDMF achieved the area under the receiver operating characteristic curve (AUC) of 93.14%, 91.30%, 97.35% (FFDM) and 93.79%, 95.16%, 99.31% (DBT) respectively for the malignant, benign and normal mass. While the HFMM further boosted the performance to AUC of malignant 94.14%, benign 95.42% and normal mass 99.56% The matthews correlation coefficient (MCC) were 73.15% and 81.02% for FFDM and DBT accomplished by MDMF, and enhanced to 81.72% when integrating the multimodality information from DBT and FFDM via the proposed HFMM. The experimental results suggested that the proposed HFMM achieved superior discriminative performance when compared with the benchmark classification algorithms and fusion architectures, rendering it a practical tool for breast mass discrimination in breast cancer screening.