Performance of Automatic Machine Learning versus Radiologists in the Evaluation of Endometrium on Computed Tomography

Objectives: In this study, we developed radiomic models that utilize a combination of imaging features and clinical variables to distinguish endometrial cancer (EC) from non-EC diagnoses on computed tomography (CT). Methods: A total of 926 patients consisting of 416 EC and 510 non-EC diagnoses were included. Uterus and the endometrium were manually segmented on CT. Fourteen feature selection and ten classification methods were manually examined to select the most optimized machine learning pipeline. Automatic machine learning using Tree-Based Pipeline Optimization Tool (TPOT) was performed. 847 patients were portioned into training, validation, testing sets, and another 79 patients were as our external testing set. The performance of the machine learning pipelines on the testing sets was compared to radiologists. Results: There was significant difference in age between the EC and non-EC groups (64.0 vs. 53.7, p <0.001). The manual expert optimized pipeline using the “Relief” feature selection method and “Bagging” classifier on the external testing set achieved accuracy of 0.73 (95% CI: 0.62-0.82), sensitivity of 0.64 (95% CI: 0.45-0.79), and specificity of 0.78 (95% CI: 0.65-0.87), while TPOT achieved accuracy of 0.80 (95% CI: 0.70-0.87), sensitivity of 0.61 (95% CI: 0.43-0.77), and specificity of 0.90 (95% CI: 0.78-0.96). When compared to all radiologists averaged, the TPOT achieved higher test accuracy (0.80 vs. 0.49, p<0.001) and specificity (0.90 vs. 0.51, p<0.001), with comparable sensitivity (0.61 vs. 0.46, p=0.130). Conclusions: Our results demonstrate that automatic machine learning can distinguish EC from non-EC diagnoses on CT imaging with higher accuracy and specificity than radiologists. Funding Statement: This work was supported by the National Cancer Institute (NCI) of the National Institutes of Health under Award Number R03CA249554 to H. Bai, by a training grant from the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the National Institutes of Health (5T32EB1680), by the National Cancer Institute (NCI) of the National Institutes of Health (F30CA239407) to K. Chang and by the 111 project of ministry of education discipline innovation and wisdom base under grant Number B18059 to Beiji Zou. Declaration of Interests: The authors declare no potential conflicts of interest. Ethics Approval Statement: The study was approved by the Institutional Review Board (IRB) at the first and third institutions. With the agreement to use the second institution data, the IRB approval of our study was waived for the second institution.