Deep Learning Algorithm for Chest Radiography: Multicenter Respiratory Outpatient Diagnostic Cohort Study

Background: Commercially available deep learning algorithms (DLAs) for chest radiography (CXR) have been evaluated with variable clinical settings. This study evaluated a DLA for detecting and localizing referable thoracic abnormalities on CXRs in a multicenter respiratory outpatient diagnostic cohort and compared the performance of physicians with and without DLA assistance. Methods: A total of 6,006 consecutive participants who visited respiratory outpatient clinics from 3 institutions and underwent both CXR and CT in 2018 were retrospectively evaluated with DLA. Observer performance tests were conducted with a subset of 230 randomly sampled participants to assess whether the algorithm enhanced physician performance. The area under the receiver operating characteristic (ROC) curve (AUC) and the area under the alternative free-response ROC (AUAFROC) were measured to evaluate the performance of the DLA and physicians in image classification and lesion localization, respectively. Findings: In the assessments performed in 6,006 participants, 4,264 referable thoracic abnormal lesions were found in 3,337 (55·6%) participants, including 1,173 pulmonary nodules/masses (20%), 919 consolidations (15%), 15 pneumothoraces (0·2%), and 2,157 other lesions (51%). DLA showed an AUC of 0·86 to 0·87, sensitivity of 0·87 to 0·90, specificity of 0·70 to 0·76, false-positive findings per image of 0·31 to 0·42, and AUAFROC of 0·86 to 0·87. For the randomly sampled dataset (n=230), the average AUC increased from 0·86 (95% CI: 0·82, 0·90) to 0·89 (95% CI: 0·85, 0·92) (P=0·003) and the average AUFROC increased from 0·80 (95% CI: 0·76, 0·84) to 0·82 (95% CI: 0·78, 0·86) (P=0·003) with the aid of DLA. The AUROC of standalone DLA was 0·91, which was significantly higher than the standalone AUROCs of pulmonologists (0·84,P=0·01) and radiology residents (0·85, P=0·03). Interpretation: The DLA showed acceptable performance and could enhance physician performance for diagnosing referable thoracic abnormalities in daily clinical practice. Funding Statement: This work was supported by grant (HI19C0847) from Korea Health Industry Development Institute. Declaration of Interests: Activities not related to the present article: KNJ and HK received a research grant from Lunit. Other authors have neither financial interests nor conflicts of interest. Ethics Approval Statement: This study was approved by the institutional review boards of all participating institutions, which waived the requirement for patient consent.