Machine Learning-Based Models Did Not Improve the Prediction of Complications after Percutaneous Coronary Intervention in Japan

Background: Whether machine-learning-based (ML) models improve the predictability of adverse periprocedural events compared with traditional risk-scoring systems remains unknown. Methods: We extracted data from a Japanese, multicenter PCI registry for patients with coronary artery disease who underwent PCI in accordance with the NCDR-CathPCI registry. We developed logistic regression (LR) and extreme gradient descent boosting (XGB) models to predict periprocedural acute kidney injury (AKI), bleeding, and in-hospital mortality, incorporating the same covariates as the original NCDR-CathPCI risk score models. The hyperparameters of the XGB models were tuned using a stratified 3-fold cross-validation. Model discrimination was evaluated using the area under receiver-operating characteristic (AUROC) curves, and the calibration was evaluated using a calibration plot. Results: In the analytic cohort, the median age was 70 (interquartile range 62, 77) years and predominantly men (79·3%). AKI, bleeding, and in-hospital mortality occurred in 9·6%, 7·8%, and 2·3%, respectively. The NCDR-CathPCI risk scores demonstrated good discrimination for each outcome with good calibration (AUROC of 0·79, 0·76, and 0·92 for AKI, bleeding, and in-hospital mortality). Compared with the NCDR-CathPCI risk scores, the LR and XGB models modestly improved model discrimination for AKI and bleeding (AUROC of 0·80, and 0·77 in LR models; 0·81, and 0·79, in XGB models) but not for in-hospital mortality (AUROC of 0·92 in LR models, and 0·93 in XGB models, respectively).The calibration curve demonstrated that the XGB model for in-hospital mortality overestimated the risk in low-risk patients. Conclusion: All of the original NCDR-CathPCI risk models for periprocedural adverse events showed an adequate discrimination and calibration within our cohort. Discrimination of bleeding and AKI risk improved modestly when ML-based models were incorporated, but whether the improvement in overall performances by ML-based models was clinically significant was unclear. Funding: The present study was funded by the Grants-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (KAKENHI; Nos. 16KK0186, 16H05215, 18K17332, and 20H03915 https://kaken.nii.ac.jp/ja/index/). Declaration of Interest: Dr. Shiraishi is affiliated with an endowed department by Nippon Shinyaku Co., Ltd., Medtronic Japan Co., Ltd., and BIOTRONIK JAPAN Inc., and received research grants from the SECOM Science and Technology Foundation and the Uehara Memorial Foundation and honoraria from Otsuka Pharmaceuticals Co., Ltd. and Ono Pharmaceuticals Co., Ltd.; Dr Kohsaka reported receiving grants from Daiichi Sankyo during the conduct of the study; and grants from Bayer and personal fees from Bristol Bayer, Bristol-Myers Squibb, and Pfizer outside the submitted work. Ethical Approval: The protocol of this study was under the principles of the Declaration of Helsinki and approved by the committee of each participating hospital.