Harnessing Event Report Data to Identify Diagnostic Error During the COVID-19 Pandemic

ABSTRACT Introduction COVID-19 exposed systemic gaps with increased potential for diagnostic error. We implemented a new approach leveraging electronic safety reporting to identify and categorize diagnostic errors during the pandemic. Methods We evaluated all safety event reports from March 1, 2020 to February 28, 2021 at our academic medical center using two complementary pathways. Pathway 1: all reports with explicit mention of COVID-19. Pathway 2: all reports without explicit mention of COVID-19 where natural language processing (NLP) plus logic-based stratification was applied to identify potential cases. Cases were evaluated by manual review to identify diagnostic error/delay and categorize error type using a recently proposed classification framework of 8 categories of pandemic-related diagnostic errors. Results We included 14,230 reports and identified 95 (0.67%) cases of diagnostic error/delay. Pathway 1 (n = 1780 eligible reports) yielded 45 reports with diagnostic error/delay (positive predictive value [PPV] = 2.5%), of which 38.1% (16/45) were attributed to pandemic-related strain. In Pathway 2, the NLP-based algorithm flagged 110 safety reports for manual review from 12,450 eligible reports. Of these, 50 reports had diagnostic error/delay (PPV = 45.5%); 94.0% (47/50) were related to strain. Errors from all 8 categories of the taxonomy were found on analysis. Conclusion An event reporting-based strategy including use of simple-NLP identified COVID-related diagnostic errors/delays uncovered several safety concerns related to COVID-19. An NLP based approach can complement traditional reporting and be used as a just-in-time monitoring system to enable early detection of emerging risks from large volumes of safety reports.