Logging in: a comparative analysis of electronic health records versus anesthesia resident-driven logbooks

Resident logbooks (RLBs) documenting clinical case exposure are widespread in medical education despite evidence of poor accuracy. Electronic health records (e.g., anesthesia information management systems [AIMS]) may provide advantages for auditing longitudinal case exposure. We evaluated the agreement between AIMS and RLBs for tracking case exposure during anesthesiology residency. We performed a historical cohort study with anesthesiology residents (2011–2018, all of whom used a RLB contemporaneously with AIMS) working in a multisite academic health sciences network. The primary outcome was total case-load logging; secondary outcomes were volumes for seven surgical specialties (general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery). Correlation of case numbers tracked by AIMS vs RLB was assessed using Pearson correlation; agreement was determined using Bland–Altman plots and intraclass correlation coefficients (ICC). Data from 27 anesthesiology residents were collected. Overall, mean (standard deviation) case numbers were generally greater with AIMS vs RLB [649 (103) vs 583 (191); P = 0.049). Total case volumes between systems had moderate correlation (r = 0.50) and agreement (intraclass correlation coefficient [ICC], 0.42; 95% CI, 0.34 to 0.59). Bland–Altman plots showed variable agreement between AIMS and RLB data [mean (SD) bias = 66 (166) cases]. For general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery, there was a range of poor to moderate agreement (ICC, 0.23–0.57) between AIMS and RLB. For anesthesiology resident case-logging, the number of cases logged in an AIMS was higher with lower variance compared with RLBs. Anesthesia information management systems vs RLB data showed low–moderate correlation and agreement. Given the additional time and resources required for RLBs, AIMS may be a superior method for tracking cases where available.