Evaluating Surgical Complexity of Endoscopic Hysterectomy: An Inter-Rater Agreement Study for Novel Scoring Tool

Study Objective Evaluate inter-rater reliability of a novel scoring tool for surgical complexity assessment of endoscopic hysterectomy. Design Validation study. Setting Academic medical center. Patients or Participants 11 academic OB/GYNS with varying years of post-training practice and surgical volumes Interventions Application of a novel scoring tool to evaluate surgical complexity of 150 sets of surgical images taken in a standardized fashion (global pelvis, anterior cul-de-sac, posterior cul-de-sac, right adnexa, left adnexa). Surgical complexity was staged on a scale of I-IV (low to high complexity). Raters were asked to assess uterine size, number and location of fibroids, adnexal and uterine mobility, need for ureterolysis, and presence of endometriosis or adhesions. Measurements and Main Results Number of post-residency years in practice for participating surgeons ranged from 2 – 15, with an average of 8 years. 8 (72.73%) OB/GYNs had completed a fellowship in minimally invasive gynecologic surgery. 6 (54.55%) reported annual volume of hysterectomy >50 cases. Raters reported 95.4% of images were satisfactory for assessment. Of the 150 sets of images, most were found to be Stage I-II complexity (Stage I: 46.4%, Stage II: 30.6%, Stage III: 16.4%, Stage IV: 6.6%). Level of inter-rater agreement regarding Stage I-II vs. III-IV complexity was moderate (k=0.493, CI [0.43-0.56]). Moderate inter-rater agreement persisted between surgeon raters with annual hysterectomy volume > 50 (k=0.484, CI [0.42-0.56]) as well as between surgeon raters with fellowship experience (k=0.498, CI [0.434-0.57]). Conclusion Current methods for assessment of surgical complexity for clinical and research purposes rely on seemingly objective markers like uterine weight, estimated blood loss, and operative time, which can be limited or flawed. In contrast, this novel scoring tool has rich and comprehensive evaluation capabilities and achieved moderate inter-rater agreement. The complexity scoring tool can be used in conjunction with or instead of traditional methods for assessment of surgical complexity of endoscopic hysterectomy.