The fragility of statistically significant findings from randomised controlled trials in the urological literature

OBJECTIVES: To use the Fragility Index to evaluate the robustness of statistically significant findings from urological randomised controlled trials (RCTs). MATERIALS AND METHODS: The 'Fragility Index' is defined as the minimum number of patients in one arm of a trial whose status would have to change from 'event' to 'non-event', such that a statistically significant result becomes non-significant. We identified all RCTs published in four major urology journals between 2011 and 2015, and we determined the Fragility Index values for those trials reporting statistically significant results of dichotomous outcomes using the Fisher's exact test. RESULTS: In all, 332 RCTs were identified, and 41 studies met the inclusion criteria. The median (interquartile range) Fragility Index was 3 (1, 4.5), indicating that an addition of only three alternate events to one arm of a typical trial would have eliminated its statistical significance. In 27/40 cases (67.5% of cases), the number of patients lost to follow-up was larger than its Fragility Index. CONCLUSIONS: The results of urology RCTs that study dichotomous outcomes and report statistically significant differences between groups are sometimes fragile and depend on few events. Urologists should interpret these RCTs cautiously, particularly when the number of participants lost to follow-up exceeds the Fragility Index. Routine reporting of Fragility Index values alongside P values may provide additional guidance about the robustness of statistically significant findings.