Defining and assessing the reproducibility of Crohn's disease endoscopic lesions: a Delphi-like method from the GETAID.

Background and aims Defining and assessing the reproducibility of Crohn's disease (CD) endoscopic lesions is essential in assessing endoscopic healing. Methods Twelve endoscopic CD experts from the GETAID defined aphthoid erosions (AE), superficial ulcerations (SU), deep ulcerations (DU), stenosis and fistulas according to a Delphi-like method. Thirty different GETAID physicians declared if they found acceptable each definition. Intra- and inter-observer agreements were investigated using 100 videos with one tagged specific lesion (AE, SU, DU or sham lesion) read by 15 independent endoscopists at baseline and one month later in a randomized order. Video quality was determined by an external reader. According to kappa estimate (κ ±standard error), intra or inter-observer agreement was qualified as "moderate" (0.4-0.6), "substantial" (0.6-0.8) or "almost perfect" (0.8-1.0). Results Among 30 different experts, 83% to 97% found acceptable the definitions retrieved from the Delphi-like method. Intra-observer κ was 0.717 (± 0.019) for SU, 0.681 (±0.027) for AE, 0.856 (±0.014) for DU showing "substantial" agreement. It was 0.801 (±0.016) for any ulceration (DU or SU). There was a high variability across readers from "moderate" to "almost perfect" agreement. Inter-observer κ was 0.548 (±0.042) for SU, 0.554 (±0.028) for AE 0.694 (±0.041) for DU and 0.705 (±0.042) for any ulceration. Inter-observer agreement increased when reading the 53 high-quality videos: 0.787 (±0.064)(p=0.001), 0.607 (±0.043)(p=0.001) and 0.782 (±0.064)(p=0.001) for DU, AE and any ulceration, respectively. Conclusion Despite variable intra-agreement level across readers, the GETAID definitions for CD endoscopic lesions provided "substantial" inter-observer agreements, especially in case of high-quality videos.