Abstract P2-01-06: How much agreement can we expect on BI-RADS mammographic findings? Observer agreement among 10 expert mammographers

Purpose: To determine the agreement between expert readers on mammographic findings and calcification patterns. Materials and Methods: Ten academic radiologists from 5 centers reviewed 250 de-identified mammographic cases without prior exams which were previously assessed as BI-RADS 4 or 5 with subsequent pathologic diagnosis by percutaneous or surgical biopsy. For benign cases diagnosed by percutaneous biopsy, 1 year of benign or negative imaging follow-up was required. Using standardized forms, each radiologist assessed the presence of any suspicious mammographic findings (microcalcifications, asymmetry (1-vew), focal asymmetry (2-view), architectural distortion), and the morphology (none, round/punctate, amorphous, coarse heterogeneous, fine pleomorphic, fine linear branching) and distribution (none, diffuse, regional, grouped, linear, segmental) of any identified microcalcifications. Agreement between radiologists for presence/absence of findings, morphology, and distribution of calcifications was determined by calculating the Kappa (k) coefficient with 95% confidence interval (95% CI). The kappa coefficient proposed strength of agreement is ≤0 = poor, .01–.20 = slight, .21–.40 = fair, .41–.60 = moderate, .61–.80 = substantial, and .81–1 = almost perfect, as established by Landis and Koch.1 Results: Of the 250 lesions, 156 (62%) were benign and 94 (38%) malignant. Agreement among the 10 expert readers was strongest for recognizing the presence/absence of calcifications (k = 0.82, 95% CI: 0.80-84), “almost perfect”). There was substantial agreement among the readers for the identification of a mass (k = 0.67, 95% CI: 0.66-69), whereas agreement was fair for the presence of a focal (2-view) asymmetry (k = 0.21, 95% CI: 0.1900.23) or architectural distortion (k = 0.28, 95%CI: 0.26-0.30). Agreement for asymmetries (1-view) was slight (k = 0.09, 95%CI: 0.08-0.11). Among the 6 categories of microcalcification distribution and morphology, reader agreement was moderate (distribution k = 0.60, 95%CI:0.59-0.61; morphology k = 0.51, 95%CI: 0.50-0.52). Conclusion: When asked to characterize suspicious mammographic findings, this sampling of 10 expert academic breast imagers across 5 centers revealed varying strength of agreement for different findings, ranging from slight to almost perfect. Strongest agreement (“almost perfect”) was found for identifying the presence or absence of microcalcifications, although agreement drops to moderate when readers are asked to specify microcalcification morphology and distribution. 1 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics.1977;33:159–174. Citation Information: Cancer Res 2013;73(24 Suppl): Abstract nr P2-01-06.