Original Research Report Kappa statistics to measure interrater and intrarater agreement for 1790 cervical biopsy specimens among twelve pathologists: Qualitative histopathologic analysis and methodologic issues

Background. As part of a program project to evaluate emerging optical technologies for cervical neoplasia, we performed fluorescence and reflectance spectroscopic examinations of patients with abnormal Papanicolaou smears. Biopsy specimens were taken from each area and measured optically, and study pathologists performed qualitative histopathologic readings. Several methodologic issues arose in this analysis: (1) the interpathologist and intrapathologist agreement between institutions for the 1790 biopsy specimens; (2) the interinstitutional agreement among the two institutions conducting the trials on 117 randomly chosen biopsy specimens; (3) the interinstitutional agreement among the two institutions and a third expert gynecologic pathologist to ensure the expert readings were comparable to those outside both institutions on 117 randomly chosen biopsy specimens; and (4) an additional three reviews of the 106 difficult biopsy specimens by all three institutions.