Weighted kappa statistics for measuring the divergence of rapid section and definitive diagnosis

Weighted Kappa statistics measure or quantify the disagreement between diagnoses given by different observers or which were established in different time periods. The study reports on 2226 frozen-section examinations performed in the period of 1986 to 1995 at the Pathology Institute of Neuchâtel, Switzerland. The two cohorts formed by the "first diagnosis" and the "definitive diagnosis" are classified in four SNOMED Morphology categories and, in case of disagreement, the weight of the error is computed according to an arbitrary scheme of 2 to 20 points. Of the diagnoses 2093 (= 94.0%) were completely identical. The kappa of the cohort is 0.91 (+/- 0.008), the weighted kappa 0.95 (+/- 0.005). Analyses of different parameters show a constantly higher value for weighted kappa than for nonweighted kappa. The 133 cases with disagreements reveal, among others, that frozen-section examination of the thyroid gland has a high risk of error, whereas biopsies of the mammary gland give the best results (rare and practically no significant disagreement). Subsequent analyses investigate diagnostic quality according to the chronological evolution and the performance of the responsible pathologists.