Multi-Institutional Evaluation of Inter-rater Agreement of Variant Classification Based on the 2017 AMP, ASCO and CAP Standards and Guidelines for the Interpretation and Reporting of Sequence Variants in Cancer

This multi-institutional study was undertaken to evaluate interrater reliability of the 2017 Association for Molecular Pathology/American Society of Clinical Oncology/College of American Pathologists guidelines for interpretation and reporting of oncology sequence variants and to assess current practices and perceptions surrounding these guidelines. Fifty-one variants were distributed to 20 participants from 10 institutions for classification using the new guidelines. Agreement was assessed using chance-corrected agreement (Cohen κ). Overall chance-corrected agreement (κ) was 0.35. To evaluate if data sharing could help resolve disagreements, a summary of variant classifications and additional information about each variant were distributed to all participants. Chance-corrected agreement improved to 0.7 after the original classifications were revised. Participants were invited to take a web-based survey regarding their perceptions of the guidelines. Only 20% (n = 3) of the survey respondents had prior experience with the guidelines in clinical practice. The main perceived barriers to guideline implementation included the complexity of the guidelines, discordance between clinical actionability and pathobiologic relevance, lack of familiarity with the new classifications, and uncertainty when applying criteria to potential germline variants. This study demonstrates noteworthy discordances between pathologists for variant classification in solid tumors when using the 2017 Association for Molecular Pathology/American Society of Clinical Oncology/College of American Pathologists guidelines. These findings highlight potential areas for clarification/refinement before mainstream clinical adoption.