Consistently high agreement between independent raters of Niemann-Pick type C1 clinical severity scale in phase 2/3 trial

Abstract Background Niemann-Pick Disease, type C1 (NPC1) is a rare neurodegenerative genetic disorder characterized by impaired intracellular transport of cholesterol and other lipids. The Niemann-Pick Disease, type C1 Severity Scale (NPC-SS) was developed to quantify neurological progression of NPC; it is used to monitor the natural history of disease progression and assess response to treatment. Objective The objective of the study was to examine the inter-rater reliability of the NPC-SS in a Phase 2/3 clinical trial. Methods Study data were from a multicenter, prospective, randomized, double-blind, trial of adrabetadex in 56 subjects with NPC1. Clinical data recorded at each study site were distributed to two independent blinded central raters to generate a severity score. A composite four-item score was utilized as the primary clinical study endpoint, while a five-item focused score has been utilized in other NPC1 trials. Inter-rater reliability was assessed using two-way mixed models for instrument stability, Cohen's kappa, weighted kappa, and percent agreement for the four-item and five-item scores. Results The frequency distribution and mean (SD) of the NPC-SS domain assessments by the raters were almost identical. Evaluation at the patient visit level showed wide variability between visits, however, weighted kappa calculation provided a lower variability between visits. The average kappa coefficients ranged between 0.69 to 0.89, indicating good to very good agreement between raters. Conclusion These results support the NPC-SS, including derived 4- and 5-item composite scores, as reliable measures for use in a clinical trial setting.