Benchmarking the Accuracy of Polygenic Risk Scores and their Generative Methods

The estimate of an individual9s genetic susceptibility to a disease can provide critical information when setting screening schedules, prescribing medication and making lifestyle change recommendations. The polygenic risk score is the predominant susceptibility metric, with many methods available to describe its construction. However, these methods have never been comprehensively compared or the predictive value of their outputs systematically assessed, leaving the clinical utility of polygenic risk scores uncertain. This study aims to resolve this uncertainty by deeply comparing the maximum possible, currently available, 15 polygenic risk scoring methods to 25 well-powered, UK Biobank derived, disease phenotypes. Our results show that simpler methods, which employ heuristics, bested complex, methods, which predominately model linkage disequilibrium. Accuracy was assessed with AUC improvement, the difference in area under the receiver operating curve generated by two logistic regression models, both of which share the covariates of age, sex, and principal components, while the second model also contains the polygenic risk score. To better determine the maximal utility of polygenic risk scores, straightforward score ensembles, which bested all methods across all traits in the training data-set, were evaluated in the withheld data-set. The score ensembles revealed that the accuracy gained by considering a polygenic risk score varied greatly, with AUC improvement greater than 0.05 for 9 traits. Many additional analyses revealed widespread pleiotropy across scores, large variations between assessment statistics, peculiar patterns amongst phenotype definitions, and wide ranges in the optimal number of variants used for scoring. If these many variable aspects of score creation can be well controlled and documented, simple methods can easily generate polygenic risk score that well predict an individual9s future liability of certain diseases.