Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation

Comparisons between observed data and model simulations represent a critical component for establishing confidence in population physiologically-based pharmacokinetic (Pop-PBPK) models. Numerical predictive checks (NPC) that assess the proportion of observed data that correspond to Pop-PBPK model prediction intervals (PIs) are frequently used to qualify such models. We evaluated the effects of three components on the performance of NPC for qualifying Pop-PBPK model concentration–time predictions: (1) correlations (multiple samples per subject), (2) residual error, and (3) discrepancies in the distribution of demographics between observed and virtual subjects. Using a simulation-based study design, we artificially created observed pharmacokinetic (PK) datasets and compared them to model simulations generated under the same Pop-PBPK model. Observed datasets containing uncorrelated and correlated observations (± residual error) were formulated using different random-sampling techniques. In addition, we created observed datasets where the distribution of subject body weights differed from that of the virtual population used to generate model simulations. NPC for each observed dataset were computed based on the Pop-PBPK model’s 90% PI. NPC were associated with inflated type-I-error rates (> 0.10) for observed datasets that contained correlated observations, residual error, or both. Additionally, the performance of NPC were sensitive to the demographic distribution of observed subjects. Acceptable use of NPC was only demonstrated for the idealistic case where observed data were uncorrelated, free of residual error, and the demographic distribution of virtual subjects matched that of observed subjects. Considering the restricted applicability of NPC for Pop-PBPK model evaluation, their use in this context should be interpreted with caution.