On Monitoring Outcomes of Medical Providers

An issue of substantial importance is the monitoring and improvement of health care facilities such as hospitals, nursing homes, dialysis units or surgical wards. In addressing this, there is a need for appropriate methods for monitoring health outcomes. On the one hand, statistical tools are needed to aid centers in instituting and evaluating quality improvement programs and, on the other hand, to aid overseers and payers in identifying and addressing sub-standard performance. In the latter case, the aim is to identify situations where there is evidence that the facility’s outcomes are outside of normal expectations; such facilities would be flagged and perhaps audited for potential difficulties or censured in some way. Methods in use are based on models where the center effects are taken as fixed or random. We take a systematic approach to assessing the merits of these methods when the patient outcome of interest arises from a linear model. We argue that methods based on fixed effects are more appropriate for the task of identifying extreme outcomes by providing better accuracy when the true facility effect is far from that of the average facility and avoiding confounding issues that arise in the random effects models when the patient risks are correlated with facility effects. Finally, we consider approaches to flagging that are based on the Z-statistics arising from the fixed effects model, but which account in a robust way for the intrinsic variation between facilities as contemplated in the standard random effects model. We provide an illustration in monitoring survival outcomes of dialysis facilities in the US.