Evaluation of Exposure-Type Stratification to Improve Poison Center Surveillance

Objective Our objective was to determine if the detection performance of current surveillance algorithms to detect call clusters is improved by stratifying by exposure category. Introduction The Centers for Disease Control and Prevention (CDC) uses the National Poison Data System (NPDS) to conduct surveillance of calls to United States poison centers (PCs) to identify clusters of reports of hazardous exposures and illnesses. NPDS stores basic information from PC calls including call type (information request only or call reporting a possible chemical exposure), exposure agent, demographics, clinical, and other variables. CDC looks for anomalies in PC data by using automated algorithms to analyze call and clinical effect volume, and by identifying calls reporting exposures to pre-specified high priority agents. Algorithms analyzing call and clinical effect volume identify anomalies when the number of calls exceeds a threshold using the historical limits method (HLM). Clinical toxicologists and epidemiologists at the American Association of Poison Control Centers and CDC apply standardized criteria to determine if the anomaly is a potential incident of public health significance (IPHS) and then notify the respective health departments and PCs as needed. Discussions with surveillance system users and analysis of past IPHS determined that call volume-based surveillance results in a high proportion of false positive anomalies. A study assessing the positive predictive value (PPV) of this approach determined that fewer than four percent of anomalies over a five-year period were IPHS. 1 A low PPV can cause an unnecessary waste of staff time and resources. We hypothesized that first stratifying call volume by exposure category would reduce the number of false positives. With the help of medical toxicologists, we created 20 toxicologically-relevant exposure categories to test this hypothesis. Methods To compare cluster detection performance between the two approaches, we used a historical testbed of hourly exposure call counts with and without initial stratification by exposure category from 10 selected PCs from Jan 1, 2006 - Jul 31, 2015. We ran the HLM for both non-stratified and stratified testbeds to estimate the monthly number of anomalies triggered (i.e., alert burden). Our target signals to assess detection performance consisted of call samples from three large public health events: the 2009 Salmonella food poisoning event from contaminated peanut butter, the 2012 Hurricane Sandy- associated carbon monoxide poisonings in New Jersey, and the 2014 Elk River contaminated water spill in West Virginia (WV). For each event, we chose 30 random calls one thousand times to obtain 1000 random sets of inject clusters. Each inject cluster was iteratively added into the testbed with and without initial stratification by exposure category. We then applied the HLM for each iteration to see if the algorithm identified the inject cluster. The sensitivity for each approach for each PC was calculated as the proportion of iterations where the algorithm identified the inject cluster. We reported median sensitivities from the ten PCs for each of the time windows of 1, 2, 4, 8, and 24 hours. Results Figure 1 summarizes results for the WV event with markers showing anomaly burden (x-axis) and sensitivity (y-axis) using the stratified ( Δ ) and the non-stratified (o) approach by different time windows (hrs). The results from the other two events are not shown but established similar patterns. Anomaly burden is shown as the estimated monthly anomaly count for each approach. For example, markers linked by the arrow show that with a 4-hour time window, the stratified approach achieves nearly perfect sensitivity with ~10 anomalies as the monthly anomaly burden while sensitivity of the non-stratified approach is below 20% with ~40 monthly anomalies. The stratified approach gave improved overall sensitivity across all time windows, and reduced anomaly burden for 1-, 2-, and 4-hour time windows. Conclusions We found a consistent detection advantage (higher sensitivity and lower anomaly burden) for the stratified vs traditional non- stratified approach for 1-, 2-, and 4-hour time windows. Further research should focus on refining the stratified approach and the specific surveillance parameters (such as time windows) that increase algorithm performance. Figure 1: Detection performance comparison: stratified vs non-stratified approach; 2014 Elk River contaminated water spill in West Virginia scenario