Causal interpretation of correlational studies - Analysis of medical news on the website of the official journal for German physicians

Background Media frequently draws inappropriate causal statements from observational studies. We analyzed the reporting of study results in the Medical News section of the German medical journal Deutsches Arzteblatt (DA). Methods Study design: Retrospective quantitative content analysis of randomly selected news reports and related original journal articles and press releases. A medical news report was selected if headlines comprised at least two linked variables. Two raters independently categorized the headline and text of each news report, conclusions of the abstract and full text of the related journal article, and the press release. The assessment instrument comprised five categories from ‘neutral’ to ‘unconditionally causal’. Outcome measures: degree of matching between 1) news headlines and conclusions of the journal article, 2) headlines and text of news reports, 3) text and conclusions, and 4) headlines and press releases. We analyzed whether news headlines rated as unconditionally causal based on randomized controlled trials (RCTs). Results One-thousand eighty-seven medical news reports were published between April 2015 and May 2016. The final random sample comprised 176 news reports and 100 related press releases. Degree of matching: 1) 45% (79/176) for news headlines and journal article conclusions, 2) 55% (97/176) for headlines and text, 3) 53% (93/176) for text and conclusions, and 4) 41% (41/100) for headlines and press releases. Exaggerations were found in 45% (80/176) of the headlines compared to the conclusions of the related journal article. Sixty-five of 137 unconditionally causal statements of the news headlines were phrased more weakly in the subsequent news text body. Only 52 of 137 headlines (38%) categorized as unconditionally causal reported RCTs. Conclusion Reporting of medical news in the DA medical journal is misleading. Most headlines that imply causal associations were not based on RCTs. Medical journalists should follow standards of reporting scientific study results.