Cochrane risk of bias tool was used inadequately in the majority of non-Cochrane systematic reviews.

Abstract Objective To analyze how many non-Cochrane systematic reviews (NCSRs) used Cochrane’s risk of bias (RoB) tool, domains they used, and whether judgments and comments about risk of bias were in line with Cochrane Handbook. Methods This was a methodological (research-on-research) study. We retrieved NCSRs from PubMed, extracted information about methods used for RoB assessment, and if they used 2011 Cochrane RoB tool, we analyzed their RoB methods and compared them with Cochrane Handbook guidance. Results We included 508 NCSRs; 431 (85%) reported they analyzed RoB, and 269 (53%) used Cochrane RoB tool. Only 16 of those 269 (5.9%) reported both a judgment and a supporting comment in the Cochrane RoB table in the manuscript (N=4) or in a supplementary file (N=12). Fifteen reviews, with 158 included trials, used judgments low/high/unclear; 41% of analyzed available judgments were inadequate, either because judgment was not in line with comment, or comment was missing. Conclusions The majority of non-Cochrane systematic reviews use Cochrane RoB tool to assess risk of bias, but the majority of them reported it incompletely, with high prevalence of inadequate judgments. Authors, editors and peer-reviewers should make an effort to improve completeness and adequacy of Cochrane RoB assessment in non-Cochrane reviews.