Usability and sensitivity of the risk of bias assessment tool for randomized controlled trials of pharmacist interventions

Background The Cochrane collaboration risk of bias assessment (RoB) tool is used in several fields to evaluate the methodological quality of studies. Its strengths and challenges are discussed. Objective To assess the sensitivity of the RoB tool in studies of pharmacist interventions. Setting DEPICT database was used to pool randomized controlled trials (RCTs) of complex interventions. Method A Guide for RoB Judgment in Pharmacy Services was created to help in the interpretation and judgment of bias criteria. The evaluation of bias (low, unclear, high risk) was performed by RCT. Sensitivity analyses were performed to assess the influence of different interpretations of eight elements of judgment in the RoB tool. Paired analysis and estimations of the effect size (95% confidence interval) of the criteria modifications compared to the original analyses were calculated. Main outcome measure Changes in the interpretations of judgment in the RoB tool. Results Overall, 8.3, 45.4, and 46.3% of the studies were determined to have low, unclear, and high risk of bias, respectively. High risk of bias was caused by attrition and detection domains. The number of studies classified with high risk of bias significantly increased for five of the eight interpretations, while unclear risk of bias increased for three interpretations (with a negligible effect size in all of them). Lack of blinding, loss of participants, and the use of subjective and self-reported outcomes were the main elements resulting in high risk of bias. Conclusion The RoB tool is useful for evaluating RCTs of pharmacist interventions if adapted criteria for judgment are used. Ignoring these adjustments produces a floor-effect with studies classified with high risk of bias.