Tips for Busy Clinicians to Gauge Conclusions from Clinical Trials

Excerpt

One of the problems clinicians face is knowing how much to trust conclusions drawn from the results of clinical trials. Trialists design studies to maximize the chance that they will detect differences in outcomes and label those differences as statistically significant. To achieve statistical significance, the difference either needs to be sufficiently large, the sample size needs to be large, or the groups compared need to have sufficiently small variation. Trialists can increase the chance of finding sufficiently large differences by selecting populations at high risk of the outcome and can decrease variation by choosing relatively homogenous populations. Clinical research is very expensive, both for patients and trialists. Because of ethical, economic, and pragmatic reasons, studies are powered to be barely statistically significant, based on estimates of the size of differences that are deemed clinically important (and to some extent expected) and variation. Design issues in the planning stages of trials such as desired power (and thus the sample size required to achieve that power) are fundamentally economic. The incremental benefit derived from not missing a true difference in outcomes (measured by estimating the value of health benefits for patients who will use an effective therapy) is balanced against the incremental cost of increasing the trial power and sample size (to look for small differences).¹ Missing a true difference means either people will not be offered effective therapies or the trial needs to be repeated. …