Where is the supporting evidence for treating mild to moderate chronic obstructive pulmonary disease exacerbations with antibiotics? A systematic review

The Helsinki Declaration emphasises the great importance of conducting experimental studies for medical progress. However, it also states that one should be very careful before embarking on randomised trials with placebo controls because research participants have a right to the best available treatment [1]. Worries about the 'unethical use of placebo' continue [2, 3]. However, what about the reverse scenario? Might there be cases where experimental treatment did not show superiority over placebo but where the placebo controls were abandoned nevertheless, thus exposing patients to adverse effects and society to healthcare expenditures not offset by any beneficial effects?

There is a large number randomised trials comparing different antibiotics (without placebo control) for the treatment of exacerbations of chronic obstructive pulmonary disease (COPD). Although it may seem plausible that antibiotics are beneficial in about 50% of the COPD patients in whom bacteria are the cause of the exacerbation [4], there is evidence indicating that antibiotics have a short-term effect only in COPD patients with severe exacerbations but not in mild to moderate exacerbations [5]. Although head-to-head comparisons of different treatment options available in clinical practice can be very useful [6], an underlying assumption of such trials is that the treatments are effective compared with a placebo [7‐9]. While conducting a series of systematic reviews of treatments in COPD, we gained the impression that the evaluation of antibiotics for COPD exacerbations had moved to head-to-head comparisons of different antibiotics with very little research in placebo-controlled trials. We set out to test this hypothesis in a more systematic way. We counted the number of randomised trials of antibiotic treatment for mild to moderate exacerbations in COPD patients published in the last 50 years, determined whether they used a placebo control or conducted head-to-head comparisons, and contrasted the number of patients studied with the results of a cumulative meta-analysis, using data from a recently published systematic review.

Our historical analysis showed that many head-to-head trials of antibiotics for mild to moderate COPD exacerbations have been conducted although evidence from randomised placebo-controlled trials never showed that antibiotics were effective at all. So, in this case, the evaluation of antibiotics did not follow the general principle that placebo-controlled trials must have shown that the treatment is better than the placebo or a sham procedure, before head-to-head trials are to be conducted [9].

Our study has some strengths and limitations. We used a comprehensive literature search to identify trials. The selection of a widely prescribed drug for a major disease makes our 'case report' meaningful, because we could base our historical analysis on a large number of trials. A limitation of systematic reviews in COPD is the evolution of definitions and classifications of COPD over the years. This raises uncertainty about the nature of the study populations included in trials published before 1995 or even before 2000. Many trials included patients with 'chronic bronchitis' who did not present evidence of chronic airflow obstruction or poor reversibility of airflow obstruction.

It should be emphasised that the results of this study apply to antibiotic treatment for mild to moderate COPD exacerbations. In patients with severe exacerbations, antibiotics show a strong effect not only on treatment failure but also on mortality [5]. Our analysis of placebo-controlled trials differs to some extent to that of the Cochrane review [26] because we did not include a study [27] that was not a randomised but, in our opinion, a matched controlled study. However, we included the trial by Sachs et al [24] missed by the Cochrane review and received the original data from the authors. However, the Cochrane review [26] also showed that antibiotics have a significant effect on treatment failure in hospitalised patients but not in outpatients.

We have focused in our analysis on one major treatment for COPD patients. It remains unclear if head-to-head trials are conducted frequently in other areas without an evidence base from placebo-controlled trials. It is conceivable that this example of antibiotics is just the tip of the iceberg. A recent systematic review summarised placebo-controlled and head-to-head trials of exercise for low back pain [28]. Head-to-head trials were conducted over the same period of time as the placebo-controlled trials. The same applies to trials evaluating antidepressants for elderly people [29, 30] or antibiotics for acute otitis media in children [31, 32]. For these widely prescribed treatments for common conditions, it remains as yet uncertain whether the conduct of head-to-head trials was scientifically and ethically justified.

Further studies are warranted to estimate the magnitude of the phenomenon that conduct of head-to-head trials is not solidly based on evidence from placebo-controlled trials. It is unclear why investigators rarely refer to existing systematic reviews to justify their trials scientifically and ethically, although access to such systematic reviews has become so easy [33]. We also need to find out what triggers investigators to conduct head-to-head trials too early. The reports on head-to-head trials on antibiotics shed some light on this tendency. High plausibility that an intervention works may be one of the most important reasons. For antibiotics, evidence is available about their in-vitro activity against the bacteria commonly found in COPD exacerbations. Intuitively, one may extrapolate this to the clinical situation. Physicians are also confirmed in their habit of antibiotic prescription because most outpatients with mild to moderate exacerbations recover within 2 weeks. Yet they do not seem to be aware of the 'natural' recovery rate (that is, without antibiotics) in these patients, which is 80% or more and equal to that in patients receiving placebo [5].

In general, the pharmaceutical industry tends to favour placebo-controlled trials in order to show large effects and to avoid direct comparison with competitors. In situations like this, however, where comparisons with placebo do not favour a drug, the pharmaceutical industry might be more interested in head-to-head trials. Once a treatment, such as antibiotics for COPD exacerbations, is established in clinical practice an attractive market is available. If a company wants to enter this market it needs to provide a trial showing clinical non-inferiority of a new antibiotic and some advantages in terms of adverse effects or costs. Hence, chance aside, even a new antibiotic lacking specific activity will not be inferior to any established antibiotic in COPD outpatients. This will make it relatively easy to get approval from regulatory agencies, as long as the drug is safe.

As mentioned above, placebo-controlled trials continue to be conducted even if the effectiveness of a drug has been definitively established. Fergusson et al [2] provided an impressive example on that problem recently. They identified all comparisons of aprotinin, which limits perioperative bleeding and reduces the need for blood transfusions, with a placebo. After 12 placebo-controlled trials, the effect estimates stabilised at a cumulative OR of 0.25 for the need for blood transfusions favouring aprotinin. Nevertheless, another 52 randomised placebo-controlled trials were conducted although that was ethically not justifiable [3]. These trials cited previous randomised controlled trials very poorly so that readers were not aware that the question had been answered definitively by previous trials.

The unjustified conduct of placebo-controlled or head-to-head trials can be prevented if funding bodies and ethical committees ask investigators to present a systematic review that justifies a new trial. Such systematic reviews are often available already, for example, published by the Cochrane Collaboration. If not, investigators need to carry out a systematic review themselves and publish it regardless of the findings in order to stimulate a public discussion about the need for new trials. Uncertainty about the effectiveness of a treatment should be discussed openly because it may also encourage patients and physicians to participate in a new trial.

Our historical analysis showed that the evaluation of antibiotics for mild to moderate COPD exacerbations has been far from optimal. Head-to-head trials have been conducted without any supporting evidence from placebo-controlled trials. This phenomenon raises important scientific and ethical concerns and more studies are warranted to find out its prevalence, mechanisms and consequences in medicine. In the meantime, public funding bodies and ethical committees in particular should encourage the careful conduct or study of pertinent systematic reviews before supporting the conduct of new trials.