Background
The promotion of randomised controlled trials (RCTs) to evaluate surgical interventions was once colourfully suggested to be the 'fifth horseman of an apocalyptical surgical fundamentalism' [
1]. While the value of the RCT design has been more readily accepted by others, it has not become the default study design for the evaluation of new surgical interventions [
2‐
4]. This article will consider the challenges to successfully conducting an RCT evaluation of a surgical intervention. Surgical interventions can be defined as those which involve physically changing body tissues and organs through manual operation such as cutting, abrading, suturing or the use of lasers. It should be noted that some authors use a wider definition of surgical trials [
3] which includes trials in a surgical context, where surgery is involved but is not one of the interventions under evaluation, for example a placebo-controlled trial of ibuprofen for pain and disability relief after hip replacement surgery [
5]. While many of the issues raised below have relevance for such trials, they have been much more readily conducted and do not face the same challenges.
Since the epochal streptomycin trial in the 1940s, the RCT design has been applied widely [
6]. As understanding of the nature of study design and of the influence of bias has grown, the RCT design has for many become the 'gold standard' of evaluation, the standard against which others are compared. Through random allocation of participants, equally distributed groups are formed in a RCT. This allows for any difference between treatment groups to be confidently inferred to be due to the treatments themselves and not any other factor, known or unknown to be related to outcome. Where equipoise exists, it can be argued from an ethical point of view that every participant is guaranteed a (random) chance of receiving the best treatment. With the requirements of regulatory bodies such as the Food and Drug Administration in the USA, the RCT has been not only expected but mandated in the pharmaceutical area. Through the strong promotion of Chalmers amongst others [
7] and the evidence-based medicine movement in general, it is common for new interventions to be evaluated in an RCT context. While recognising RCTs as the 'gold standard', it has been suggested that the design has a more limited role in assessing surgery than for drug interventions [
2,
8,
9]. However, findings from alternative (non-randomised) study designs cannot be given the same confidence, due to the substantial risk of the introduction of bias. Comparisons of randomised and non-randomised studies have shown that the results can be divergent, in direction as well as magnitude [
10]. While many surgeons accept the need in principle for RCTs, they struggle to reconcile their personal involvement with their surgical experience.
Randomised comparisons of surgical interventions have been performed for many years [
7]. A number of surgical interventions have been shown to be ineffective and later discarded, in some cases following randomised comparisons [
11]. Internal mammary artery ligation was a popular surgical procedure until two small RCTs, some 20 years after the intervention was proposed, reported no benefit over placebo surgery. There is, however, some evidence that the growth in the number of RCTs being conducted in surgery has stagnated and fallen behind other clinical areas [
8,
12]. A review of one surgical journal found an increase in the number of RCTs from 1990 to 2000 [
13]. However, only 3.4% of all articles in leading surgical journals were RCTs [
14]. Furthermore, of the RCTs published in these journals during a ten-year period, less than one half of the RCTs were a randomised comparison of a surgical intervention against an alternative. The contrast with other areas can be seen by comparing audits of the evidence base for clinical practice in internal medicine and several surgical specialties. Whereas one half of interventions in internal medicine were judged to be based upon evidence from RCTs, two surgery audits reported a quarter or less [
15‐
17].
While there are genuine challenges to conducting surgical trials, concerns about the quality of surgical trials are often more general and more readily addressable. Reviews of urological and orthopaedic trauma trials, which recognised the difficulties inherent in conducting surgical trials, found poor compliance with the consort statement for reporting RCTs, echoing the findings of an earlier review, and may reflect generally poorer trial methodology [
18‐
20]. A lack of understanding about RCTs in surgical communities and the need for better epidemiological and statistical training of surgeons have been noted [
3,
21]. Historical and ethos-related reasons as to why the surgical community has been slow to change have been suggested [
4].
The difficulties of conducting an RCT of a surgical intervention have long been recognised [
21‐
25]. Surgical RCTs may have suffered as understanding of bias has grown, as the maximum safeguards against bias are difficult to achieve, particularly compared with pure pharmaceutical treatments. A review of surgical operative questions in the gastrointestinal specialty suggested that while there was, at least in principle, substantially more scope for RCTs, there were many questions where an RCT was judged not to be feasible [
26]. A number of barriers to conducting surgical RCTs have been suggested [
2,
3,
9,
27,
28]. Stirrat and colleagues produced a helpful summary of the issues involved and some suggestions to overcome them [
9]. More recently, two other groups have done similarly [
3,
28]. An overview of the issues and possible solutions will be reviewed under three headings: the timing of the evaluation, defining the research question and trial design issues. It should be noted that a systematic review and corresponding meta-analysis can play an invaluable role in trial design.
Broadly, surgical trials can be classified into three generic types: exploratory, explanatory and pragmatic trials. Exploratory trials allow early assessment of new interventions. An explanatory trial seeks to assess whether the intervention can work under favourable conditions. In contrast, a pragmatic trial seeks to inform clinical decision making by evaluating an intervention in a realistic clinical setting. General characteristics of the three types are shown in Table
1, though in practice a trial may have some characteristics that reflect more than one type or which defy easy categorisation. In particular, a continuum between explanatory and pragmatic trials exists [
29]. It should be noted that the evaluation of surgery, by its complex nature, naturally fits a more pragmatic framework. By adopting a generally pragmatic approach, the impact of many of the trial design issues is diluted. For example, legitimate variations in surgical/centre practice or some patients not receiving surgery as initially planned can be incorporated into the evaluation, as opposed to attempting to preclude them from occurring. Such an approach evaluates a realistic management strategy involving the surgical intervention. As a consequence, the findings from a trial with this approach will be more widely applicable.
Table 1
Characteristics of three generic forms of surgical trials
Aim of evaluation | To explore the impact of the intervention | To assess whether the intervention is efficacious | To assess whether the intervention is effective |
Patient population | Initially those presumed to be most likely to benefit, though later modification is allowed | Narrow inclusion criteria of patients expected to be most suited to treatment | Broad inclusion criteria reflecting variations in clinical practice between centres |
Surgical setting | Surgeon(s) with substantial generic surgical expertise | Surgeon(s) with expertise in the intervention under evaluation | Surgeons from multiple centres representing different levels of expertise |
Intervention definition | Freedom to develop and refine | Tight definition and strictly controlled | Broader definition incorporating variations which reflect clinical practice |
Outcomes of main focus | Surgical process and short-term clinical outcomes | Short-term (sometimes surrogate) clinical outcomes | Longer-term clinical and patient-reported outcomes (such as quality of life measures) |
Competing interests
The author declares that he has no competing interests.
Authors' contributions
JC wrote the manuscript and has read and approved the final version.