Introduction
Challenges for outcome measurement in RCTs in surgery
Choosing relevant outcomes to suit the trial design and purpose
Trial | Aim | Primary outcome | Secondary short-term outcomes | Secondary long-term outcomes | Comments | ||||
---|---|---|---|---|---|---|---|---|---|
Clinical | PRO | Clinical | PRO | Process | Economic | ||||
To investigate post-operative drainage in the management of chronic subdural haematoma after burr-hole evacuation | Rate of reoperation to treat recurrent chronic subdural haematoma | 30-Day mortality Hospital stay (days) MRS and GCS at discharge Gross focal neurological deficit at discharge | N/A | Mortality at 0.5 years MRS and GCS at 0.5 years Gross focal neurological deficit at 0.5 years Mobility status at 0.5 years | N/A | N/A | N/A | Single-centre RCT with outcomes focusing on clinical endpoints. This limits generalisability | |
To evaluate the technical and oncological safety and efficacy of laparoscopically assisted surgery for colorectal cancer | Several were defined: Resection margins Proportion of Dukes’ C2 tumours In-hospital mortality 3-year overall survival, disease-free survival and local recurrence rates | 30-Day and 3-month complication rates Transfusion requirements | HRQL at 2 weeks and 3 months | 3- and 5-year distant recurrence rates 3- and 5-year wound/port site recurrence rates 5-year overall survival 5-year disease-free survival 5-year local recurrence | HRQL at 0.5, 1.5 and 3 years | N/A | N/A | This very pragmatic trial had a sample size selected to provide an overall picture of several short- and long-term outcomes. It was not powered for specific endpoints | |
To assess whether five surgical caesarean techniques were associated with improved outcomes for women and babies | A composite outcome: Death Maternal infectious morbidity* Further operative procedures Blood transfusion up to 6-week follow-up visit | Individual components of composite outcome Pain Interventions for post-partum haemorrhage Still birth Other severe maternal morbidity Apgar score <3 at 5 min Laceration of baby during caesarean section Death of baby at 6 weeks Operation length Hospital stay Admission to ITU Hospital readmission within 6 weeks | Assessed in follow-up study [62]: Reproductive status, subsequent pregnancies, death or serious morbidity of child at 3 years | Women’s health and mortality (requested at face to face interview) at 3 years | N/A | N/A | Multicentre, multinational pragmatic RCT design. Authors state outcomes were specifically selected to provide guidance for clinical decision-making and composite components were chosen based on primary objective of the trial | ||
To compare the long-term effects of endovascular and open repair of large aneurysms | All-cause mortality Aneurysm-related mortality within 30 days | Graft-related complications Graft-related interventions 30-day operative mortality | HRQL at 1 and 3 months | Graft-related complications Graft-related interventions Adverse events Renal function (up to 8 years) | HRQL at 1 year | Cost-effectiveness based on QALY (EQ-5D) Resource use annually up to 8 years | Multicentre pragmatic trial designed to provide long-term data, although HRQL outcomes not measured for duration of trial | ||
To compare short-term outcomes of laparoscopic and open resection of colorectal cancer within an enhanced recovery programme | Hospital stay | Convalescent hospital stay Readmission hospital stay Major morbidity†
Requirement for opioid analgesia Antiemetic administration Performance indicators for mobility/strength at 2 and 12 days, 6 and 12 weeks Sleep and oxygen saturation at 2, 6 12weeks | HRQL at 2 and 6 weeks Resource use at 2 weeks and 3 months | N/A | N/A | N/A | N/A | Single-centre RCT with small sample size designed to assess short-term outcomes only | |
To evaluate laparoscopic fundoplication surgery with medical management for the treatment of (GORD) | Validated PRO with the REFLUX questionnaire | SF36, EQ-5day & REFLUX HRQL score at 3 months Surgical complications | Mortality Use of anti reflux drugs at 3 months and 1 year | SF36 and EQ-5D score at 1, 2, 3, 4 and 5 year REFLUX HRQL and REFLUX symptom score at 2, 3, 4 and 5 years | N/A | N/A | Multicentre pragmatic RCT with PRO primary outcome. N.B. REFLUX questionnaire has several scales to form an overall score | ||
Comparison of CABG with PCI for the treatment of patients with left main coronary disease three-vessel disease | Composite: rate of MACCE at 1 year (all-cause mortality, stroke, myocardial infarction, repeatrevascularisation) | Rate of MACCE at 1 month | HRQL at 1-month post procedure | Rate of MACCE at 0.5, 3 and 5 years Rates of composite components Rates of stent thrombosis or graft occlusion | HRQL at 0.5, 1, 3 and 5-year post procedure | N/A | Cost and cost-effectiveness at 1, 3 and 5 years | Multicentre, multinational pragmatic RCT. Practical interpretation limited as components of composite outcome were not weighted to reflect their importance or relative frequency |
Selecting and reporting appropriate outcome measures for RCTs in surgery
Assessing and measuring outcomes with minimal risk of bias
Trial | Interventions | Primary outcome | Type of outcome | Personnel blinded | Methods used for blinding |
---|---|---|---|---|---|
Raviele et al. 2004 [52] | Permanent pacing versus placebo for recurrent tilt-induced vasovagal syncope | Recurrence rate of syncope at 1 year assessed by patient diaries | PRO | Patients Surgeons Clinical staff | Pacemakers implanted into both treatment arms using the same method; a single person in each centre was responsible for the programming of the pacemaker to ON in active mode with rate drop response, or OFF in the inactive mode. |
Sung et al. 2003 [53] | Endoscopic treatment versus irrigation alone for non-bleeding vessels or adherent clot in gastroduodenal ulceration | Recurrence of bleeding before discharge and at 30 days | Independent adjudication panel | Patients Clinical staff Outcome assessors | Intervention group underwent routine endoscopic treatment for bleeding ulcer including irrigation, suction, heater probe, or mini-snare. The control group received irrigation of ulcer base but no manipulation with heater probe, snare or suction; post-operative care by blinded clinical team; endoscopist not involved in post-operative care or outcome assessment; criteria for re-bleed pre-defined and assessed by blinded panel. |
Moseley et al. 2002 [49] | Arthroscopic lavage and debridement versus arthroscopic lavage only versus sham procedure for osteoarthritis of the knee | Knee pain at 24 months assessed using Knee-Specific Pain Scale (developed for this trial) | PRO | Patients Nurses Clinical staff | Patients allocated to lavage and debridement or lavage only underwent general anaesthesia and endotracheal intubation. Patients in the control arm underwent short-acting intravenous sedation and opioid analgesia with spontaneously breathed oxygen-enriched air. Patients allocated to lavage and debridement received a standard arthroscopic procedure. Patients allocated to lavage only underwent an identical procedure except that no debridement was performed (unless an unstable meniscal tear was identified in which case this was excised). For patients in the control arm, the knee was prepped and draped as usual and the 1-cm incisions performed. No instrumentation was performed but the surgeon manipulated the knee as per arthroscopic debridement. Post-operative care was provided by blinded clinical staff according to a standardised pathway; the surgeon was not involved in post-operative care or outcome assessment. |
Vitek et al. 2003 [54] | Pallidotomy versus medical therapy for Parkinson’s disease | Average change in unified Parkinson’s disease rating scale at 6 months | Assessor reported | Independent assessors | Two independent outcomes assessors blind to treatment allocation collected all outcome data; all patients wore hats to mask scars during baseline and follow-up outcomes collection; there was no contact between assessors and participants between follow-up appointments; assessors were not involved in the routine care of participants; patients were asked not to inform assessors of treatment allocation. |
Quinn et al. 2002 [55] | Suturing versus conservative management of hand lacerations | Cosmetic appearance at 3 months | Assessor reported | Independent assessors | Assessment of wounds made photographically with no knowledge or contact with participant at 3 months after treatment by two independent doctors. |
Gervaz et al. 2010 [56] | Laparoscopic versus open (lower midline) approach for sigmoid colectomy to treat diverticular disease | Composite pain assessed daily for 4 days; time to first flatus/bowel movement | PRO | Patients Nursing staff | Sterile, opaque dressing to cover entire lower abdomen for 4 days |
Ezra et al. 2004 [57] | Vitrectomy alone versus vitrectomy plus autologous serum transfer versus conservative management for full thickness macular hole | Composite closure of macular hole and visual acuity (time point unclear) | Assessor reported | Independent assessors | Closure of hole accessed via digitalised photographs of fundoscopy and digitalised images of flourescein angiography. These assessments were made separately as vitrectomy is obvious on fundoscopy. This way partial blinding between the arms was maintained because serum transfer cannot be seen; blinded assessment by a separate assessor of visual acuity |
Summary and conclusion
Clinical outcome measure |
An outcome measured in clinical practice, e.g. survival or surgical complications such as anastomotic leak, including the specific measurement variable (e.g. systolic blood pressure), analysis metric (e.g. change from baseline, final value, time to event), method of aggregation (e.g. median, proportion) and timepoint [58]. |
Patient reported outcome (PRO) |
An outcome reported directly by the patient themselves without interpretation from an observer [59]. Examples include assessments of health status and quality of life (e.g. physical ability, symptom severity). PROs are typically recorded in a self-completed questionnaire. |
Patient-reported outcome measure |
A measure of a PRO including domain (e.g. anxiety), specific measurement tool (e.g. name of questionnaire) and other levels of specification required for clinical outcomes (analysis metric and method of aggregation) [60]. |
Hospital/process-related outcome |
A metric related to the organisation or individual involved in the patients’ care rather than the effect of the intervention on the patient’s health, e.g. length of hospital stay or number of tests conducted. |
Resource use measure |
A metric to quantify the cost of care, including direct financial expenses as well as staff time. |
Surrogate outcome |
A measure that is not of direct practical importance but is believed to reflect an outcome that is important; often a physiological or biochemical marker that can be relatively quickly and easily measured, and that is taken as being predictive of an important clinical outcome [3]. |
Composite outcome |
An outcome that consists of two or more component outcomes [5]. |
Primary outcome |
The outcome of greatest importance [3]. |
Secondary outcome |
An outcome used to evaluate additional effects of the intervention deemed a priori as being less important than the primary outcomes [3]. |
Explanatory outcome |
Outcomes that are measured to provide additional information about an intervention, but may be without an a priori hypothesis. |