We recently conducted a systematic review of the methodological quality of randomised controlled trials (RCTs) in plastic surgery. In accordance with convention, we are here separately reporting a systematic review of the reporting quality of the same RCTs.
Methods
MEDLINE® and the Cochrane Database of Systematic Reviews were searched by an information specialist from 1 January 2009 to 30 June 2011 for the MESH heading ‘Surgery, Plastic’. Limitations were entered for English language, human studies and randomised controlled trials. Manual searching for RCTs involving surgical techniques was performed within the results. Scoring of the eligible papers was performed against the 23-item CONSORT Statement checklist. Independent secondary scoring was then performed and discrepancies resolved through consensus.
Results
Fifty-seven papers met the inclusion criteria. The median CONSORT score was 11.5 out of 23 items (range 5.3–21.0). Items where compliance was poorest included intervention/comparator details (7 %), randomisation implementation (11 %) and blinding (26 %). Journal 2010 impact factor or number of authors did not significantly correlate with CONSORT score (Spearman rho = 0.25 and 0.12, respectively). Only 61 % declared conflicts of interest, 75 % permission from an ethics review committee, 47 % declared sources of funding and 16 % stated a trial registry number. There was no correlation between the volume of RCTs performed in a particular country and reporting quality.
Conclusions
The reporting quality of RCTs in plastic surgery needs improvement. Better education, awareness amongst all stakeholders and hard-wiring compliance through electronic journal submission systems could be the way forward. We call for the international plastic surgical community to work together on these long-standing problems.
Introduction
Randomised controlled trials (RCTs) represent the criterion standard in evaluating healthcare interventions. However, RCTs can yield biased results if they lack methodological rigour [1], especially where surgical techniques are involved [2]. Readers need complete, clear and transparent information in order to assess a trial accurately. Unfortunately, many trials fail to provide critical information in published reports [3‐5]. Inadequately reported RCTs are associated with bias in estimating the effectiveness of interventions and with poor methodology [6‐12].
The Consolidated Standards of Reporting Trials statement was originally developed in 1996 [13] to aid reporting of RCTs, and an extension for non-pharmacological interventions (CONSORT NPT) was published in 2008 [14]. It consists of a 23-item checklist and flow diagram.
Anzeige
Our team have previously assessed the methodological quality of recent RCTs in plastic surgery, concluding that it requires improvement [15]. However, although flaws in methodology limit the validity and generalizability of study results, without accurate reporting, shortcomings in study design and implementation can be compounded and become difficult to accurately assess. Studies looking at methodological quality are usually separate from reporting quality [16]. Given this previous work, the purpose of this new study is to systematically review the reporting quality of recent RCTs in plastic surgery using CONSORT NPT criteria. This new work will allow the plastic surgical community to take stock of where reporting quality has been in recent years.
Material and methods
Search methods
The search technique was the same as used for this team’s previous work assessing the methodological quality of plastic surgery RCTs [15]. An information specialist (trained in database searching by NHS Evidence, who conducts approximately 100 searches per year and is the staff trainer) based at the lead authors Plastic Surgery Unit searched MEDLINE® and the Cochrane Database of Systematic Reviews from 1 January 2009 to 30 June 2011 for the Medical Subject Headings (MESH is the NLM controlled vocabulary thesaurus used for indexing articles for PubMed) ‘Surgery, Plastic’ or ‘Reconstructive Surgical Procedures’ with ‘or’ used as a Boolean operator and with the ‘explode’ function activated. We chose this time period since the CONSORT NPT statement was only published in 2008 and hence it would be unfair to hold RCTs prior to this period to a standard that did not exist at the time of writing. Limitations were set for the English language, human studies and randomised controlled trials.
Results were then manually searched by four of us (ED, MS, CFC and EE) for relevant RCTs involving surgical techniques. Papers involving purely pharmacological therapies in all arms, cost analyses, study protocols, interim or non-randomised studies, short communications and RCTs involving virtual or simulated procedures were excluded.
Scoring
The papers were then scored by one of three primary scorers (ED, MS, EE) against the 23-item CONSORT NPT checklist with each item being given an equal weighting. Items 4 and 11 were subdivided into four and three sub-parts, respectively; a full point was only gained if all sub-parts were fulfilled; otherwise, the appropriate fraction was awarded. The resulting mark out of 23 was termed the ‘CONSORT score’. Following this initial round of scoring (ED scored 2009, MS scored 2010 and 2011), all papers were then re-scored by a single secondary scorer (CFC). Discrepancies were then resolved by consensus (between ED, MS and CFC), and if that could not be reached, they were referred to the lead author (RA) for a final judgement. Evaluators were not blinded to the country of origin of the authors.
Anzeige
Compliance with individual items of the statement was analysed (by summating the number of articles fulfilling that item divided by the total number of included articles) as well as the relationship between CONSORT score and year of publication, geographical origin (for the RCT), the number authors and the ISI 2010 impact factor for the journals in which the RCTs were published. These additional correlates were chosen to give information about whether improvements are occurring over time and whether volume and perceived markers of RCT quality (like impact factor of the journal in which it is published) correlate with actual reporting quality as defined by CONSORT.
Secondary analyses
In addition to the CONSORT score, papers were assessed for whether they fulfilled seven additional criteria referred to in the CONSORT NPT and whether they mention conflicts of interest, sources of finding, a trial registry number and ethical approval.
Statistical analysis
In line with previous work [15], inter-rater agreement was assessed using the Kappa score. Data were analysed using non-parametric descriptors such as median, and correlations were calculated using Spearman rho, using SPSS version 20.
MEDLINE; 3 [Limit to: Publication Year 2009–2011 and English Language]; 11,395 results
5.
MEDLINE; 4 [Limit to: (Publication Types Randomized Controlled Trial) and Publication Year 2009–2011 and English Language]; 254 results
From the initial set of 254 papers retrieved from MEDLINE, 63 were selected following a manual search and abstract assessment by the authors (ED, MS, CFC). Subsequent to complete download of all 63 papers, six were excluded for being a study protocol, purely pharmacological or theoretical, retrospective or an interim study. This resulted in 57 RCTs which met the inclusion criteria (seven were multicentre), published across 28 journals. All RCTs compared treatment interventions and none related to diagnosis. No further relevant trials were found in the Cochrane Database of Systematic Reviews (Fig. 1).
Fig. 1
PRISMA flow diagram, illustrating how papers were selected (adapted from Moher et al. [17])
×
The median CONSORT score was 11.5 out of 23 items (range 4.5–21.0) with a Kappa score 0.80. There was a slight trend for improvement over the 3-year period, on average 1.5 CONSORT points per year (Spearman rho 0.999) (see Table 1).
Table 1
Summary statistics for CONSORT scores per year
Year
Number of RCTs
Median CONSORT score
Range
2009
26
9.9
4.5–17.3
2010
21
11.5
5.3–21.0
2011a
10
12.9
9.0–15.0
aTo 30 June 2011 only
Compliance with individual items of the CONSORT
Compliance was highly variable for the different CONSORT items. This is shown in Fig. 2. Compliance was the poorest for items related to intervention/comparator details (7 %), randomisation implementation (11 %) and blinding (26 %), as shown in Table 2.
Fig. 2
Compliance of the 57 RCTs with the individual items of the CONSORT statement
Table 2
Compliance of RCTs with individual items of the CONSORT statement (ranked in order of increasing fulfilment)
Criteria
Abbreviated description
Compliance (%)
New item (23)
Details of the experimental treatment and comparator as they were implemented
7
10
Who generated the allocation sequence, who enrolled participants and who assigned participants to their groups
11
11
Whether or not participants, those administering the interventions and those assessing the outcomes were blinded to group assignment
26
16
Number of participants (denominator) in each group included in each analysis and whether analysis was by ‘intention-to-treat’
27
1
How participants were allocated to interventions—is it mentioned in the title
31
7
How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules
31
13
Flow of participants through each stage (a diagram is strongly recommended)
38
18
Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those prespecified and those exploratory
38
8
Method used to generate the random allocation sequence, including details of any restriction (e.g. blocking, stratification)
40
9
Method used to implement the random allocation sequence, clarifying whether the sequence was concealed until interventions were assigned
40
21
Generalizability (external validity) of the trial findings
45
4
Precise details of the interventions intended for each group and how and when they were actually administered
52
14
Dates defining the periods of recruitment and follow-up
56
6
Clearly defined primary and secondary outcome measures
62
19
All important adverse events or side effects in each intervention group
64
15
Baseline demographic and clinical characteristics of each group
65
3
Eligibility criteria for participants and the settings and locations where the data were collected
69
5
Specific objectives and hypotheses
71
20
Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision
75
12
Statistical methods used to compare groups for primary outcome(s); methods for additional analyses, such as subgroup analyses and adjusted analyses
76
17
For each primary and secondary outcome, a summary of results for each group and the estimated effect size and its precision
78
22
General interpretation of the results in the context of current evidence
89
2
Scientific background and explanation of rationale
There was poor fulfilment of additional criteria and only 61 % declared conflicts of interest, 75 % permission from an ethics review committee, 47 % declared sources of funding and just 16 % stated a trial registry number (Table 3).
Table 3
Compliance of RCTs with additional criteria
Additional criteria
Compliance (%)
AD1—eligibility criteria for care providers
7
AD2—details on the centres’ volume
5
AD3—number of care providers performing the treatment in each group
38
AD4—number of participants treated by each care provider
31
AD5—details on patients’ expectancies or preference for the treatments at baseline
2
AD6—baseline data of care providers
0
AD7—details on care providers’ compliance with the planned procedure
4
Ethical approval
75
Sources of funding
47
Trial registry number
16
Conflicts of interest
61
Anzeige
CONSORT score and number of authors
There was no correlation between number of authors and CONSORT score (Spearman rho = 0.12, see Fig. 3).
Fig. 3
CONSORT score against number of authors
×
CONSORT score and impact factor
There was no correlation between journal impact factor and CONSORT score (Spearman rho = 0.26, see Fig. 4).
Fig. 4
CONSORT score against ISI 2010 impact factor
×
Geographical distribution of RCTs and CONSORT compliance
There was no correlation between the volume of RCTs conducted in a particular country and CONSORT score (Fig. 5).
Fig. 5
A bar chart of CONSORT score and number of RCTs against country
×
Discussion
Over the last decade, there have been increasing calls for utilising an evidence-based medicine approach within surgery [18, 19]. However, the evidence base in plastic surgery is still dominated by case series, and calls for higher levels of evidence (including RCTs) are gathering pace [20, 21]. Poor reporting ‘short circuits’ proper critical appraisal prevents inclusion in systematic reviews and meta-analyses, and resulting clinical judgements could be misleading and potentially dangerous.
Anzeige
Previous reviews of surgical RCTs have shown that, in 20 %, the conclusions were not justified by the data [22]. Solomon et al. found that the lowest quality RCTs were those that involved a surgical technique, were published in a surgical journal and where a surgeon was the principal author [23].
Research in the field of RCT reporting quality has pointed to the consistent absence of the same key quality data: sample size calculations, randomisation sequence generation method and implementation, post-randomisation verification of balance in known confounders, allocation concealment, blinding, intention-to-treat analysis and participant flow charts. Our study is the first to assess the compliance of recent RCTs in plastic surgery with the CONSORT NPT criteria, and the results support this pattern of poor compliance with a low median CONSORT score of 11.5. This is similar to the CONSORT scores (out of 22) found in earlier work by one of us (RAA), with 11.1 for Urological RCTs, 10.3 for cardiac surgery, 10.9 for general surgery, 11.9 for hepatobiliary, 10.8 for orthopaedic and 12.0 for vascular surgery [19]. Furthermore, the additional CONSORT NPT criteria were severely lacking (Table 3). Whilst they are not core criteria, the authors recognise them as important for RCTs of interventions. Poor reporting has also been linked with poor methodology as shown by Taghinia et al. [24] and others [8‐12].
Studies with more authors have been correlated with higher citation levels [25]. One would anticipate that more authors on a study would lead to better reporting quality; however, our research did not support this, consistent with earlier work [19]. There was also no correlation between CONSORT score and the impact factor of the journal in which the RCT was published, again consistent with earlier work by one of us (RA) [19]. The impact factor is heralded by many plastic surgeons and editors [26] as the reflection of increasing journal and article quality. Yet successive studies are now showing that it has no bearing on the quality of reporting of one of the highest levels of evidence—the RCT. Sinha et al. [27] identified the top three ranked surgical journals by impact factor and found that, of 42 RCTs analysed, only 40 % had a Jadad score ≥3 and there was no significant difference between CONSORT-endorsing and non-endorsing journals. Our data showed no link between volume of RCTs from a particular geographic region and CONSORT score. This suggests that the problems of poor reporting are indeed global and not confined to a few select countries.
Within plastic surgery specifically, several studies have found that RCTs need improvement in reporting quality. McCarthy et al. [28] analysed level 1 studies in five plastic surgery journals from 1978 to 2009. They found that only 39 % reported randomisation technique. In their analysis of 96 RCTs in plastic surgery published between 2004 and 2008, Veiga et al. [29] found that 29 % appropriately described allocation concealment. Momeni et al. [30] analysed 172 RCTs from three plastic surgical journals during 1990–2005 and found that only 12 % reported on their allocation concealment and 37 % described participant dropouts. Karri looked at 133 RCTs published across three plastic surgery journals from 1980 to 2004 [31]. Sample size calculation was only reported in 12.8 % of trials, randomisation methodology in 29.3 %, allocation concealment in 18.8 %, blinding of investigator/assessment in 51.9 % and study limitations in only 33.8 % trials. Veiga Filho et al. in 2005 examined 34 RCTs in plastic surgery and concluded that they were overall of low quality with 59 % scoring two points or less on a Jadad score [32]. In addition, they concluded that the process of randomisation in the studies was appropriately designed and conducted but that authors did not report it. Follow-up work by the same group in 2011 concluded that RCT quality as measured by Jadad criteria had ‘significantly increased’ [29].
Anzeige
In 2005, the International Committee of Medical Journal Editors implemented a policy [33] that required registration of all clinical trials prior to enrolment of the first patient (part of the aim being a reduction in publication bias from the non-publication of negative studies). In 2007, this became part of US regulatory requirement with section 801 of the Food and Drug Administration Amendments Act [34]. In 2008, trial registration became an ethical requirement with the updated Declaration of Helsinki mandating it [35]. Despite this, only 16 % of RCTs in our sample stated their trial registry. The low reporting of conflicts of interest, sources of funding and ethical approval is concerning; such information is pivotal scientific and scholarly transparency and integrity.
On the positive side, there was an upward trend in the median CONSORT score, rising three points over the 3-year period 2009–2011. However, since all the CONSORT items are mandatory for reporting, this increase is unlikely to be due to a serious appreciation and compliance with CONSORT specifically. It is more likely to be due to general improvements in reporting that may be occurring or simply random.
The limitations of our study include restriction to the English language, searching only MEDLINE® and analysing papers by publication date rather than submission date, since publication lag times for journals vary. The English language has increasingly become the lingua franca of science with an estimated 80–90 % of papers in scientific journals written in English [36]. This has been coupled with initiatives to offer translation services of key papers [37]. We feel that this strategy would still capture all the relevant papers. Whilst we did not blind our evaluators to the country of origin of the authors, we do not feel there was any inherent bias against a particular country and scores were recorded in a dispassionate manner.
Further work is needed to assess barriers to compliance with CONSORT amongst key stakeholders in the process: authors, journal reviewers and editors, funders, institutions and readers. Meticulous planning, involvement of a trial methodologist, biostatistician and compliance with CONSORT NPT in the eventual paper would be key for those conducting and reporting RCTs. Journal editors and peer reviewers as guardians of the scholarly literature have an important gatekeeper role here [38, 39].
We support previous calls for the better education of plastic surgeons at all levels in clinical research methods, evidence-based medicine and improved funding/support of plastic surgical RCTs. McCulloch et al. [40] have called for the deployment of alternative prospective designs, such as interrupted time series studies, to be used when randomised trials are not feasible—we support their call and indeed suggest that registry studies may be useful in some instances.
One solution is to hard-wire compliance with CONSORT by making the checklist a mandatory item for submission if a manuscript is submitted as an RCT. It can also be published as a supplementary item online allowing for greater transparency and scrutiny by readers. Such a policy was adopted by the International Journal of Surgery in January 2013 [41].
Conclusion
The reporting quality of RCTs in plastic surgery requires improvement. Perceived surrogate markers of quality such as number of authors and impact factor of the journal had no relationship on CONSORT compliance. We suggest ways this could be improved including better education, awareness amongst all stakeholders and hard-wiring compliance through electronic journal submission systems.
Acknowledgments
The authors would like to thank Patricia Rey, Information Specialist at Queen Victoria Hospital NHS Foundation for her guidance and meticulous search of the literature to retrieve the initial set of papers.
Conflict of interest
RA is on the Editorial Board of the International Journal of Surgery.
Open Access
This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (
https://creativecommons.org/licenses/by/2.0
), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Mit der Zeitschrift Die Chirurgie erhalten Sie zusätzlich Online-Zugriff auf weitere 43 chirurgische Fachzeitschriften, CME-Fortbildungen, Webinare, Vorbereitungskursen zur Facharztprüfung und die digitale Enzyklopädie e.Medpedia.
Bis 30. April 2024 bestellen und im ersten Jahr nur 199 € zahlen!
Nach der Katheterablation von Vorhofflimmern kommt es bei etwa einem Drittel der Patienten zu Rezidiven, meist binnen eines Jahres. Wie sich spätere Rückfälle auf die Erfolgschancen einer erneuten Ablation auswirken, haben Schweizer Kardiologen erforscht.
Schmerzen im Unterbauch, aber sonst nicht viel, was auf eine Appendizitis hindeutete: Ein junger Mann hatte Glück, dass trotzdem eine Laparoskopie mit Appendektomie durchgeführt und der Wurmfortsatz histologisch untersucht wurde.
Derzeit wird empfohlen, eine Therapie mit GLP-1-Rezeptoragonisten präoperativ zu unterbrechen. Eine neue Studie nährt jedoch Zweifel an der Notwendigkeit der Maßnahme.
Die Ureterstriktur ist eine relativ seltene Komplikation, trotzdem bedarf sie einer differenzierten Versorgung. In komplexen Fällen wird dies durch die roboterassistierte OP-Technik gewährleistet. Erste Resultate ermutigen.
Update Chirurgie
Bestellen Sie unseren Fach-Newsletterund bleiben Sie gut informiert.
Das Karpaltunnelsyndrom ist die häufigste Kompressionsneuropathie peripherer Nerven. Obwohl die Anamnese mit dem nächtlichen Einschlafen der Hand (Brachialgia parästhetica nocturna) sehr typisch ist, ist eine klinisch-neurologische Untersuchung und Elektroneurografie in manchen Fällen auch eine Neurosonografie erforderlich. Im Anfangsstadium sind konservative Maßnahmen (Handgelenksschiene, Ergotherapie) empfehlenswert. Bei nicht Ansprechen der konservativen Therapie oder Auftreten von neurologischen Ausfällen ist eine Dekompression des N. medianus am Karpaltunnel indiziert.
Das Webinar beschäftigt sich mit Fragen und Antworten zu Diagnostik und Klassifikation sowie Möglichkeiten des Ausschlusses von Zusatzverletzungen. Die Referenten erläutern, welche Frakturen konservativ behandelt werden können und wie. Das Webinar beantwortet die Frage nach aktuellen operativen Therapiekonzepten: Welcher Zugang, welches Osteosynthesematerial? Auf was muss bei der Nachbehandlung der distalen Radiusfraktur geachtet werden?
Inhalte des Webinars zur S1-Leitlinie „Empfehlungen zur Therapie der akuten Appendizitis bei Erwachsenen“ sind die Darstellung des Projektes und des Erstellungswegs zur S1-Leitlinie, die Erläuterung der klinischen Relevanz der Klassifikation EAES 2015, die wissenschaftliche Begründung der wichtigsten Empfehlungen und die Darstellung stadiengerechter Therapieoptionen.
