Systematic ReviewA low proportion of systematic reviews in physical therapy are registered: a survey of 150 published systematic reviews
Introduction
Systematic reviews (SRs) provide the best evidence to contribute to decision-making about the implementation of healthcare interventions.1 Although these studies are conducted with explicit and transparent methods, discrepancies might occur between the protocol and the publication. For example, authors might adapt the methods so that the SR generates more positive and statistically significant results, especially because there is a tendency for some scientific journals to preferentially publish manuscripts with statistically significant results.2 This may affect the validity of the results by introducing bias, such as outcome reporting bias.3, 4, 5 Outcome reporting bias is defined as the selective reporting from a subset of original outcomes, based on results.3 One of the strategies suggested to reduce this bias is the prospective registration of protocols for SRs.6
Protocol registration has been increasingly recommended for clinical trials7 and SRs.8 A registry for protocols of SRs was first proposed by the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement in 2009,8 which resulted in subsequent development and implementation of the International Prospective Register of Systematic Reviews (PROSPERO). A protocol provides transparency and makes explicit the hypotheses, methods and analysis of the SR that is to be conducted. According to the Cochrane Handbook for Systematic Reviews of Interventions,1 a prospectively registered protocol reduces authors’ biases by publicly documenting the a priori planned methods. When necessary, changes may occur between protocol and publication. However, any changes should be: decided upon without calculating their effect on the results, applied as an amendment to the registered protocol at the time of the decision, and reported with explanation in the manuscript.1
A previous study has demonstrated that nearly one-third of a sample of the SRs registered on PROSPERO show discrepancies between the primary outcomes registered in the protocol and the primary outcome reported in the publication.9 Other studies in several specific fields have also revealed discrepancies between protocols and published SRs.10, 11, 12 As the prevalence of discrepancies differs between different fields of research, it is important to assess this issue in other disciplines, such as physical therapy. Discipline-specific data may also indicate which strategies might be most beneficial to control these discrepancies.
Launched in 1999, the Physiotherapy Evidence Database (PEDro) indexes published practice guidelines, SRs and randomized controlled trials to support an evidence-based approach in physical therapy. PEDro is one of the most complete databases for physical therapy publications.13, 14 Pinto et al.15 analyzed 200 randomized controlled trials sampled from PEDro and identified that many: were not prospectively registered; were not registered at all; and/or had discrepancies between the registered protocol and the published report.15 However, a search of GoogleScholar using the search terms regist-, systematic review, and physiotherapy or physical therapy did not identify any studies addressing the extent of registration of SRs in physical therapy. Therefore, the primary aims of the study were: (a) to estimate the proportion of SRs of physical therapy interventions that are registered, (b) to assess the methodological quality of (un)registered SRs of physical therapy interventions, and (c) to investigate whether outcome reporting bias is present in those SRs that have a registered protocol. As a secondary aim, we explored whether registration is associated with characteristics of SRs, including the geographical location of the authors, the impact factor of the journal, funding, and spin.
Section snippets
Methods
This study was a survey of SRs of physical therapy interventions. PEDro was used as the source of the SRs because it is considered one of the most complete database of SRs of physical therapy interventions.13 From the total sample of SRs indexed in 2015, we randomly selected 150 reports using a random number function in Microsoft Excel software. The full texts were restricted to publications written in English, Italian, Portuguese and Spanish. The full-text published report of each SR was
Results
At the time this study was conducted (September, 2016), PEDro indexed 646 SRs that had been published in 2015. We selected a random sample of 150 published SRs, which equated to 23% of the SRs. Cochrane reviews accounted for 15 of the 150 reviews. The prevalence of registration among the SRs was 19% (29 SRs). The characteristics of the whole sample and of the registered and unregistered SRs specifically are presented in Table 1.
Fig. 2 describes the association between the SRs’ characteristics
Discussion
This survey found a low prevalence of registration of SRs in the physical therapy field, with only one fifth being registered. Registration was more likely among funded SRs and among SRs published in journals with a high impact factor. Importantly, registered SRs had significantly higher quality compared to unregistered SRs. Our findings also indicate that although a third of the registered SRs demonstrated discrepancies between protocol and publication, no evidence of outcome reporting bias
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflicts of interest
The authors declare no conflicts of interest.
Acknowledgements
C.B.O., I.R.L., D.O.S. and R.V.B. are funded by São Paulo Research Foundation (FAPESP) [grant numbers 2016/03826-5, 2015/17777-3, 2015/11534-1, 2015/00406-2].
References (33)
- et al.
Many scenarios exist for selective inclusion and reporting of results in randomized trials and systematic reviews
J Clin Epidemiol
(2013) - et al.
Clinical trial registration: a statement from the International Committee of Medical Journal
Lancet
(2004) - et al.
Indexing of randomised controlled trials of physiotherapy interventions: a comparison of AMED, CENTRAL, CINAHL, EMBASE, hooked on evidence, PEDro, PsycINFO and PubMed
Physiotherapy
(2009) - et al.
AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews
J Clin Epidemiol
(2009) - et al.
Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0
(2011) - et al.
Publication bias in clinical trials due to statistical significance or direction of trial results
Cochrane Database Syst Rev
(2009) - et al.
Bias in meta-analysis due to outcome variable selection within studies
Appl Stat
(2000) - et al.
Dissemination and publication of research findings: an updated review of related biases
Health Technol Assess
(2010) - et al.
Publishing protocols of systematic reviews: comparing what was done to what was planned
JAMA
(2002) - et al.
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
Ann Intern Med
(2009)
A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study
J Clin Epidemiol
Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions
Cochrane Database Syst Rev
Discrepancies in outcome reporting exist between protocols and published oral health cochrane systematic reviews
PLoS ONE
Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study
BMJ
CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions
Phys Ther
Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials
Phys Ther
Cited by (18)
Aerobic training reduces blood pressure and waist circumference and increases HDL-c in metabolic syndrome: a systematic review and meta-analysis of randomized controlled trials
2018, Journal of the American Society of HypertensionCitation Excerpt :The protocol for this systematic review was previously registered in an international database of systematic reviews in health and social care (registration number CRD42016033862). Protocol registration provides explicit hypotheses, methods and analysis of the systematic review.23 In addition, it reduces authors’ biases by publicly documenting the planned methods.24
Updating systematic reviews
2018, Journal of PhysiotherapyOverall confidence in the results of systematic reviews on exercise therapy for chronic low back pain: a cross-sectional analysis using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) 2 tool
2020, Brazilian Journal of Physical TherapyCitation Excerpt :In contrast to our findings, the study by Martins et al.75 reported that, only 22.5% of the reviews were rated as ‘very poor’. Oliveira et al.76 evaluated the methodological quality of systematic reviews of physical therapy interventions using the AMSTAR tool, most of the reviews were rated as ‘low’ (median 3, 0–11 scale) for the orthopaedics PEDro subdiscipline. The three systematic reviews in which the overall confidence in the results were rated as ‘High” in our study evaluated the effectiveness of physical conditioning exercises,64 Yoga11 and Pilates10 in LBP patients.
Trials and tribulations of transparency related to inconsistencies between plan and conduct in peer-reviewed physiotherapy publications: A methodology review
2024, Journal of Evaluation in Clinical PracticeLocal anesthetics as a tool for Staphylococcus spp. control: a systematic review
2024, Brazilian Journal of Microbiology