Results from clinical studies on acupuncture for stroke rehabilitation are contradictory. The reason for the inconsistent findings especially lie in the transparency and accuracy of randomized controlled trials (RCTs) reports. This study aims to analyze the quality of reporting and its correlates in RCTs on acupuncture for stroke rehabilitation.
Methods
Quality of reporting for included papers was assessed against a subset of criteria adapted from the CONSORT 2010 statement and STRICTA. An overall quality score (OQS) and a combined key methodological index score (MIS) was calculated for each trial. Then, factors associated with OQS and MIS were identified.
Results
A total of 15 RCTs were included in full text. The median OQS based on the CONSORT statement and STRICTA was 8 and 12, respectively. The significant predictors for CONSORT OQS was funding source, for STRICTA was year of publication. With regard to the MIS, no variable was associated with improved methodological quality.
Conclusions
Our study found that the overall quality of reporting on RCTs of acupuncture for stroke rehabilitation was general or good. But some items’ reporting was found where information was insufficient or inadequate in most studies which needed substantial improvement.
The online version of this article (doi:10.1186/1472-6882-14-151) contains supplementary material, which is available to authorized users.
Lixing Zhuang, Jun He contributed equally to this work.
Competing interests
The authors declare that they have no conflicts of interest concerning this article.
Authors’ contributions
ZLX was responsible for conception and design of the review, carried out the literature search, performed data analysis, and drafted the manuscript. HJ and ZX performed data extraction and assessment of risk of bias, participated in conception and design of the review, and critically revised the man ± uscript. LLM participated in conception and design of the review, and critically revised the manuscript. All authors read and approved the final manuscript.
Background
Stroke is a leading cause of death in both middle- and high-income countries, responsible for 22% of all deaths worldwide [1]. The risk factors leading to increasing stroke incidence are an aging population, dietary changes, and work-related stress, etc. In China, stroke is the second most common cause of death in cities and the third most common cause of death in rural areas [2]. Also, stroke is a major cause of disability and dependency. Poststroke disability with a high incidence (>50%) brings a heavy burden to patients and their caregivers [3]. In the United States, the total societal and healthcare costs have risen from $53.6 billion in 2004 [4] to $68.9 billion in 2009 [3]. Clearly, patients with stroke urgently need safe and effective treatments for alleviating the burden, which drives people to search for conventional treatment to improve the effect of stroke rehabilitation, such as acupuncture.
For more than 3000 years, practitioners in China have used acupuncture to treat various diseases, including the sequelae of stroke. Acupuncture is widely used to improve motor, sensation, speech and other neurological functions in patients with stroke. Compared to other conventional interventions, acupuncture is relatively simple, inexpensive and safe which has been well accepted by Chinese patients and also increasingly practiced in some Western countries [5]. Results from clinical studies on acupuncture for stroke rehabilitation, however, are contradictory. Although some reviews indicate that too little evidence exists to prove the efficacy of acupuncture for stroke patients [6‐9], a series of studies suggest that this modality is an effective method for improving disabilities due to stroke [10‐13]. Researchers have called the quality of these meta-analyses into question, however, and the likely potential for publication bias suggests a need for additional research on the subject. Currently, it is necessary to evaluate the quality of reporting on randomized controlled trials (RCTs) of acupuncture for stroke rehabilitation, for quality of reporting is essential for guiding journal peer-review decisions, for experts’ recommendations, and to conduct unbiased meta-analysis, as it influences our interpretation of evidence.
Anzeige
The Consolidated Standards for Reporting Trials (CONSORT) statement developed in 1996 [14] aimed to facilitate critical appraisal and interpretation of RCTs by authors, providing them guidance about how to improve the reporting of their trials, as well as by peer reviewers and editors, helping them identify reports with potentially biased results. The CONSORT statement has been further revised and is published as the CONSORT 2010 statement. Combined with the CONSORT statement, the acupuncture-specific Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) were designed to help authors fully report the acupuncture intervention in clinical trials [15, 16]. The combination of both tools helps us evaluate the completeness and transparency of RCTs reports on acupuncture for stroke rehabilitation according to their items.
However, to the best of our knowledge, there are no data about the quality of RCTs reporting in acupuncture for stroke rehabilitation. The present study aims to (i) describe the characteristics of published RCTs reports on acupuncture for stroke rehabilitation; (ii) assess the overall quality and key methodological items of published articles of RCTS on acupuncture for stroke rehabilitation; and (iii) determine factors associated with better reporting quality.
Methods
Search strategy
We searched the Cochrane Stroke Group Trials Register (March 2013), the Chinese Stroke Trials Register (March 2013), the Chinese Acupuncture Trials Register (March 2013) and the Trials Register of the Cochrane Complementary Medicine Field (March 2013). In addition, we searched the following bibliographic databases: MEDLINE (1966 to March 2013), EMBASE (1980 to March 2013), CINAHL (1982 to March 2013), AMED (the Allied and Complementary Medicine Database, 1985 to March 2013), the Chinese Biological Medicine Database (CBM-disc, 1979 to March 2013), China National Knowledge Infrastructure (CNKI, 1979 to March 2013), Wanfang databases (1982 to March 2013) and VIP database (1992 to March 2013). The following search terms were used in Chinese and English: poststroke, hemorrhagic, ischemic, cerebrovascular disorders, acupuncture, electro-acupuncture, needle, scalp acupuncture, auricular acupuncture, acupuncture point, meridian, acupoint and point injection, etc.
Inclusion & exclusion criteria
Types of studies
RCTs comparing acupuncture with at least one control group that used placebo, sham treatment or conventional treatment in patients with subacute (one to three months since onset) or chronic stroke (over three months since onset) were included in this research. RCTs were included if invasive acupuncture was used as the sole intervention or as an adjunct to another standard treatment for stroke rehabilitation and if the control group received the same concomitant treatments as the acupuncture group. A randomization sequence generated from a random number table, calculator or computerized random number generator was considered authentic.
Types of participants
Patients of any gender, sex or ethnicity with ischemic or hemorrhagic stroke in the subacute or chronic phases were eligible. Stroke must be diagnosed according to the World Health Organization definition (rapidly developed clinical signs of focal (or global) disturbances of cerebral function, lasting more than 24 hours or leading to death, with no other apparent cause than of vascular origin [17]; or magnetic resonance imaging (MRI) or confirmed by computerized tomography (CT ).
Types of interventions
Acupuncture therapy will be defined as body acupuncture, scalp acupuncture, electro-acupuncture, warm needling, or ear acupuncture, etc. Studies that compare different types of acupuncture were excluded. Placebo acupuncture refers to a needle attached to the skin surface (not penetrating the skin but at the same acupoints) [18]. Sham acupuncture refers to: 1) a needle placed in an area close to but not in the acupuncture points [18]; 2) subliminal skin electro stimulation via electrodes attached to the skin [19].
Assessment of quality of reporting
Rating of overall reporting quality
An overall quality score (OQS) with 15 items from the CONSORT 2010 statement [20] and 17 items from the STRICTA [21, 22] were used (Table 1). Each item was scored 1 if it was reported and 0 if it was not clearly, or definitely not stated. The CONSORT discussion section items were excluded because we considered them too subjective to evaluate.
Table 1
Overall quality of reporting rating using items from the CONSORT Statement (n = 15)
Item
Criteria
Description
No. of positive trials
%
95% CI
Cohen’sк coefficient
95% CI
1
“Randomized” in the title or abstract
Study identified as a randomized controlled in the title or abstract
15
100
-
1.00
-
2
Background
Adequate description of the scientific background and explanation of rationale
12
80
57 to 100
0.74
0.50 to 0.98
3
Trial design
Description of trial design (such as parallel, factorial) including allocation ratio
10
67
40 to 94
0.71
0.45 to 1.00
4
Participants
Description of the eligibility criteria for participants
13
87
67 to 100
0.78
0.40 to 1.00
5
Interventions
Details of the interventions intended for each group
11
73
48 to 99
0.82
0.70 to 1.00
6
Outcomes
Definition of primary (and secondary when appropriate) outcome measures
4
27
1 to 52
0.85
0.75 to 0.99
7
Sample size
Description of sample size calculation
2
13
−6 to 33
0.83
0.68 to 1.00
8
Randomization
Description of the method used to generate the random sequence
9
60
32 to 88
0.76
0.60 to 0.99
12
Statistical methods
Description of the statistical methods used to compare groups for primary outcomes, subgroup analyses, or adjusted analyses
11
73
48 to 99
0.73
0.40 to 1.00
13
Flow chart
Details on the flow of participants through each stage of the trials (No. of patients randomly assigned, receiving intended treatment, completing the protocol and analyzed)
5
33
6 to 60
0.84
0.68 to 1.00
14
Recruitment
Dates defining the periods of recruitment and follow-up
6
40
12 to 68
0.65
0.38 to 0.99
15
Baseline data
An outline of baseline demographic and clinical characteristics of each group
14
93
79 to 100
0.68
0.42 to 1.00
17
Outcomes and estimation
For each primary and secondary outcome, a summary of results for each group is given, and the estimated effect size and its precision (e.g. 95% CI)
4
27
1 to 52
0.75
0.52 to 1.00
18
Ancillary analyses
Clear statement of whether subgroup/adjusted analyses were prespecified or exploratory
3
20
−3 to 43
0.68
0.35 to 0.99
19
Harms
Description of all important adverse events in each group
5
33
6 to 60
0.85
0.72 to 1.00
Rating of key methodological items
Concealment of allocation, appropriate blinding (either practitioner, participant or assessor), and analysis according to intention-to-treat (ITT) principle are highly related to potential sources of systematic bias and thus distortion in the estimation of the effect. Consequently, these three important key methodological items from the CONSORT 2010 Statement have been assessed separately. We then developed three “yes”/”no” items, so that emphasis was placed on quality of reporting rather than adequacy of trial design. A combined key methodological index score (MIS) was calculated for each trial by combining the scores of these three factors (range, 0 to 3).
Data extraction & analysis
Each article was reviewed by two independent investigators. They extracted useful information by modified CONSORT and STRICTA checklists (Table 1, 2 and 3). After both raters finished their review, we calculated Cohen’s к-statistics to assess agreement between two reviewers. Agreement was judged as poor if к ≤ 0.20; fair if 0.20 lower than к ≤ 0.40; moderate if 0.40 lower than к ≤ 0.60; substantial if 0.60 lower than к ≤ 0.80; good if к higher than 0.80; and perfect if к = 1. Discrepancies were reviewed in detail and subsequently settled by consensus.
Table 2
Overall quality of reporting rating using items from STRICTA (n = 15)
Item
Criteria
Description
No. of positive trials
%
95% CI
Cohen’sк coefficient
95% CI
1
Acupuncture rationale
(1a) Style of acupuncture (e.g., Traditional Chinese Medicine, Japanese, Korean, Western medical, Five Element, ear acupuncture, etc.)
12
80
57 to 100
0.65
0.38 to 0.98
(1b) Reasoning for treatment provided, based on historical context, literature sources and/or consensus methods, with references where appropriate
12
80
57 to 100
0.78
0.60 to 1.00
(1c) Extent to which treatment was varied
10
67
40 to 94
0.72
0.48 to 0.95
2
Details of needling
(2a) Number of needle insertions per subject per session (mean and range where relevant)
10
67
40 to 94
0.75
0.60 to 0.99
(2b) Names (or location if no standard name) of points used (uni-/bilateral)
12
80
57 to 100
0.83
0.65 to 1.00
(2c) Depth of insertion, based on a specified unit of measurement or on a particular tissue level
8
53
25 to 82
0.92
0.81 to 1.00
(2d) Responses sought (e.g., de qi or muscle twitch response)
9
60
32 to 88
0.63
0.38 to 1.00
(2e) Needle stimulation (e.g., manual or electrical)
13
87
67 to 100
0.74
0.50 to 0.99
(2f) Needle retention time
14
93
79 to 100
0.78
0.62 to 1.00
(2 g)Needle type (diameter, length and manufacturer or material)
10
67
40 to 94
0.70
0.53 to 1.00
3
Treatment regimen
(3a) Number of treatment sessions
10
67
40 to 94
0.62
0.33 to 0.98
(3b) Frequency and duration of treatment sessions
10
67
40 to 94
0.69
0.43 to 1.00
4
Other components of treatment
(4a) Details of other interventions administered to the acupuncture group (e.g., moxibustion, cupping, herbs, exercises, lifestyle advice)
3
20
−3 to 43
0.85
0.68 to 0.99
(4b) Setting and context of treatment, including instructions to practitioners, and information and explanations to patients
5
33
6 to 60
0.64
0.35 to 0.98
5
Practitioner background
(5) Description of participating acupuncturists (qualification or professional affiliation, years in acupuncture practice, other relevant experience)
8
53
25 to 82
0.76
0.62 to 1.00
6
Control or comparator interventions
(6a) Rationale for the control or comparator in the context of the research question, with sources that justify the choice(s)
9
60
32 to 88
0.78
0.65 to 1.00
(6b) Precise description of the control or comparator. If sham acupuncture or any other type of acupuncture-like control is used, provide details as for items 1–3 above
9
60
32 to 88
0.73
0.56 to 1.00
Table 3
Reporting quality of key methodological items (n = 15)
Item
Criteria
Description
No. of positive trials
%
95% CI
Cohen’sк coefficient
95% CI
9
Allocation concealment
Description of the method used to implement the random allocation sequence assuring the concealment until interventions are assigned
5
33
6 to 60
0.68
0.45 to 1.00
11
Blinding
Whether or not participants, those administering the interventions, or those assessing the outcomes were blinded to group assignment
9
60
32 to 88
0.75
0.52 to 1.00
16
Intent-to-treat analysis
No. of participants in each group included in each analysis and whether it was done by “intention to treat”
5
33
6 to 60
0.82
0.68 to 1.00
Anzeige
To identify factors associated with the overall quality of publications, we used the OQS as the outcome variable which was modeled using linear regression. Only variables that were significant at P ≤ 0.10 in the univariate models were used in a multivariate regression model for selecting significant variables (P ≤ 0.05). To identify factors associated with methodological quality, we used the MIS as the outcome variable in the regression analyses. As the outcome variable can be considered as a count, we relied on ordinal regression model and adjusted the variance empirically.
Linear and ordinal regression analysis was performed using SPSS version 20.0. Database of RCTs in acupuncture for stroke rehabilitation are provided in Additional file 1 and 2.
Results
The RCTs selection process is outlined in Figure 1. After identification, screening, eligibility determining of the literatures, a total of 15 relevant RCTs were included in the final analysis.
Figure 1
Diagram flow of the randomized controlled trials articles selection process.
×
Characteristics of the included trials
Year distribution of publication
Counting the number of articles, frequency, which refers to acupuncture for stroke rehabilitation in RCTs, was found to increase over time: from 2 (13.33%) in 1997–2001 to 8 (53.33%) in 2007-December, 2012.
Publication language
Of the 15 RCT papers, 10 (66.67%) were written in English. The remaining 5 papers (33.33%) were published in Chinese.
Nationality of authors
A total of 7 (46.67%) included papers were written by authors in Chinese research institutes; 7 (46.67%) were reported by international researchers; the remaining one paper (6.66%) was collaborations of Chinese and international researchers.
Funding source
Seven articles (46.67%) reported their sources of funding. Funding was obtained from provincial/municipal or international institutes.
Choice of comparator interventions
Interventions of nine trials (60.0%) comprised of sole acupuncture, followed by acupuncture plus other therapies (6 reports; 40.0%).
Quality of reporting
Rating of overall reporting quality
The ratings of overall quality of reporting based on the CONSORT statement were listed in Table 1. When the 15 RCTs were considered, the median OQS was 8, with a minimum of 1 and a maximum of 15. Poor reporting existed in terms of “outcomes”, “sample size”, “outcomes and estimation”, “ancillary analyses” with positive rate of less than 30% (Table 1). All of the items had a substantial, good or perfect agreement (Table 1).
The ratings of overall quality of reporting based on STRICTA were listed in Table 2. When the 15 RCTs were considered, the median OQS was 12, with a minimum of 3 and a maximum of 16. Poor reporting existed in terms of “4a: Details of other interventions” and “4b: Setting and context of treatment” with positive rate of 20% and 33% (Table 2). All of the items had a substantial or good agreement (Table 2).
Rating of key methodological items
Allocation concealment, blinding, and analysis by intent to treat were reported in 5 (33%), 9 (60%), and 5 (33%) of the 15 RCTs, respectively (Table 3). The median MIS was 1 with a minimum of 0 and a maximum of 3 (Table 3). Among the 15 studies, 5 (33%) did not report any of the three key methodological items. All of the items had a substantial or good agreement (Table 3).
Exploratory analysis: factors associated with better reporting quality
In univariate analyses, nationality of authors and funding source were associated with an increased OQS based on the CONSORT statement. After adjustment, the multivariate linear regression model suggested that funding source remained independent and significant predictors of overall quality. Specifically, the mean OQS based on the CONSORT statement increased by about 4.05 for manuscripts with funding source (95% CI, 0.44 to 7.67; P < 0.05) (Table 4).
Table 4
Multivariate linear regression analysis for factors associated with better OQS from the CONSORT statement (n = 15)
Variables
β
S.E.
t
P
95% CI
Constant
2.32
2.59
0.90
0.39
−3.28 ~ 7.92
Funding source
4.05
1.67
2.43
0.03
0.44 ~ 7.67
In univariate analyses, only year of publication was associated with an increased OQS based on STRICTA. Specifically, the mean OQS based on STRICTA increased by about 0.48 for manuscripts published in the period of one year increment (95%CI, 0.00 ~ 0.96; P ≤ 0.05) (Table 5).
Table 5
Univariate analysis for factors associated with better OQS from STRICTA (n = 15)
Variables
β
S.E.
t
P
95% CI
Constant
−958.16
446.54
−2.15
0.05
−1922.84 ~ 6.53
Year of publication
0.48
0.22
2.17
0.05
0.00 ~ 0.96
With regard to the MIS, in ordinal regression analyses, no variable was associated with improved methodological quality (P > 0.05).
Discussion
To the best of our knowledge, this study is the first investigation on the quality of reporting on RCTs of acupuncture for stroke rehabilitation according to the revised CONSORT and STRICTA guidelines. We found evidence that quality of reporting of the overall CONSORT items was general with median OQS 8 which was more than half of its total score 15; quality of reporting of the STRICTA items was good with median OQS 12 which was more than 70% of its total score 17; however, it is poor quality for the reporting of key methodological items with median MIS 1 which was 33% of its total score. Similar results were reported in some published studies [23, 24]. Some studies found that the quality of reporting in RCTs on Chinese medicine remained poor (mainly in key methodological items), but had improved over time [25, 26]. None of the variables we looked at in our ordinal regression model were significant predictors of improved methodological quality. We noted that the overall OQS based on the CONSORT Statement was correlated with funding source and STRICTA score was correlated with year of publication.
The purpose of reporting guidelines such as CONSORT and STRICTA is to increase the transparency of study methods and ultimately improve the overall quality of research. However, poorly reported trials make it ambiguous for other readers to assess the validity of the results and may mislead medical policy-makers in their decisions. Our study found that the overall quality of reporting was general or good between 1997 to March 2013. There might be multiple reasons. First, most of included reports are published after 2001 when STRICTA and revised CONSORT statement was published, especially, STRICTA score was correlated with year of publication. In the evaluation of acupuncture trials performed by Prady et al. [27], the authors concluded that reporting of CONSORT items had improved after the introduction of CONSORT but that the introduction of STRICTA did not improve the reporting of STRICTA items. However, another evaluation of auriculotherapy trials [23] came to similar conclusions with ours. Maybe the introduction of STRICTA and CONSORT statement improved the overall quality of reporting in our research. Second, more and more clinicians were training in study design and reporting of RCTs, and some better quality research papers had been submitted and published in international journals which had a more rigorous peer-review process. Our finding that the overall OQS based on the CONSORT statement was correlated with funding source, also suggested clinical trials with funding have more capacity to provide assurance for the better quality of study design and reporting of RCTs.
However, some items’ reporting was found where information was insufficient or inadequate in most studies. These areas are as follows: “allocation concealment”, “analysis by intent to treat”, “outcomes”, “sample size”, “outcomes and estimation“ and “ancillary analyses” based on the CONSORT statement; “details of other interventions” and “setting and context of treatment” based on STRICTA.
A generated allocation schedule should be implemented by using allocation concealment, a critical mechanism that prevents foreknowledge of treatment assignment and thus shields those who enroll participants from being influenced by this knowledge [20]. Trials in which the allocation sequence had been inadequately or unclearly concealed yielded larger estimates of treatment effects than did trials in which authors reported adequate allocation concealment. Intention to treat was defined as the inclusion of all patients randomly assigned in the analysis, regardless of whether they actually satisfied the entry criteria, the treatment actually received, and subsequent withdrawal or protocol deviations [25, 27]. ITT is generally favored because it avoids bias associated with non-random loss of participants. Unfortunately, we found only 33% of the trial reports provided a description of ITT or allocation concealment, making it very difficult to judge the validity of their findings.
Authors should indicate how the sample size and primary outcome was determined. Theoretically, clinical significant difference between interventions was detected by a high power if a trial had enough number of subjects. The researcher should note the primary and second outcome on which the calculation was based. For each primary and secondary outcome, results for each group, the estimated effect size and its precision (e.g. 95% confidence interval) presentation were recommended. However, no more than 30% of included reports presented the primary and second outcome, sample size calculation and outcome estimation, etc.
Anzeige
As for STRICTA, “details of other interventions” referred to the auxiliary techniques, prescribed self-treatment and lifestyle advice provided by the practitioner. All additional components, whether carried out by the practitioner or patient and whether integral or adjunctive to the acupuncture needling, should be described clearly [21]. “Setting and context of treatment” could also provide important additional components to treatment. For patients, the context factor includes some information given by practitioners might modify the trial’s outcome; for practitioners, it includes some instructions given by patients after prescribing or proscribing explanations to them might modify practitioners’ normal practice [21]. But the number of reports provided this information was small. As different acupuncturists providing treatment to different treatment arms will influence generalisability of the trial results, the details of other interventions and the background of both groups should be explained.
One limitation we should point out is that, due to language barrier, we didn’t search for any manuscripts published in non- Chinese or English journals. This is an area for worthwhile future study. It remains unknown whether searching in other language journals would have altered the constitution of our sample or results.
Conclusions
Our study found that the overall quality of reporting on RCTs of acupuncture for stroke rehabilitation was general or good. however, it is poor quality for the reporting of key methodological items. our results stress the need for researchers involved in RCTs of acupucture for stroke to improve the methodological quality of their research through the introduction of STRICTA and CONSORT statement.
Acknowledgments
The authors would like to express sincere thanks to our colleagues, staff of Guangzhou University of Chinese Medicine and its First Affiliated Hospital for their support.
Anzeige
Open Access
This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License (
https://creativecommons.org/licenses/by/2.0
), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Competing interests
The authors declare that they have no conflicts of interest concerning this article.
Authors’ contributions
ZLX was responsible for conception and design of the review, carried out the literature search, performed data analysis, and drafted the manuscript. HJ and ZX performed data extraction and assessment of risk of bias, participated in conception and design of the review, and critically revised the man ± uscript. LLM participated in conception and design of the review, and critically revised the manuscript. All authors read and approved the final manuscript.
