This study aimed to evaluate the quality of reports about randomized controlled trials (RCTs) of scalp acupuncture (SA) for the treatment of vascular dementia (VD).
Method
A systematic search of reports published through to December 2015 was performed in eight databases. The quality of RCTs that used SA as an intervention for VD was evaluated based on the 2010 Consolidated Standards for Reporting of Trials (CONSORT) and 2010 Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) guidelines. Thirteen items from the CONSORT guideline were scored to give an overall quality score (OQS, range 0–13), and a combined key methodological index score (MIS) (range 0–5) of five key methodological items was measured. The OQS of 17 items from the STRICTA guideline (range 0–17) was also measured.
Results
In total, 26 reports were evaluated. The median OQS based on the CONSORT guideline was 8 (minimum 5, maximum 11), and “trial design,” “sample size,” “ancillary analyses,” and “harms” had a positive rate of less than 10%. The median MIS was 2 (minimum 0, maximum 5), with “allocation concealment and implementation,” “blinding,” and “intent-to-treat analysis” having a positive rate of less than 15%. The median OQS based on the STRICTA guideline was 12 (minimum 8, maximum 14), with “extent to which treatment was varied (1c),” “number of needle insertions per subject per session (2a),” and “setting and context of treatment (4b)” having a positive rate of less than 10%.
Conclusions
The overall quality of reports on RCTs of SA treatment for VD was moderate to low. The quality of methodological items was markedly lower than that of other items. The CONSORT and STRICTA guidelines should be used more frequently to standardize the quality of RCT reports of SA treatment for VD.
Abkürzungen
CONSORT
Consolidated Standards of Reporting Trials
ITT analysis
Intention-to-treat analysis
MIS
Methodological index score
OQS
Overall quality score
RCT
Randomized controlled trial
STRICTA
Standards for Reporting Interventions in Controlled Trials of Acupuncture
WHO
World Health Organization
Background
Vascular dementia is the second most common cause of dementia after that caused by Alzheimer’s disease [1‐3]. It is thought to result from cognitive impairment caused by changes resulting from cerebrovascular disorders [4] and from various outcomes of ischemic and hemorrhagic encephalopathy that induce mental and physical disabilities [5].
According to a report from the World Health Organization (WHO), there were currently estimated to be 35.6 million people with dementia worldwide in 2012, which will double by 2030 and more than triple by 2050 [6]. Dementia is a challenge to patients, caregivers, and healthcare providers [7] and carries a heavy financial burden, with the annual cost of care per patient ranging from 17,000 to 55,200 US$ for serve dementia [8].
Anzeige
Scalp acupuncture therapy stimulates the lesion area on the scalp to produce therapeutic effects. Although this treatment modality has been performed for more than 1000 years, the treatment has advanced in recent years [9], and the WHO published the Standard International Acupuncture Nomenclature in 1991 [10]. The efficacy of scalp acupuncture for vascular dementia has been confirmed by numerous empirical clinical studies [11‐13]. However, a more reliable scientific method should be developed to validate the effectiveness of scalp acupuncture for vascular dementia.
Some researchers have implemented randomized controlled trials (RCTs) as a means to validate the therapeutic effects and efficacy of acupuncture, and this study design is considered optimal to assess the effectiveness of an intervention [14]. However, despite the effective design of an RCT, inappropriate study methods can affect the reliability and validity of outcomes [15]. Thus, research methodology is a critical factor that determines overall study quality [16]. Hence, there is an urgent need to assess the quality of RCTs based on systematic quality control and assessment for all stages of clinical trials, from planning and execution to analysis [17].
The Consolidated Standards of Reporting Trials (CONSORT) guideline, which was developed in 2001 and updated in 2010, was intended to assist in the assessment and interpretation of parallel-group RCTs so that users could identify biased outcomes and ultimately improve RCT reporting [18]. The Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) guideline was designed in 2001 and revised in 2010 to assist in the reporting of clinical studies on acupuncture [19]. The combination of these guidelines helps researchers to assess the completeness and transparency of RCTs [20].
The aim of this study was to evaluate the quality of reports on domestic and foreign RCTs in which scalp acupuncture was used for the treatment of vascular dementia, based on the CONSORT and STRICTA guidelines, in order to provide preliminary data from clinical trials on scalp acupuncture for vascular dementia.
Anzeige
Methods
Collection of the RCTs
We searched for all reports about RCTs in which scalp acupuncture was used for the treatment of vascular dementia that were published before the end of December 2015 in eight databases (PubMed, EMbase, Cochrane Library, China National Knowledge Infrastructure, National Institute of Informatics Scholarly and Academic Information Navigator, National Digital Science Library, Korean Traditional Knowledge Portal, and Korean Studies Information Service System). The search terms were “vascular dementia, dementia,” and “scalp acupuncture.”
Literature selection/exclusion criteria
Types of studies
RCTs in which a control group of patients with vascular dementia who received a placebo, sham treatment, or conventional treatment was compared with an experimental group treated with scalp acupuncture were included. Non-randomized, cross-over RCTs, case reports, and case-control studies were excluded.
Types of participants
The subjects of RCTs were patients who were diagnosed with vascular dementia based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, the U.S. National Institute of Neurological Disease and Stroke and the Association Internationale pour La Recherche et l'Enseignement en Neurosciences, or definitively via computed tomography or magnetic resonance imaging, regardless of age, sex, and other demographic factors.
Types of intervention groups
The types of scalp acupuncture interventions included were body electro-acupuncture and body acupuncture, accompanied by drug therapy using Western or Chinese medicine or physical therapy such as exercise and rehabilitation.
Assessment of the quality of the reports
This study assessed the quality of the reports based on the 25-item CONSORT 2010 guideline and the 6-item STRICTA 2010 guideline. Each item on the checklists detailed in these reports was answered with a “yes” or “no.” Two raters independently performed the assessments with reference to the explanation provided by the CONSORT guideline and STRICTA guideline. Their assessments were compared, and any discrepancies were determined by consensus with the third author [21, 22].
Rating of overall reporting quality
Each of 13 items included in the CONSORT guideline was scored to compute an overall quality score (OQS) (range 0–13) (Table 1) [23‐25]. The discussion section items of the CONSORT guideline were excluded, due to the difficulty of objectively evaluating them [26‐28]. Each of 17 items in the STRICTA guideline was also scored (range 0–17) (Table 2). For scoring of the quality of items, 1 point was given if the information for each item was stated in the study, and 0 points if the information was not stated or was unclear [19, 29].
Table 1
Rating of overall quality using items from the CONSORT guideline (n = 26)
Item
Criteria
Description
Number of positive trialsa
Percentage
Cohen’s к coefficient
95% CI
1
“Randomized”
in the title or abstract
Study identified as a randomized controlled in the title or abstract
25
96
1.00
1.00
2
Background
Adequate description of the scientific background and explanation of rationale
14
54
0.68
0.39 to 0.97
3
Trial design
Description of trial design (such as parallel, factorial) including allocation ratio
2
8
1.00
1.00
4
Participants
Description of the eligibility criteria for participants
26
100
1.00
1.00
5
Interventions
Details of the interventions intended for each group
26
100
1.00
1.00
6
Outcomes
Definition of primary (and secondary when appropriate) outcome measures
19
73
0.78
0.49 to 1.07
7
Sample size
Description of sample size calculation
1
4
0.65
-0.03 to 1.32
12
Statistical methods
Description of the statistical methods used to compare groups for primary outcomes, subgroup analyses, or adjusted analyses
23
88
0.78
0.36 to 1.20
13
Flow chart
Details on the flow of participants through each stage of the trials (number of patients randomly assigned, receiving intended treatment, completing the protocol and analyzed)
22
85
0.62
0.12 to 1.12
14
Recruitment
Dates defining the periods of recruitment and follow up
21
81
0.71
0.32 to 1.10
17
Outcomes and estimation
For each primary and secondary outcome, a summary of results for each group is given, and the estimated effect size and its precision (for example, 95% CI)
20
77
0.75
0.42 to 1.08
18
Ancillary analyses
Clear statement of whether subgroup/adjusted analyses were prespecified or exploratory
0
0
1.00
1.00
19
Harms
Description of all important adverse events in each group
2
8
0.63
0.13 to 1.12
CONSORT Consolidated Standards of Reporting Trials. aPositive trials, the information for each item was stated so 1 point was given
Table 2
Rating of overall quality using items from the STRICTA guideline (n = 26)
Item
Criteria
Description
Number of positive trialsa
Percentage
Cohen’s к coefficient
95% CI
1
Acupuncture rationale
(1a) Style of acupuncture (e.g., traditional Chinese medicine, Japanese, Korean, Western medical, five element, ear acupuncture, etc.)
26
100
1.00
1.00
(1b) Reasoning for treatment provided, based on historical context, literature sources and/or consensus methods, with references where appropriate
22
85
0.84
0.52 to 1.15
(1c) Extent to which treatment was varied
1
4
1.00
1.00
2
Details of needling
(2a) Number of needle insertions per subject per session (mean and range where relevant)
2
8
1.00
1.00
(2b) Names (or location if no standard name) of points used (uni-/bilateral)
26
100
1.00
1.00
(2c) Depth of insertion, based on a specified unit of measurement or on a particular tissue level
17
65
0.72
0.43 to 1.02
(2d) Responses sought (e.g., de qi or muscle twitch response)
20
77
0.88
0.66 to 1.10
(2e) Needle stimulation (e.g., manual or electrical)
13
50
0.52
0.19 to 0.86
(2f) Needle retention time
21
81
0.87
0.61 to 1.12
(2 g) Needle type (diameter, length and manufacturer or material)
22
85
0.84
0.52 to 1.15
3
Treatment regimen
(3a) Number of treatment sessions
23
88
0.78
0.36 to 1.20
(3b) Frequency and duration of treatment sessions
25
96
1.00
1.00
Other components of treatment
(4a) Details of other interventions administered to the acupuncture group (e.g., moxibustion, cupping, herbs, exercises, lifestyle advice)
15
58
0.67
0.37 to 0.97
(4b) Setting and context of treatment, including instructions to practitioners, and information and explanations to patients
1
4
0.65
-0.03 to 1.32
5
Practitioner background
(5) Description of participating acupuncturists (qualification or professional affiliation, years in acupuncture practice, other relevant experience)
22
77
0.74
0.43 to 1.08
6
Control or comparator interventions
(6a) Rationale for the control or comparator in the context of the research question, with sources that justify the choice(s)
9
35
0.76
0.51 to 1.02
(6b) Precise description of the control or comparator. If sham acupuncture or any other type of acupuncture-like control is used, provide details as for items 1–3 above
26
100
1.00
1.00
STRICTA Standards for Reporting Interventions in Controlled Trials of Acupuncture. aPositive trials, the information for each item was stated so 1 point was given
Rating of key methodological items
In the CONSORT guideline, the key items pertaining to methodology (i.e., “randomization,” “allocation concealment,” “blinding,” “baseline characteristics,” and “intention-to-treat (ITT) analysis”) were assessed separately as they were related to potential factors of bias [30‐32]. A combined key methodological index score (MIS) (range 0–5) of these five key methodological items was measured (Table 3). For each of the studies, 1 point was given when the reports had information on each item, and 0 points were given if the information was not stated or was unclear.
Table 3
Quality of key methodological items (n = 26)
Item
Criteria
Description
Number of positive trialsa
%
Cohen’s к coefficient
95% CI
8
Randomization
Description of the method used to generate the random sequence
14
54
0.68
0.39 to 0.97
9 and 10
Allocation concealment and implementation
Description of the method used to implement the random allocation sequence assuring the concealment until interventions are assigned
3
12
0.84
0.52 to 1.15
11
Blinding
Whether or not participants, those administering the interventions, or those assessing the outcomes were blinded to group assignment
3
12
1.00
1.00
15
Baseline data
An outline of baseline demographic and clinical characteristics of each group
11
42
0.92
0.77 to 1.07
16
Intent-to-treat analysis
No. of participants in each group included in each analysis and whether it was done by “intention to treat”
3
12
0.71
0.32 to 1.10
aPositive trials, the information for each item was stated so 1 point was given
Data extraction and analysis
Two raters independently performed the assessments with reference to a discussion about the definitions of each item and the instructions provided in the CONSORT and STRICTA guidelines. Their assessments of the reports were compared, and any discrepancies were resolved by consensus with the third author.
Cohen’s к-statistic was quantified to assess the inter-rater agreement. We defined agreement of 0.20 as “poor,” that greater than 0.20 but less than 0.40 as “low,” that greater than 0.40 but less than 0.60 as “moderate,” that greater than 0.60 but less than 0.80 as “substantial,” that greater than 0.80 as “good,” and agreement of 1 as “perfect” [23]. Items upon which there was any disagreement were specifically examined to reach an agreement. Cohen’s к-statistic analysis was performed using SAS software, version 9.3 (SAS Institute, Inc., Cary, NC, USA).
In order to evaluate the overall quality of reported RCTs and relevant factors, OQS was used as a dependent variable modeled using linear regression. Only variables with p ≤ 0.10 on univariate analysis were included in the multivariate regression model to identify significant variables (p ≤ 0.05). To analyze the factors related to methodological quality, the MIS was used as an outcome variable in regression analysis. Linear and ordinal regression analysis was performed using SPSS software version 20.0 (SPSS, Chicago, IL, USA).
Results
The RCT selection process for this study is depicted in Fig. 1. After identification, screening, and determining the eligibility of the reports, a total of 26 relevant RCTs were included in the final analysis.
Fig. 1
Flowchart of the article selection process. RCT randomized controlled trial
×
Features of the included RCTs
A number of RCTs on scalp acupuncture for vascular dementia were reported in the last 10 years. Most of these studies were reported in the last 8 years, with one in 2006 (3.8%), one in 2007 (3.8%), four in 2008 (15.4%), three in 2009 (11.5%), one in 2010 (3.8%), two in 2011 (7.7%), five in 2012 (19.2%), seven in 2013 (26.9%), two in 2014 (7.7%), and none in 2015 (0%) (Fig. 2).
Fig. 2
Number of publications
×
Anzeige
Twenty-two reports were written in Chinese (84.6%) and four reports were in English (15.4%), but all of the 26 reports were published in China by Chinese researchers. With regard to the types of interventions in the control group, among the 26 RCTs evaluated, Western medicine was used in 20 studies (76.9%), body electro-acupuncture was used in three studies (11.5%), and body acupuncture was used in one study (3.8%), with a sample size ranging from 40 to 241 subjects. Twelve studies were conducted with external funding (46.2%).
Quality of the reports
Rating of overall quality
Table 1 shows the ratings of overall quality based on the CONSORT guideline. The median OQS of the 26 RCTs was 8 (minimum 5, maximum 11). The items that were relatively more neglected in the reports were “trial design,” “sample size,” “ancillary analyses,” and “harms,” showing less than 10% of a positive rate. The two raters reached a substantial (items 2, 6, 7, 12, 13, 14, 17, and 19), perfect agreement (items 1, 3, 4, 5, and 18) for all items with the exception of “outcomes and estimation” (Table 1).
Table 2 shows the ratings of overall quality based on the STRICTA guideline. The median OQS of the 26 RCTs was 12 (minimum 8, maximum 14). Items that were relatively neglected by RCTs were “extent to which treatment was varied (1c),” “number of needle insertions per subject per session (2a),” and “setting and context of treatment (4b),” with less than 10% of a positive rate. The two raters reached substantial (items 2c, 3a, 4a, 4b, 5, and 6a), good, or perfect agreement (items 1a, 1b, 1c, 2a, 2b, 2d, 2f, 2 g, 3b, and 6b) for all items with the exception of “needle stimulation (2e)” (Table 2).
Rating of key methodological items
The median MIS value for the five items (items 8, 9, 10, 11, 15, and 16) was 2 (minimum 0, maximum 5). Particularly, “allocation concealment and implementation,” “blinding,” and “intent-to-treat analysis” had less than 15% of a positive rate (Table 3). The two raters reached substantial (items 8 and 16), good (items 9, 10, and 15), or perfect agreement (item 11) for all items (Table 3).
Exploratory analysis: factors associated with better quality of the reports
In a single-factor analysis of variance, all the factors related to the CONSORT and STRICTA guidelines were found to have no mutual association. Ordinal regression analysis showed that the variables to enhance the quality of the methodology were not associated with the MIS (p > 0.05).
Discussion
In this study, a comprehensive and systematic search of domestic and foreign RCTs on scalp acupuncture for the treatment of vascular dementia was performed, and the quality of these reports was assessed based on the CONSORT and STRICTA guidelines. These two guidelines are useful tools that were developed to identify gaps in the reporting of RCTs and have been widely used to assess the quality of reports on acupuncture studies [20].
Based on the CONSORT guideline, the median OQS of the 26 RCTs was 8, which was greater than 50% of the total score, and that of the STRICTA items was 12, which was greater than 70% of the total score. However, the “trial design,” “sample size,” “ancillary analyses,” and “harms” items of the CONSORT guideline and the “extent to which treatment was varied (1c),” “number of needle insertions per subject per session (2a),” and “setting and context of treatment (4b)” items of the STRICTA guideline were inappropriately addressed or neglected in most of the reports. This finding is in line with the results of other quality assessment studies [20, 25, 29, 30].
It is important for researchers to clearly state the design of clinical trial studies (randomized, parallel, double-blind, placebo-controlled, sham-controlled, etc.), including the details of the randomization, as indicated in the “trial designs” item of the CONSORT guideline. Even in cases of a parallel randomization ratio such as 8:8 for two groups, it is important to clearly provide the allocation ratio to improve the quality of reporting. The importance of trial design is especially highlighted among non-universal clinical trials with large sample sizes or complex analyses [31].
Anzeige
Sample size calculation is required to maintain a balance between the statistical considerations and the differences in therapeutic effects for intervention groups. A clinically significant difference between the intervention group and the control group generates reliable outcomes if the sample size is sufficiently large [20]. However, it may be difficult to achieve the planned sample size because recruiting qualified subjects within a limited time period is challenging [33, 34]. RCTs with a small sample size are often prone to biases or are insufficient in measuring the therapeutic advantages [26]. Therefore, researchers should pay more attention to sample size measurements by consulting with professional clinical statisticians in order to identify highly reliable and significant differences between control and intervention groups.
With regard to the “ancillary analyses” item, performing multiple analyses of the same data may introduce biases that lead to exaggeration in the interpretation of the study results [35], and reporting of analytical outcomes that were not prespecified in the research protocol leads to selective reporting bias for subgroup analyses [36]. Therefore, authors should report the results of prespecified analyses for high reliability, and clearly state the reason for and purpose of conducting subgroup analyses.
A randomized trial is the optimal design to generate efficacy and safety data, but it is difficult to detect rarely occurring “harms” through such a design. Many RCTs have provided inappropriate reports of abnormal responses [37] or low-quality reports of abnormal responses [38]. Furthermore, very few studies mention severe abnormal responses or subjects who dropped out because of abnormal responses [39]. Information about the risk and the benefits of an intervention is required to help the participants of clinical trials make reasonable and balanced decisions. The occurrence and characteristics of abnormal responses affect the acceptance and usefulness of a particular intervention [20].
The “extent to which treatment was varied (1c)” item had a 3% of a positive rate in this study, which is similar to the results of a previous study [22]. Variations between treatments in each clinical trial should be minimized by using a standardized protocol, and researchers should describe the degree of personalized treatment to both the patient and practitioner [40]. The “number of needle insertions per subject per session (2a)” item had an 8% positive rate in this study. Researchers should strictly report the total number of needles used and as many other related details as possible [40]. The “setting and context of treatment (4b)” item is an additional important component to report [41]. As changes in medical practice or context of treatment of the patients affect the outcome of experiments [29], researchers must report information that is provided to the patients about the interventions for treatment and control groups [20].
Anzeige
The median MIS, which represents the quality of methodological reports of the CONSORT guideline, was markedly low at 2, and the majority of studies had insufficient or missing information on “allocation concealment and implementation,” “blinding,” and “intent-to-treat analysis.” These key methodological items play an important role in preventing selection bias, performance/detection bias, and attribution bias [42]. An inappropriate study design could lead to exaggeration of the clinical outcome [23, 24, 43, 44], which has been reported in a few other quality assessment studies [45, 46]. In reports of RCTs of Chinese medicine, the quality of the reporting of the methodology is poor, warranting the use of CONSORT guidelines. We recommend that clinical researchers should be professionally trained with regard to the design and reporting of RCTs, and that the studies should be stringently peer-reviewed to improve the overall quality of research articles for their submission and publication in international journals.
Although we have comprehensively and systematically assessed 26 RCTs, there are a few limitations in our study. First, it was difficult to search for studies published in languages other than English and Chinese, such as those published in Japanese or other Asian languages. Nevertheless, most of the studies on scalp acupuncture treatment are published in Chinese or English. Second, only articles published between 2006 and 2015 were included in this study, which may be associated with the increased incidence of geriatric diseases caused by an upsurge of the elderly population in the last 10 years.
Conclusions
This study confirmed the quality of reporting of RCTs on scalp acupuncture for the treatment of vascular dementia. The CONSORT and STRICTA guidelines should be more widely used to enhance the quality of the reporting of RCTs in the future. The present study findings suggest that more reliable research on scalp acupuncture for the treatment of vascular dementia, including larger sample sizes, is warranted in the future.
Acknowledgements
The authors would like to express sincere thanks to our colleagues, staff of College of Korean Medicine Dong-Shin University, and DongShin University Gwangju Oriental Hospital, and Chonnam National University Medical School and Hospital for their support.
Funding
This study was supported by the Convergence of Conventional Medicine and Traditional Korean Medicine R&D program funded by the Ministry of Health and Welfare through the Korea Health Industry Development Institute (KHIDI) (HI14C0862).
Authors’ contributions
KJH, CJB, SJC, CMR, NCS, and HJY conceived and designed the experiments. YYN, CMR, and SMY performed the experiments. YYN and SMY analyzed the data. PJH and PGC contributed reagents/materials/analysis tools. YYN and KJH wrote the paper. KJH and SJC resolved disagreements and bias. All authors read and approved the final manuscript.
Competing interests
The authors declare that they have no competing interests.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
