An overview of systematic reviews of complementary and alternative therapies for fibromyalgia using both AMSTAR and ROBIS as quality assessment tools

Fibromyalgia (FM) is a chronic pain disorder characterised by widespread pain [1]. It has been described as a ‘central sensitization syndrome’ caused by biological abnormalities in the central nervous system [2] and is often associated with other conditions such as irritable bowel syndrome and depression.

The recently revised FM diagnostic criteria (2011), approved by the American College of Rheumatology (ACR), use a FM Symptom Scale by combining the Widespread Pain Index (WPI) and Symptom Severity Scale (SS) (Wolfe et al. 2011 [3]). The WPI assesses 19 general body areas for pain occurring in the preceding 2 weeks. The severity of the person’s fatigue, unrefreshed waking, cognitive symptoms and general somatic symptoms are rated by the SS for a score ranging from 0 to 12 [3].

FM is reported to affect between 1 and 4% of the population [4]. The use of the new criteria has reduced the gender ratio form 7:1 to 2:1 female to male ratio, which is similar to other chronic pain conditions [5].

FM can develop at any age, including childhood, and there does not appear to be any variation in prevalence with regard to country, culture or ethnic group. Surprisingly, there does not appear to be any variation in industrialised/non-industrialised countries [6].

Medication is currently the main form of treatment; there is strong evidence of an effect for several drugs like antidepressants (e.g. amitriptyline) and muscle relaxants (e.g. cyclobenzaprine) [7, 8]. However, adverse effects of medication are frequently experienced [9‐12]. FM is difficult to treat within primary care, and people with FM often turn to complementary and alternative medicine (CAM) therapies; therefore, it is a condition that has received much attention from CAM researchers [13]. Prior research has found that around 90% of people with FM have used at least one form of CAM to manage their symptoms [14‐17].

CAM has been defined as ‘…diagnosis, treatment and/or prevention which complements mainstream medicine by contributing to a common whole, by satisfying a demand not met by orthodoxy or by diversifying the conceptual frameworks of medicine’ (Ernst et al.) ([18], p. 506). This review focuses on eight common CAMs which have featured in several CAM surveys [19‐21]: acupuncture, hypnotherapy, homoeopathy, osteopathy, chiropractic, herbal medicine, reflexology and aromatherapy (see Appendix 1 for further details on each therapy).

There are two main aims within this overview. The first is to update the synthesis of reviews of CAM literature on FM and establish what evidence is currently available with regard to the efficacy of several CAM practices used in its treatment. As systematic reviews (SR) are often considered the least biased source of evidence to evaluate the efficacy of a particular intervention, this overview will focus on SRs for FM.

The second aim is to provide a robust assessment of the evidence in this area using two complementary quality assessment tools: AMSTAR [22] and ROBIS [23].

Taking a look at previous overviews from the last 5 years, in 2012, Terry et al.’s [1] overview of reviews of CAM for FM identified five systematic reviews. The reviews found some evidence of beneficial effects for acupuncture, homoeopathy, hydrotherapy and massage, whilst no evidence for therapeutic effects for chiropractic treatment of FM symptoms. However, no quality assessment of the individual reviews was performed.

In 2015, Launche et al. [24] also published a synthesis of CAM for FM reviews. The AMSTAR scale [22] was used to assess the quality of the review. In contrast to our overview, Lauche et al. [24] did not restrict the type of CAM, whereas we restricted to the most common CAMs. In addition, we wanted to apply a more rigorous risk of bias assessment to the systematic reviews identified; AMSTAR focuses on the methodological quality of the reviews rather than risk of bias, so we wanted to compensate for that.

In our overview, all eligible systematic reviews of FM were assessed using both the AMSTAR scale [22] and the ROBIS tool [23]. This will provide an up-to-date and rigorous overview of evidence of CAM for FM.

Two reviewers (RP, VL) independently extracted data and summarised the review in a characteristic table (see Table 1). Data was extracted from full-text reviews using a standardised data extraction form. The extraction form was piloted prior to starting the overview and refined. Disagreements were resolved through discussion with a third reviewer (RC). Information was extracted from each included review on author, date of review, country, list of studies included in the individual review, intervention and comparator summary, number of participants, diagnosis criteria, meta-analysis results or summary of main between-group results, whether a sensitivity or subgroup analysis was conducted, risk of bias assessment and adverse events.

We extracted the mean and standard deviation (SD) of continuous variables and any between-group statistical analyses. We reported the standard mean difference (SMD) and 95% confidence intervals (CI) and results of any tests of heterogeneity reported in the relevant meta-analyses. If ‘pain’ was measured alongside another outcome (e.g. discomfort) and recorded as a single variable, we would extract the data and highlight this in the table and text.

Due to the expected overlap of studies and heterogeneity between reviews (particularly with regard to interventions and comparator arms), we conducted a narrative synthesis of the findings rather than pooling of meta-analyses from the included reviews.

In our protocol [PROSPERO CRD42016035846], we said we would not apply any language restrictions; however, it was decided that we would only include English language papers as the ROBIS tool would be a complex tool to ask someone to extract data with.

The search strategy yielded 568 potentially relevant papers for inclusion. After 125 duplicate titles were removed, 443 remained. Once screened, 98 papers were identified as potentially eligible and full-text copies were retrieved and reviewed by the two reviewers (RP, VL) (see Fig. 1 for flow diagram). From these papers, 15 were included in this overview, and the reasons for excluding articles are presented in Table 4 in Appendix 2. Results of the included studies are presented in Table 1. The summarised AMSTAR scores are presented in Table 2, and the summarised ROBIS scores are presented in Table 3. The justification statements for ROBIS are presented in Table 6 in Appendix 4.

The 15 included reviews were published between 2003 and 2014 and originated from seven countries. The included systematic reviews investigated the following therapies: homoeopathy (n = 2), acupuncture (n = 7), chiropractic (n = 1), herbal medicine (n = 1) and multiple CAMs (n = 4).

We identified seven systematic reviews [43‐49] that assessed acupuncture for FM alongside the four multiple CAM reviews [30‐34]. One of the earlier reviews was conducted by Mayhew and Ernst [43] and included five RCTs [50‐54] (n = 316) of various forms of acupuncture versus sham acupuncture (non-stimulation of acupuncture point or stimulation at traditional needle location). A meta-analysis was not performed, but the authors reported that three of the five included studies [51‐53] found an effect of acupuncture. These effects were, however, mostly short-lived and, therefore, of debatable value [43]. Of the remaining two trials that did not favour acupuncture, one [54] was considered well designed and of good methodological quality using the Jadad scale [39].

Langhorst et al.’s [44] pooled analysis included seven studies [50‐52, 54‐57] (n = 242) and found strong evidence for the reduction of pain (SMD −0.25; 95% CI −0.49 to −0.02; P = 0.04, I ² = 1%) at post-treatment compared to sham or simulated acupuncture. The methodological quality was assessed by the 11-item van Tulder score [58]). Sensitivity analyses demonstrated a significant effect on pain at post-treatment in studies with high risk of bias whereas the effect on pain at post-treatment in studies of low risk of bias did not demonstrate an effect.

Martin-Sanchez et al. [45] found, from a pooled analysis of four studies [51, 52, 54, 55] (n = 257), a SMD between acupuncture and sham groups from baseline of 0.02 (95% CI −0.24 to 0.28) with regard to pain intensity, but with wide confidence intervals which included the null value. Between-study homogeneity was in evidence (P = 0.41, I ² = 0%) for this comparison. No assessment of quality or risk of bias was reported.

The meta-analyses conducted by Yang et al. (2014) [46] were inaccurate as they used the control group twice in the same analyses for two studies [53, 55]. Thus, we chose not to report the results from the meta-analyses here.

Cao et al. (2013) [47] found that acupuncture had no better effect than sham acupuncture with regard to pain relief in people with FM, as shown in pooled analysis of seven arms from five trials [52, 54, 55, 57, 59]. The change in VAS score was reported as SMD −0.09 (95% CI −0.32 to 0.14, P = 0.44 I ² = 2%) and the VAS results at post-treatment were SMD −0.22, (95% CI −0.51 to 0.07, P = 0.13, I ² = 26%). However, a pooled analysis of four trials [60‐63] showed acupuncture was better than antidepressants in reducing VAS pain scores: SMD −0.60 (95% CI −0.93 to −0.27, P = 0.0004, I ² = 22%). The small sample size, scarcity of studies for each comparison, and lack of an ideal sham weakens the level of evidence and its clinical implications. The only analyses we have reported here was that conducted on studies that compared acupuncture alone which did not incorporate mixed therapies in the meta-analyses. Two out of the 16 trials were evaluated as low risk of bias [55, 63], four [50, 54, 62, 63] were rated as having unclear risk of bias, whilst the other ten trials were evaluated as high risk of bias. Nine trials described randomization [51, 54, 55, 59‐62, 64, 65], and six trials reported adequate allocation concealment [50, 51, 55, 59, 60, 64]. Three trials blinded both patients and outcome assessors [51, 55, 63]. Five trials reported the number of dropouts [51, 57, 63‐65], and none of these trials used intention-to-treat analysis.

Deare et al.’s [48] Cochrane review identified eight RCTs and one quasi-RCT [66]. This is one of the most up-to-date systematic reviews on acupuncture. Pain severity (VAS 100 mm) showed a reduction in pain for those treated with real acupuncture compared with no acupuncture at the end of treatment (mean difference (MD) −22.40 points on a 100-point VAS scale; 95% CI −40.98 to −3.82, P = 0.02, favouring acupuncture). This was based on just one study [67]. Pain severity using pooled analysis of six studies of the VAS, numerical rating scale (NRS), the Westhaven Yale Multi-dimensional Pain Inventory (MPI) [68] and MPQ found no difference between groups in reducing pain ((N = 286) SMD −0.14; 95% CI −0.53 to 0.24, P = 0.48, I ² = 54%). A short-term benefit of acupuncture over antidepressants was found in one study [66]; VAS = −17.3 on a 100-point scale; 95% CI −24.1 to −10.5. All studies except one were at low risk of selection bias; five were at risk of selective reporting bias (favouring either treatment group); two were subject to attrition bias (favouring acupuncture); three were subject to performance bias (favouring acupuncture) and one to detection bias (favouring acupuncture). Using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) tool [42], they established there was low- to moderate-level evidence that compared with no treatment and standard therapy, acupuncture improved pain and stiffness in people with FM. There was moderate-level evidence that the effect of acupuncture does not differ from sham acupuncture in reducing pain. Electro-acupuncture was better than manual acupuncture for pain and stiffness reduction, although the effects were not maintained at 6 months follow-up.

With regard to the eight CAMs we were interested in, our overview is in agreement with Lauche et al.’s [24] work which suggested that acupuncture had the best evidence of effectiveness for FM, conflicting results for chiropractic and inconclusive results for homoeopathy and phytotherapy (herbal medicine). In addition, some reviews that we identified were missing from Lauche’s overview [29, 46, 47, 49]. It is unclear from their inclusion/exclusion criteria why these four reviews would have been excluded. Thus, our overview provides a more up-to-date overview of the selected CAMs.

Our overview also drew similar conclusions to Terry et al.’s [1]. They also found some evidence of beneficial effects arising from both acupuncture and homoeopathy for the treatment of FM symptoms, whilst no evidence for therapeutic effects from chiropractic interventions was found.

To date, AMSTAR is one of the main scales for assessing quality of a systematic review. It is quick and easy to complete, and there was good inter-rater reliability (kappa = 0.70, agreement 83.6%). In general, there was consistency between ROBIS and AMSTAR. Five reviews [28, 44, 46‐48] achieved a high overall rating (scores >6) with the AMSTAR scale (although AMSTAR is not designed to have a final score). These five reviews also all scored low risk of bias on ROBIS. There were discrepancies on rating for three reviews; Mayhew and Ernst [43] achieved a low risk of bias but scored just 3 on AMSTAR, Yang et al. [46] achieved a high risk of bias but scored 6 on AMSTAR and de Souza Nascimento et al. [75] achieved a low risk of bias score on ROBIS but scored 4 on AMSTAR.

If a meta-analysis was included, this made rating domain 4 of the ROBIS tool easier to complete. Narrative syntheses were much harder to rate on this particular domain. There is little information in the ROBIS guidance document on how to score the signalling questions where no quantitative synthesis has been done or where the small number of studies included in the quantitative synthesis does not permit exploration of the data with regard to heterogeneity, robustness of the finding and quality. De Silva et al. [32] was an interesting review. Despite scoring high or unclear for domains 1–4 they still achieved a low score overall; this was because they did not overemphasise their findings and were able to critique their shortcomings of the review process. This highlights one of the strengths of ROBIS.

One author evaluated their own work [RP: 28] and one of the developers of ROBIS (PD) was involved in the applying ROBIS to assess the included reviews. Another of the developers of ROBIS (RC) was involved in the write up of the report. Although the search strategy was comprehensive, it is possible that some relevant reviews may not have been identified. In addition, a limitation of the overview is that several of the included reviews would be considered out of date (more than 5.5 years) [85]. Some reviews were excluded, due to language restrictions we imposed. This was due to requiring a trained person in the ROBIS tool to complete the assessment. This meant two potential reviews were excluded due to language [86, 87] (see Table 4 in Appendix 2). Despite these issues we believe the systematic approach to this overview minimises bias. Difficulties in using ROBIS may have led to errors in interpretation; lower inter-rater reliability was achieved than when using AMSTAR. In addition, CAM papers tend to be published in lower impact journals and often restricted by word count. Earlier reviews did not tend to score so highly on either tool probably because reporting criteria have changed over time.