Complementary and alternative medicine recommendations for depression: a systematic review and assessment of clinical practice guidelines

A systematic review was performed to identify guidelines for the treatment and/or management of depression following standard methods [28] and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria [29]. A protocol was registered with PROSPERO; registered number: CRD42019132295. Eligible guidelines that provided CAM recommendations were evaluated using the well-accepted and validated Appraisal of Guidelines, Research and Evaluation II (AGREE II) tool [30] twice, once for the overall guideline and a second time for the CAM sections of the guideline. The AGREE II instrument is comprised of 23 items organized within six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence.

The Population, Intervention, Comparison and Outcomes (PICO) framework was used to define the eligibility criteria for the depression guidelines. Eligible populations were adults aged 18+ years with unipolar or major depression. Guidelines that focused solely on geriatric populations however were excluded. In terms of interventions, only guidelines that focused on the treatment and/or management of depression were deemed eligible for the purpose of identifying whether any CAM therapies were mentioned or recommended. With respect to Comparisons, we aimed to assess the quality of depression guidelines, specifically commenting on CAM sections. In other words, we aimed to determine the proportion of depression guidelines that made CAM recommendations and the quality by which they did so. Outcomes included AGREE II scores to assess of the quality of the content and format of the overall and CAM sections of the guidelines. Eligible guideline were in the English language, accessible to the public or via the McMaster library system, and developed by non-profit organizations including academic institutions, government agencies, disease-specific foundations, specialty/expert groups, international organizations or professional associations or societies. This timeframe provided developers with a minimum of five years since the publication of AGREE II and thus time to apply the tool which outlines methods for the generation of high-quality guidelines. Publications that were not evidence-based clinical practice guidelines focusing on the treatment and/or management of depression were deemed ineligible. For example, consensus statements, protocols, abstracts, conference proceedings, letters or editorials, studies based on primary studies that evaluated depression management or treatment or studies that focused on depression curriculum, education, training, research, professional certification or performance were not eligible. It is important to note that the AGREE II tool was used to assess only those eligible guidelines that provided CAM therapy recommendations. Thus, depression guidelines that had no CAM mention or only mentioned CAMs without making any CAM recommendations were not assessed using AGREE II. This allowed for the comparison between AGREE II scores of the overall guideline and specifically the CAM sections. Only demographic information was collected and described for the eligible guidelines that only made mention of CAM therapies and did not make any CAM therapy recommendations.

On October 11, 2018, a search of three databases, MEDLINE, EMBASE, and CINAHL, was conducted from 2008 to October 9, 2018 inclusive. A sample search strategy is provided in Supplementary File 1. In addition, the Guidelines International Network (G-I-N), a website containing a library or repository of guidelines [31] was searched with keywords reflective of the eligibility criteria of this study, including “depression”, “major depressive disorder”, and “depressive disorder”. Finally, the NCCIH website was also searched as it provided a single list of CAM guidelines [https://​nccih.​nih.​gov/​health/​providers/​clinicalpractice​.​htm]. To identify the eligible sources, a two-step screening process was conducted by ZN and HN individually; they each screened the titles and abstracts of all sources, after which they each screened the full texts of the sources. Each screening step was standardized by JYN who helped to resolve discrepancies in the selection of eligible sources between ZN and HN.

The following data were extracted and summarized from each of the eligible guidelines: date of publication; country of first author; type of organization that published the guideline (academic institutions, government agencies, disease-specific foundations, specialty/expert bodies or groups, international organizations, or professional associations or societies); and if there was any mention of CAM. Moreover, the following data were also extracted: the specific CAM therapies that were mentioned if present; recommendations of CAM therapies if present; CAM funding sources; and whether any CAM providers were members of the guideline development group or panel. Associated tools and/or resources to support the implementation of the guidelines were sought from each guideline’s developer website. Data from these resources were used to assess applicability. Some CAM therapies mentioned in the eligible guidelines of this study were not widely known as or as easily defined as either standard non-CAM or CAM treatments by ZN and the other researchers, so the literature was searched to seek an understanding of each therapy and determine it as either CAM or non-CAM [12, 17, 32‐50], while consulting the NCCIH and literature that has previously provided an operational definition of CAM [16, 51].

The quality of the eligible guidelines was assessed using the AGREE II instrument following standardized methods described in the user manual [30]. All 23 items within the 6 domains of the AGREE II tool were scored on a 7-point Likert scale, where “1” reflects “strongly disagree” and “7” reflects “strongly agree”. Next, the quality of the overall guideline was rated on the same scoring system for an overall assessment score and then it was determined if the guideline should be recommended for use or not. Initially, a pilot test was performed in which all three authors applied the AGREE II tool to appraise three separate guidelines, after which inconsistencies in scores were discussed and resolved to standardize scoring. The AGREE II tool was then independently applied to the eligible guidelines of this study by ZN and HN. Those guidelines with CAM recommendations were assessed using the AGREE II tool twice to assess the overall guideline and then to assess the CAM sections specifically. The AGREE II questions were modified for application to the CAM sections of the guidelines specifically, as described in Supplementary File 2. Discrepancies between the scores of ZN and HN were discussed and resolved in consultation with JYN, without unduly modifying scores. Next, the average appraisal scores, average overall assessments, and scaled domain percentages of the guidelines were generated. Each appraiser’s average of all 23 scored items for each guideline was calculated separately and those two values were then averaged to generate the average appraisal score of each guideline. The overall assessment scores of both appraisers were averaged for each guideline to generate the average overall assessments. Scaled percentages were calculated for each domain to allow for comparisons between the different aspects of each guideline. The scaled domain percentage was calculated for each of the six domains by taking the sum of the individual scores of the two appraisers within the domain, scaling it to the minimum and maximum possible score for that domain, and then transforming it into a percentage. For each domain, the average of the guidelines’ scaled domain percentages was also generated.

Searches retrieved 1043 total items, 931 were unique, and 863 titles and abstracts were eliminated, leaving 68 full-text guidelines that were considered. Of those, 49 were not eligible, because they were not clinical practice guidelines (n = 16), they were not focused on depression (n = 1), they addressed depression associated with another condition or disease (n = 5), there was a more recent version of the guideline available (n = 4), they were not in English (n = 9), they were published prior to the eligible timeframe (n = 5), they did not focus on the target population (n = 7), they were not treatment focused (n = 1), or they were withdrawn/invalid guidelines (n = 1), leaving 19 guidelines eligible for review [52‐70]. Thus, a total of 19 guidelines were eligible, of which 1 mentioned CAM but did not make any CAM recommendations [68], and 2 made no mention of CAM [69, 70]. The remaining 16 guidelines made CAM therapy recommendations; these guidelines were assessed using the AGREE II tool.

Eligible guidelines were published from 2008 to 2018 in the USA (n = 5), Canada (n = 4), Germany (n = 3), the UK (n = 2), Australia (n = 1), China (n = 1), Poland (n = 1), Scotland (n = 1), and Spain (n = 1) [52‐70]. The guidelines were funded and/or developed by professional associations or societies (n = 5), academic institutions (n = 2), international organizations (n = 2), specialty/expert groups (n = 7), a combination of a specialty/expert body and a government agency (n = 1), a combination of a specialty/expert group and an international organization (n = 1), and a specialty/expert group and an academic body (n = 1).

CAM recommendations were made in 16 guidelines [52‐67] and included the following therapies: St. John’s Wort (Hypercium perforatum) (n = 9), SAMe (S-adenosyl-L-methionine) (n = 4), exercise/physical activity (n = 12), yoga (n = 1), acupuncture (n = 3), specialized forms of acupuncture including electro-acupuncture, scalp acupuncture, filiform needle acupuncture, body acupuncture involving regulating mind and soothing the ‘Gan’, scalp electro-acupuncture (n = 1), ear (pressure) therapy/ pressing of ear points with herbal seeds (n = 1), relaxation training/therapy (n = 1), (bright) light therapy (n = 5), (guided) self-help/self-management (n = 5), bibliotherapy (n = 3), mindfulness based therapy/mindfulness based cognitive therapy (MBT/MCBT) (n = 6), acceptance and commitment therapy (ACT) (n = 2), sleep deprivation/wake therapy (n = 2), saffron (Crocus sativus) (n = 1), omega-3/ polyunsaturated fatty acids (n = 4), vitamin D (n = 1), folate/L-methylfolate (n = 3), inositol (n = 1), dehydroepiandrosterone (DHEA) (n = 1), tryptophan (n = 1), acetyl- L-carnitine (ALC) (n = 1), lavender (Lavendula) (n = 1), and rose root (Rhodiola rosea) (n = 1). The aforementioned characteristics of these 16 guidelines are described in Table 1 and a summary of the CAM recommendations in these eligible guidelines are presented in Fig. 2. One of the 16 guidelines had a CAM funding source, and 5 guidelines included active or retired CAM providers as part of the guideline development group.

There was one guideline that only made mention of CAM without making any CAM recommendations [68]. Regarding the one psycho-therapeutic CAM mentioned, which was acceptance and commitment therapy (ACT), only a brief statement was made indicating that this is an available treatment option for depression without any comment on supporting evidence. A detailed discussion comparable to the non-CAM treatments in this guideline was made for acupuncture, exercise, and St. John’s Wort. A detailed description of the adverse effects and risks/side effects of St. John’s Wort was specifically discussed as well. The remaining CAM therapies, including omega-3 fatty acids, SAMe, fatty acids, yoga, and meditation, were vaguely mentioned with brief statements on the quality of evidence relative to the non-CAM treatments which were discussed in more detail.

The average appraisal scores and average overall assessments regarding use for each of the 16 guidelines with CAM recommendations are presented in Supplementary File 3. From the overall average appraisal scores of each of the guidelines, the lowest score was 3.4 while the highest was 6.0, using the seven-point Likert scale (where “7” reflects strongly agree that the criteria for that item is fulfilled); only 1 guideline received an average appraisal score lower than 4.0, 10 guidelines received an average appraisal score greater than 4.5, and 5 guidelines received an average appraisal score greater than 5.0. Meanwhile, the lowest average overall assessment score was 3.3, while the highest was 6.0, of the 16 guidelines that were assessed; only 1 guideline had an average overall assessment score lower than 4.0, 11 guidelines achieved a score of 4.5 or greater, and 5 guidelines achieved a score of 5.0 or greater.

The average appraisal scores and average overall assessments regarding use for each of the 16 guidelines with CAM recommendations are shown in Supplementary File 3. From the average appraisal scores for the CAM sections of each of the 16 guidelines, the lowest score was 1.9 and the highest score was 5.8 on the seven-point Likert scale; 3 guidelines received an average appraisal score lower than 3.0, 11 guidelines received an average appraisal score of 4.0 or greater, and 2 guidelines received an average appraisal score greater than 5.0. For the CAM average overall assessments of the 16 guidelines, the lowest score was 2.0, while the highest was 6.0; 4 guidelines achieved an average overall assessment score of 3.0 or lower, 11 guidelines achieved a score of 4.0 or greater, and 2 guidelines achieved a score of 5.0 or greater.

The two appraisers only recommended 1 of the 16 guidelines as Yes without modifications [65]. For 13 of the 16 guidelines, both appraisers concurred in their decision regarding the overall recommendation; one guideline was rated as No by both appraisers [52], and 11 were rated as Yes with modifications by both appraisers [53, 54, 56, 58‐63, 66, 67]. Finally, 3 guidelines were recommended as Yes and Yes with modifications [55, 57, 64].

The two appraisers only recommended 1 of the 16 guidelines as Yes without modifications [65]. For 13 of the 16 guidelines, both appraisers concurred in their decision regarding the overall recommendation; five guidelines were rated as No by both appraisers [52, 56, 60, 63, 67] and 8 were rated as Yes with modifications by both appraisers [53‐55, 57, 58, 61, 62, 66]. Finally, 2 guidelines were recommended by the two appraisers as No and Yes with modifications [59, 64].

The scaled domain percentages of the overall guidelines were generated: for the scope and purpose domain, the lowest score was 41.7%, while the highest score was 100.0%; for the stakeholder involvement domain, the lowest score was 36.1%, while the highest was 88.9%; for the rigor-of-development domain, the lowest score was 20.8%, while the highest was 83.3%, for the clarity-of-presentation domain, the lowest score was 75.0%, while the highest was 100.0%; for the applicability domain, the lowest score was 8.3%, while the highest was 72.9%; and for the editorial independence domain, the lowest score was 25.0%, while the highest was 95.8%.

The scaled domain percentages of the CAM sections of the guidelines were also generated: for the scope and purpose domain, the lowest score was 33.3%, while the highest was 100.0%; for the stakeholder involvement domain, the lowest score was 8.3%, while the highest was 77.8%; for the rigor-of-development domain, the lowest score was 4.2%, while the highest was 83.3%; for the clarity-of-presentation domain, the lowest score was 13.9%, while the highest was 100.0%; for the applicability domain, the lowest score was 2.1%, while the highest was 58.3%; and for the editorial independence domain, the lowest score was 25.0%, while the highest was 95.8%.

With regard to the overall scaled scores, the overall objectives and health questions were specific and clear for all the guidelines except one [52]. Moreover, for those guidelines with high overall scaled scores, the purpose and health intent were well defined. Meanwhile, most guidelines had CAM scaled scores that were equal to their overall scaled scores in this domain, with the exception of 4 guidelines [52, 57, 60, 64, 67]. For those guidelines that had the same CAM and overall scaled score, the overall objectives encompassed CAM therapies or did not explicitly exclude them. The guidelines with lower CAM scaled scores generally had vague, if any, health questions specific to CAM as compared to specific health questions listed for the non-CAM therapies [52, 57, 60, 64, 67]. Meanwhile, the target populations were generally well-defined for all guidelines and there was no distinction made between the target population for CAM versus non-CAM therapies.

Most guidelines described the guideline development group members well; two guidelines fulfilled all of the criteria including the names, affiliations, expertise and geographical locations of the development members [52, 55], while the remaining were missing at least one of the criteria [53, 54, 56‐67]. A few guidelines included CAM experts; some provided the same level of description for CAM experts as the other members [54‐56, 58, 63, 66], and one guideline had CAM experts listed as some of the consulted stakeholders, with unclear involvement and little to no description [65]. Only two guidelines described the views and preferences of the target audience in detail [61, 65], while most of the guidelines described them vaguely [52, 55‐57, 59, 62, 64, 66]. In terms of the CAM subsections, most guidelines had no difference between their description level of the views and preferences of CAM therapies in the target population compared to the non-CAM therapies, except for two which provided less detail [59, 61]. A few guidelines described both the intended audience and how the guideline may be used [55, 57, 60‐62, 65], while the remaining provided a description of either the intended audience or usage, except for one which was missing a description of both [52]. There was no difference between the overall and the CAM scores for the description of the target users as the target users were stated for the entire guideline and not for specific therapies.

The majority of the guidelines used well-defined systematic methods to search for evidence for the overall guidelines [53‐57, 61, 63‐66], while some lacked a comprehensive search strategy [52, 58‐60, 62, 67]. Some of the guidelines clearly defined their criteria for selecting the evidence [55‐57, 63‐65], while the remaining provided minimal detail. In terms of the CAM sections, most guidelines clearly used systematic methods to search for CAM-related evidence except for two [52, 63]. One guideline described the criteria for selecting evidence pertaining to CAM therapies in less detail compared to that of the non-CAM therapies [64]. The strengths and limitations of the overall evidence of almost all of the guidelines were well-defined, while one had a vague description [63], and one lacked a description of the strengths and limitations of the evidence altogether [52]. There was only one guideline that described the strengths and limitations of CAM evidence vaguely relative to the overall evidence [63]. The methods for formulating the recommendations were clearly described for most guidelines [53, 54, 56, 64, 65, 67], and the quality of the CAM sections for this item was the same as the overall guidelines. The health benefits, side effects, risks and trade-offs for the recommendations were described by most guidelines to varying degrees. In regards to the CAM sections, a few guidelines failed to list the benefits or harms for all of the CAM therapies, only listing them for a few [55, 57, 63] and one guideline did not mention any consideration of health benefits, side effects, and risks in formulating CAM recommendations [52]. All the guidelines made an explicit link between the overall recommendations and supporting evidence. There was only one guideline that did not provide an explicit link between the CAM recommendations and the supporting evidence [52], while the remaining guidelines did. Some guidelines vaguely stated the involvement of an external review [55, 57, 61, 64‐66], and two guidelines clearly stated that no external review was conducted [53, 54]. None of the guidelines explicitly had CAM experts involved in an external review process, except for one [66]. Most guidelines provided a statement that the guideline will be updated [55, 57, 58, 61, 62, 64‐66], with one outlining a detailed methodology for the updating procedure. The CAM scores were the same as the overall scores for the updating procedure.

The overall recommendations of all guidelines were specific and unambiguous, except for one [66]. On the other hand, the CAM recommendations for many guidelines were vague or had limited detail [52, 55‐57, 60, 61, 63, 67]. All guidelines provided the different options of non-CAM therapies for the management and/or treatment of the depression clearly. In contrast, some guidelines presented limited or vague CAM therapy options within the scope of the guideline [52, 56, 60, 63, 67]. The overall key recommendations were easily identifiable for many guidelines [52, 55‐59, 61‐67]. Some guidelines did not present their CAM recommendations clearly [52‐54, 56, 57, 60, 63, 67].

One guideline described facilitators and barriers for the implementation of both the overall and the CAM-specific recommendations in detail [64], while some failed to address the facilitators and barriers clearly for both sections [54, 56, 60, 62]. The remaining guidelines vaguely addressed these factors for both sections, with a few guidelines simply listing some factors that applied to all treatment modalities for their CAM sections [52, 53, 55, 57‐59, 61, 63, 65‐67]. The majority of the guidelines provided advice and/or tools to facilitate the integration of the overall recommendations into practice except for two [56, 59]. There were some guidelines that did not provide any advice or tools specifically for CAM recommendations [52, 56, 59, 62, 63]; two guidelines only provided an implementation section that addressed advice for all treatments in general thereby encompassing CAM recommendations [58, 60]. A few guidelines described the resource implications of implementing the overall recommendations [53, 55, 57, 59, 61‐66]. In regards to the CAM sections, two guidelines sufficiently reported the resource implications for CAM recommendations [55, 66], while some guidelines addressed the implications vaguely with generalized statements applying to all modalities [53, 57, 62, 64, 66]. Some guidelines vaguely described monitoring or auditing criteria for the overall sections [53, 54, 56, 58, 63, 66, 67], while the others described it more clearly. Some guidelines did not adequately report any monitoring criteria for CAM-specific recommendations [52, 53, 55, 56, 58, 60, 63] and one guideline reported the criteria thoroughly for both the overall and CAM sections [64].

Some guidelines did not explicitly state that the funding body did not influence the content of guidelines [52‐54, 58, 61, 65‐67], and two guidelines disclosed the influence of the funding body in place of that statement [57, 64]. Two guidelines simply stated that there was no “commercial funding body” involved in the guideline development [60, 62], and two guidelines did not comment on funding [55, 63]. Competing interests of the guideline development members were addressed and reported in the majority of the guidelines, with most of them failing to report the methods used to seek competing interests, their potential influence on the development process, and/or the types of competing interests considered [53‐56, 58‐67], except for one guideline that sufficiently reported all the criteria [57], and one guideline which simply disclosed a statement that there were no competing interests amongst the development group members [52].

Strengths of this study include the use of a comprehensive systematic review to identify eligible guidelines for the treatment and/or management of depression, as well as the use of the AGREE II instrument, a valid and well-accepted appraisal instrument for guideline quality assessment [30]. Limitations include the fact that only two appraisers assessed guidelines using AGREE II, as opposed to four appraisers as recommended by the instrument’s user manual. To mitigate this and standardize scoring, all three authors independently appraised three guidelines in an initial pilot test and discussed the results and application of the instrument. Furthermore, following the appraisal of all guidelines assessed for this study, any discrepancies were discussed, and scores were carefully reviewed. Finally, this study did not capture any guidelines published outside of the English language, and thus does not account for guidelines that may capture recommendations for traditional medicines or CAM endemic to non-English speaking regions of the world.