Background
It is currently estimated that more than 70 % of North Americans have tried at least one form of complementary and alternative medicine (CAM), [
1‐
3] collectively spending billions of dollars annually on these therapies [
4,
5]. CAM has been defined as “a group of diverse medical and health care interventions, practices, products or disciplines that are not generally considered part of conventional medicine” [
6]. The National Center for Complementary and Integrative Health (NCCIH) further defines a non-mainstream practice used
together with conventional medicine as “complementary”, a non-mainstream practice used
in place of conventional medicine as “alternative”, and the coordinated delivery or use of conventional and complementary approaches as “integrative” [
6]. This study henceforth refers to therapies that fall into all of these categories as CAM.
The past several decades have seen a sharp increase in research on CAM given the strong patient-driven market [
7]. Examples of well-studied CAM therapies that show potential benefit include chiropractic spinal manipulation for low back pain and headaches [
8‐
12], and acupuncture for different types of pain [
13‐
18]. Recognizing such benefits, academic institutions are increasingly incorporating CAM into medical education, research and practice [
11]. However, a variety of factors appear to influence whether and how CAM is used. Patients may not discuss their use of CAM with health care professionals out of fear of being judged or not seeing this as important to disclose, potentially leading to contraindications with other treatment [
19‐
22]. Many health care professionals were not exposed to CAM in their medical training [
23], are unfamiliar with CAM therapies, and find it challenging to discuss use or disuse of CAM with their patients [
24,
25]. This is exacerbated by the fact that CAM is comprised of many different and unrelated types of therapies and schools of thought about their use [
26]; and the reliability of evidence about safety and effectiveness varies between CAM therapies [
27‐
29]. Given all of these factors, concerns have been raised about legal and ethical issues pertaining to the recommendations that health care professionals offer their patients about using or not using CAM therapies [
24,
30]. Hence, patients and health care professionals may benefit from credible, knowledge-based resources upon which to base discussions and decisions about use of CAM.
Health care professionals often rely on evidence-informed clinical practice guidelines to understand whether use of a given therapy is recommended, and as a basis for informed and shared decision-making with patients about associated risks and benefits [
31]. Research on a variety of clinical topics has identified that overuse, underuse or misuse of therapies may be associated with guidelines that are of poor quality [
32], and the quality of guidelines has been proven to vary considerably [
33]. Few studies have examined CAM guidelines. Content analysis of 10 guidelines on cardiovascular disease and type II diabetes revealed that CAM-relevant information was brief, in some cases unclear, inconclusive and lacking in direction for health care professionals [
34]. Analysis of 65 National Institute for Health and Clinical Excellence guidelines available in 2009 found that, among 17 guidelines that mentioned CAM, it was not clinically relevant to most; in 14 of 48 guidelines that did not mention CAM, available evidence on the safety and effectiveness of relevant CAM therapies had not been included [
35]. Therefore, no research has thoroughly evaluated the credibility of CAM guidelines. An understanding of the nature of CAM guidelines available to support informed and shared decision-making among patients and providers would help to identify whether such resources are absent and thus needed, or how they could be improved, thereby guiding future guideline development and associated research. The purpose of this study was to assess the quantity and quality of CAM guidelines.
Discussion
To identify credible, knowledge-based resources upon which patients and health care professionals can base discussions and decisions about use of CAM, the purpose of this research was to assess the quantity and quality of CAM guidelines. This study identified 17 guidelines (nine specific CAM therapy, eight mixed CAM therapies) published in 2003 or later that were relevant to a variety of conditions and diseases. Quality as assessed by the 23-item AGREE II instrument varied widely across guidelines overall and by domain; two guidelines scored 5.0 or higher in both average appraisal score and average overall assessment [
46,
49], and three guidelines scored 3.5 or lower in both of these metrics [
45,
52,
55] (1 = strongly disagree; 7 = strongly agree that criteria are met).
To our knowledge, no previous studies have assessed the quantity and quality of guidelines on CAM therapies. Thus, we believe that this is the first study to assess the credibility and nature of CAM guidelines. The findings are similar to those of guidelines on other clinical topics. In this study of CAM guidelines, the scaled domain percentages from highest to lowest were clarity of presentation (85.3 %), scope and purpose (83.3 %), rigour of development (61.2 %), editorial independence (60.1 %), stakeholder involvement (52.0 %) and applicability (20.7 %). In a previous study we found that, among 137 guidelines on a wide variety of clinical topics published from 2008 to 2013, the scaled domain percentages were ordered in similar fashion from highest (clarity of presentation 76.3 %) to lowest (applicability 43.6 %) [
33]. Previous studies that examined a total of 654 guidelines published from 1980 to 2007 [
57,
58], and 1,046 guidelines produced between 2005 and 2013 by 130 Australian guideline developers [
59] also reported similar findings. Therefore the variable and sub-optimal quality of guidelines is not a unique phenomenon.
Notable strengths of this study included the use of a comprehensive systematic review to identify eligible CAM guidelines and the use of the validated AGREE II instrument by which to assess their quality, which is the internationally-accepted gold standard for appraising guidelines [
38]. The interpretation of these findings may be limited by the fact that guidelines were independently assessed by two appraisers instead of four as recommended by the AGREE II instrument to optimize reliability. To mitigate this and standardize scoring, ARG, JYN and LL conducted an initial pilot-test during which they independently appraised the same two guidelines, then discussed the results and achieved consensus on how to apply the AGREE II instrument. Following appraisal of the 17 guidelines, ARG met with JYN and LL to discuss and resolve any uncertainties without unduly modifying legitimate discrepancies. This review does not address all CAM therapies; three therapies were chosen (herbal medicine, acupuncture, chiropractic or osteopathic manipulation) because they were identified as having the largest evidence base, and were therefore considered more likely to be the subject of guidelines [
39]. We may not have identified all guidelines that included these three types of CAM therapy because, to establish a feasible scope, we did not search for guidelines on specific clinical topics and then peruse them for CAM-related content, and we did not search all CAM journals or the Guidelines International Network guideline library. We included CAM topics for which there was likely to be available evidence such as guidelines. Many patients use CAM lacking supporting evidence, therefore, it may be useful to examine guidelines on a broader range of CAM topics to evaluate the basis for recommending those therapies.
By describing the quantity and quality of CAM guidelines, this study revealed that few CAM guidelines are available to support informed and shared decision-making among patients and health care professionals. This likely reflects the lack of research on CAM therapies. Others have identified numerous factors that challenge CAM research including negative attitudes about CAM therapies [
60‐
65] and a lack of targeted funding [
66‐
69]. However, this is expected to change given that CAM therapies continue to be used by more than 40 % of the population in some regions of the world [
70,
71]; and patients continue to use CAM despite documented risks associated with some CAM therapies [
22,
70‐
75]. As research emerges, so too will guidelines that focus on CAM therapies [
10].
This study also revealed that the quality of CAM guidelines varied across domains within individual guidelines, and across different guidelines. This finding is relevant to those who will produce CAM guidelines in the future, and to developers of existing CAM guidelines that, when updated, could be improved. Apart from the AGREE II instrument, numerous principles, frameworks, criteria and checklists are available to help guideline developers, including CAM guideline developers, to generate the highest-quality products [
76‐
81].
Conclusions
This study identified 17 guidelines published since 2003 on CAM therapies including herbal medicines, acupuncture, and chiropractic or osteopathic manipulation. Appraisal of these guidelines with the AGREE II instrument revealed that quality varied within and across guidelines. Some of these guidelines that achieved higher AGREE II scores and favourable overall recommendations could be used by patients and health care professionals as the basis for discussion about the use of these CAM therapies. In future updates, guidelines that achieved variable or lower scaled domain percentage and overall recommendations could be improved according to specifications in the AGREE II instrument, and with insight from a large number of resources that are available to support guideline development and implementation [
75‐
80]. However, the fact that few CAM guidelines are available to support informed and shared decision-making between patients and health care professionals may continue to foster underuse of beneficial CAM therapies, and overuse or contraindicated use of other CAM for which there is no proven benefit or potential associated risks. This finding justifies the need for greater research on the safety and effectiveness of CAM therapies. Future research should also identify CAM therapies other than those reviewed here which are supported by sufficient evidence to serve as the basis for guideline development.
Acknowledgements
Not applicable.