Characteristics and quality of clinical practice guidelines for depression in adults: a scoping review

We included all CPGs, defined as a document that aimed to state recommendations, that fulfilled the following criteria: assessed screening, diagnosis or management of depression in adults; were published or totally/partially updated in the last 5 years (January 2014- May 2018); full-text were available in English or Spanish; and used systematic reviews of the evidence to guide their recommendations. We decided to include only CPGs based on systematic reviews, based on the current CPG definition which states that they should be designed based on a systematic review of the evidence [12].

We excluded those CPGs that assessed specific types of depression such as bipolar or psychotic depression, or specific types of populations such as depression in patients with cancer or in older people after a stroke.

We performed a comprehensive search in eighteen databases. Our search strategy included terms related to depression and guidelines/practice guidelines. Searchers were performed by two independent researchers (JHZT and DVZ), and the last update was run in June 2018 (see Additional file 1).

Two independent researchers (JHZT and DVZ) evaluated if the CPGs met the eligibility criteria for inclusion. When there were discrepancies, a consensus was reached after debating them among all the authors.

The following characteristics were extracted from the CPGs: authors; year of publication; country; involvement of patients or their representatives in the CPG development process; methodology used to reach recommendations; methodology used for grading the strength of recommendations; usage of minimally important difference (MID) when evaluating the effect of interventions; and the number of recommendations and good clinical practice (GCP).

We defined a recommendation as “all the statements in favor or against an intervention based on systematic reviews of the evidence, which typically include a formal assessment of the benefits and drawbacks of available treatment options” [13]. All the statements that synthetize opinions from an organized group of experts (expert consensus) and aim to describe “customary and expected care to be offered to patients” in situations where little to no evidence is available were considered as GCP [13].

We defined MID as a measure of the “smallest change in patient-reported outcomes of interest that patients perceive as important” [14].

To assess the quality of CPGs we used the Appraisal of Guidelines Research and Evaluation II (AGREE-II), which has 23 items distributed in six domains (scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence). Each guideline was rated by two researchers. When a difference in two or more points in each item was found, the item was discussed to get to a consensus. Otherwise, we used the mean of the two raters for each item. Lastly, we followed the AGREE-II Instrument guideline to calculate the scores for each domain [15].

We considered that when a CPG had a total score ≥ 70% it had adequate quality, we also used the same cutoff for each of the domains of the AGREE-II Instrument. This cutoff point was taken from a previous study that evaluated the quality of depression CPGs in children [8]. Likewise, we considered that when a CPG had a score ≥ 70% in the third domain (rigor of development) of the AGREE-II Instrument, the CPG had an adequate rigor of development.

This study explores the characteristics, scope, and quality of CPGs for depression in adults that based their recommendations on systematic reviews and were published between January 2014 and May 2018. We included eleven CPGs from seven countries on four continents, from which two reported the patient involvement in the design or validation of the CPGs, six provided recommendations on screening, five on diagnosis, eight on pharmacological treatment, nine on psychological treatment, nine on other non-pharmacological treatments. Regarding the quality assessment, 4/11 CPGs reached a score ≥ 70% in the overall assessment of the AGREE-II instrument, and 3/11 CPGs reached a score ≥ 70% in the rigor of development domain. In addition, only 5/11 CPGs shared their search strategy, while only 4/11 listed the selected studies they used to reach recommendations, and 7/11 CPGs did not clearly state which methodology they used to translate evidence into a recommendation.

The involvement of patients or their representatives in the development of CPGs is considered important as it is supposed to complement scientific evidence to reach more acceptable and implementable recommendations [29]. Thus, many guidelines development groups recommend its inclusion in every step through the development of CPGs, including the definition of the scope and objectives, the definition of the review questions, the developing of recommendations (sharing their preferences regarding the assessed interventions), and the review of the final version of the CPG [30].

However, we found that 2/11 CPGs reported that patients had participated in the design or validation of the CPGs. This low patient involvement is similar to that found in other studies. One study that evaluated 62 Dutch guidelines assessed patients’ participation in the development process through three items (patients’ participation, identification of the patient’s input in the CPG, and the emphasis of patients’ participation in the individual patient level), and found that only 1/62 CPGs fulfilled satisfactorily these items [31]. The CPG that fulfilled these criteria was the Dutch guideline for depression [32]. Moreover, a study evaluated the patient involvement in guidelines in 101 organizations that publish CPGs in G-I-N North America and National Guideline Clearinghouse and found that only 8% of them require the patient or public involvement on guideline development groups, while 15% sometimes require it or describe it as optional [33].

When focusing on the third domain of the AGREE II Instrument (rigor of development), few CPGs (3/11 = 27%) reached a score ≥ 70%. This is similar to a systematic review of CPGs for depression in children and youth that found that only 4/17 = 17.6% achieved a score ≥ 70% in this domain [8]. Other studies have reached different results regarding the percentage of CPGs that achieved a score ≥ 70% in the third domain on the AGREE-II instrument. i.e.: 1/11 = 9.1% for Heparin-induced thrombocytopenia CPGs [34], 15/115 = 13.0% for kidney transplantation CPGs [35], 5/30 = 16.6% for fertility preservation in young women undergoing gonadotoxic treatment CPGs [36], 6/17 = 35.3% for pediatric traumatic brain injury CPGs [37], and 5/12 = 41.7% for treatments for oral cancer CPGs [38].

Our results indicate that few of the CPGs achieve an adequate methodological quality, which could lead to recommendations that are not based on the best available evidence. This situation could be due to the fact that developing a high-quality CPG demands many financial resources, time, highly specialized personnel, and health system support [39‐41]. In addition, some CPGs may fulfill an adequate rigor of development but attained a low score in the AGREE-II instrument because the development process was not adequately reported [42]. To avoid this, the guideline development groups could apply AGREE-II or another instrument to verify the adequate reporting of their CPGs.

To state a recommendation, two basic steps are needed: the selection of evidence and the methodology used to translate evidence into a recommendation [5]. We evaluated some characteristics in order to understand how these steps were performed.

Regarding the selection of evidence, only 5/11 CPGs shared their search strategy, while only 4/11 listed the selected studies they used to reach recommendations. Not sharing these information prevents the readers from adequately evaluating if there was any bias in the selection of evidence used to guide the recommendations, and prevents the replication and corroboration of the searches performed.

Regarding the methodology used to translate evidence into a recommendation, 7/11 CPGs did not clearly state the methodology used. A clearly defined methodology is necessary to understand what criteria were used and how the developing group judged each criterion to reach a recommendation. This allows users to understand how subjectivity and possible competing interest of the guideline developing group may have influenced on its recommendations, and help decide if recommendations can or should be implemented in their own settings [43]. Inconsistent recommendations are not rare, as shown by a systematic review that assessed the recommendations stated in CPGs for depression treatment, which found inconsistencies in the recommendations for the second and third line of pharmacological treatment [7]. For CPGs with inadequate methodology, it is necessary to evaluate the suitability of its use, and be careful when considering the implementation of its recommendations.

Our study is not free from limitations. We only collected guidelines published in English or Spanish, so our findings could not be representative of CPGs published in other languages. The CGPs quality was assessed using the AGREE-II instrument, based on the reporting of the CPGs, so guidelines with inadequate reporting could be classified as deficient, despite their actual quality. Lastly, there are not validated cut-off points for the AGREE-II instrument so the discrimination between CPGs with adequate and inadequate quality could be inaccurate.

However, to our knowledge, this is the first study that has evaluated the characteristics and quality of CPGs for depression in adults. This evaluation has some important strengths: we used a systematic search strategy involving eighteen databases to find available CPGs for depression in adults, we used the AGREE-II instrument that provides a standard methodology to critically appraise the quality of CPGs, and we performed independent appraisals by two researchers.