Critical appraisal of tobacco dependence treatment guidelines

Tobacco use and dependence is a leading preventable cause of morbidity and mortality globally, and is strongly linked to numerous diseases and healthcare burden [1]. Worldwide, the age standardized prevalence of daily smoking was 25% for men and 5.4% for women from 1990 to 2015 [2]. Although concerted efforts have been put in place to reduce the global smoking prevalence, many countries continue to record high smoking rates, resulting in increased disease burden and healthcare expenditures [2, 3]. Tobacco cessation interventions play an important role in addressing tobacco-related health risks and mortality. Studies have shown that when pharmacotherapy and behavioral interventions are used among adults either singly or in combination, this results in higher success rates [4‐7]. Therefore, the role for healthcare providers in implementing tobacco cessation interventions within their clinical practice settings becomes unequivocally important. Clinicians use tobacco cessation tools and resources such as clinical practice guidelines (CPGs) to provide tobacco use dependence management.

CPGs are important resources used by clinicians when making evidence-based decisions to improve the quality and outcomes of the care delivered to patients. CPGs are attractive due to their concise nature and capacity to provide consistent care by clinicians [8]. On the other hand, CPGs have their limitations when recommendations are influenced by expert opinion and practice experiences. In addition, CPGs are adopted into patient care without undergoing robust critical appraisal to assess their validity and methodological quality [9]. Evidence-based CPGs for the treatment of tobacco use dependence are of varied scope and quality, making it challenging for clinicians to select and apply the best evidence-based recommendations. Having high quality critically appraised tobacco cessation guidelines is important for clinicians to apply the best evidence when treating their patients for tobacco use dependence.

This study aimed to critically appraise current tobacco cessation guidelines and to determine the ones with the highest quality for potential utilization in clinical practice.

No approval was necessary.

Following the search of the databases, 7715 records were identified in addition to 26 records identified through organizational websites and systematic review references. After removing duplicates and screening (titles, abstracts, full-texts), a total of 24 guidelines related to tobacco cessation that satisfied the study eligibility criteria were identified and evaluated (Fig. 1).

Table 2 describes the characteristics of the CPGs included in the review. The 24 guidelines developed by a wide range of organizations (from governmental to professional bodies) were published between 2006 and 2018. Different sources of guideline-development funding were reported; some guidelines were sponsored by pharmaceutical companies, while others received governmental funding [12‐19]. Two guidelines were sponsored by Pfizer, one guideline by GalaxoSmithKline, and one guideline by Boehringer Ingelheim Pharmaceuticals Inc as unrestricted grants while many did not report the funding source [17, 20‐29]. All the included guidelines were targeted to healthcare providers and focused on patients who are tobacco smokers. Some guidelines included recommendations related to subpopulations including: children and teenagers (14 guidelines) [13, 14, 16, 17, 19, 22, 24‐26, 28, 30‐33]; patients with mental illnesses (12 guidelines) [13, 14, 16, 17, 19, 24, 25, 28, 30‐33]; and pregnant women (13 guidelines) [13, 14, 16, 17, 19, 24‐26, 28, 31‐34]. Moreover, four guidelines covered patients undergoing surgical interventions as a subpopulation [17, 22, 30, 33], while two guidelines included prisoners [14, 25].

The majority of the guidelines (n = 14) [12, 15, 16, 18, 19, 21, 23, 27, 29‐32, 34, 35] reported implementing a systematic search for primary literature through multiple resources and databases. The NICE guidelines utilized the highest number of databases and organizations’ websites compared to other guidelines [21, 23, 29]. The NCCN guideline searched only one database in addition to consensus meetings [27]. Other CPGs evaluated did not report the sources of the evidence used [13, 14, 17, 20, 22, 24‐26, 28, 33]. Other characteristics of the reviewed CPGs are provided in Table 2.

The quality assessment of the 24 included guidelines is provided in Table 3 Seven guidelines [18, 19, 21, 23, 29, 34, 35] were judged to be of high quality with an overall score of > 70% based on the AGREE II instrument. The NICE guideline [29] had the highest AGREE II quality score, while the guidelines by the Japanese Circulation Society (JCS) Joint Working Group and the National Center for Smoking Cessation and Training (NCSCT) had the lowest AGREE II quality scores [22, 25]. Regarding the overall scores of guidelines on the domains of the AGREE II instrument, the highest scores were for Domain 4 (clarity of presentation) (70.95%) and Domain 2 (Stakeholders’ involvement) (56.37%). The domain with the lowest average score was Domain 5 (Applicability) (45.05%) as shown in Fig. 2.

Considering the overall recommendations, two guidelines [25, 28] were not recommended, while six were recommended with no modifications [18, 19, 21, 27, 29, 35].

The ICC was calculated for each guideline and it reflected a very strong agreement between the reviewers, except for two guidelines [18, 23]. The average ICC score across all guidelines was 0.817 (range 0.221–0.975).

Tobacco use is one of the major public health threats worldwide. It is a major risk factor for cardiovascular diseases, respiratory diseases, cancers, and deaths [36]. In the last several years, several efforts have been exerted to reduce the burden of this epidemic. Despite these, a systematic and an organized approach for the treatment of tobacco use dependence is needed in different healthcare settings. Several CPGs for the treatment of tobacco use are developed and available for potential clinical use. However, data regarding the quality, rigour of development, and applicability of these guidelines are scarce. This review attempted to identify and appraise existing CPGs for tobacco cessation.

Twenty-four CPGs for the treatment tobacco use dependence met the eligibility criteria for the study and were assessed using AGREE II criteria. There was a great variability between the different guidelines in terms of their quality based on items and domains assessment. Seven guidelines were considered of high quality with an overall score of > 70% [18, 19, 21, 23, 29, 34, 35]. The guideline with the highest overall ranking score (87.56%) was the NICE guideline for stop smoking interventions and services [29]. While this guideline excelled in all domains of the Agree II criteria as compared to other guidelines, it is plausible that the developer had a superior reporting system. It is worth noting that before implementing any of the guideline recommendations, it is important to see the adaptability of these recommendations to the context of the practice setting. The guidelines with the lowest overall quality scored had consistently low scores across all domains, especially in stakeholder involvement, rigor of development, and editorial independence domains.

The domains with the lowest quality appraisal scores were ‘rigour of development’ with an average score of 50.21% and ‘applicability’ with an average score of 45.05%. These findings are consistent with previous studies’ findings [37‐39]. There was agreement on only six guidelines to be recommended for use without modifications [18, 19, 21, 27, 29, 35], while 11 guidelines were recommended for use in practice with modifications [12‐15, 20, 23, 24, 26, 30, 31, 34].

On the ‘rigour of development’ domain, which is of utmost importance for the quality of guidelines, only two guidelines scored over 90%. They are the NICE guidelines for “smoking: stopping in pregnancy childbirth” and NICE guideline for “stop smoking interventions” [21, 29]. Of the 24 guidelines, 11 scored less than 50% on this domain [13, 14, 16, 17, 20, 22, 25, 26, 28, 32, 33]. Under the “rigour of development” domain in the AGREE II tool, there are eight items: use of systematic methods to search for evidence, description of criteria used for selecting the evidence, description of strengths and limitations of evidence body, description of methods used for formulating recommendations, consideration of benefits, side effects, and risks in formulating recommendations, having an explicit link between recommendations and supporting evidence, review of the guideline by experts prior to its publication and provision of a procedure for updating the guideline [11]. The items with the lowest scores were in relation to the description of criteria used for selecting the evidence, description of the methods used for formulating guideline recommendations and provision of a procedure for updating the guideline. For instance, 12 guidelines did not explicitly describe the criteria that they have used for including/excluding evidence [13, 14, 16, 17, 20, 22, 24‐26, 28, 32, 33], 11 guidelines did not describe what methods or systems were adopted to reach the final guideline recommendations and decisions [12‐14, 17, 20, 22, 25, 26, 28, 32, 33], and 15 did not state the procedure for updating the guidelines with the latest research evidence [13‐17, 20, 22, 24‐26, 28, 30, 32, 33, 35]. On the other hand, the majority of the guidelines had explicitly added the link between the guideline recommendations and the research evidence based on which the guidelines have made their recommendations. In addition, in most instances, guidelines summarized their recommendations in tables along with the strength ratings of the evidence. Having clinical guidelines of high methodological rigor is crucial [8, 40]. There is a need to develop tobacco cessation guidelines while applying rigorous methodologic strategies. Without describing the exact criteria and methods used to generate the evidence, the users of guidelines would not be able to decide whether the recommendations are built on robust or weak evidence. Furthermore, guidelines should include a system to monitor the updates in evidence to ensure that recommendations are pertinent and timely [41].

In relation to the applicability of guidelines, the highest score obtained was for NICE guidelines for stop smoking interventions with a score of 81.25% [29]. Of the 24 guidelines assessed, 14 guidelines scored less than 50% [12, 14‐17, 20, 22, 26‐28, 31‐34]. This domain is largely based on the availability of implementation tools, presence of cost analyses, and resource descriptions required for implementation. In many cases, these considerations may not be fully understood before publication and may not be applicable to all settings where the guideline may be implemented. It is likely, for example, that costs and resources of implementation may differ between institutions, healthcare settings, cities, jurisdictions, and countries [11]. The item with the lowest score under this domain was for the resource implications of applying the recommendations. Eleven guidelines did not identify the resources that are required to apply the recommendations [14, 16, 17, 19, 22, 26, 28, 31‐34]. These recourses could be financial, human or physical resources. The item with the second lowest score was for inclusion of monitoring and/or auditing criteria for guideline implementation [12, 15, 16, 21, 22, 26, 28, 31, 32]. Furthermore, 6 guidelines scored poorly on the item related to the provision of advice and/or tools on how to implement recommendations in practice [12, 16, 20, 26, 28, 32]. Guidelines are not self-implementable and should be adapted to the context of the setting where they are going to be applied taking into consideration the setting’s cultural, financial and environmental factors. Evidence suggests that it is expected that clinicians and patients would benefit from guidelines containing application tools [42]. These tools could overcome many of the patient, provider, institutional and system-level barriers that could face guideline implementation. Guidelines should also include explicit criteria that originate from the main guideline recommendations to help monitoring and measuring the application of the guideline recommendations.

The guidelines that scored the highest on the domain of “scope and purpose” were WHO recommendations for the prevention and management of tobacco use and second-hand smoke exposure in pregnancy [18], NICE guideline for stopping smoking in pregnancy and after childbirth [21], and the U.S. Preventive Services Task Force Recommendation Statement [34]. These guidelines explicitly stated the objectives of the guidelines, and the patient population to whom the guidelines can be applied. While four out of 24 guidelines did not clearly describe the scope and objectives of the guidelines, their benefits, and their outcomes [24, 28, 32, 33], and targeted population [25, 28, 33, 35].

In terms of “stakeholder involvement” domain, the overall average score for the domain was 56.37%. The top performing guidelines for this domain were the Clinical Practice Guideline for Treating Tobacco Use and Dependence by the U.S. Public Health Service [35] with a score of 94.44%, NICE guidelines for stopping smoking in pregnancy and after childbirth [21] with a score of 100%, and NICE Stop smoking interventions and services [29] with a score of 97.22%. On the other hand, 10 guidelines scored less than 50% on this domain; for these guidelines, information about the stakeholder(s) involved in developing the guideline was not clear. In addition, the opinions, experiences and expectations of the target population or patients were not sought and the target users of the guideline were not defined. It is strongly recommended that the stakeholders’ opinions especially of patients would be sought when developing guideline recommendations. Involvement of patients in the decision-making process is associated with improved application of guidelines and better health outcomes [43].

In terms of clarity of presentation, the key recommendations in 15 out of the 24 guidelines were clearly either presented as a separate table or textually embedded within the guidelines. Although important for readability and usability of the guideline, it could be argued that other domains (e.g. rigour of development) may be of greater importance to guideline quality. As for editorial independence, to avoid any potential for bias, guideline developers should demonstrate that the views of the funding body have not influenced the content of the guideline and should state any conflict of interest they may have. Eight guidelines did report these two items, respectively [17, 20, 22, 24‐26, 28] and [14, 16, 17, 20‐22, 25, 26].

This study has implications for practice including pharmacy practice and future research. In terms of practice, users of guidelines, especially clinicians and pharmacists, should be aware of the commonly identified issues raised by this study, in order to best assess the appropriateness of adapting guidelines in practice. Users including pharmacists should also be aware of specific flaws and limitations of any guideline, specifically if these occur in domains of great importance to their population (e.g. patient preferences vs. inclusion of implementation tools). Moreover, pharmacy organizations should carefully appraise the quality of smoking guidelines before endorsement. From research perspective, the barriers to creating guidelines that are deemed of high quality by the AGREE II criteria should be explored with respect to smoking cessation. It could be possible, for example that the quality of evidence available as a whole is not at the same level as other clinical conditions. In such cases, domain scores will likely be inevitably lower and beyond the control of the guideline developer. Future research should focus on identifying how the use of AGREE II criteria in guidelines creation and reporting may influence guideline development, guideline implementation by practitioners, and any effects on patient care decisions.

This study has some limitations some of which are inherent to any systematic review. The literature search may not have identified all available guidelines on tobacco dependence treatment. However, the extensiveness of the search and inclusion of supplementary search strategy in addition to searching electronic databases might have resulted in identifying all major guideline that were available in the published literature. The second limitation is that we only included guidelines published in English. Therefore, guidelines published in other languages were excluded and not assessed. Furthermore, as per the AGREE II guidelines, it is preferred to have four reviewers per guideline; however, due to the small number of study investigators, only two investigators independently reviewed each guideline. This may not be a limitation, however, as the agreement between raters was high. Our findings were consistent with previous studies and this likely reflects the robustness of the AGREE II tool.

In conclusion, the systematic search yielded 24 clinical practice guidelines that targeted tobacco cessation. Seven guidelines were considered of high quality with an overall score of over 70% based on the AGREE II instrument. Clarity of presentation was the main area of strength in these guidelines. However, the rigour of development and applicability were the major weaknesses in these guidelines. There is a need to improve the guideline development process and reporting in the field of tobacco cessation. Future developers of guidelines should develop guidelines in line with the AGREE II domains and items [11]. Description of the criteria used for selecting evidence and of the methods used for formulating guideline recommendations and explanation of the process for updating guidelines should be included. Seeking the patients’ opinions and expectations and inclusion of application tools for guidelines’ implementation to the daily healthcare practice should also be considered to improve the quality of guidelines.