System change interventions for smoking cessation
Abstract
Background
System change interventions for smoking cessation are policies and practices designed by organizations to integrate the identification of smokers and the subsequent offering of evidence‐based nicotine dependence treatments into usual care. Such strategies have the potential to improve the provision of smoking cessation support in healthcare settings, and cessation outcomes among those who use them.
Objectives
To assess the effectiveness of system change interventions within healthcare settings, for increasing smoking cessation or the provision of smoking cessation care, or both.
Search methods
We searched databases including the Cochrane Tobacco Addiction Group Specialized Register, CENTRAL, MEDLINE, Embase, CINAHL, and PsycINFO in February 2016. We also searched clinical trial registries: WHO clinical trial registry, US National Institute of Health (NIH) clinical trial registry. We checked ‘grey' literature, and handsearched bibliographies of relevant papers and publications.
Selection criteria
Randomized controlled trials (RCTs), cluster‐RCTs, quasi‐RCTs and interrupted time series studies that evaluated a system change intervention, which included identification of all smokers and subsequent offering of evidence‐based nicotine dependence treatment.
Data collection and analysis
Using a standardized form, we extracted data from eligible studies on study settings, participants, interventions and outcomes of interest (both cessation and system‐level outcomes). For cessation outcomes, we used the strictest available criteria to define abstinence. System‐level outcomes included assessment and documentation of smoking status, provision of advice to quit or cessation counselling, referral and enrolment in quitline services, and prescribing of cessation medications. We assessed risks of bias according to the Cochrane Handbook and categorized each study as being at high, low or unclear risk of bias. We used a narrative synthesis to describe the effectiveness of the interventions on various outcomes, because of significant heterogeneity among studies.
Main results
We included seven cluster‐randomized controlled studies in this review. We rated the quality of evidence as very low or low, depending on the outcome, according to the GRADE standard. Evidence of efficacy was equivocal for abstinence from smoking at the longest follow‐up (four studies), and for the secondary outcome ‘prescribing of smoking cessation medications' (two studies). Four studies evaluated changes in provision of smoking cessation counselling and three favoured the intervention. There were significant improvements in documentation of smoking status (one study), quitline referral (two studies) and quitline enrolment (two studies). Other secondary endpoints, such as asking about tobacco use (three studies) and advising to quit (three studies), also indicated some positive effects.
Authors' conclusions
The available evidence suggests that system change interventions for smoking cessation may not be effective in achieving increased cessation rates, but have been shown to improve process outcomes, such as documentation of smoking status, provision of cessation counselling and referral to smoking cessation services. However, as the available research is limited we are not able to draw strong conclusions. There is a need for additional high‐quality research to explore the impact of system change interventions on both cessation and system‐level outcomes.
PICOs
Plain language summary
Do organizational changes to support people quitting smoking improve services and help more people to quit?
Background
Smoking is a cause of many health problems, including cancers, heart and lung diseases. Health professionals (e.g. doctors, nurses, pharmacists, dentists, etc.) may be able to reduce this harm by helping smokers to quit during a clinic visit. However, it may be difficult for health care providers to recognize smokers. They may also feel they cannot deliver good support as they do not have enough time, skills, training, budget or resources. A change within health professionals' wider organization may help to improve their involvement in care to help people to stop smoking, and in turn improve the chances of them quitting smoking. These changes may include introducing a system for asking patients if they smoke and recording smoking status on the patient health records; providing health care providers with training, budget or resources to help them deliver more effective quitting support; identifying a dedicated staff member to provide quitting support; introducing new rules to restrict smoking or support quitting activities; introducing advice to quit smoking into routine care; and paying health workers for delivering cessation support. This review aimed to find out whether implementing these organizational changes improves health professionals’ involvement in quit smoking activities, and whether it helps smokers stop smoking. We assessed the following activities by health professionals to evaluate their involvement in quit smoking activities: 1) asking about tobacco use; 2) documenting smoking status on patient health records; 3) advising smokers to quit; 4) counselling to quit; 5) providing medicines to quit smoking; and 6) referring to other centers such as quitlines (a telephone support service for smokers to quit) where smokers can obtain further help.
Study characteristics
We searched for studies published up to February 2016.
Our search identified seven studies which investigated changes made to the way organizations offered stop‐smoking support in healthcare settings; six studies were conducted in the USA and one in Spain. Two studies evaluated the changes implemented in primary care clinics and another two studies evaluated changes in dental clinics. One study each evaluated changes in community pharmacy, Veterans Affairs Medical Center and pediatric practice. All included studies were supported or funded by government agencies.
None of the studies implemented all the recommended changes or activities listed above. Five studies implemented four types of changes and two studies implemented three types of changes. Identifying all smokers, training health professionals and advising smokers to quit were part of all included studies.
Key results
Of the seven studies identified, only four evaluated the effect of organizational changes on study participants' quitting status. Of these, two found that organizational changes helped people to quit smoking, but the other two studies reported that they did not, and hence no conclusion could be drawn on this factor. Activities such as counselling to quit, recording smoking status in patient health records, and referring smokers to an outside stop‐smoking clinic improved after those changes were made. Of the three studies which evaluated factors such as asking about tobacco use and advising to quit, two reported that organizational changes could improve both of those factors.
Quality of the evidence
The overall quality of the evidence was judged to be low because of the small number of available studies and inadequate study designs. More well‐conducted studies are needed to fill this knowledge gap. Some studies are already underway but need to be evaluated in detail to include in the review.
Authors' conclusions
Summary of findings
| System change interventions for tobacco control (primary cessation outcome) | |||
| Patient or population: Smokers | |||
| Outcomes* | No of Participants | Quality of the evidence | Comments |
| Cessation outcome | 7142 | ⊕⊝⊝⊝ | 2 of the 4 studies significantly favoured the intervention. Mixed effect and low quality evidence preclude drawing conclusions |
| NB. Illustrative comparative risks and relative effects columns have been removed, as only narrative syntheses were conducted due to the presence of significant heterogeneity among studies. | |||
| GRADE Working Group grades of evidence | |||
| 1Self‐reported abstinence was verified only in one study, and one study reported higher dropout rate in one group. | |||
| System change interventions for tobacco control (secondary outcomes) | |||
| Patient or population: Smokers | |||
| Outcomes* | No of Participants | Quality of the evidence | Comments |
| Provision of cessation counselling | 10,949 | ⊕⊕⊝⊝ | 3 of the 4 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Asking about tobacco use | 2615 | ⊕⊕⊝⊝ | 2 of the 3 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Provision of cessation advice | 3003 | ⊕⊕⊝⊝ | 2 of the 3 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Quitline referral | 3006 | ⊕⊝⊝⊝ | All 3 studies significantly favoured the intervention. However, the low quality of the evidence precludes drawing conclusions |
| Quitline enrolment | 1191 | ⊕⊝⊝⊝ | Both studies significantly favoured the intervention. However, the low quality of the evidence precludes drawing conclusions |
| Prescription of NRT or other pharmacotherapy | 2615 | ⊕⊕⊝⊝ | Of the 2 studies, 1 significantly favoured the intervention. Mixed effect and low‐quality evidence preclude drawing conclusions |
| NB. Illustrative comparative risks and relative effects columns have been removed, as only narrative syntheses were conducted due to the presence of significant heterogeneity among studies *We have not included data from one study as the data were collected as counts (no denominator) | |||
| GRADE Working Group grades of evidence | |||
| 1Included studies had high risk of detection bias. | |||
Background
Description of the condition
The consequences of tobacco use are well recognized and understood, as are the benefits of smoking cessation (Critchley 2012; Ebbert 2005; Peto 2000; Taylor 2002). Smoking cessation not only leads to significant and immediate health benefits, but also decreases most of the related risks within a few years of quitting (WHO 2011). Even people who quit later in life gain benefit. For example, among smokers who quit at the age of 65 years, men gain two years of life on average and women gain three (Taylor 2002). Quitting smoking is associated with a 36% reduction in risk of all‐cause mortality among people with coronary heart disease, which is significant when compared with other secondary preventive therapies such as lowering cholesterol (Critchley 2012). Given the high prevalence of smoking, even a modest improvement in smoking cessation rates could translate to major health and economic benefits.
Addressing tobacco use within a healthcare setting requires clinical, programme, and system‐level changes. According to clinical practice guidelines for treating tobacco use and dependence, all healthcare organizations should develop plans to support consistent and effective identification, documentation and treatment of tobacco smokers (Fiore 2008). As a minimum requirement, all healthcare providers should ask the tobacco use status of the people who consult them, should briefly advise all smokers to quit, and refer them to quitline or other smoking cessation services (Revell 2005).
Even though there are guidelines and evidence to provide smoking cessation services at every clinical encounter, reports suggest that healthcare providers are not delivering the recommended levels of support to those who smoke (Braun 2004). Previous studies have reported suboptimal rates of smoking cessation services, delivered by different types of healthcare professionals, in various healthcare settings (Aquilino 2003; Braun 2004; Thorndike 1998). The levels of smoking cessation support in hospitals are also low (Freund 2005; Freund 2008). It is evident that even in high‐income countries, current healthcare systems are not well organized to address the issue of smoking.
Description of the intervention
“System change smoking cessation interventions describe specific strategies that healthcare administrators, managed care organisations, and purchasers of health plans can implement to treat tobacco dependence” (AHRQ 2012).They involve systematic identification of smokers and subsequent offering of evidence‐based cessation treatments (Fiore 2007). Fiore 2007 has suggested six system‐level strategies to facilitate treatment of tobacco dependence:
-
Implement a system for identifying smokers and documenting tobacco‐use status in every clinic and hospital;
-
Provide education, resources and feedback to promote provider interventions;
-
Dedicate staff to provide smoking cessation treatment and assess its delivery in staff performance evaluations;
-
Promote hospital policies that support and provide smoking cessation services;
-
Provide evidence‐based tobacco dependence treatments (both counselling and pharmacotherapy);
-
Reimburse providers for the delivery of effective tobacco dependence treatments and include these services among their defined duties.
How the intervention might work
The barriers to providing effective smoking cessation support include: lack of support from the organization, perceived objections from patients, lack of systems for identifying smokers, lack of staff time and skill, perceived inability to change practices, perceived lack of efficacy of tobacco dependence treatments, and the cost of providing care (Wolfenden 2009). A strategic system change approach may be effective in addressing these multidimensional factors associated with low smoking care provision and may improve the number of people achieving abstinence. Outcomes for chronically‐ill people will improve only when healthcare systems reconfigure themselves to address the needs and concerns of their patients (Wagner 1998). Tobacco smoking is a chronic relapsing condition that often requires ongoing medical and behavioural interventions, and is consequently considered a chronic health condition (Hudson 2010). A system‐level change may therefore be essential to deal with the issue.
Why it is important to do this review
System change interventions are multicomponent, and may vary significantly in their intensity, content and delivery. It is not clear which types of approach are more effective than others. A summary of this evidence is critical as, to our knowledge, no systematic review assessing the effectiveness of such interventions has yet been published. This review is intended to identify various system change interventions for smoking cessation, across various healthcare settings, and to evaluate the effectiveness of such approaches at both the participant and organizational levels.
Objectives
To assess the effectiveness of system change interventions within healthcare settings, for increasing smoking cessation or the provision of smoking cessation care, or both.
Methods
Criteria for considering studies for this review
Types of studies
Randomized controlled trials (RCTs), cluster‐RCTs with at least two intervention sites and two comparator sites, and quasi‐RCTs. Interrupted time series studies (ITS) are also eligible for inclusion if they have a clearly‐defined point in time when the intervention occurred and at least three data points before and three after the intervention.
Types of participants
People who smoke and are receiving care in a healthcare setting; and the staff working in these healthcare settings.
Types of interventions
System change interventions for smoking cessation are policies and practices designed by organizations to integrate the identification of all smokers and the subsequent offering of evidence‐based smoking cessation treatments (pharmacological or non‐pharmacological, or both) into the routine delivery of health care (Fiore 2007). Thus we include interventions which have been developed for identifying people who smoke, documenting smoking status and providing tobacco dependence treatment in various healthcare settings (primary, secondary or tertiary care).
We considered studies using the components of the system change model described in Fiore 2007, detailed in the Description of the intervention section of this review.
We did not consider studies focusing only on training health professionals, identification of smokers (electronic health records) or providing smoking cessation counselling, without a system change approach. As identification of all smokers and providing treatment to them are the crucial components of system change smoking cessation interventions, we considered only studies which aimed to identify all smokers accessing the system and to provide treatment. We excluded studies of interventions with research personnel involvement in intervention delivery. We also excluded studies targeting a single type of health professional within the health service (unless the organization included only one health profession, e.g. a pharmacy employing only pharmacists without any pharmacy technicians or pharmacy assistants), as organizational‐level interventions should target most health professionals in the system in order to deliver tobacco cessation interventions at every clinical encounter. Hence the system change intervention should be designed to integrate the provision of smoking cessation services within the routine delivery of health care.
Types of outcome measures
Primary outcomes
Following the standard methodology of the Cochrane Tobacco Addiction Group, the primary outcome was abstinence from smoking at longest follow‐up, assessed as point prevalence (defined as prevalence of abstinence during a time window immediately preceding the follow‐up) or continued or prolonged abstinence (defined as abstinence between quit day or predetermined grace period and a follow‐up time). We used the strictest available criteria to define abstinence, preferring continuous or prolonged abstinence over point prevalence abstinence, and biochemically‐validated abstinence over self‐reported abstinence.
Studies that did not assess smoking cessation were also eligible for inclusion if they reported any secondary outcome and met the other inclusion criteria. We examined and reported the classification of primary and secondary outcomes in each included study.
Secondary outcomes
Secondary outcomes include process outcomes measured as organizational‐level outcomes or outcomes of care:
-
Organizational‐level outcome measures, including the number of smokers identified with smoking status documented and number of health professionals trained or dedicated to providing cessation support.
-
Outcomes of care, including the number of smokers counselled, given self‐help materials, offered nicotine replacement therapy (NRT) or other pharmacotherapy, nomination of a quit date, referral to specialist smoking cessation services (such as telephone quitline support) or other health professionals, and provision of a follow‐up appointment.
Search methods for identification of studies
Electronic searches
We searched the following databases:
-
Cochrane Tobacco Addiction Group Specialized Register, 1st February 2016;
-
Cochrane Central Register of Controlled Trials (CENTRAL), 2016 Issue 1;
-
MEDLINE/MEDLINE In Process (via OVID) 1946 to 15th February 2016;
-
Embase (via OVID), 1947 to 15th February 2016;
-
PsycINFO (via OVID), 1806 to 15th February 2016;
-
CINAHL (via EBSCO), 1938 to 15th February 2016.
Search strategies included both key words and MeSH/Emtree. We did not apply any language restrictions. The search strategy for MEDLINE is presented in Appendix 1.
Searching other resources
We also searched clinical trial registries (the WHO clinical trial registry and the US National Institute of Health (NIH) clinical trial registry) for ongoing studies. We sought additional studies by screening the references of relevant reviews and identified studies (citation tracking). We also used personal bibliographies and communication with experts in the field to identify any other studies.
Data collection and analysis
Selection of studies
DT implemented the search strategy and merged search results using reference management software (EndNote, Thomson Reuters). We reviewed the titles and abstracts of studies for possible inclusion, and subjected those selected to full‐text assessment. We linked together multiple reports of the same study. Two authors (DT and JG) independently assessed all the full‐text articles retrieved, and included those studies meeting the inclusion criteria in the review. We resolved any discrepancies by discussion with the third author (BB), a content area expert who acted as an arbiter for disagreement about the intervention or content of the study. Another arbiter (MJA), who is an expert in clinical trials and meta‐analysis, checked any methodological discrepancies. We noted characteristics of the studies excluded (after full‐text assessment), including the reason for exclusion.
Data extraction and management
Two authors (DT and JG) independently extracted data, using a pilot‐tested standardized data collection form. We entered the collected data into Review Manager 5 (RevMan). We contacted authors of the studies by email, where data were not available or unclear.
We extracted the following information from each of the selected studies:
-
lead and corresponding authors’ information;
-
date of publication;
-
location and setting;
-
methods of recruitment and inclusion criteria;
-
methods of randomization, allocation concealment and blinding;
-
study design, duration and follow‐up details;
-
characteristics of participants (e.g. age, sex and smoking status);
-
specific details of the intervention (type, duration, content, format and delivery of intervention, use of pharmacotherapy, adherence to therapy and information about the providers);
-
control group component;
-
number of participants in each arm;
-
outcome measures and definitions, including any biochemical validation, and time point at which they were measured and reported;
-
Results: estimate of effect with confidence interval and subgroup analysis (summary data of intervention and control group were entered separately into RevMan) and missing data;
-
funding and declarations of interest for the primary investigators;
-
authors’ conclusions;
-
additional comments and information.
If studies were reported in more than one publication (e.g. different time points of the study), we extracted the data from all publications onto separate data collection forms and combined them. If there was one full journal article and multiple conference abstracts available, we considered only the journal article. We resolved any disagreements in the data collection process by discussion with a third author (MJA).
Assessment of risk of bias in included studies
Two authors (DT and JG) independently assessed the risk of bias in included studies, with any disagreements resolved by discussion and consensus, and by consulting a third author (MJA) where necessary.
We assessed studies with a separate control group (RCTs, cluster‐RCTs and quasi‐RCTs) using the seven standard Cochrane criteria:
-
-
sequence generation;
-
allocation concealment;
-
blinding of participants and personnel;
-
blinding of outcome assessment;
-
incomplete outcome data;
-
selective outcome reporting;
-
other potential bias.
-
We judged each criterion for bias on a three‐point scale ‘low risk’, ‘high risk’ and ‘unclear risk’ (Higgins 2011), and constructed a 'Risk of bias' table.
-
‘Low risk’, when there was a low risk of bias across all key domains.
-
‘Unclear risk’, when there was an unclear risk of bias in one or more of the key domains.
-
‘High risk’, when there was a high risk of bias in one or more of the key domains.
For each included study, we provide a summary assessment of risk of bias.
Measures of treatment effect
We present the intervention effect for each outcome descriptively, summmarizing nominal variables using numbers and proportions. Wherever possible, we have produced a risk ratio (quitters in treatment group/total randomised to the treatment group) / (quitters in control group/total randomised to the control group) for the outcome of each individual trial.
Unit of analysis issues
In the case of trials with repeated observations, we considered the longest follow‐up for the analysis (Higgins 2011). We assessed all reported secondary outcomes only at a single time point.
Although heterogeneity precluded us from performing a meta‐analysis for this review, we had considered using adjusted estimates of effect measures for analyses of cluster‐randomized trials. Had such data been unavailable, we would have conducted an approximate analysis where the required information could be obtained (Higgins 2011).
Dealing with missing data
We report the number of participants lost to follow‐up by group, where available. For the primary outcome, we used an intention‐to‐treat analysis approach where possible. This assumes that people lost to follow‐up continue smoking (West 2005). However, we also extracted and reported the strategies used in each study to deal with missing data for the primary outcome measure, and where adjustments have been made as part of the study analyses we also report the summary statistics directly from the study reports.
Assessment of heterogeneity
We explored heterogeneity visually using tables and forest plots, by comparing the effect sizes of studies grouped according to potential effect modifiers. These included:
-
Type of intervention (e.g. identification of smokers, documentation of smoking status, treatment, training of health professionals, feedback of services, etc.);
-
Intensity of intervention (e.g. counselling, pharmacotherapy, both counselling and pharmacotherapy, duration of intervention, etc.);
-
Type of health professional involved;
-
Setting (primary, secondary and tertiary);
-
Study design (RCTs, cluster‐RCTs, quasi‐RCTs or ITS studies);
-
Quality of studies.
We assessed statistical heterogeneity using the Chi2 test for homogeneity (with significance defined at the alpha‐level of 10%) and quantified using the I2 statistic (Higgins 2011). We considered pooling the data using a meta‐analysis where heterogeneity was less than 50% (Higgins 2011).
Assessment of reporting biases
We did not systematically assess publication bias, because of the limited number of studies included, in accordance with the Cochrane Handbook (Higgins 2011). If sufficient studies are available in future updates, we will assess publication bias using funnel plots.
Data synthesis
We conducted a meta‐analysis for the primary outcome measure using a random‐effects model. However, due to the presence of significant heterogeneity (I2 = 78%), we deemed it inappropriate to present pooled effects, and we therefore present a narrative synthesis of the included studies. We report major characteristics and results for each trial.
Subgroup analysis and investigation of heterogeneity
We did not conduct any subgroup analyses, due to the limited number of included studies and the decision not to pool them.
Sensitivity analysis
We did not conduct any sensitivity analyses, due to the limited number of included studies and the decision not to pool them.
'Summary of findings' table
Following standard Cochrane methodology, we created 'Summary of findings' tables. We created one for the primary cessation outcome, and a second for the secondary outcomes. We considered it important to summarize both the primary cessation outcome and the secondary outcomes, because of the two‐phase nature of system change interventions, with the intervention potentially influencing health professional behaviour, which can go on to potentially affect the patient‐level outcome. Focusing on both sets of outcomes could help to identify why the intervention is successful or not. Also following standard Cochrane methodology, we used the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness and publication bias) to assess the quality of the body of evidence for each outcome, and to draw conclusions about the quality of evidence within the text of the review. As it was impossible to carry out statistical analyses, we provide a narrative summary of outcomes in both 'Summary of findings' tables.
Results
Description of studies
See: Characteristics of included studies; Characteristics of excluded studies
Results of the search
The database search yielded 9278 titles. We found three additional studies through other sources (handsearching and trial registry searching). After removing duplicates, we screened 7004 titles and abstracts, reviewed 159 full‐text articles, and shortlisted 48 studies. Of those, we excluded 39 after thoroughly reviewing the full text, with reasons recorded in the Characteristics of excluded studies table. We found two ongoing studies through the searches (Bonevski 2016; Ostroff 2014); we will perform a full eligibility check once the studies are complete. See Figure 1 for an illustration of eligibility decisions.
Study flow diagram.
Included studies
We included studies if the intervention was designed to integrate the identification of smokers and the subsequent offering of evidence‐based tobacco dependence treatment into routine care.
This review includes seven cluster‐RCTs. All the studies except one (Cabezas 2011 in Spain) were conducted in the USA. The settings of the included studies were: two in primary care clinics (Cabezas 2011; Rothemich 2010), two in dental clinics (Gordon 2010; Little 2009), and one each in a community pharmacy (Patwardhan 2012), a Veterans Affairs medical centre (Joseph 2004) and a paediatric practice (Winickoff 2014). Winickoff 2014 focused on parents attending an outpatient paediatric practice. None evaluated a system change intervention for inpatient smokers. One study had two reports (Winickoff 2013 and Winickoff 2014) which we collated and included in this review as Winickoff 2014.
Intervention
We provide a summary of intervention components implemented by each study as an additional table (Table 1).
| Identification/ documentation of smoking status | Smoking cessation Training/ resources/feedback to providers | Dedicated staff to support cessation activities | Policies to improve access to cessation interventions | Free smoking cessation treatment from the organization | Reimburse clinicians for providing smoking cessation support | |
| Cabezas 2011 | Yes | Yes | No | No | Yes | No |
| Gordon 2010 | Yes | Yes | No | No | Yes | No |
| Joseph 2004 | Yes | Yes | No | Yes | Yes | No |
| Little 2009 | Yes | Yes | No | Yes | Yes | No |
| Patwardhan 2012 | Yes | Yes | No | Yes | Yes | No |
| Rothemich 2010 | Yes | Yes | No | Yes | Yes | No |
| Winickoff 2013 | Yes | Yes | No | Yes | Yes | No |
All included studies used the services of existing staff to provide the intervention. None of the studies incorporated all six system change strategies. Five studies implemented four system‐level strategies (Joseph 2004; Little 2009; Patwardhan 2012; Winickoff 2014) and two studies implemented three strategies (Cabezas 2011; Gordon 2010). Identifying all smokers, training staff and providing evidence‐based treatment were components of all seven studies.
Four studies (Little 2009; Patwardhan 2012; Rothemich 2010; Winickoff 2014) implemented a system of identifying smokers. Rothemich 2010 used a vital sign stamp to mark paper patient records; Little 2009 used a new field in the electronic health record; and Winickoff 2014 used a specific action sheet attached to medical records to identify smokers. In Patwardhan 2012, dental technicians identified smokers by asking about their tobacco use and documented the status on a form that was then attached to their prescription to notify pharmacists. In Joseph 2004, various strategies were recommended for implementation across intervention sites to improve identification of smokers and documentation of smoking status by health professionals, including the ‘smoking as a vital sign’ approach, use of an electronic clinical reminders system and adaptation of a note template to include smoking status. Finally, two studies (Cabezas 2011; Gordon 2010) identified smokers either by office staff or by clinicians, who asked about tobacco use, but without reporting the methods of documentation.
All included studies provided clinicians with training. The duration of training ranged from 30 minutes to 20 hours. Two studies (Little 2009; Rothemich 2010) provided feedback to the clinicians and practices. Little 2009 derived data from electronic health records on rates of tobacco use assessment, advice, counselling, referral offers and referral acceptances, which were used to deliver feedback, and provided monthly performance feedback at provider, clinic and cross‐clinic levels. In Rothemich 2010, feedback was provided by the quitline service at both patient and practice levels. Patient‐level feedback included the number of counselling sessions completed by the participant, smoking status at last contact, difficulties in contacting participants and reasons for any unsuccessful enrolment or early termination. Practice‐level feedback was provided quarterly and included the volume of referrals and summary data such as readiness to quit, quit attempts and smoking status.
None of the studies reported an intervention which involved a 'champion' co‐ordinating tobacco dependence services. However, a core implementation group, which included study staff, professional leaders and administrators of each of the intervention clinics, was present in Little 2009 to facilitate the implementation of intervention components.
All the studies included provision of cessation advice by clinicians to all identified smokers, except one (Patwardhan 2012) that did not provide cessation advice to those who had already decided to quit smoking. Smokers were instead referred to a specialist quitline service. Five studies (Gordon 2010; Little 2009; Patwardhan 2012; Rothemich 2010; Winickoff 2014) referred smokers to smoking cessation services external to the organization. All seven studies included pharmacotherapy (NRT or prescription medications, or both) in the intervention.
In four studies (Cabezas 2011; Gordon 2010; Joseph 2004; Winickoff 2014) pharmacotherapy was provided within the organization. In the other studies, the provision of pharmacotherapy was co‐ordinated from smoking cessation services external to the organization.
None of the included studies reported reimbursement to the providers for delivery of smoking cessation care.
Control group
In four studies (Cabezas 2011; Gordon 2010; Little 2009; Winickoff 2014) usual care was the control condition. In Joseph 2004, each control clinic received five copies of the AHCPR smoking cessation guideline (AHCPR 1996). In Patwardhan 2012, control group pharmacies received an informal presentation on quitline services, quitline cards and enrolment in Fax to Quit (FTQ) services (a free service). In Rothemich 2010, a traditional tobacco use vital sign stamp (only smoking status recorded) was used in control clinics.
Outcomes
Primary outcomes
Four studies (Cabezas 2011; Gordon 2010; Joseph 2004; Winickoff 2014) reported cessation outcomes. All these studies used self‐reported cessation, except for Winickoff 2014, which used cotinine‐validated abstinence.
Secondary outcomes
All the studies except Cabezas 2011 reported system‐level outcomes (process outcomes). One study (Gordon 2010) assessed system‐level outcomes only for the intervention participants, and another (Patwardhan 2012) collected system‐level outcome data from healthcare providers. Reported system‐level outcomes included the number of people asked about tobacco use, number of smokers advised to quit, number of smokers counselled to quit, number of smokers referred to a specialized smoking cessation clinic, and provision of pharmacotherapy.
Excluded studies
We excluded studies if the study design did not meet the criteria for inclusion, if there was substantial involvement of research personnel in the provision of smoking cessation care, or if the intervention targeted a single health professional (or profession in the case of multidisciplinary health services) within the service. We also excluded studies if the intervention targeted a specific population instead of providing support to all smokers attending the clinic. Specific reasons for exclusion can be found in the Characteristics of excluded studies table.
Risk of bias in included studies
We present rhe 'Risk of bias' assessment for included studies in the 'Risk of bias' table, under each individual Characteristics of included studies table. We also present these results in graphical form in Figure 2 and Figure 3.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
Allocation
Five of the seven studies (71%) (Cabezas 2011; Joseph 2004; Patwardhan 2012; Rothemich 2010; Winickoff 2014) adequately described the random sequence generation and were considered to have low risk of bias for this domain.
Blinding
Blinding of both participants and personnel (to avoid performance bias) was not possible due to the study design in any of the included studies, except for Patwardhan 2012, where the study personnel were not aware of the existence of two groups and therefore were considered to be effectively blinded. We rated three studies (Gordon 2010; Joseph 2004; Little 2009) at low risk of detection bias. Joseph 2004 assessed outcomes using a blinded outcome assessor; Little 2009 collected data from electronic health records; and Gordon 2010 collected outcome data using a postal survey.
Incomplete outcome data
Four studies (Cabezas 2011; Gordon 2010; Joseph 2004; Winickoff 2014) reported cessation outcomes and included follow‐up data. In Cabezas 2011, although the dropout rate was high, it was similar in both groups (43.3% in intervention and 44.8% in control), and hence we rated it at low risk of bias. Gordon 2010 reported moderate dropout rates, but the rate was higher in the intervention group (30.7% versus 26.1%; P < 0.01), so we judged this study to be at high risk of bias for this domain. Although not statistically significant, the follow‐up rate in Winickoff 2014 was marginally higher in the control group (64.5% versus 72.4%; P = 0.11) and we therefore rated it at high risk of bias. Joseph 2004 did not report dropout rates by group and we therefore considered this to be at unclear risk of bias.
We judged three studies (Little 2009; Patwardhan 2012; Rothemich 2010) which reported only system‐level outcomes to be at low risk of bias, as there were no follow‐ups involved.
Selective reporting
Only Cabezas 2011 had a published protocol and was therefore rated at low risk of bias for this domain. We considered all other studies to be at unclear risk of bias, as there was no means of checking whether selective reporting had taken place.
Other potential sources of bias
Three studies which reported cessation outcomes used unverified self‐report data.
Effects of interventions
See: Summary of findings for the main comparison System change interventions for tobacco control; Summary of findings 2 System change interventions for tobacco control
See: summary of findings Table for the main comparison for a summary of the main comparisons in this review.
Primary outcome
Four studies (Cabezas 2011; Gordon 2010; Joseph 2004; Winickoff 2014) evaluated the effect of a system change intervention on smoking cessation. Of these, two studies (Cabezas 2011; Gordon 2010) found that the quit rate was higher in the system change intervention group than in the control group. In Cabezas 2011, the primary end point, one‐year self‐reported continuous abstinence at two‐year follow‐up, was significantly higher in the intervention group than in the control group, at 8.1% (120/1482) versus 5.8% (78/1345). The adjusted odds ratio (OR) reported by the study was 1.5 (95% confidence interval (CI) 1.05 to 2.14). We calculated the unadjusted intention‐to‐treat risk ratio (RR) to be 1.41 (95% CI 1.07 to 1.86). Gordon 2010 reported significant improvement in both point prevalence abstinence (11.3% [158/1394] versus 6.8% [79/1155]; unadjusted RR 1.66, 95% CI 1.28 to 2.15) and prolonged abstinence (5.3% [74/1394] versus 1.9% [22/1155]; unadjusted RR 2.79, 95% CI 1.74 to 4.46) at six‐month follow‐up. In Joseph 2004 and Winickoff 2014, the intervention did not result in better cessation rates at one‐year follow‐up (11.4% [32/280] versus 13.2% [39/295]; unadjusted RR 0.86, 95% CI 0.56 to 1.34; and 4.3% [24/556] versus 4.1% [26/635]; adjusted OR 1.07, 95% CI 0.64 to 1.78, respectively). The design effect (clustering effect) was taken into account in the analyses conducted by Cabezas 2011 and Winickoff 2014.
In the original analyses, Gordon 2010 used multiple imputation to handle missing data, Cabezas 2011 and Winickoff 2014 considered all lost to follow‐up as smokers, and Joseph 2004 evaluated smoking status based only on available data (complete‐case analysis).
Secondary outcomes
We provide a summary of secondary outcomes in an additional table (Table 2).
| Study | Asking about tobacco use | Documentation of smoking status | Advice to quit | Counselling to quit | Initiation of NRT or other pharmacotherapy | Quitline referral | Quitline enrolment |
| Cabezas 2011 | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed |
| Gordon 2010 | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed |
| Joseph 2004 | No difference between groups (76.0% vs 74.3%; P = 0.71) | Favoured intervention (60.7% vs 67.0%; P < 0.001) | Not assessed | No difference between groups (73.9% vs 71.8%; P = 0.60) | No difference between groups (14.7% vs 18.0%; P = 0.38) | Not assessed | Not assessed |
| Little 2009 | Not assessed | Not assessed | Not assessed | Favoured intervention (69% vs 3%; P < 0.01) | Not assessed | Not assessed | Not assessed |
| Patwardhan 2012* | Favoured intervention (636 vs 5; P < 0.001) | Not assessed | Favoured intervention (25 vs 3; P < 0.01) | Not assessed | Not assessed | Favoured intervention (240 vs 85; P = 0.02) | Favoured intervention (81 vs 8; P < 0.001) |
| Rothemich 2010 | Not assessed | Not assessed | No difference between groups (58.2% vs 55.3%; P = 0.39). | Favoured intervention (34.4% vs 27.7%; P = 0.001) | Not assessed | Favoured intervention (21.4% vs 8.7%; P < 0.001) | Not assessed |
| Winickoff 2013 | Favoured intervention (59.4% vs 32.6%; P < 0.001) | Not assessed | Favoured intervention (50.5% vs 26.9%; P < 0.001). | Favoured intervention (54.7% vs 19.2%; P < 0.001) | Favoured intervention (18.5% vs 2.4%; P < 0.001) | Favoured intervention (37.2% vs 9.3%; P < 0.001) | Favoured intervention (4.1% vs 1.1%; P < 0.01) |
*data collected as counts (no denominator)
Asking about tobacco use
Of the three studies (Joseph 2004; Patwardhan 2012; Winickoff 2014) that evaluated the effect of a system change intervention on identification of smokers, two reported significant improvements in the intervention group. In Patwardhan 2012, 636 people (measured as counts; no denominator) were screened for tobacco use in all experimental‐group pharmacies, compared with five in all control‐group pharmacies (P < 0.001). Winickoff 2014 also favoured intervention (59.4% versus 32.6% screened; P < 0.001). However, in Joseph 2004, the intervention did not improve the identification of smokers (76.0% versus 74.3% screened; P = 0.71).
Tobacco screening was part of standard care for both groups in two studies (Little 2009; Rothemich 2010), with similar screening rates across the intervention and control groups prior to the start of the studies.
Documentation of smoking status
One study (Joseph 2004) reported the effect of the system change intervention on documentation of smoking status. Before intervention, control sites were significantly more likely to document smoking status than intervention sites (63.1% versus 55.7% respectively; P < 0.001). However, the direction of this difference was reversed after the intervention was implemented (60.7% versus 67.0% respectively; P < 0.001).
Advice to quit
Of the three studies (Patwardhan 2012; Rothemich 2010; Winickoff 2014) that evaluated the effect of a system change intervention on the number of smokers advised to quit, two reported significant improvements in the intervention group. In Patwardhan 2012, 25 smokers (measured as counts; no denominator) were advised to quit in the experimental‐group pharmacies compared with three in the control‐group pharmacies (P < 0.01). Winickoff 2014 also favoured intervention (50.5% versus 26.9%; P < 0.001). However, in Rothemich 2010 the intervention did not improve the rate of advice, with no significant difference in quitting advice across groups (58.2% intervention versus 55.3% control; P = 0.39).
In one study (Little 2009), the provision of advice to quit was standard practice for both groups prior to the start of the study and pre‐study rates were similar across groups.
Counselling to quit
Four studies (Joseph 2004; Little 2009; Rothemich 2010; Winickoff 2014) evaluated the effect of an intervention on the number of smokers subsequently counselled to quit. One study (Little 2009) reported the combined effect on both counselling and referral to quitline services. Rothemich 2010 reported the effect on the discussion of methods to quit. Three of these studies reported significant improvements in the rate of counselling in the intervention group; (Little 2009: 69% intervention versus 3% control; P < 0.01), (Rothemich 2010: 34.4% intervention versus 27.7% control; P = 0.001), and (Winickoff 2014: 54.7% intervention versus 19.2% control; P < 0.001). However, in Joseph 2004 the intervention did not improve counselling rates (73.9% intervention versus 71.8% control; P = 0.60).
Initiation of NRT or other pharmacotherapy
Two studies (Joseph 2004; Winickoff 2014) evaluated the effect of a system change intervention on the prescription of NRT. Winickoff 2014 reported a significant improvement in the intervention group versus control group (18.5% versus 2.4% respectively; P < 0.001). However, Joseph 2004 reported no significant difference between groups (14.7% intervention versus 18.0% control; P = 0.38).
Quitline referral and enrolment
Rates of quitline referral were assessed by three studies (Patwardhan 2012; Rothemich 2010; Winickoff 2014), and all reported significantly higher rates in the intervention arm. In Rothemich 2010 (21.4% intervention versus 8.7% control; P < 0.001) and in Winickoff 2014 (37.2% intervention versus 9.3% control; P < 0.001), a higher proportions of intervention participants were referred to a quitline. In Patwardhan 2012, 240 intervention participants received a quitline card compared to 85 control participants (P = 0.02; measured as counts; no denominator).
Two studies which evaluated quitline enrolment also favoured the intervention. In Winickoff 2014, a higher proportion of intervention participants were enrolled in a quitline programme following intervention (4.1% intervention versus 1.1% control; P < 0.01). In Patwardhan 2012, 81 intervention participants were enrolled in a quitline compared to eight in the control group (P < 0.001; measured as counts; no denominator).
Provided self‐help materials
One study (Gordon 2010) assessed the rate of receipt of reading materials among intervention participants (66.5% received reading materials), but this was not measured for the control participants, so we can report no between‐group comparisons.
Discussion
Summary of main results
This review includes seven cluster‐RCTs evaluating the effect of system change interventions on cessation or system‐level outcomes, or both. The seven studies were heterogeneous for types of settings, interventions, providers and outcome measures. When we attempted meta‐analysis this was corroborated by significant statistical heterogeneity, and we therefore do not report pooled effect estimates for any outcomes.
On the basis of the available evidence, it is difficult to draw any firm conclusions about the success of system change interventions in changing cessation practice or enhancing quit rates. The evidence was equivocal for the primary outcome of smoking cessation. However, all studies which evaluated our secondary outcomes, such as documentation of smoking status, quitline referrals and enrolment favoured the intervention. Three of the four studies which evaluated the provision of cessation counselling also supported the intervention. Of the three studies which evaluated outcomes such as asking about tobacco use and advising to quit, two favoured the intervention in both cases. The evidence for using NRT as part of a system change intervention was also uncertain (of the two studies, one favoured the intervention and the other did not show any difference between groups).
When interpreting the findings of this review, it is important to consider the two‐phase nature of the system change intervention: system change interventions directly promote the delivery of smoking cessation services (i.e. process outcomes), which could subsequently lead to improvement in smoking cessation outcomes among smokers. Hence improvement in smoking cessation is the second phase of the intervention effect. This might be one of the reasons that most of the evaluated process outcomes (secondary outcomes) were improved; however, the effect on the cessation outcome remains inconclusive.
Overall completeness and applicability of evidence
The results should be interpreted with caution, for the following reasons:
-
We could include only seven studies in this review, of which only four evaluated the primary outcome and only a few evaluated each of the secondary outcomes;
-
Clinical practice guidelines for treating tobacco use and dependence (AHRQ 2012) recommend implementing all components of the system change approach; however, none of the included studies implemented all the components of the system change intervention;
-
Although all studies included some components of a system change intervention, such as assessment of smoking status, training clinicians and assisting smokers, the intensity and extent varied widely among studies;
-
None reported reimbursing clinicians or dedicating a staff member to the provision of smoking cessation care;
-
Although guidelines (AHRQ 2012) recommend regular education and training of all staff on providing smoking cessation support, none of the included studies provided ongoing education, and the duration of smoking cessation training also varied widely across studies;
-
Only four studies provided pharmacotherapy from the clinic.
The majority of studies were conducted in the USA and none was conducted in low‐ or middle‐income countries. Hence, the generalizability of the findings to low‐ and middle‐income countries is unknown.
Quality of the evidence
The quality of evidence for all the outcomes, including the primary outcome, was low or very low, as reported in summary of findings Table for the main comparison, so that we can draw no robust conclusions about how useful system change interventions are. There was a high risk of bias in many studies. Four studies evaluated cessation outcomes, but only one had biochemical verification of the self‐reported data. There was inconsistency in most of the outcomes evaluated, which is likely to be due to differences in settings, populations, providers and interventions. Effect sizes also differed greatly between studies. We therefore judge that the overall quality of evidence of the included studies was low.
Potential biases in the review process
The search strategy used in this review was carefully developed, and reviewed by experts in the field, including the review group Information Specialist. We conducted a comprehensive search of a large number of databases. One author went through all references identified by the electronic searches, excluding papers that clearly were not eligible, and two authors independently assessed all potentially eligible titles and abstracts against the eligibility criteria to ensure that no important references were missed. We also searched reference lists of included studies. Despite all of this, we cannot rule out the possibility of missing some important studies. There is always a potential risk of publication bias. Unfortunately, because we found few studies for inclusion in this review, we could not systematically assess publication bias.
Agreements and disagreements with other studies or reviews
Various components of the system change approach, such as training health professionals, using electronic health records for identifying smokers, advising and counselling smokers to quit and providing pharmacotherapy have been evaluated separately in other Cochrane Reviews. However, none of them evaluated the effectiveness of a system change approach. Levy 2004 estimated that such strategies could reduce smoking prevalence by 2% to 3.5% at a population level. AHRQ 2012 guidelines also promote the use of a system change approach to address tobacco smoking.
Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.
Risk of bias summary: review authors' judgements about each risk of bias item for each included study.
| System change interventions for tobacco control (primary cessation outcome) | |||
| Patient or population: Smokers | |||
| Outcomes* | No of Participants | Quality of the evidence | Comments |
| Cessation outcome | 7142 | ⊕⊝⊝⊝ | 2 of the 4 studies significantly favoured the intervention. Mixed effect and low quality evidence preclude drawing conclusions |
| NB. Illustrative comparative risks and relative effects columns have been removed, as only narrative syntheses were conducted due to the presence of significant heterogeneity among studies. | |||
| GRADE Working Group grades of evidence | |||
| 1Self‐reported abstinence was verified only in one study, and one study reported higher dropout rate in one group. | |||
| System change interventions for tobacco control (secondary outcomes) | |||
| Patient or population: Smokers | |||
| Outcomes* | No of Participants | Quality of the evidence | Comments |
| Provision of cessation counselling | 10,949 | ⊕⊕⊝⊝ | 3 of the 4 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Asking about tobacco use | 2615 | ⊕⊕⊝⊝ | 2 of the 3 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Provision of cessation advice | 3003 | ⊕⊕⊝⊝ | 2 of the 3 studies significantly favoured the intervention. The low quality of the evidence precludes drawing conclusions |
| Quitline referral | 3006 | ⊕⊝⊝⊝ | All 3 studies significantly favoured the intervention. However, the low quality of the evidence precludes drawing conclusions |
| Quitline enrolment | 1191 | ⊕⊝⊝⊝ | Both studies significantly favoured the intervention. However, the low quality of the evidence precludes drawing conclusions |
| Prescription of NRT or other pharmacotherapy | 2615 | ⊕⊕⊝⊝ | Of the 2 studies, 1 significantly favoured the intervention. Mixed effect and low‐quality evidence preclude drawing conclusions |
| NB. Illustrative comparative risks and relative effects columns have been removed, as only narrative syntheses were conducted due to the presence of significant heterogeneity among studies *We have not included data from one study as the data were collected as counts (no denominator) | |||
| GRADE Working Group grades of evidence | |||
| 1Included studies had high risk of detection bias. | |||
| Identification/ documentation of smoking status | Smoking cessation Training/ resources/feedback to providers | Dedicated staff to support cessation activities | Policies to improve access to cessation interventions | Free smoking cessation treatment from the organization | Reimburse clinicians for providing smoking cessation support | |
| Cabezas 2011 | Yes | Yes | No | No | Yes | No |
| Gordon 2010 | Yes | Yes | No | No | Yes | No |
| Joseph 2004 | Yes | Yes | No | Yes | Yes | No |
| Little 2009 | Yes | Yes | No | Yes | Yes | No |
| Patwardhan 2012 | Yes | Yes | No | Yes | Yes | No |
| Rothemich 2010 | Yes | Yes | No | Yes | Yes | No |
| Winickoff 2013 | Yes | Yes | No | Yes | Yes | No |
| Study | Asking about tobacco use | Documentation of smoking status | Advice to quit | Counselling to quit | Initiation of NRT or other pharmacotherapy | Quitline referral | Quitline enrolment |
| Cabezas 2011 | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed |
| Gordon 2010 | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed | Not assessed |
| Joseph 2004 | No difference between groups (76.0% vs 74.3%; P = 0.71) | Favoured intervention (60.7% vs 67.0%; P < 0.001) | Not assessed | No difference between groups (73.9% vs 71.8%; P = 0.60) | No difference between groups (14.7% vs 18.0%; P = 0.38) | Not assessed | Not assessed |
| Little 2009 | Not assessed | Not assessed | Not assessed | Favoured intervention (69% vs 3%; P < 0.01) | Not assessed | Not assessed | Not assessed |
| Patwardhan 2012* | Favoured intervention (636 vs 5; P < 0.001) | Not assessed | Favoured intervention (25 vs 3; P < 0.01) | Not assessed | Not assessed | Favoured intervention (240 vs 85; P = 0.02) | Favoured intervention (81 vs 8; P < 0.001) |
| Rothemich 2010 | Not assessed | Not assessed | No difference between groups (58.2% vs 55.3%; P = 0.39). | Favoured intervention (34.4% vs 27.7%; P = 0.001) | Not assessed | Favoured intervention (21.4% vs 8.7%; P < 0.001) | Not assessed |
| Winickoff 2013 | Favoured intervention (59.4% vs 32.6%; P < 0.001) | Not assessed | Favoured intervention (50.5% vs 26.9%; P < 0.001). | Favoured intervention (54.7% vs 19.2%; P < 0.001) | Favoured intervention (18.5% vs 2.4%; P < 0.001) | Favoured intervention (37.2% vs 9.3%; P < 0.001) | Favoured intervention (4.1% vs 1.1%; P < 0.01) |
| *data collected as counts (no denominator) | |||||||
