A comparison of patient recall of smoking cessation advice with advice recorded in electronic medical records

The delivery of brief smoking cessation advice during routine general practice consultations, a simple intervention usually lasting no more than one or two minutes, has been shown to be effective in increasing cessation rates[1], and is one of the most cost-effective means to reduce the burden of smoking[2]. However, in the past, large discrepancies have been reported between the proportion of patients who recall having received cessation advice and the proportion of patients with advice documented in their medical records; in one study, cessation advice was recorded in the notes of just 30.9% of patients who reported having received advice[3].

The introduction of the Quality and Outcomes Framework (QOF) in April 2004 provided, for the first time, a financial incentive for GPs to document in smokers' medical records that they have been offered cessation advice, worth approximately £4,500 to each general practice per year[4]. Specifically, since 2004 GPs have been rewarded for recording having offered cessation advice to smokers with specified chronic health conditions (asthma, chronic obstructive pulmonary disease, chronic kidney disease, coronary heart disease, diabetes mellitus, hypertension, schizophrenia, bipolar affective disorder and other psychoses and stroke or transient ischemic attack) at least every 15 months[4]. A tripling of rates of advice recorded in medical records occurred in the year following the introduction of the QOF, with no concomitant increase in prescribing of nicotine replacement therapy or bupropion for which there is no QOF incentive[5]. One explanation for this is that the QOF simply increased GPs' propensity to document cessation advice offered, rather than actually increasing their rate of advice-giving[5]. To investigate whether this might have occurred we compared advice recorded in a large dataset of primary care records with that recalled in a survey of National Health Service (NHS) patients in England in the years since the introduction of the QOF.

The Health Improvement Network (THIN) is a database containing the electronic primary care medical records of almost seven million patients from 446 general practices throughout the UK, and is broadly representative of the UK population in terms of patient demographic characteristics[6]. For each year from 2000 to 2009, all patients from the THIN dataset who were over the age of 16 and registered with an English practice on an index date of 1st July in that year were identified. Each patient's year of birth, sex and the Strategic Health Authority (SHA) within which their GP surgery was located were identified. Patients' medical records were searched for Read codes[7] documenting the delivery of smoking cessation advice to that patient, and, for each year, the proportion of patients with a recording of cessation advice in the 12 months prior to the index date was calculated.

The Primary Care Trust (PCT) Patient Surveys monitor patients' experiences of NHS services[8]. In 2004, 2005 and 2008, a simple random sample of patients was selected from each PCT in England, and a postal questionnaire administered asking whether the respondent had 'definitely' or 'to some extent' received cessation advice from a health professional at their GP surgery within the last 12 months. Completed questionnaires were received from 122,113 patients in 2004, 116,939 in 2005 and 69,470 in 2008 (response rates of 47.4%, 45.4% and 38.3% respectively).

Previous work using the Patient Survey has shown that the provision of smoking cessation advice by primary health care professionals varies with patient sex and age[9]. Consequently, as Patient Survey respondents and patients in the THIN dataset have different demographic characteristics, directly comparing 'raw' data on smoking cessation advice received by patients in each source is not appropriate. Therefore, we used the following standardisation procedure to enable comparison of data from THIN and the Patient Surveys. For 2004, 2005 and 2008, age group, sex and SHA-specific rates of patients reporting having received smoking cessation advice within the last 12 months at least 'to some extent' were calculated from Patient Survey responses. These rates were applied to strata of the THIN population (similarly defined by age group, sex and SHA) at the corresponding index date using indirect standardisation[10], producing estimates for annual rates of recalled cessation advice that might be expected from THIN patients, based on Patient Survey responses (referred to as 'predicted recall rates'). Predicted recall rates were then compared graphically with the actual cessation advice rates documented in THIN patients' medical records.

This study was approved by the Leicestershire and Rutland Research Ethics Committee.

All analyses were completed using STATA version 11.0 (STATA Corp, College Station, TX).

The number of patients aged 16+ registered with a THIN practice in England increased from 1.8 million in July 2000 to 2.0 million in July 2009. At each index date, 49% of patients were male, with a mean age of 47 years (interquartile range 32-61). Figure 1 shows, for years studied, the proportions of patients within THIN who had smoking cessation advice documented in their medical records in the previous 12 months and, for Patient Survey years, predicted recall rates (estimates for annual rates of recalled cessation advice within THIN patients, based on the results of the Patient Survey).

The proportion of THIN patients with cessation advice documented in their medical records increased considerably over the study period, from 1.2% of patients in 2000 to 10.9% in 2009, with the majority of this increase occurring between 2003 and 2005. However, although similar in 2004, the predicted recall rate was subsequently lower and increased less over the survey period (6.6% of patients in 2004; 8.3% in 2008). In 2004 there was good agreement between recording of cessation advice in THIN and predicted recall rates, but in both 2005 and 2008 agreement between recorded and predicted recall of cessation advice was much less strong with recall rates being much lower.

This study is, to our knowledge, the first to compare, at a population-level, smoking cessation advice recorded in medical records with patients' recall of such advice as reported in large surveys; in 2004 there was close agreement between both data sources but this decreased substantially in 2005 and 2008.

Some of the longitudinal changes in the proportions of patients recalling cessation advice, or having this documented in their medical records, may be due to changes in population smoking prevalence. Ideally, this study would have assessed recorded and recalled advice within smokers only, but the ability to confidently identify smokers in THIN was poor in the early years of this analysis[11] and it was impossible to identify respondents who were smokers from the 2008 Patient Survey. To allow interpretable, annual comparisons the results presented are, therefore, based on annual denominators of all patients (for THIN) and all respondents (Patient Survey). However, between 2004 and 2005, the period when the gap between patient-reported and documented advice appears in Figure 1, there was little change in smoking prevalence in England[12], and thus changes in prevalence are unlikely to explain the divergent data.

The relatively low Patient Survey response rates raise the possibility of response bias, with smokers or patients recalling advice perhaps being more or less likely to complete the survey; if the latter is the case, the Patient Survey may underestimate the proportion of patients who recall having received cessation advice. However, the response rates in 2004 and 2005 are very similar and the characteristics of respondents completing the survey in these two years are unlikely to have changed substantially. Again, therefore, it seems reasonable that other factors also account for the divergence in recorded and recalled advice rates.

The findings presented here are contrary to those from research studies, which showed more patients recalling receiving advice than had this documented in their medical records[13]. Historically, GPs may not have documented all cessation advice delivered to smokers, though when asked a majority claim to have done this[14]; with the introduction of the QOF, from 2004 onwards GPs may simply be documenting more of the advice that they give[5]. The failure to observe large increases in patients' recall of cessation advice, alongside no increase in rates of prescribing of stop smoking medications[5], tends to support this. The divergence between rates of recording of advice and patient recall seen in Figure 1 is less easy to explain, unless there was an increase in the amount of advice being delivered in such a way that patients did not perceive it as such. GPs have different approaches to advice giving[15], and thus advice documented in patient records could reflect simply the briefest mention of smoking and not be of sufficient duration or intensity to be recalled as 'advice' by smokers[16]. Alternatively, GPs could be recording offers of advice that were not actually made or which were refused; if the latter occurred, patients would not necessarily report receiving advice whereas the offer could legitimately be recorded in their medical records. To our knowledge there is no research which validates smoking cessation Read Codes, so it is not possible to be sure of the degree to which codes used in routine clinical practice represent the nature and extent of the advice delivered to smokers; this is an area for further research.

In conclusion, this study finds substantial increases in the number of patients with a record of cessation advice having been delivered in their primary care medical records, and a smaller increase in the proportion reporting having received advice. However, the discrepancies between these data sources and the inherent difficulties involved in interpreting each mean we cannot be sure that, since the introduction of the QOF, the proportion of smokers being advised to quit by primary care health professionals has actually improved as much as the improved documentation rates suggest. Our findings call into doubt the effectiveness of the financial incentives introduced to encourage GPs to deliver cessation advice to smokers and add to the limited body of evidence evaluating the effect of pay for performance schemes on clinical care and patient outcomes[17].