Distributing questionnaires about smoking to patients: impact on general practitioners' recording of smoking advice

Questionnaires distributed to patients as they attend doctors are frequently used in primary care studies with behavioural outcomes[1]. In particular, such data collection methods have been employed in trials of smoking cessation[2, 3] or health promotion interventions[4] and in some studies investigating the impact of training physicians in smoking cessation methods on doctors' clinical behaviour and patients' smoking[5]. However, little research has investigated the validity of such instruments for these uses. Post-consultation questionnaires are known to over-estimate rates of smoking cessation advice given by general practitioners (family physicians – GPs)[6, 7] and hospital physicians[8, 9], and false positive reports of advice increase as time elapses following patients' consultations[7, 10]. There is even less research which investigates whether giving pre- or post consultation questionnaires to patients affects doctors' consulting behaviour and after extensive literature searching, the authors could identify no studies on this topic. Consequently, we analyse data from a primary care smoking cessation study, to investigate whether or not distributing questionnaires on smoking behaviour to patients before and after they see a general practitioner influenced doctors' recording of smoking cessation advice in patients' medical records.

We obtained and inspected medical records for 77.9% (5276/6775) of all adult patients who attended general practitioners' surgery sessions and the number of medical records inspected for each GP varied between 48 and 306. Overall, 6.4% (336/5276) of records contained a note that advice against smoking had been given. 60.2% (3178/5276) of records inspected were from women and the mean [SD] age of patients whose records were inspected was 40.7 [22.3], with age data being unavailable for 4 patients.

51.9% (2739) of medical records were from surgery sessions during which researchers distributed questionnaires on smoking behaviour and the remainder were from surgery sessions where no data collection occurred. The distribution of age and gender of patients was similar between the two groups, but the distribution of data collection surgeries differed between the 'after' period of SIS (when GPs could claim the new payment) and the 'before' period when this was not available; a researcher distributing questionnaires was present in 1271 out of 2767 (45.9%) patients seen during the 'after' period, and 1266 out of 2509 (50.5%) seen during the 'before' period. The level of agreement between both RAs and a general practitioner for the presence or absence of a note of discussion of smoking in the medical records was good[12] with the Kappa values (95% CI) for agreement between a general practitioner and the two RAs being 0.79 (0.6 to 0.98) and 0.85 (0.66 to 1.0) respectively.

Recording of advice against smoking in medical records was significantly more likely when a researcher distributed questionnaires in the surgery session; when a researcher was present 8.0% (220/2739) of medical records had a note that smoking had been discussed versus 4.6% (116/2537) when no researcher was present. (Odds ratio 1.80, 95% CI 1.38 to 2.35, p < 0.001).

Recording of advice against smoking in medical records was also more likely in female than male patients (6.9% of women's notes included a record of smoking advice versus 5.6% of men's), and in the middle two quartiles of age (9.8 and 7.5%) than in the youngest (< age 24, 3.8%) or oldest (≥ age 59, 4.3%) patients. Additionally, patients' medical records from the period of SIS before the new health promotion payment became available to general practitioners were slightly more likely to have a record of advice (6.9%) than those from the period during which this could be claimed (5.8%). This suggests that the new health promotion payment had no impact on recording of smoking advice in medical records.

Table 1 shows that, after adjustment for age, sex and for whether or not the new health promotion payment could be claimed at the time the selected consultation occurred, the odds ratio for the recording of advice against smoking in patients' medical records during surgery sessions where questionnaires were distributed compared to those where no data collection occurred was 1.78 (95% CI, 1.36 to 2.34).

This analysis demonstrates that when researchers distribute questionnaires about smoking to patients before and after they see their general practitioner and the doctor is not blind to questionnaire contents, he or she is almost twice as likely to record discussion of smoking in the medical records. The impact of the researcher had an independent effect which was not affected by patients' age or sex or the availability to general practitioners of a new smoking-related health promotion payment which was being evaluated[13].

The authors searched but could find no papers investigating the impact on general practitioners' behaviour of questionnaires distributed to patients, so we believe that this study is original. Additionally, data presented here was collected at a time when patients in the UK were not required to give consent for medical researchers to inspect their medical records for research purposes (i.e. to 'opt in' to this type of research). If the study were repeated today, researchers would need to contact patients and obtain their consent before inspecting records, introducing potential bias into this process. A weakness of the study is that researchers were only able to obtain 78% of the medical records for patients who attended selected surgery sessions and we can say nothing about the records which were not obtained. Researchers relied on clerical staff to find records in addition to performing their usual daily work tasks and it is possible that this unpaid work did not receive high priority, resulting in the relatively high rate of non-located records. It is unlikely, however, that researchers' inability to locate medical notes would be related to doctors' recording of discussing smoking, so data from records which researchers could not access would not be expected to affect study findings. Finally, it is possible that unmeasured factors may have affected doctors' documentation of the smoking advice that they delivered to patients. For example, doctors are more likely to advise smokers' who are interested in stopping or who present with smoking-related problems[14]. It is possible, therefore, that doctors' propensity to discuss smoking when it may be relevant to patients' presenting problems, or with 'interested' smokers, results in more discussion of smoking between GPs and smokers than non-smoking patients. If we had collected data on patients' smoking status in both study groups, then we would have been able to adjust our findings for any influence that this may have had. However, our method for selecting comparison group surgeries should have ensured that these were similar and seems unlikely that smoking prevalence would differ significantly between patients attending the two groups of surgeries, such that this could explain study findings.

General practitioners do not document all of the advice against smoking which they deliver to patients[6], so questionnaires may have prompted more discussion of smoking by GPs, more recording of advice or both of these outcomes. One could differentiate between these possibilities by either directly observing or recording what actually occurred in consultations, but recording of consultations can also influence consulting behaviour[15]. Researchers who plan to use pre- or post-consultation questionnaires need to consider the potential impact of these on doctors' behaviours. It is likely that questionnaires addressing other topics (e.g. alcohol or dietary intake) may have similar impacts on other aspects of clinical behaviour (e.g. discussions about alcohol or recording of these). Such impacts are likely to be most important in studies where recording of doctors' behaviours in medical records are used as outcomes. In these kinds of study, questionnaires use could result in higher overall rates of outcomes than anticipated, affecting sample sizes need to show differences between groups.

Researchers also need to be cautious when using pre-consultation questionnaires in primary care trials that involve allocating participants to "normal care" treatment groups. Some primary care smoking cessation studies have used "normal care" treatment arms[2, 16‐18], but questionnaires distributed to general practitioners in 'normal care' trial arms would be expected to increase doctors' recording of smoking status and may even influence doctors' delivery of smoking advice to patients. As doctors' brief advice against smoking is an effective smoking cessation intervention[19], such questionnaires could potentially increase cessation rates in 'normal care' trial arms, interfering with the power of any trial to detect differences between trial groups.

This study provides direct evidence for the value of blinding study participants, including health professionals, to the contents of research questionnaires[20]. However, in applied health services research this is not always possible because doctors will often wish to see questionnaires to be reassured that data collection instruments are not too long and will not cause offence to patients[21]. Further research into the impact of questionnaires distributed to patients for research purposes is required and where blinding of research participants is not possible, researchers must consider the likely implications of this.

When researchers distribute questionnaires to patients before and after they consult with GPs and GPs are aware of questionnaire content, this can affect GPs' behaviour. Giving out questionnaires which asked about patients' smoking habits resulted in GPs recording more advice against smoking in patients' medical records. This increase could have been due to an increase either in the actual amount of smoking cessation advice delivered or in the recording of advice given. Study findings require consideration when designing research studies that use data recorded in medical records as outcome measures.