Background
Variation in decision-making is apparent amongst health professionals in a range of health care settings. In the United Kingdom (UK) there is wide variation in referral to secondary care at general practice and practitioner level [
1], with two fold differences between the top and bottom deciles of general practices for hospital admissions, after standardising for population age, sex and deprivation [
2,
3]. In the United States the percentage of cases classified as urgent in one study varied from 11% to 63% for four nurses, two accident and emergency doctors and two family practitioners [
4]. Some of the variation in practice has been explained by differences in case mix. For example socio-demographic patient factors such as social class and the proportion of the population chronically ill explained about half of the variation in hospital admission levels between general practices in the UK [
2]. However, attempts to explain the remaining variation by exploring the effect of health professional characteristics on decision-making have resulted in little more of the variation being explained [
1,
2,
5,
6]. Investigators of variation in decision-making have recommended moving away from examining the demographic characteristics of health professionals and turning instead to exploring psychological and sociological factors [
2], in particular health professional risk-taking in the face of uncertainty [
7].
Computerised decision support systems (CDSS) are sometimes used by health professionals with the aim of supporting safety and consistency in their clinical decision-making. In the United Kingdom CDSS is routinely used in NHS Direct in England and Wales, and NHS 24 in Scotland, both of which offer 24-hour national telephone clinical assessment by nurses. One might expect there to be less variation in clinical decision-making between nurses in these services because of the use made of CDSS. However, protocols and computer programmes do not necessarily remove variability in health care [
8,
9], and in fact we have previously noted that the level of variation in nurses' decision-making in NHS Direct has been similar to that of health professionals in other services [
10].
Attempts have been made to explain variation in decision-making when CDSS is used. The type of software, and the length of nursing experience of individuals using the software, explains some of the variation in decision-making [
10]. Within the same study, an attempt was also made to explore the decision-making process between nurse and CDSS, and possible influences on it, using qualitative interviews with nurses [
10,
11]. This helped to explain why variation occurred in decisions made, even when CDSS was used. It was found that, although the CDSS recommends the action which patients should take, nurses can explicitly override this recommendation, or they can influence the recommendation made by the CDSS through the way in which they choose to navigate the system [
11]. Within this qualitative study, differences in nurses' attitudes to risk was identified as a possible explanatory factor for variation in the decisions made. Some nurses were very concerned about the risks of under-triaging and missing a serious illness, while others were concerned about over-triaging and overloading a busy service [
10]. Nurses viewed the CDSS as a safety net but they also recognised its limitations and the need for nurse expertise. There was variation between nurses in terms of the emphasis they placed on their own professional expertise and intuition, and the expertise embodied within the CDSS. Given the perceived safety net role of the CDSS, a low risk approach for some nurses was to adhere to the software recommendations. Finally, nurses' talk revealed differences in tolerance of uncertainty in their clinical decision-making, which has been identified elsewhere as a possible explanatory factor of variation in decision-making [
7].
In light of this, the aims of this study were to assess nurse attitudes to risk in telephone clinical assessment using CDSS, and to examine how far measured differences in attitudes between nurses explained the variation in decisions made. The study was undertaken in NHS 24. As well as offering a 24-hour service, NHS 24 is the frontline service for all out of hours general practice services and therefore a large proportion of calls to it are typically made in the evenings or at weekends. Nurse advisors use CDSS to clinically assess and triage callers and can advise them to self-care, to contact their general practitioner or out-of-hours service immediately or later, or to attend accident and emergency departments urgently or as an emergency via a 999 ambulance. That is, they offer advice on the urgency with which help should be sought, pass that call on if required, or offer advice about the management of a health problem. Typical examples would be a parent of a young child calling to ask whether the child needs to see a doctor immediately, or an adult calling to ask for advice on managing a self-limiting condition. To help nurses develop the skills required for such telephone triage, NHS 24 runs an extensive in-house training programme covering the use of the CDSS system, and the identification and response to potentially high risk clinical situations. In this study the dichotomous outcome variable of whether or not calls were sent to self-care was chosen because advice to 'self-care only' presents both callers and nurses with the clearest risk – that the patient will not be seen by a health professional and may be falsely reassured and their care delayed [
10]. Our hypotheses were that nurses who were more concerned about the risk of missing a serious illness, nurses who relied more on the software, and nurses with low tolerance of uncertainty in their clinical decision-making, would be more risk averse and send fewer calls to self care.
Methods
In April 2005, staff in NHS 24 compiled a list of the 542 nurse advisors employed in the service and sent a two-sided questionnaire to each nurse using the internal postal system. Questionnaires were accompanied by reply paid envelopes and returned directly to the research team. Up to two reminders were sent. Anonymised routine data on all calls undergoing clinical assessment during a six month period in 2005 were obtained from NHS 24, including date of call, time of call, age and sex of patient, triage decision, and identification code of the nurse advisor taking the call. This unique nurse identifier was used to match the triage decision data for an individual nurse to their responses to the questionnaire.
The questionnaire
A validated tool was required to measure attitudes to risk in nurses undertaking telephone assessment. This tool needed to address clinical decision-making in the face of uncertainty [
7], and the other aspects of risk identified in the qualitative study of decision making in nurse telephone assessment [
10,
11]. Two potential instruments were identified. However, one was not valid for this study because in attempting to measure doctors' tolerance of clinical uncertainty, researchers developed a tool which actually measured doctors'
reactions to uncertainty, that is the stress related to uncertainty and the reluctance to disclose uncertainty [
12]. The other tool – 'the Grol instrument' – had content validity for this study because it measured attitudes to risk-taking in medical decision-making among family practice doctors [
13], specifically attitudes towards the risk of missing a serious illness when seeing mostly minor problems. This seemed highly relevant to nurse telephone assessment in NHS 24. We found no validated instruments which measured attitudes to risks specific to telephone assessment so we generated a further 10 items from the qualitative study of clinical decision-making in telephone assessment [
10,
11].
A questionnaire was constructed which asked questions about the background and work practices of the nurse, as previously asked in a study of NHS Direct [
10]. It included questions on length of nursing experience, type of nursing experience prior to joining NHS 24, and length of time worked in NHS 24. Nurses were asked to provide their unique NHS 24 nurse identifier. Additionally, there were 15 statements about attitudes to risk, each with a five point Likert scale ranging from "strongly agree" to "strongly disagree." This section of the questionnaire included the five item 'Grol instrument' on attitudes to risk in clinical decision-making [
13], with minor changes made to make it relevant to NHS 24 nurses, and the 10 items generated from our previous work [
10,
11].
Cognitive testing of the questionnaire was undertaken by asking four NHS Direct nurses to complete the questionnaire in the presence of a researcher and discuss problems with comprehension and reasons why they selected options. This resulted in changes to the instructions, and the options available for some questions. No changes were made to the risk section of the questionnaire but it was noted that nurses found it difficult to decide which option to select for some individual items. They explained that this was due to the complexity of decision-making and the wide range of situations faced by telephone assessment nurses. The questionnaire was then piloted on 25 nurses in an NHS Direct site. Item completion rates, and the spread of data within individual items, were good for the 13 responses received.
Analysis
First, a descriptive analysis was undertaken of the nurse characteristics. The adapted Grol instrument score was calculated to identify the proportion of nurses with a 'no risk-taking' attitude and to compare that with published data on doctors [
13]. Then an exploratory factor analysis was undertaken in SPSS version 12 using the 15 items on risk. The aim of the factor analysis was to identify sets of items which tapped into different aspects of attitudes to risk in telephone triage, such as 'tolerance of uncertainty'. Then STATA version 9 was used to test the explanatory power of nurse-level factors in a multi-level logistic regression. The dependent variable was whether a call was sent to self care or not. The independent variables included in the analysis were age of patient, sex of patient (male, female), time of day of call (in-hours Monday-Friday, out-of-hours evenings and weekends), three variables on nurse background and experience, and the fifteen risk items. Ideally, in comparing the decisions of different nurses, adjustments should be made for case-mix to take account of the type and severity of health problems. However, this information is not routinely available from NHS 24 and therefore case-mix adjustment was only possible using the age and sex of the patient, and the time of the call. In NHS 24 calls are allocated sequentially to nurses on duty and therefore there should be no systematic differences in the case-mix of different nurses except that due to the time of day or night they work. We restricted the analysis to include only nurses who had assessed at least 100 calls during the period examined.
Discussion
This study has confirmed that there is variation in decision-making among telephone assessment nurses using CDSS. The attitudes of NHS 24 nurses to risk are comparable to those of some European family doctors; 59% of nurses exhibited a 'no risk-taking' attitude to clinical decision-making, compared with 54% of British doctors, 60% of Belgian doctors and 24% of Dutch doctors [
13]. There were a number of statistical associations between nurses' attitudes to risk and the proportion of calls they sent to self care. However, the lack of consistency of relationship within and across items reduces the credibility of these associations. The strong correlations between referral behaviour and attitudes to risk in a study of family doctors using some of the same items were not consistently found here [
13]. The only items with any consistency were those which mentioned the computer algorithms. Safety is embodied in the computerised decision-support software [
11] and therefore we would expect nurses who felt strongly about the need to use the software to be risk averse and be less likely to use the option of self care. This was the case for three of the four items which mentioned the computer, with some gradient of relationship within items.
Those nurses whose background was mainly in community nursing were less likely to send callers to self care than nurses from a mainly acute hospital background. The effect of type of nursing background on telephone assessment decisions has previously been examined for NHS Direct, and no relationship found, although nurses with longer lengths of nursing experience were more likely to send callers to self care [
10]. In our study we found no association between length of nursing experience and clinical assessment, although the odds ratios in our study showed some increase with length of nursing experience. These differences between NHS Direct and NHS 24 are surprising given the similarity of the services and of the research methods used to explore the effect of nurse characteristics on triage decisions. However, one important difference between NHS 24 and NHS Direct is the extent to which each provides out-of-hours services for general practice. Over 90% of NHS 24 calls in this study were out-of-hours (compared with 70% of NHS Direct calls [
10]) and in addition NHS 24 takes all general practice out-of-hours calls. Also, the proportion of calls sent to self care in NHS Direct was 40% [
10] compared with 16% for NHS 24 in this study. This contextual difference may account for differences between findings for the two services. Indeed the findings may be specific to other contextual issues such as the type of decisions being made. The majority of the decisions here were made in the context of triaging for out of hours general practice services rather than, for example, supporting a patient to make values-sensitive decisions [
17].
The development and testing of an instrument to measure attitudes to risk in telephone assessment was not an aim of this study, but was undertaken given the lack of an existing instrument. Although we followed established methods of instrument development, taken together our items did not perform well psychometrically [
15]. Our attempts to measure attitudes to risk may, in retrospect, have been insufficiently conceptualised. The risk environment faced by telephone triage nurses is complex, and includes a range of competing risks whose salience may vary from nurse to nurse. For example, the nurse must consider patient safety, patient convenience, management demands for efficiency, the workload of services, and their own professional integrity and reputation. In many situations these risks are in conflict: avoiding one may mean facing another instead. Given such a context, even if a nurse can be said to have an 'aversion to risk', it is unclear how this will impact on their assessment behaviour, since they will be able to choose to avoid only some of the risks in the complex environment they face. The items in our risk measure attempted to address some of the competing risks evident from our previous qualitative work, but may not have been sufficiently closely related to how nurse decision-making actually occurs. We suggest that future quantitative work in this area will benefit from a deeper understanding of the risk environment as perceived by telephone assessment nurses. The use of observation of nurse advisors, or clinical vignettes, to determine nurses' attitudes to risk may be a more useful way forward than the instrument used here.
Competing interests
The author(s) declare that they have no competing interests.
Authors' contributions
JM, COD, and DH conceived the study. JM and AOC designed the study. IA facilitated acquisition of data. AOC analysed the data. All authors contributed to interpretation of data and guided further analysis. AOC drafted the manuscript and all authors made critical contributions to further drafts. All authors read and approved the final manuscript.