Measuring empathic, person-centred communication in primary care nurses: validity and reliability of the Consultation and Relational Empathy (CARE) Measure

Ten out of 55 randomly selected GP Practices within a 40 mile radius of the study office responded positively to an invitation to participate in the study. The Practices represented three Scottish Health Board areas and combined provided the opportunity to collect data on the consultations of 20 nurses (19 practice nurses and 1 nurse practitioner). The aim was to collect self-completed patient questionnaires for 50 consecutive consultations with each participating nurse. The sampling strategy ensured that patients from a range of socioeconomic levels would be included in the study, but we did not have sufficient funding to specifically sample from high and low deprivation areas as we did in our original validation work [21, 22]. The consultation number was based on previous work that demonstrated the required sample size to effectively discriminate between GPs [22].

Practice receptionists gave consecutive adult patients (16 years or older) a questionnaire when they checked in for their appointment with a participating nurse. This is the same approach that we have used in our previous validation studies with GPs [21, 22]. Patients completed the questionnaire immediately after the consultation and placed it in a sealed box in the waiting room. The questionnaire contained:

Data were collected between September 2012 and October 2013.

The 10 CARE Measure items are rated on a 5-item response scale from 1 = poor to 5 = excellent. The overall score is the sum of the ten items with 10 being the lowest possible score and 50 the highest. Up to two not applicable (N/A) responses or missing values are allowed and these are replaced by the average item score.

As we wanted to directly compare the findings of the current study with our previous work with GPs [21, 22] we did not attempt to ‘weight’ the scores.

Ethical approval was obtained from the National Research Ethics Service (NRES) Committee North West, Preston (Reference 12/NW0607, on 2/8/2012). All nurses who participated in the study provided written informed consent. Informed consent was not required from the patients who decided to complete the questionnaire, because the questionnaire was anonymous and did not ask for identifiable information. Completion of the questionnaire was voluntary and the optional nature of the study was explained in the information on the front of the questionnaire.

The data were analysed using SPSS (version 21) and urGENOVA software via its associated wrapper program GS4 for the reliability analyses [29, 30] as was done in our previous studies [25, 26]. Descriptive methods were used to describe the sample, calculate the CARE Measure and PEI scores, and check the variability in the data. As data distributions were skewed, differences between groups were assessed through non-parametric tests. The perceived relevance and face validity of the CARE Measure was assessed by analysing the number of not applicable and missing values for each of its 10 items as well as the patients’ rating of the importance of the 10 items, as in previous studies [22‐27]. Construct validity was examined through factor analysis (principal component analysis with varimax rotation and Kaiser normalisation) and correlations (Spearman’ rho) between the CARE Measure items, and PEI and the overall satisfaction measure. The internal reliability of the CARE Measure was assessed through Cronbach’s alpa. The ability of the measure to discriminate between nurses was assessed using Generalisability-theory (G-Theory) and associated Decision D studies [22, 25, 26].

Overall, almost three-quarters of patients (73 %) perceived the CARE Measure items as very important to their current consultation. Older patients and patients with worse self-reported overall health tended to score the importance of the items higher than younger patients (p = 0.000) and patients with better health status (p = 0.033) (Table 2). No statistical differences were found for the other patient characteristics.

‘Not applicable’ responses in the CARE Measure amounted to only 6 % of the total possible ‘not applicable’ responses (444/7740 items; see Table 3). For CARE Measure items 1 to 8, the average number of not applicable responses was 3 % (n = 181), ranging from 0.6 % for item 1 to 5 % for item 2. The highest number of “not applicable” responses were recorded for items 9 (n = 112, 15 %) and 10 (n = 151, 20 %), which relate to ‘taking control’ and ‘making a plan of action’ respectively. The total number of responses for which there was missing data was 124, representing less than 2 % of the total possible number of missing responses (i.e. 124/7740). The missing responses were evenly distributed across the CARE Measure items.

Three or more ‘not applicable’ or missing responses were given by 76 (9.8 %) patients. Patients with worse self-reported health had less occurrences of three or more ‘not applicable’ or missing responses than patients with better self-reported health status (χ ² = 5.860, p = 0.53). No statistical associations were found for the other patient characteristics (results not shown).

Explanatory factor analysis on the CARE Measure items, PEI and satisfaction questions showed three factors with the CARE Measure items on one factor with high loadings (0.883-0.967), indicating a robust internal structure of the CARE Measure (Table 4). The PEI items loaded on the second factor, and the satisfaction measures on the third factor. The three factors explained 83 % of the variance.

Correlations between the CARE Measure scores and patient enablement and overall satisfaction supported construct (convergent) validity in relation to overall satisfaction (Spearman’s rho 0.54, p = 0.000), and as expected less (divergent) with patient enablement (Spearman’s rho 0.19, p = 0.000).

The CARE Measure showed weak but significant positive relationships with consultation length (Spearman’s rho 0.12, p = 0.002) and how well patients knew the nurse (Spearman’s rho 0.22, p = 0.000). The patient characteristics of age, gender, self-perceived overall health, living arrangements, employment status and language spoken at home were unrelated to the CARE Measure score.

In terms of the CARE Measures ability to discriminate between nurses a moderate level of agreement (inter-rater reliability) between patients was found with 50–60 patient ratings per nurse (Table 5). As expected, the reliability of the measure increased with the number of completed questionnaires per practice nurse.

The measure showed high internal reliability (Cronbach’s alpha coefficient 0.97). Removal of any item weakened this internal reliability (results not shown).

Overall, the mean CARE Measure score for all practice nurses (45.9) was somewhat higher than in the primary care setting with GPs (40.9) [22]. Additionally, the primary care nurses required a slightly higher number (60) of completed CARE Measures than the GPs for whom 50 CARE Measure scores were sufficient to estimate reliably the mean CARE Measure score for an individual GP [22]. The reason for the higher number of required patient questionnaires for practice nurses is that the practice nurses had less variability in the data. The high level of perceived importance of the CARE Measure items to everyday practice nurse consultations in this study (73 %) was similar to those with general practitioners (76 %) [22]. In the latter, it was also found that older patients and patient with worse health status (in terms of long-standing illness, psycho-social or emotional problems) tended to rate the items as more relevant. The high perceived relevance of the measure to clinical encounters has also been found in similar studies outside primary care [25, 26] and in international settings [24, 27]. The finding that most “not applicable” responses were given to item 9 (helping you to take control) and item 10 (making a plan of action with you) was also in agreement with previous studies [22‐26]. As the majority of consultations were routine nursing appointments it could be the case that within that context empowerment and shared-decision making were perceived as less relevant by some patients. Finally, the significant but weak positive associations between estimated consultation length, how well patients know the nurse and CARE Measure scores were similar to those shown in previous studies [22, 25]

A strength of the study was that it builds on earlier work on the CARE Measure and adds to the body of knowledge on its reliability and validity across healthcare disciplines. It was also a reasonably large number of patients, though the number of nurses was lower than we had hoped for. The study had some limitations. First, the participating nurses were volunteers and the study may have been open to sampling bias. This could have led to the high consultation scores and limited the range in performance and the resulting reliability of the inventory. Secondly, one practice withdrew from the study (due to circumstances unrelated to the study) and this reduced the number of participating nurses from 20 to 17. A larger sample size, as employed in previous studies [22, 25, 26], may have captured a wider potential spectrum of possible population performance among nurses and may have found it easier to discriminate between different nurses and demonstrate higher levels of reliability. Another limitation was that it was underestimated how many patients would not return the questionnaires. This resulted in some nurses having collected less than 50 questionnaires even though more than 50 questionnaires had been handed out to patients.

As the CARE Measure was originally developed and validated in primary care with GPs, the findings of this study suggest that the CARE Measure can be used reliably with primary care practice nurses as well. Practice nurses working in general practice tend to have serial consultations with patients allowing them to establish ongoing relationships with them. In this study, most patients felt that they knew the nurse quite well or very well. Further work is required across nursing to establish if the CARE Measure is also reliable, valid, acceptable and feasible in other nursing settings, such as secondary care or acute care community clinics such as sexual health, and also in allied healthcare professionals such as physiotherapists, occupational therapists, podiatrists, and so on.

Although capturing patients’ views on health professionals’ interpersonal skills is now widely regarded as an important feature of high quality health care systems, the evidence that such feedback in itself leads to change in professionals consulting behaviour (and thus improves scores) is equivocal [33]. In order to support healthcare practitioners to improve or maintain their CARE Measure scores and/or the ability to provide an empathic service, earlier work developed [34, 35] and piloted [36] the CARE Approach framework. This inter-disciplinary resource is derived from the CARE Measure and wider literature and covers the four interactive components of Connecting, Assessing, Responding and Empowering with the aim of fostering empathy and patient-centredness in clinical encounters [34, 35]. The use of the CARE Measure as well as the CARE Approach feeds into current healthcare policies and professional codes of conduct on maintaining, enhancing and monitoring empathic, person-centred care [15‐19].