An instrument assessing patient satisfaction with day care in hospitals

We composed a questionnaire for day care patients, based on the COPS [13]. The COPS is a short core questionnaire to measure patient satisfaction, based on the needs of clinical patients and outpatients of academic hospitals. The questionnaire was developed to compare satisfaction scores between hospitals, and to identify opportunities for quality improvement.

The COPS consists of six dimensions, each dimension is covered by two, three or four questions: Admission procedure (3 items), Nursing care (2 items), Medical care (2 items), Information (4 items), Autonomy (3 items) and Discharge and aftercare (3 items).

For the COPS-D we included the six dimensions (17 items) of the COPS, assuming that these dimensions are as relevant to day care patients as they are to clinical and outpatients. In addition, we asked quality staff members of five general hospitals that had indicated a need for a day care patient satisfaction questionnaire to indicate which questions they found necessary to add in order to measure patient satisfaction in their day care organisation. Based on the needs formulated we added two new dimensions to the existing six dimensions from the COPS: Operation room (6 items) and Pre-admission visit (4 items). See Appendix 1 for the items used. In total the COPS-D consists of 27 questions. The same answering categories were used as in the COPS: a 5-point Likert-scale (1 = unsatisfied, 2 = somewhat satisfied, 3 = rather satisfied, 4 = quite satisfied and 5 = very satisfied). A dimension score is composed by adding the item scores and dividing the resulting total score by the number of items.

Besides the COPS-D, the questionnaire includes an overall rate for satisfaction with the patient’s treatment and stay in the hospital (range 0 unsatisfied to 10 very satisfied), and questions to assess patients’ background characteristics (i.e., age, gender, level of education) and a rating of perceived health status (bad, moderate, good, very good, excellent).

The COPS-D was tested in the five participating hospitals between November 2005 and May 2008 by sending a questionnaire to a sample of day care patients. We randomly selected 200 patients of each participating day care specialty who had visited the day care facility within the last six months. They received the questionnaire at home, accompanied by a letter from the hospital informing them about the questionnaire’s background. The questionnaire could be returned to an independent research institute in a pre-stamped envelope. A reminder was sent after two weeks. A helpdesk using phone and email was installed for patients needing support.

First we calculated the correlations of the newly added dimensions Operation room and Pre-admission visit with the original COPS-items. If these correlations were higher than 0.7, the items in the dimensions Operation room and Pre-admission visit might be measuring the same concept as the original dimensions in the COPS could be deleted without loss of information.

We tested the construct validity of the eight dimensions using a confirmatory principal component factor analysis with varimax rotation and eigenvalues greater than 1 [15]. We calculated extraction communalities as estimates of the variance in each variable accounted for by the factor. Small values indicate variables that do not fit well with the factor solution. The communality fits with a threshold of more than 0.4 [14]. We also investigated the Measures of Sampling Adequacy (MSA). This statistical analysis tests whether the sample fits the a priori defined model. If values are lower than 0.5 this may indicate that the variable does not seem to fit with the structure of the other variables.

Construct validity was also tested by calculating Spearman’s rank correlation coefficients of the eight dimensions with each other and with the overall satisfaction. A ρ-value ≥ 0.5 is considered to represent a strong correlation; 0.35 to 0.5 a moderate correlation; and 0.2 to 0.34 a weak correlation [16].

We tested the reliability of each dimension by calculating the Cronbach’s α. The α should preferably be higher than 0.7 [17, 18]. In addition, we calculated the inter-item correlations and the item-total correlations (ITC) corrected for item overlap (item internal consistency). The inter-item correlations within a dimension should preferably be ≥ than 0.45. If the inter-item correlations are high (0.6 or 0.7), this indicates that 3 to 5 items will suffice in the dimension. If the inter-item correlations tend to be low (0.3 or 0.4), more items must be added to the dimension with a minimum of 7 to yield acceptable α’s [17]. Too high inter-item statistics (≥ 0.9) can indicate a redundancy of an item. Item-total statistics show the Cronbach’s α if an item is deleted: if this α is higher, the item should preferably be deleted. Also the item internal consistency should be larger than 0.4 [18]. We reported floor and ceiling effects to assess the skewness of the scores. The floor effect refers to the percentage of patients giving the worst possible score (namely 1 = unsatisfied). The ceiling effect refers to the percentage of patients giving the highest possible score (namely 5 = very satisfied).

Next, we tested the item discriminant validity (IDV). We correlated the items with the dimensions. The items should correlate more strongly with the dimension they are supposed to fit in than with the other dimensions [19].

Next, we checked the external validity of the sample used: whether the results can be extrapolated to the population of day care patients [20]. We compared the figures of the Dutch National Medical Registration (LMR) [21] on day care admissions in 2008 of the same eight specialties on gender and age with our sample. We expect that the results can be extrapolated based on our sample.

Furthermore, we tested known group differences using an ANOVA with a Bonferronni post hoc analysis. We expect elderly, lower educated and healthier patients to report higher overall satisfaction on all dimensions. We also expect that gender does not have a significant effect on all dimensions [12, 22, 23].

Finally, we tested the acceptability of the questionnaire by checking response rates and the missing values. The answering category ‘not applicable’ was excluded from the analyses. Items with a relatively high number of missing values (more than 10%) must be avoided [24, 25] and might be left out of the basic questionnaire as they may not be applicable or relevant.

Data were analyzed used IBM SPSS 15.0.

The COPS-D was sent to 8355 patients discharged from a day care unit from the five general hospitals. In total, 3802 patients returned and completed the questionnaire. The average response rate was 46% (range from 38% till 60%) See Table 1 for patient characteristics.

The confirmatory factor analysis confirmed the structure of eight dimensions. 73% or more of the variance was explained by the dimensions (range 73,3% to 89,8%, mean 79%,) see Table 2. All items showed an extraction communality ≥ 0,45 (range 0,634 to 0,898). All items showed a MSA ≥ 0.5 (range 0,651 to 0,889).

Table 3 shows that good Cronbach’s α’s were found for the eight dimensions (range 0,816 to 0,906). The item internal consistency is supported by levels higher than the threshold of 0,40 (range 0,589 to 0,825). Also, all inter-item correlations were higher than 0,45 but not too high (<0,9) (range 0,511 to 0,797). The item internal consistency being high affirmed that the number of items in the dimensions is sufficient.

We also found a ceiling effect in our data: the percentage of patients giving the highest possible score is much higher (range 11,6% to 30,4%) than the percentage of patients who gave the worst possible score (range 0,1% to 1,0%). The item discriminant validity (IDV) shows that all items correlate more highly with the dimension they fit in than with the other dimensions (range 0,342 to 0,729).

The correlation of the dimensions with the other dimensions and with the overall satisfaction is given in Table 4.

The Spearman ρ-correlation with the overall satisfaction was significant at the 0.01 level for all dimensions. Also, all correlations could be considered as strong (>0,5) (range 0.527 – 0.623). The dimension Information correlated most strongly with patient ratings of overall satisfaction (ρ = 0,623). The inter-dimension correlation was also significant at the 0.01 level for all dimensions. All correlations except one (0.477) could be considered as strong (range 0.529 – 0.762). The strongest inter-dimensional correlation is the one between the dimensions Information and Discharge, the weakest correlation is the one between Admission and Medical care.

The results concerning the external validity and known group differences of the COPS-D are given in Table 5 and 6. Table 5 shows that our sample consists of less children than the LMR-data. Gender and age appeared to be comparable to the total Dutch day care population.

Table 6 shows significant differences regarding gender, age, education and health status. Comparison by gender showed significant differences for two dimensions, nursing care (male 4,12 (SD = 0,74), female 4,06 (SD = 0,80), p = 0,022) and medical care (male 4,10 (SD = 0,86), female 4,03 (SD = 0,88), p = 0,024). Older patients are more satisfied on all eight dimensions. Similarly, healthier patients were more satisfied consistently. Lower educated patients were more satisfied, except with the dimensions Pre-admission visit and Autonomy.

One of the remaining 24 items of the COPS-D had 12% missing values: ‘Transfer of information to external professionals’. All the other items had less than 8% missing values (range 0% to 7,9%).

The factor analysis confirmed the relevance of the eight dimensions of the questionnaire. The extraction communalities were higher than the threshold, as were the MSA.

Patients’ overall satisfaction showed strong correlations (Spearman’s ρ >0,5) with the eight dimensions. From the literature [32] we know that overall satisfaction rates most highly correlate with factors associated with patients interaction with the hospital staff. This study showed that the dimension with the highest correlation with overall satisfaction in day care was the dimension Information. These findings correspond for example with the results of patient satisfaction research on pre-assessment clinics, where the provision of information was also an important indicator of patient satisfaction [29, 33], and in day care for neurological patients, where patient satisfaction was not related to a new diagnosis or treatment, but rather to the amount of information and emotional support during the day care stay [33]. Patient information has become crucial in health care because it is necessary to enable the patient to take part in medical decisions and the resulting care provision [34].

The items in the COPS-D with the strongest correlation with overall satisfaction were ‘Transfer of information between professionals’ (ρ = 0,58), ‘Reception at the day care department’ (ρ = 0,57) and ‘Information about further treatment’ (ρ = 0,57). Next in strenght were ‘Expertise’, ‘Attention’ and ‘Information of the nurses’ (ρ = 0,55) and ‘Doctors’ (ρ = 0,53).

Apparently, the procedural items are more important to day care patients than to clinical patients, for whom interaction and attention of the professionals were most strongly related with their overall satisfaction [32]. Literature supports the finding that exchange of information between health care professionals and patients is essential, also because of time constraints and limited patient contact [5]. This might be an interesting field of future research. Especially the consequences for organisation of and attitude of professionals in day care centres deserve attention.

Although our sample involves less children (age 0–19 year) than the total Dutch day care population, the results of the two groups are comparable regarding gender and age. Therefore we assume that the results can be extrapolated to Dutch day care patients in general.

We found, as expected, that older and healthier patients are more satisfied with respect to all dimensions. We also found patients with lower education levels are more satisfied, except for the dimensions Pre-admission visit and Autonomy and that gender does not have a significant effect on satisfaction scores on all dimensions. These findings are in line with the literature [12, 22, 23].

Clearly missing values are to be avoided. Our analysis showed that one of the remaining 24 items in this questionnaire had over 10% missing values. There is no strict rule regarding the maximum number of missing values to be considered acceptable. The number of missing data may be affected by a number of factors: the nature of the variable, the specialty a patient visited or the patient’s treatment [24, 25].

The item ‘Transfer of information to external professionals’ clearly is not applicable to all patients. Twelve per cent of the day care patients did not answer this item on the questionnaire. This might depend on the specialty or treatment of the patient. Twenty per cent of the dermatology patients did not answer this item, as well as up to 7% of the patients coming for ear nose throat-surgery.

During our pilot study it appeared that day care organisation varied widely between Dutch hospitals. Day care centres differed in name, organisation of the department, employees and (medical) treatments or operations. For example, there are day care centres for surgery, for radiotherapy, psychiatric treatment, dialysis or diagnostics. Therefore, we suggest that this item is only added to the questionnaire if transfer to external professionals is indeed applicable. If this item is deleted, the Cronbach’s α still will be good (0,82 rather than 0,86). Another possibility for future research is to add the answering category ‘Not applicable’, because this might be the reason a relatively high percentage of patients did not answer the question.

As patient satisfaction is seen as indicator of quality of care and satisfaction may depend on the type of hospitalisation, it is reasonable to assume there is a difference in satisfaction between different kinds of hospital care [32, 35]. Day care patients receive a different kind of care than clinical patients. The logistics and atmosphere of day care departments are different in the inpatient clinic. In general, day care patients are in better health than clinical patients. Moreover, health status influences patient satisfaction [23, 32, 36], as do individual conditions, treatments and preferences [37]. It can be expected that day care patients are more satisfied with day care facilities than clinical patients with inpatient care [30, 38, 39]. This corresponds with findings about patient satisfaction about day care admission for neurological second opinions or tertiary referrals [33]. Because the Clinical COPS, Outpatient COPS and COPS-D contain several identical questions, a comparable study can identify possible differences between the three types of care. This is an interesting topic of future research.

A number of limitations of the study design must be mentioned.

First, we could not study the characteristics of the non-responders, because of anonymity. Although our response rate is reasonable [14, 40], extremely (dis)satisfied patients may not have returned the questionnaire. However, former research showed that the impact of non-response bias on satisfaction questionnaires of hospitalized patients is relatively small [34, 41]. Also the external validity results showed that although our sample involves somewhat less children than the population of day care patients in the Netherlands, the groups are comparable regarding gender and age. Therefore, we assume that our non-response bias is limited.

Secondly, although the COPS is entirely based on the needs of clinical patients [13], day care patients were not specifically involved in constructing the COPS-D. The adaption is based on suggestions of professionals in hospitals that provide day care. We have assumed that the six general dimensions of Admission, Nursing care, Medical care, Information, Autonomy and Discharge and aftercare were also important to day care patients. We indeed found high correlations between these dimensions and the day care patients’ overall satisfaction. Still, there might be other aspects of day care which are important but not yet covered in the COPS-D. In-depth interviews and focus groups could further establish the content validity of this questionnaire for day care patients.

One of the arguments against assessing patient satisfaction is the skewed score distribution found regularly [34, 42]: most patients are satisfied with the care they receive and only very few are dissatisfied. We indeed found a ceiling effect in our data: the highest percentage of the maximum score given addresses medical care (given by 30,4% of the patients). Nevertheless, high satisfaction figures do not mean that there is no room for improvement [36]. In this study we also see differences in means between the dimensions: e.g., comparing the mean score for Admission (M = 4,22) and the mean score for Discharge (M = 3,72). Thus, there is still room for improvement.

· the reception

· the personal attention of the nurse

· the expertise of the nurse

· the information and instruction

· the reception at the day care centre

· the rapidity of being able to speak to by the staff

· the degree of support of the staff

· the reception at the Operation Room

· the personal attention of the operation staff

· the expertise of the operation staff

· the personal attention of the nurses

· the expertise of the nursing staff

· the personal attention of the doctors

· the expertise of the doctors

· the clarity of information given by nurses

· the clarity of information given by doctors

· the way information was transferred from one person to another

· the rapidity of learning research results

· the degree of encouragement to be self-sufficient

· the degree to which you could participate in treatment decisions

· the privacy you were given such as in conversations with doctors during physical examinations and during visiting times?

· the information provided about further treatment

· the transfer of information to external professionals, such as your G.P.

· the discharge procedure