Background
An Intensive Care Unit (ICU) is a place full of uncertainty and stress for both patients and for their relatives. Many researchers have suggested that relatives of ICU patients have a need for accessibility, support and information [
1‐
3]. Nurses usually respond to these needs intuitively, based on individual experience, rather than in an evidence-based way [
4]. Furthermore, relatives rarely ask for support and attention which may, in the long term, result in psychological distress [
5,
6]. Therefore, it is necessary to better tailor the quality of care to relatives. Furthermore, according to the quality standards of the Dutch Society of Intensive Care (NVIC), every ICU needs to monitor the quality of care, including the satisfaction of the relatives with respect to the care [
7,
8]. However, an evidence-based, valid, and reliable Dutch instrument that evaluates concrete experiences and perceived quality of care from the perspective of the relatives is still lacking.
To bridge the gap of this lacking instrument, we had some thoughtful considerations translating and adapting existing questionnaires. The ‘Critical Care Family Needs Inventory’ (CCFNI) is a questionnaire with 45 items to measure the needs of relatives in the ICU [
9] in a French and an English version. However,
need as a unique starting point is not sufficient to confirm simply and straightforward which interventions may have positive effects in the support of relatives [
10]. Thus, the CCFNI does not adequately assess the quality of care as perceived by relatives. Another frequently used questionnaire to evaluate the satisfaction of the ICU patients’ relatives is the ‘Family Satisfaction in the ICU survey’ (FS-ICU) [
11,
12]. Their items are based on an existing framework that measures patient satisfaction, in combination with items related to end-of-life care. The FS-ICU 24 seems a valid, reliable and feasible instrument for determining the
satisfaction of relatives in ICU. Quite apart from the fact that satisfaction of patients might not at all correlate with the satisfaction of the relatives [
13], it is preferable to measure
experiences rather than satisfaction as they give more objective and specific information for quality improvement [
14]. The utilized concept of satisfaction might raise some bottlenecks such as ceiling effects, cognitive dissonance and socially desirable answers. A discrepancy model, which describes satisfaction as a result of expectation minus the perceived experience, could overcome these problems [
15]. Because of this conceptual difference, the FS-ICU was not used to translate and adapt the items of the questionnaire. The ‘Critical Care Family Satisfaction Survey’ (CCFSS) was assessed as a reliable and valid tool to measure the satisfaction of relatives as well [
16]. Yet, both instruments, the FC-ICU 24 and the CCFSS, have a disadvantage when being implemented in the Netherlands, as they have been developed and used in a non-Dutch situation. Therefore, it is likely that some items will be rated as being more or less important by relatives in different countries or even on different continents [
17]. For example, perceptions related to decision making might have fundamental culture specific differences on overall responsibilities of the medical team or the relatives. In addition, questions in this domain seemed multi-interpretable and difficult to translate in the exact meaning of the original questionnaire. Therefore, it was desirable to develop a measurement instrument that specifically evaluates the quality of care from the perspective of relatives in ICUs in the Netherlands in a logical follow-up of all previous studies.
This paper describes the development of a valid, reliable and feasible measuring instrument in the quality of care for practical use in ICUs in the Netherlands. The development process was based on standards for determining the experiences with provided care from a client group’s perspective, according to the Consumer Quality Index (CQI) method [
18]. The CQI instruments are theoretically founded by the CAHPS® instruments and QUOTE® methodology, both based on a discrepancy model. To meet a sufficient quality of care, the expectations regarding the quality should be in accordance with the perceptions of the actual experiences according to these methodologies [
15]. This questionnaire, the CQI ‘Relatives in Intensive Care Unit’ (CQI ‘R-ICU’), has been developed in a close cooperation between the University of Applied Sciences of Arnhem and Nijmegen, the Open University of the Netherlands and three hospitals (Erasmus University Medical Centre Rotterdam and the regional medical centers Kennemer Gasthuis Haarlem and Ziekenhuis Gelderse Vallei Ede). The Medical Ethics Committee of Erasmus MC judged that the research proposal (MEC-2011-189) complied with the Dutch law on Medical Research in Humans (WMO).
The strength of the CQI questionnaire is that it addresses the conceptual and methodological problems associated with satisfaction surveys, and that relatives were directly involved in the instrument’s development. The questionnaire focuses on “reports” of facts and experiences of the quality of care rather than on subjective ratings of satisfaction [
14,
15,
19]. An important step in the development of a CQI is determining the measurable aspects of care (quality indicators), whereby many authors have adopted a structure, process and outcome indicator [
20‐
22]. The aim of this study is to develop an appropriate set of quality indicators which measures all the domains in the quality of care relating to relatives in the ICU.
Discussion
The aim of this study was the development and psychometric evaluation of a new Dutch questionnaire to measure experiences of ICU patients relatives’ with the quality of care, the CQI ‘R-ICU’. At first, the underlying aspects of the total care quality assessment for relatives of ICU patients were determined. From the focus group discussions it appeared that relatives found information on the patient’s situation of utmost importance. Furthermore, they considered the following aspects to be essential: to be involved, honest communication, way of approaching, attention, time of the health care providers, emotional support, participating in the process of caring, and being present during medical visit rounds. The participants ranked second in importance: care at first entrance, visiting hours, privacy, and waiting room.
The a-priori grouping, with the structure indicator ‘Organization’ and the process indicators ‘Communication’, ‘Care for relatives’ and ‘Participation’ were not found in the factor analyses. Alternatively, a clustering of items was found on two process factors; Communication and Participation. We need to note that these factors include the same items as preliminarily defined in the aspects of the quality indicators, although they also contain items of the structure indicator Organization (privacy, noise, waiting room, possibilities to visit). The items ‘noise’, ‘possibility to visit’, and ‘privacy’ showed a limited internal consistency (ICT < 0.3), and were consequently removed from the factor. The item ‘waiting room’ was excluded due to the number of missing values. The item ‘information about parking and parking fees’ was classified in the factor Participation, but it also had an ICT < 0.3. All these items seemed to fit with the factor Organization. When we performed a factor analysis with four factors, the factor Organization only included 2 items ‘privacy’ and ‘possibilities for visiting’ with a very low alpha (α = .49).
Following the CQI guideline, [
18] several items (digital information, having fixed person, contact and support with other professionals, waiting room, transfer from ICU, support group, after care appointment) were not included in the factor analysis. These items were either not categorized as an experience question, skewed, had a high number of missing values because of ‘skip and go’ answers. Although some items (e.g., doctors listen carefully) had too many missing values, we had to include the item as they were required questions. Most of these items were reformulated and have been included in the CQI ‘R-ICU’ 4.0 for the discriminatory phase study, which will be more appropriate to determine its underlying structure.
The short length of stay of the patient at the ICU might have been another important reason for missing values. In these cases, relatives often indicated that they were not able to answer all the questions. In current study, relatives were included based on being admitted for ≥ 24 h at the ICU. In other studies, family of patients were included only when admitted for ≥48 h at the ICU [
26] or >6 h [
30]. This variable length of stay is important for the analysis of the discriminatory phase, in order to decide whether it is necessary to design a special CQI ’ R-ICU’ for the short-stay group.
In current study, the overall score of relatives’ experiences with the quality of care at the ICU was high, which matches findings from other studies [
29‐
32]. Relatives were most satisfied about organizational and environmental aspects (e.g., no noise of staff, visiting hours, shift in medical staff, accessibility fixed person) and aspects of communication (e.g., no conflicting information, understandable) and they were dissatisfied about informational aspects (e.g., digital and written information, support groups, meals), participation aspects (e.g., care for patient, keeping diary) and supportive aspects (after care personal interview). Both in current and another study [
29], relatives wanted the physician to be more available for regular person-to-person calls. It is noteworthy that the question ‘do doctors listen carefully’ had a high rate of missing values (6.1%); relatives noted that they had not seen the doctor. A possible explanation was that these patients were only admitted for 1 day and the relatives had missed the doctor’s visit. This item will be revised with a response category ‘did not see a doctor’.
The oldest group (>65) had a significantly higher overall score and a significantly higher score regarding the communication with nurses. The relatives of the non-survivors had a significantly higher score regarding the communication with doctors. These results correspond to results from other studies [
33,
34], that recommend that younger patients and their families may need more support around end-of-life preparation and discussion of treatment preferences.
Current study showed that items related to contact with professionals for psychosocial care, aftercare and support groups, were relatively less important. In recent literature, the opposite was observed [
35‐
37]. It may be possible that relatives only recognize the impact of the stressful period and the need for psychosocial care at a later stage [
35]. Possible explanations for this may be the of measurement, the kind of relationship, severity of illness, and current situation of the patient.
Practical implications
Two strengths of the development of the CQI ‘R-ICU’ questionnaire are that it attempts to overcome the conceptual and methodological problems associated with previous satisfaction surveys and that the relatives were directly involved in the instrument’s development. The questionnaire focuses on “reports” of the quality of care rather than on highly subjective ratings of satisfaction [
15]. These individual-reported measures are essential to quality improvement programs as they will provide feedback regarding person-centeredness in daily practice to healthcare professionals and policy makers. It is therefore essential to involve the individual as an active partner in professional care and treatment [
38]. This means seeing them as valuable persons, working alongside professionals to get the best outcome. The CQI ‘R-ICU’ is a helpful instrument to learn from the relatives as partners in caring for the ICU patient and use these reports to advance quality improvement effort in the ICU. Although this instrument has been developed in the Netherlands, the method could be applied in all healthcare settings in an international perspective. It seems a general and robust measurement instrument, even more applicable if the discriminative phase has been reported. This study, already conducted in 21 hospitals nationwide, will provide further knowledge on the applicability in different settings such as cardiology, general mixed, and specific ICUs. The translated English version of the CQI ‘R-ICU’ 5.0 can be found on
http://blog.han.nl/acute-intensieve-zorg/files/2009/07/Engelse-vertaling-CQI-5.0-Naasten-op-de-IC-1.pdf.
Limitations
Although the study had a strong design, with qualitative and quantitative result to rely on, some limitations to the results could be made. First, the respondents might have provided socially desirable answers, which were more positive than actually experienced. Second, a selection bias may have occurred, only the most satisfied individuals returned the questionnaire. Third, this study was performed in six hospitals resulting in a relatively small number of respondents to evaluate the whole population of ICU patients relatives’ in the Netherlands. Therefore, the results are not generally applicable. A nationwide multicenter study, which has already been conducted, is necessary to confirm or disapprove the identified results.
Acknowledgements
We would like to thank all the relatives who participated in the study. We also thank John van Vlijmen, Ina van Goor, Leonie Claes MSc and Esther Bakker PhD for their support and effort in this research process. Finally, we would like to thank Joke Mintjes RN, PhD for her comments on the article.