The consumer quality index (CQ-index) in an accident and emergency department: development and first evaluation

Healthcare services have shown an increasing interest in the quality of care they provide [1]. After clinical outcome evaluations, evaluations based on the patient’s perspective have become more prominent since the introduction of patient-centred care [2]. International organisations such as the Organization for Economic Cooperation and Development (OECD) and the World Health Organization (WHO), have emphasised the importance of the patient’s perspective in the evaluation of healthcare delivery. National and cross-national comparisons of patients’ experiences are important for identifying areas in need of improvement [3]. Patients’ experiences provide information on which healthcare professionals, patients, and health-insurance companies may base their decisions. Furthermore, it enables the government and the Health Care Inspectorate to monitor the quality of healthcare. Finally, the standardized measurement of patients’ experiences enables evaluations of research of research projects.

It is preferable to measure patients’ experiences rather than their satisfaction, as they have shown to be more objective and to yield more detailed information for quality improvement [4]. One theory is that satisfaction is a multi-dimensional concept, partly based on expectations and personal preferences. This complicates the objective measurement of the quality of care. When a product fails to match expectations, the quality will be judged as unsatisfactory [5].

In the Netherlands, the Consumer Quality Index (CQ-index), a standardized method for developing surveys and measuring healthcare quality from the patient’s perspective, was introduced in 2006 in order to promote patient-centred care. In order to obtain reliable and valid questionnaires, the development process has been prescribed in a manual and it is guided and controlled by a scientific advisory board. The content validity is ascertained during a qualitative phase which includes a literature search, interviews with experts, and patient focus groups. After this phase a pilot study on the CQ-index should be performed to determine internal consistency. The CQ-index is characterised by combining patients’ experiences with the relative importance of each experience item resulting in a list of priorities for improvement of quality of care. Several CQ-indices for a variety of community services, care settings and condition-specific patients’ groups have been developed, such as the rheumatoid arthritic questionnaire, the cataract questionnaire, the hip and knee questionnaire, and breast-cancer questionnaire [2, 6, 7]. Different patient groups turned out to have different priorities, which stressed the need for specific questionnaires [8]. In emergency medicine, two CQ-indices for general practitioners (office hours and out-of-hours), and a CQ-index for maternity services are available. The CQ-indices for ambulance services and dispatch centres are under development. This study completes the set of questionnaires for emergency services with the development of the CQ-index for the accident and emergency department.

In the Netherlands, general practitioners are positioned as gatekeepers, also in the case of emergency care. Emergency care by GPs is provided from local GP-practices during working hours, and out-of-hours in regional GP-cooperatives [9]. Patients need to consult their GP for referral to accident and emergency departments (A&Es) in hospitals. However, the number of self-referrals to A&Es is growing [10]. Patients transported by ambulance in need of emergency care are brought directly to the A&E [11, 12]. A&Es are often the place where patients form their first impression of a hospital and a positive experience may influence decisions about future visits and personal patient recommendations [13]. Measuring the quality of care in the A&E as experienced by patients may provide valuable information, for instance for identifying areas in need of improvement.

The goal of this study is to develop and pilot test a CQ-index for the A&E department (CQI A&E). This questionnaire aims to measure healthcare performance in the A&E as experienced by the patient.

The aim of this study has been to construct and test a questionnaire to measure patients’ experiences with the accident and emergency department, according to the guidelines of the Consumer Quality index [15]. This first version of the Consumer Quality index for the accident and emergency department (CQI A&E) consisted of 84 questions. 52 questions were phrased as ‘experience questions’. The response rate (47%) of the pilot test was comparable to other postal surveys involving A&E patients [14, 19]. However, disease-specific CQ-indices showed better response rates (68%–84%) [2, 6, 7, 20, 21]. To determine the construct of the questionnaire, two exploratory factor analyses were performed. The first analysis was performed including thirteen items. Three domains were constituted (‘attitude of healthcare professionals’, ‘environment and impression of the A&E’, ‘respect for and explanation to the patient’). A second analysis comprising all 52 experience questions was performed with the aim of including more questions on experiences deemed important by patients. Five domains ‘attitude of healthcare professionals’, ‘information and explanation’, ‘environment of the A&E’, ‘leaving the A&E’, and ‘general information and rapidity of care’ were constructed, covering 24 items. Two out of three domains in the first EFA were internally consistent, whereas all domains in the second EFA were internally consistent. Internal consistency of domains increases by increasing the number of respondents. Despite of lower response numbers in the domains of the second EFA, the internal consistency exceeded the internal consistency of the first EFA. The percentage of explained variance of the five domains decreased five percent compared to the explained variance of the three domains of the first analysis.

Two main goals for CQI-data can be distinguished. The first goal is to compare quality of care between healthcare providers. A strict (following the CQI-guidelines), statistically correct EFA was performed in order to generate the information needed for making a valid comparison between A&Es. Researchers, the Health Care Inspectorate, health-insurance companies, hospital boards and the Ministry of Health are the intended users of these outcomes. However, the main customer in healthcare is the patient. Therefore, the second goal is to acquire the information needed for quality improvement within a healthcare institution. To this end, an alternative EFA was performed. To include more content of the questionnaire in the domains, more questions were added. We think this information may help A&E managers and others to start evaluating quality improvement projects. Both goals represent a different way of constructing domains in questionnaires. Following all criteria in the CQI guidelines, domains are constructed using the perspective of a reflective measurement model [22, 23]. Only items with a 4-point Likert scale were included. Items that did not fit into any domain, and domains that did not fulfil the statistical criteria, were omitted [24]. The qualitative phase was carried out in order to detect all aspects related to healthcare performance in the A&E. Each aspect is a unique part of the provided care and together they form the construct ‘quality of care’. To end up with a few statistically related items neglects the broad range of the aspects. Therefore, a formative measurement model may be better suited to construct domains in experience questionnaires. The latter theory concerns the construction of domains based on content and not solely on strict statistical criteria. In the second analysis, we did not try to construct domains solely from a formative perspective, but we tried to include as many experience questions as possible, while still achieving internally consistent and interpretable domains. We included all domains that came up in the second EFA, thereby doubling the content of the questionnaire included in internally consistent domains. Although we only relaxed one of the criteria of CQI-guidelines, we think that these domains are better suited for quality improvement and that they are also suited for benchmarking. The improvement scores provide concrete tailor-made information, which can be helpful for management and staff.

In accordance with most CQ-indices, the domains on communication, information, attitude of healthcare providers (often within the communication domain) and the environment of the health service, are part of the CQI A&E. Domains regarding accessibility and leaving the organisation are also often found [6, 7, 20, 25]. There are a lot of similarities to other CQI instruments, whereas the similarities with satisfaction questionnaires are few. The A&E satisfaction questionnaire Quality Patient Perspective discussed patient participation [19]. The Swedish A&E Patient Satisfaction Survey revealed 3 factors: caring, teaching and clinical competence [26].

The most important aspects in healthcare performance in the A&E from the patient’s perspective dealt with competence of professionals, hygiene and expectations. It has to be determined whether these importance scores are valid across study populations.

In an English importance study, with 16 participants who had visited the accident and emergency department the most important aspect was confidence and trust in the doctors and nurses. Secondly, ‘being treated with respect and dignity’, and thirdly ‘explanation about condition and treatment’ were important items. Interestingly, waiting time did not feature in the top 20 of most important aspects in our study. As regards the top three of least important aspects, our study concorded with the English study on aspects such as refreshments in the waiting room and not being asked details about the patient’s condition or illness too often. However, in the English study the number of respondents was limited, and inclusion criteria broader.

The study has same limitations. Firstly, we used the pilot study dataset of respondents of only one hospital. In the next phase of the development, the stability of the domain structure will be assessed in a dataset of 20 A&Es, and therefore the presented domains are preliminary outcomes. The discriminative capacity of the CQI A&E will be assessed in that phase as well. Secondly, in this study a consecutive sample was used for the psychometric analysis. All patients who had visited the A&E within an average week were included. The gender and age profile of the respondents was representative for the A&E population of the research hospital. Respondents and non-respondents were comparable as regards age, gender, day or time of attendance and symptoms. Therefore, it is unlikely that the low response rate has affected the outcomes. A significant difference between both groups was found for the triage code. The least severely injured patients (blue triage code) were underrepresented within the respondents group. These patients are often discharged without experiencing all aspects of healthcare performance and perhaps did not think of themselves as a ‘true’ A&E patient. This might have introduced selection bias among respondents.

The study protocol had advantages such as sending the questionnaires to the patient’s home, instead of administering them in person in the A&E, which prevents selection bias caused by healthcare professionals. Also, all patients received the questionnaire within one week after their A&E visit, which limits the recall bias. However, patients’ symptoms might have evoked recall bias.

The Consumer Quality index for the accident and emergency department measures patients’ experiences of A&E healthcare performance. Preliminary psychometric characteristics of the CQI A&E are good, but further research on reliability and validity is needed. Depending on the viewpoint, exploratory factor analysis results in two or five internally consistent domains. The 5-domain structure seems preferable, as this includes more content of the questionnaire while maintaining internal consistency. Furthermore, the improvement scores of each item provide information that makes it possible to identify aspects that require consideration in order to increase quality of care. The preliminary outcomes and the discriminative capacity have to be confirmed in future research by means of the CQI A&E.