Consumer Quality Index Cataract Questionnaire (CQI Cataract)
The development of the Dutch CQI Cataract was based on two different families of questionnaires [
22]. Firstly, items from the QUOTE-Cataract [
5,
6] were used. The QUOTE-Cataract is a reliable, valid, and feasible instrument for assessing the quality of care from the perspective of cataract patients and is part of the QUOTE family of surveys [
5‐
11,
22,
23]. These questionnaires conceptualize patients' experiences with quality of care in two dimensions: performance and importance [
7]. Performance refers to the actual experience of patients with the quality aspects, and importance relates to the fact that people see some quality aspects as more significant than others. It reflects what people see as desired qualities in health care. The answering formats of importance items were: not important, fairly important, important, and extremely important. The original QUOTE answering formats for the performance items were: no, not really, on the whole yes, and yes. However, response options of the performance categories were adjusted to fit in with the internationally accepted four-point Likert-scale answering structure ranging from 'never' to 'always'. Some original QUOTE items could not be adjusted to fit the four-point scale, and were therefore transformed into a dichotomous variable (no/yes).
Besides the QUOTE-Cataract, the Dutch H-CAHPS measuring patients' experiences with quality of hospital care was used to generate items [
19]. This questionnaire is part of the CAHPS family [
12‐
14,
16‐
18], and was shown to be a reliable, valid and feasible instrument for assessing the quality of hospital care from Dutch patients' perspectives [
19]. Answering categories are based on a four-point Likert scale ranging from 'never' to 'always', or based on the three-point scale: 'not a problem', 'a small problem', and 'a big problem'. Items measuring quality of hospital care from the patient's perspective were selected.
Selecting items from the H-CAHPS and QUOTE-Cataract questionnaire resulted in the CQI Cataract, which consists of two questionnaires, i.e. the CQI Cataract Experience and the CQI Cataract Importance. The CQI Cataract Experience contains general items (e.g. age, education, ethnicity, and patient's health), three global ratings (of ophthalmologist, nurses and hospital), and 41 performance items referring to the actual experience of patients with the quality aspects (e.g. How often did the ophthalmologist treat you with respect). The global ratings range from 0 to 10, with a score of 10 indicating the best possible score.
The CQI Cataract Importance also comprises demographical items, and in addition consists of importance items asking how important cataract patients value the 41 quality aspects of the CQI Cataract Experience with answering categories ranging from not important to very important (e.g. My ophthalmologist treats me with respect). The outcome of the CQI Cataract is valuable, because it shows which quality aspects patients find important and how they evaluate these aspects.
Analytic approach
In this paper we focus on patients' experiences with quality of hospital care and therefore, we only used the CQI Cataract Experience and selected the 41 items measuring quality aspects of hospital care. To evaluate the validity of this questionnaire, an exploratory factor analysis was conducted and item-total correlations correcting for item overlap were calculated. When variables are measured on a dichotomous (yes/no) scale, linear factor analysis (e.g. common factor analysis) may yield biased estimates of the factor structure [
24,
25]. Therefore, we did not include 21 dichotomous items measuring quality aspects.
The 20-item exploratory factor analysis was performed with a direct oblimin rotation. This oblique rotation was preferred to an orthogonal rotation (i.e. varimax), because it takes correlations between factors into account. An oblique rotation could also result in independent factors if that provides a better fit. The number of factors was determined by Kaiser's criterion [
26]. In general, factor loadings are considered meaningful when they exceed 0.30 or 0.40 [
27]. Therefore, items were only assigned to a factor if the magnitude of their factor loading exceeded 0.40. Item-total correlations (ITC) correcting for item overlap were calculated to evaluate the construct validity [
28]. Correlations greater than 0.40 indicate good construct validity [
29]. To get insight into the multidimensionality of the instrument, inter-factor correlations were computed. Correlations of less than 0.70 indicate that the constructed factors can be seen as separate scales [
30].
Secondly, Cronbach's alpha coefficients of the different domains were calculated to evaluate the internal consistency of the questionnaire [
31]. An alpha exceeding the value of 0.70 indicates that the scale is reliable [
29]. After assigning the items to the different scales, mean sum scores were calculated by summing the responses to the items and dividing these sum scores by the number of items filled in. The higher the score on the domain, the more positive the patient's experience with quality of care.
Thirdly, to evaluate whether part of the variation in patients' evaluations of care is related to the hospital in which they were operated, multilevel analyses were performed.
Individual characteristics of patients were taken into account as case-mix adjusters to estimate the contribution of each characteristic. One of the goals of the questionnaire is to understand individual variations within hospitals. Therefore, it is important to investigate whether survey results might be influenced by factors that are not distributed randomly across hospitals, and if so, to adjust for differences in patient mix when making comparisons between hospitals. The case-mix adjusters consisted each of multiple answering categories and were recoded into dichotomous variables. The variable age consisted of eight answering categories ranging from "18–24 years" to "80 years and older", and was recoded into a variable consisting of two age groups "18–74 years" and "75 years and older". The variable education consisted of 11 categories ranging from no education to post academic education. The answering categories "no education" and "primary education" were recoded into the category "low education". All other educational levels were categorized as "high education". Self-reported health consisted of five answering categories, ranging from "excellent" to "poor". The three categories "excellent", "very good" and "good" were recoded into the category "good health" and "moderate" and "poor" were recoded into "bad health". We excluded respondents with missing values on age (N = 97), gender (N = 18), education (N = 183), and self-reported health status (N = 125).
Finally, only hospitals with a minimum of 10 patients in our dataset were included in the analyses, and therefore we had to exclude 176 respondents from 57 hospitals. In total, 599 respondents were excluded, resulting in 4,036 respondents from 57 different hospitals. The mean number of patients per hospital in our dataset was 45, with a minimum of 15 and a maximum of 141 patients. Three separate multilevel analyses were carried out on the following three domains of the CQI Cataract Experience: communication with ophthalmologist, communication with nurses, and communication about medication. Furthermore, we performed multilevel analyses on the three global ratings of ophthalmologist, nurses and hospital, because we hypothesised that these ratings may vary between hospitals.
The MLwiN software package was used [
31], which deals with data that are hierarchically structured [
32]. This means that the data are nested, i.e. ordered in such a way that one dependent variable is measured at the lowest level and exploratory variables at the same and higher levels. In our data, individual patients (level 1) are nested within hospitals (level 2). Our hypothesis is that experiences of patients measured at the first level depend partly on the hospital in which they were operated (second level). This should result in the fact that patients within the same hospital should agree more on experiences with quality of care than patients from different hospitals. The intra-class correlation (ICC) is an index of the ratio of the within-hospital variation and the between-hospital variation [
33]. Values of the ICC range between 0 and 1. An ICC of zero indicates that the variance in patients' experiences of quality of care cannot be explained by the hospital in which they were operated.
The multilevel models used in the analyses can be viewed as hierarchical systems of regression coefficients. Regression coefficients and variance components are estimated based on the observed data. We fitted two different, nested models to the data. The first model is a random-intercept model in which no explanatory variables are included (Model 1). In this model, the variance of the dependent variable is partitioned into variance that can be attributed to the individual level, and to the hospital level.
Just like in regression analysis, explanatory variables can be used in the random-intercept model to try to explain part of the variability of the dependent variable [
34]. These variables can be entered in the model as level-one (patients' characteristics) and level-two explanatory variables (hospital characteristics) and have the same interpretation as unstandardized regression coefficient in multiple regression models [
34]. In the second model, individual characteristics (age, gender, education, and self-reported health), and one hospital characteristic (type of anaesthesia) were entered into the equation (Model 2).
Three different types of anaesthesia were used in the 57 Dutch hospitals, i.e. injection, topical preoperative drops and general anaesthesia. Currently, there is no consensus as to the optimal approach to anaesthesia [
35]. However, since 2000, the use of topical preoperative drops has increased immensely [
36], because topical anaesthesia bear no risk of injection-related complications and allow for a more rapid recovery after surgery [
37], which may influence the experience of patients with the quality of care. Entering the percentage of preoperative drops used as anaesthesia in the hospital enables us to investigate whether patients are more positively about hospitals that use preoperative drops as anaesthetics compared to hospitals using other anaesthetics. Model 2 estimates how much of the variance is explained at the patient and hospital level after correcting for these individual and hospital variables and investigates whether survey results might be influenced by factors that are not distributed randomly across hospitals. Regression coefficients were estimated to get insight into the contribution of each characteristic.
Although we did not take the dichotomous variables into account in the factor analyses, they may be able to measure differences between hospitals in patients' experiences with quality of care. Of the dichotomous quality aspects, we selected one dichotomous item which was rated by patients as most important according to the CQI Cataract Importance and performed a logistic multilevel analysis. This information item asked patients if someone informed them about what to do in case of an emergency after the cataract operation. As with the previous multilevel analyses, first the random-intercept model was fitted to the data, followed by the model in which the individual and hospital characteristics were taken into account. Logistic multilevel analysis does not estimate regression coefficients, but calculates the odds ratios (OR). Furthermore, ρ is calculated, which can be interpreted as the ICC in linear multilevel analyses.