Measurement properties of EQ-5D-3L and EQ-5D-5L in recording self-reported health status in older patients with substantial multimorbidity and polypharmacy

The OPERAM clinical trial is a two-arm, cluster-randomised controlled trial of a structured medication review assisted by a software-based decision support system versus usual care, funded by the European Union Horizon 2020 programme (trial identifier: NCT02986425) [14]. The trial was conducted in four centres in Belgium, Ireland, Netherlands, and Switzerland with a follow-up period of 12 months. The trial participants were 2,008 people aged 70 years or above with both multimorbidity (experiencing 3 or more chronic conditions concurrently) and polypharmacy (5 or more different regular drugs) [14]. The trial intervention was based on the so-called ‘Systematic Tool to Reduce Inappropriate Prescribing’ (STRIP) assistant, which is deployed using a clinical decision support system [27]. STRIP is a structured method for performing customised medication reviews and to detect potentially inappropriate prescribing [14], based on STOPP/START version 2 criteria for potentially inappropriate medications (STOPP) and potential prescribing omissions (START) [28]. Baseline characteristics of the OPERAM trial participants included a mean age of 80 years, 55% being male, 24% being university educated, and presenting with a mean of 11.0 comorbidities at baseline. The chronic comorbid conditions most commonly reported by OPERAM trial participants at baseline were hypertension (n = 1309; 65%), hypercholesterolemia (n = 725; 36%) and atrial fibrillation (n = 724; 36%). The OPERAM trial had broad eligibility criteria to improve representativeness for the population of interest, and external validity.

During the 6 month and 12 month participant telephone interviews, which took place between 2017 and 2019, all trial participants were asked to verbally complete questionnaires read out to them by the trial primary researcher, to elicit a range of trial outcomes (including the Barthel Index, EQ-VAS and EQ-5D-5L). The EQ-5D-5L was included in the OPERAM trial as part of a pre-planned, within-trial health economic analysis, but also to assess HRQoL clinically [14] (primarily by using the EQ-VAS). In addition, after initial trial recruitment had been completed, the 3L questionnaire was administered in the same way to a subset of participants of the OPERAM trial at the 6 month and 12 month follow-up time points. We chose these time points to collect EQ-5D data for assessment of responsiveness, as we judged that a 6 month interval was sufficient time for clinically important changes in patient’s health to occur. We did not use a longer time period, as it would have exacerbated the generation of missing data due to the substantial mortality rate in the target population. Based on the standard operating procedure for the administration of trial questionnaires, the 3L was administered at the end of the telephone interview. For patients with potential difficulty in concentrating, the 5L was the first questionnaire administered, followed in sequence by (1) the Morisky Medication Adherence Scale (MMAS-8), (2) the Barthel Index and (3) Beliefs about Medicines Questionnaire (BMQ).

In the course of the 6 month follow-up interviews, patients were consecutively added to the present study until a maximum of 75 participants was reached for each country [implying a planned maximum of 300 participants in total; 300 being comparable to the sample size of other responsiveness studies comparing 3L and 5L [16]]. In this study nested within a multinational clinical trial, we used the combined sample across all countries to ensure sufficient statistical power. However, for the responsiveness analyses, we also carried out subgroup analyses at the country level to check for potential differences in the responsiveness of the instruments between countries. Questionnaires were completed by patients or proxies on behalf of the patient, usually a family member or other responsible individual [i.e. nursing home employee (if applicable) or the patient’s GP [14]] if the patient presented with cognitive impairment or was otherwise unable to respond. However, our present analysis was restricted to participants who self-completed the EQ-5D measures at 6 and 12 months. It was considered necessary to remove participants for whom a proxy EQ-5D report was obtained, as they were shown to have a markedly different health profile compared to participants who self-completed all EQ-5Ds, reflected by them having statistically significant lower 6 month Barthel Index score (i.e. greater impairments in activities of daily living; p < 0.001). Another reason for removing proxy EQ-5D responses was that these can be divergent from self-completed EQ-5D responses [29]. The inclusion of the proxy responses might have led to a situation where observed differences between 3 and 5L could partially be driven by the proxy responses, with no sufficient possibility to distinguish this. Therefore, we regarded it as more appropriate to focus on the responses directly provided by patients. Ethical approval for the study was obtained at the four OPERAM clinical sites.

To be consistent and because no equivalent value set exists for Switzerland, we used German EQ-5D value sets for all analyses. The German time trade-off value set was used to calculate 3L scores (utilities) [30], and the German cross-walk algorithm was used to calculate 5L scores [31]. The German crosswalk algorithm maps 5L responses onto the German 3L value set to calculate 5L scores.

All psychometric analyses were restricted to participants who self-completed all items of the 3L and 5L instruments at 6 months and 12 months.

Descriptive statistics were calculated for the study sample, including participant characteristics and the distribution of participants across all of the levels and dimensions of the 3L and 5L at baseline (6 month responses) and follow-up (12 month responses) [26]. Volume and patterns of missing data for the 3L and the 5L were assessed. We also calculated correlation coefficients between the 3L and 5L index scores, between the 3L and VAS, and between the 5L and VAS. A very high correlation between the 3L and 5L index scores might indicate the instruments produce similar results and imply that they could be used interchangeably [32].

The consistency of the EQ-5D at 6 months (i.e. first measurement time point) was evaluated by cross-tabulating within-participant 3L and 5L responses. An inconsistent response was defined as a 5L response that was two or more levels away from the corresponding 3L response [6]. For example, an inconsistent response would be established for a participant who reported level 1 (no problems) using 3L but reported level 3 (some problems) or worse for the same dimension using 5L. An exception to this rule was made for the mobility item. Here, we considered responses from participants who reported with the 3L some problems in walking about, and also reported with the 5L being unable to walk about, to not be inconsistent. This is because the 3L mobility item is categorised into a person having “no problems in walking about”, “some problems in walking about” and being “confined to bed”. Patients who report being unable to walk about with the 5L, may not necessarily be confined to bed and may therefore instead logically report having “some problems in walking about” with the 3L.

The proportions of inconsistent responses for each of the dimensions were computed. For consistent responses, the redistribution properties of the 5L were also assessed in the cross-tabulation. For example, we were able to assess the redistribution of participants who reported ‘some problems’ for a 3L dimension, across the ‘some problems’, ‘moderate problems’ and ‘severe problems’ levels of the corresponding 5L dimension.

The proportion of participants who reported ‘no problems’ for each dimension of the 3L and the 5L was assessed. We also examined the proportion of participants who reported no problems for all dimensions of 3L and 5L (i.e. index scores of 1). McNemar’s test was used to test whether there were statistically significant differences in ceiling effects between the measures for each dimension [33]. A previous study of the general German population found that approximately 39% of respondents aged 70–79 years and 7.6% of respondents aged 80+ years reported ‘no problems’ for all 5 items of the EQ-5D-5L [34].

The discriminant validity of the EQ-5D-3L and 5L was assessed by computing Spearman’s rho between each of the EQ-5D items, and the Barthel Index at 6 months [33]. The Barthel Index is a measure of individual performance in activities of daily living (ADLs) widely used in the field of rehabilitation, consisting of 10 items. Barthel Index scores range from 0 (indicating ‘total’ dependency in ADLs) to 100 (indicating no dependency in ADLs) [35]. Spearman’s rho effect sizes of between 0.20 and 0.35 were considered weak, between 0.35 and 0.50 moderate and > 0.50 strong [33]. We assessed discriminant validity for the 3L and the 5L by testing the hypothesis that Spearman’s rho for the EQ-5D anxiety/depression or pain/discomfort items with the Barthel Index would be lower than for the other EQ-5D items. This is because the other EQ-5D items (mobility, self-care, usual activities) measure functioning, thereby being expected to correlate better with the Barthel Index which measures ADL-related functioning [36].

The responsiveness of the EQ-5D-3L and 5L measures to changes in the Barthel Index and the EQ-VAS over time (i.e. between 6 and 12 months) was assessed by using an anchor-based analysis [8]. The Barthel Index and EQ-VAS were also secondary outcome measures in the OPERAM trial [14], due to their perceived responsiveness in the OPERAM population. The EQ-VAS is a visual analogue scale measure of a person’s self-assessed health with status ranging from 0 to 100 [37]. The ‘anchor’ measures (Barthel Index and EQ-VAS) were each sub-divided into three categories to reflect whether (1) the participant’s score for the anchor measure improved clinically, (2) did not change in a clinically important way, or (3) clinically worsened between 6 and 12 months. The threshold for a clinically important change was determined using a literature-based minimal clinically important difference (MCID) estimate of 8 points for the EQ-VAS [37], and any change in the total score of the Barthel Index can be considered clinically important [38]. Standardised effect sizes (Cohen’s D) were calculated for changes in EQ-5D scores between 6 and 12 months. Cohen’s D effect sizes of between 0.2 and 0.5 were considered small, 0.5 and 0.8 moderate and > 0.8 large [39]. A high degree of responsiveness of the EQ-5D-3L/5L measures would be indicated through their demonstrated ability to detect change in the anchor measures, i.e. positive effect sizes (moderate or large) for the EQ-5D when there is an improvement in the anchor measure and negative effect sizes when there is a worsening in the anchor measure. In our study, both the EQ-5D-3L and EQ-5D-5L were administered in full, including their VAS parts that are introduced slightly differently. We assessed responsiveness of the EQ-5D-3L to change in the 3L-VAS measure, and responsiveness of the EQ-5D-5L to change in the 5L-VAS measure, as we observed differences between 3L-VAS responses and 5L-VAS responses elicited at the 6 month time point, in 15 out of the 224 participants in our sample (although we observed no differences between 3L-VAS responses and 5L-VAS responses at the 6 month time point in 209 out of the 224 participants, suggesting that broadly, the VAS can still be considered a common anchor measure for our analysis). The 3L-VAS and the 5L-VAS measures are for all essential purposes, identical measures.

Informativity of the EQ-5D-3L and the 5L measures were assessed at 6 months using the Shannon index (H′) and the Shannon evenness index (J′) [40]. H′ was calculated for each dimension of the 5L using the formula: H′ = − (proportion_none*log2(proportion_none) + proportion_some*log2(proportion_some) + proportion_moderate*log2(proportion_moderate) + proportion_severe*log2(proportion_severe) + proportion_extreme/unable*log2(proportion_extreme/unable), and similarly calculated for the 3L dimensions [21]. Higher H′ values indicate that responses to the dimension are more evenly spread across the different categories of the dimension, and consequently suggest greater informativity. The formula for the Shannon evenness index is: J′ = H′/H′max. The value of H’max for the 3L is log2(3) = 1.58 and for the 5L is log2(5) = 2.32. Unlike H′ values, J′ values lie on a common 0 to 1 scale allowing for direct comparison of results from the 3L with the 5L.

At the 6 months follow-up in the OPERAM study, 256 (83%) of patients reported the EQ-5D measures themselves, 45 (15%) had the EQ-5D measures reported by proxy by their next of kin, and 8 (2%) had the EQ-5D measures reported by proxy by some other individual (unspecified). Of the 256 participants, 224 participants also self-reported EQ-5D measures at 12 months, and with full completion of all 3L and 5L items at 6 and 12 months. This sample of 224 participants was used for all analyses and included participants who reported inconsistent responses. Age, gender, education level and comorbidity characteristics of the sample analysed for this study, were broadly similar to the characteristics of the overall OPERAM trial population (described in the methods section).

Summary statistics are provided in Table 1, showing that 56% of participants were male, 28% were university educated, the highest level of education was completed high school for 46% of participants, 26% of participants did not complete high school and 5% had spent some time in the 6 months before the trial started living in a nursing home. The average participant was experiencing a median of 10 coexistent chronic conditions upon entering the OPERAM trial. A small index score reduction of 0.01 (rounded) was observed between 6 and 12 months for both the 3L and 5L.

In this sample at 6 months, 41 unique health states were represented using the EQ-5D-3L, and 99 states using the EQ-5D-5L. Spearman’s rho at 6 months between the 3L and 5L index scores was 0.88 (95% CI: 0.84 to 0.90), between the 3L index scores and 3L-VAS was 0.41 (95% CI: 0.30 to 0.52), and between the 5L index score and 5L-VAS was 0.44 (95% CI: 0.32 to 0.54).

Missing data was similar between both instruments (see footnotes of Appendix Tables 7 and 8).

With both the 3L and 5L, it was observed that there was a small reduction between the 6 and 12 month time points in the rate of participants reporting "no problems" in their ability to undertake usual activities [from 73 to 68% with the 3L (Appendix Table 7), and from 64 to 61% with the 5L (Appendix Table 8)]. There were no statistically significant changes at the 5% level in responses between 6 and 12 months, for any of the 3L and 5L dimensions (Appendix Tables 7 and 8). Whilst the pattern of change as indicated by 3L and 5L between the two time points is broadly similar, there were important differences. Notably, it was observed that for mobility and anxiety/depression, the direction of change was different between 3 and 5L (positive for 3L and negative for 5L for both items; see Appendix Table 9).

We assessed presence of inconsistent responses between 3 and 5L, i.e. 5L responses that differed by ≥ 2 levels with the same person’s 3L response (highlighted in Appendix Table 10). There were 28 (3%) inconsistent responses between the 3L and 5L reported across items (7 (3%) inconsistent responses for the mobility item, 4 (2%) for the self-care item, 7 (3%) for the usual activities item, 4 (2%) for the pain/discomfort item and 4 (2%) for the anxiety/depression item). The 28 inconsistent responses were elicited from 26 participants in total.

A high presence of ceiling effect was observed for the self-care item for both instruments (84% of participants reported "no problems" for self-care with the EQ-5D-3L and 83% with the EQ-5D-5L). There was a substantially higher degree of ceiling effect with the EQ-5D-3L index score (29%) than with the EQ-5D-5L index score (22%), which was a statistically significant difference (p < 0.001) (Table 2).

For comparison, 4 participants (2%) reported a VAS score of 100 at 6 months (indicating they have the ‘best health they can imagine’). All 4 of these participants also reported full health with both the 3L and 5L at 6 months. 81 participants (36%) reported a Barthel Index score of 100 at 6 months (indicating they have no dependency in ADLs). Of these, 58 participants reported full health with the 3L, and 44 participants reported full health with the 5L at 6 months.

For discriminant validity, we assessed the correlation between the EQ-5D items and Barthel Index (Table 3). There were no statistically significant differences at the 5% level between the 3L and 5L items, in terms of how correlated they were with the Barthel Index (absence of statistically significant differences was demonstrated from all 95% confidence intervals for the 3L items overlapping with the 95% confidence intervals for the corresponding 5L items). Although the difference was not statistically significant, it is observed that the negative correlation between the mobility domain and the Barthel index was larger in magnitude for the 5L. We found that out of all items of the 3L and 5L, the pain/discomfort and anxiety/depression items had the weakest correlation with the Barthel Index.

142 participants (64%) reported no change in their total Barthel Index score between 6 and 12 months. Responsiveness analysis demonstrated both EQ-5D measures were responsive to changes in the Barthel Index from 6 to 12 months (Table 4). Evidence of responsiveness of both measures to changes in the Barthel Index, was demonstrated both in the overall sample (Table 4), as well as in each of the country-specific subgroups (Appendix Tables 11, 12, 13, 14). Furthermore, compared to each other, both 3L and 5L measures were similar in their responsiveness to changes in the Barthel Index. For both the 3L and 5L, Cohen’s D effect sizes changed from a moderate positive effect when the Barthel Index improved to a small negative effect when the Barthel Index worsened.

Both the 3L and 5L demonstrated some degree of responsiveness to changes in the VAS from 6 to 12 months (Table 5). Compared with each other, both 3L and 5L measures demonstrated similar responsiveness to changes in the VAS. There was a small positive improvement in 3L and 5L scores as the patient’s VAS scores improved. This improvement in the overall sample appeared to be driven by improvements in 3L and 5L scores in the Netherlands and Ireland (Appendix Tables 17, 18). However, there was no statistically significant change in in 3L and 5L scores for the patients whose VAS scores worsened from 6 to 12 months.

Shannon’s evenness indices indicated that the 3L and 5L were informative for mobility, usual activities and pain/discomfort dimensions, although less informative for self-care and anxiety/depression dimensions (Table 6). The EQ-5D-3L was slightly more informative with respect to self-care (EQ-5D-3L J′ = 0.48; EQ-5D-5L J′ = 0.42), and the EQ-5D-5L was substantially more informative with respect to mobility (EQ-5D-3L J′ = 0.69; EQ-5D-5L J′ = 0.86).