International comparison of experience-based health state values at the population level

Health state values are used to weigh the different dimensions of a particular health state, such as physical functioning and mental health.¹ By rendering the overall aggregation of health they make up a key methodological step in the evaluation of health effects. The concepts and methods used to generate health state values have been studied and discussed continuously, mainly considering the following questions: which health dimensions should be valued, how to elicit these values, and whose values should be used [1‐3]? It has been shown that such conceptual and methodological choices affect outcomes at the population level as well as for patient groups defined by diseases [4‐7]. Consequently, it is important for health care decision makers to be aware of the choices underlying the value sets they base their decisions on. A recent comparative study on diabetes patients, for example, concluded that “the choice of tariff might have substantial impact on funding decisions” [8].

Health state values have been elicited using choice-based experiments in which respondents are asked to make trade-offs between living in a less than perfect health state and living in full health.² However, concerns have been raised regarding this approach. It requires respondents to assess hypothetical health states (HHS) and, in that case, healthy respondents may focus on the health problem they are asked to imagine, overlooking other health domains and underestimating adaptation. At the same, patients may have adapted to their health problems and be unable to predict their valuation of (or recall how they valued) being in full health [3, 9]. It has been argued that a different approach is needed which reflects the degree to which health states actually affect people, instead of respondents’ choices or ex-ante preferences regarding different hypothetical health states [3]. In recent years, several population-based studies used such an experience-based approach to elicit health state values for Germany, the U.S.A., Sweden and China [10‐13]. The experience-based approach involves a generic rating by individuals on how they feel at a particular moment, complemented with concurrent descriptive information about their health status.³ Leidl et al. were the first to estimate a value set based on experienced health states (EHS) in Germany, relating respondents’ EQ-VAS rating of their own health (on a 0,100 scale) to their health status as described by the health dimensions of the EQ-5D-3 L [10]. The results indicated that such EHS-based valuations can differ from valuations of HHS.

To the best of our knowledge, EHS-based value sets were derived on a national basis only. A concern at this stage is the still limited number of countries for which EHS-based value sets are available. As a consequence, researchers in other countries could decide to use these (foreign) EHS-based value sets in their national-level analyses, similar to several studies that were based on HHS-based value sets [14‐16]. However, as valuations may differ between countries, so may economic evaluations and population health assessments based upon them. Therefore, the transferability of these studies and their usefulness for national-level policy making depends on the cross-country comparability of health preferences. It can be argued that value sets should represent national preferences since reimbursement decisions mostly use a national perspective. More generally, health systems may be expected to produce health outcomes in accordance with the preferences of the population they serve and whose means are put in use.

From a theoretical point of view, it may be expected that health state valuations differ between countries [17‐19]. Economic and geographical circumstances and social support systems vary between countries, affecting the way people perceive and value health limitations. In addition, the valuation of health states may be influenced by culturally or religiously defined preferences related to health. At least, this was found in some empirical studies on HHS-based value sets [15, 20‐27]. In general, these studies concluded that cross-country variation in health state values cannot be ignored, even though the magnitude of the differences varied between studies and valuation methods. For example, Badia et al. found statistically significant differences between Spanish and UK respondents for 35% of the health states valued [20]. Spanish respondents placed significantly greater value on the functional dimensions mobility and self-care and lower value on pain and anxiety, compared to British respondents. Similarly, Norman et al. showed that mobility problems were considered more important among Japanese respondents compared to respondents from the UK, whereas opposite results were found for pain and anxiety [25]. An important limitation of cross-country comparisons of HHS-based value sets is the methodological variation between studies regarding e.g. the number and choice of health states valued by respondents and the algorithm used to establish the value set. For example, it was shown that methodological variation in the transformation of negative values explained a substantial part of the difference between the UK and US versions of HHS-based value sets for the EQ-5D-3 L value sets [28].

In this study, we aimed to expand the evidence on differences in health state values between countries. The study is the first to compare EHS-based valuations across countries. We analyse data from EQ-5D-3 L population surveys conducted in fifteen countries between 1993 and 2002. Similar to previous national studies [10, 29], we investigate the relationship between respondents’ EQ-VAS rating of their own health (outcome variable) and their descriptive health profile using the EQ-5D-3 L. The current study provides better generalizability of the results in comparison to previous international comparisons of health state values, by including 15 countries. Furthermore, using EHS-based valuations reduces methodological variation in terms of transformation and estimation procedures, compared to previous comparisons of HHS-based value sets. We focus on two research questions: (1) Does the mean observed EQ-VAS per health state (i.e., a combination of health dimensions) differ between countries? (2) Does the estimated value of particular health dimensions vary across populations, both in terms of the size of their impact and the ranking of dimensions? Results of the first part are expected to show the extent to which EHS-based valuations of particular health states vary across countries. Results of the second part are expected to demonstrate to what extent individual health dimensions are valued differently.

In this study, we investigated EHS-based valuations from a population perspective, using pooled data from 15 countries and standardized, novel methods. The study is the first to compare valuations based on experienced health states across countries. Previous international studies focused on cross-country differences in HHS-based value sets (see e.g. [7, 21]). We first studied the mean EQ-VAS ratings associated with particular health states and found that these varied between countries. Differences were most evident for health states with fewer problems and for countries at the low-end and high-end on the EQ-VAS scale. Next, we studied the impact of the individual health dimensions and found that different populations appear to rank the dimensions and problem levels of the EQ-5D-3 L in different ways. For example, the impact of having some or extreme problems in the pain dimension was relatively high (compared to other dimensions) for Armenian and Swedish respondents, while it was low for respondents from New Zealand. In most countries, having some or extreme problems in the usual activities dimension had the largest impact, which shows that similarities between countries were present too. The magnitude of the health dimensions’ coefficients also varied between countries, which will translate into non-negligible differences in terms of health outcomes. As illustrated in the results section, the variation in coefficients may very well generate a 7-point difference on the EQ-VAS 0–100 scale, which was considered a minimally important difference from a clinical perspective in one study [33]. Estimations for all possible combinations of health dimensions also illustrated that systematic differences in terms of valuation may exist (Fig. 2). At the same time, even though individual health dimensions may be valued differently across countries, we did find a positive correlation between the valuation of mobility, self-care and usual activities. No correlation was found between the valuation of pain or anxiety and all other dimensions (a previous study found a similar pattern for Spanish respondents, see [20]). This indicates that the pain/discomfort and anxiety/depression dimensions represent different types of health problems compared to mobility, self-care and usual activities, which are valued differently as a result. At the level of extreme problems, differences between countries were less clear and more often not significant.

In spite of our standardized approach, we cannot exclude that methodological differences caused some of the variation between countries. The underlying population-surveys varied regarding the year in which the survey was conducted, the interview mode (postal or face-to-face interview), and the sampling procedure. In the regression models, interview mode was found to affect EQ-VAS ratings yet it did not change cross-country differences substantially, in particular regarding the relative importance of the different dimensions (see Additional file 2). Differences in study year may have affected the results, in case the VAS valuation of health changes over time due to e.g. changes in the health or social care system that affect valuation. To our best knowledge, there is no evidence on this issue regarding the samples used in this study, however. Differences in sampling procedures are described in Additional file 1. Not all studies reached the aim of including a representative sample of the underlying population, but differences in the distribution by age and gender across studies were taken into account in the regression model. After adjusting for differences in the distribution of these respondent characteristics and the interview mode, cross-country differences remained similar. Therefore, we argue that differences in health values between countries cannot be ignored. Interestingly, these differences may not necessarily reflect differences in the economic position of countries. Both wealthier and less-wealthier countries were found to value, for example, the reduction by some or extreme problems in mobility much heavier than other countries in the sample.

The existence of cross-country differences in EHS-based valuations resembles the results from previous studies on HHS-based value sets, yet our findings also differ in some respects. Remarkably, some or extreme problems with usual activities was associated with a large reduction of the EQ-VAS in all countries, whereas this dimension was much less important in most HHS-based value sets [24]. This may confirm the finding from Leidl et al.’s national studies that the two approaches may generate value sets with different characteristics at the population level. As argued in the introduction, EHS-based value sets can be used as an alternative for the traditional HHS-based value set. The EHS-based value set developed for Germany has, for example, been used to test the validity of the EQ-5D-3 L in specific patient groups [6, 34‐37]. Recently, EHS-based value sets have also been used in clinical studies in Sweden, in line with the recommendation of the Swedish guidelines on economic evaluation to use the patient’s perspective [8, 38, 39]. In case the approach will be applied in an international setting, it becomes important to take cross-country differences in health values into account. For example, multinational clinical trials planning to use EHS-based values may not rely on a single value set from one country but should regard the need to adapt values to decision-specific contexts by using a respective value set, and to control for eventual sensitivity of results when basing evaluation on this country-specific valuation. In addition, our study confirms that researchers should be cautious when transferring health outcomes, based on EHS-based value sets, from foreign studies into national calculations. This may lead to invalid conclusions for their target population. Results also confirm that a simple adjustment formula does not seem to exist, because respondents in one country did not attach greater or smaller value to all dimensions. Rather, this pattern fluctuated between the different health dimensions and levels. Eventually, the findings warn decision makers both against using original EQ-VAS valuations alone without considering eventual adaptation to country context, as well as against the unreflected transfer of results derived from value sets of other countries.

The results must be interpreted with the following limitations in mind. The main methodological issues related to the EQ-VAS instrument are context bias, end-of-scale bias and response spreading [2, 40]. The use of the EQ-VAS to establish health state values has been criticised because of a perceived lack of theoretical foundation, yet Parkin and Devlin showed that it does have a theoretical foundation in (psychometric) measurement theory [40]. Context bias means that the value of a particular health state depends on which health state it is compared with. This relates to experiments in which respondents value multiple health states (HHS-based value set), yet in this study we used the single EQ-VAS rating associated with the current health state of the respondents. Dolan and Kahneman argued that the usefulness of EQ-VAS-type ratings also depends on any other comparisons respondents make at time of the assessment, e.g. between themselves and other people [9]. Based on our data we however could not assess whether this led to systematic cross-country variation and should be considered a measurement distortion. By focusing on respondents’ valuation of their current health state, this study could not consider the anchor point death as in HHS-based valuation studies. In HHS-based approaches, the value of death is commonly defined as zero and used as anchor point to adjust for differential response behaviour. There is also methodological discussion on how to further develop such anchoring in HHS-based valuation [41]. Previous population level studies with and without anchoring yet indicated that the difference may be limited [42]. For the examples of patients with heart disease and with inflammatory bowel disease, it has further been found that non-anchored VAS-based value sets correlate better or the same with accepted clinical measures as anchored, utility-based value sets [35, 37]. When calculating quality-adjusted survival in the experience-based approach, death is also zero because of zero survival time. Not attributing a value to death in the experience-based approach implies that negative valuations for health states do not exist, in contrast to traditional QALY calculations. Another point, end-of-scale bias refers to respondents avoiding the end-points of the EQ-VAS-scale. The latter may have affected our cross-country comparisons (the regression coefficients) if respondents in country A were more inclined to avoid end-points compared to respondents in country B. Although a substantial proportion of the respondents did report a EQ-VAS rating of (around) 100, the issue could not be tested with the data at hand.

In addition to these EQ-VAS-related issues, we should note that we did not include interaction terms between the different EQ-5D-3 L dimensions and levels. This would allow the effect of e.g. mobility to vary by different levels of self-care. However, previous studies on health valuation showed mixed results regarding model fit improvement after the inclusion of such interaction terms [24]. Moreover, adding multiplicative terms increases data requirements and makes interpretation of the model results much more complex. Another limitation was the limited sample size of some of the surveys. More importantly, the number of respondents with extreme problems in any or several dimensions was limited (the surveys did not include institutionalized persons and certain health problems may have hindered more severely ill people from participation). Therefore, there were relatively little EHS-based values for these dimensions, which reduced the precision of the estimates. In addition, it was unclear whether all types of respondents, according to the characteristics that may affect health valuation, were represented in the surveys. There is little evidence on the impact of respondent characteristics on health valuation though and we tested the impact of differences in distributions by age and sex. Finally, we applied a standardized methodology to an international data set provided by the EuroQol Group. For Sweden and Germany EHS-based value sets have been developed elsewhere using significantly larger and more recent samples [10, 11]. This should be considered before using respective data from this study. We cannot exclude that for some of the countries analyzed specification tests would suggest a different model-specification to be better. However, the main goal of this study was to compare the valuation of dimensions across countries. For reasons of comparability, we preferred using an identical regression framework for all samples.

In sum, we explored international differences in EHS-based valuations. Decision makers, who want to focus on patient or individuals’ valuations when considering the effectiveness of medical interventions, may find EHS-based valuations a useful alternative to HHS-based valuations. When determining the effectiveness of a medical intervention, patients and physicians typically consider what has actually happened rather than preferences regarding what could occur. As one important consequence, the use of such EHS-based valuations has the potential to better integrate the assessment of medical effectiveness and cost-effectiveness. Overall, the results indicate that EHS-based valuations differ between countries, at least for the countries included in this study and for the dimensions and levels covered by the EQ-5D-3 L. This may depend upon different valuations of health problems, though systematic patterns explaining these differences between countries, or identification of clusters of countries similar in valuation remain to be recognized. Since health state values are an important input parameter in population health comparisons and evaluations of health interventions, these findings should be taken into account when nationally interpreting valuations from other countries, but also in international comparative studies. We recommend future research to focus on the causes of differences in health valuation through a single international study (at one point in time) possibly complemented with qualitative research on the determinants of health values.