Obtaining preference scores for an abbreviated self-completion version of the Teen-Addiction Severity Index (ASC T-ASI) to value therapy outcomes of systemic family interventions: a discrete choice experiment

Economic evaluations in health care often take the form of cost–utility analysis, in which outcomes are captured in terms of quality-adjusted life-years (QALYs) [1, 2], as measured with generic health-related quality of life (HRQOL) instruments like the EQ-5D [3] or SF-6D [4]. This implicitly reflects that many curative health care interventions primarily aim to improve health and longevity of patients. However, in certain health care sectors, the aim of interventions may not (solely or primarily) be to improve health, but to (also) improve broader aspects of quality of life that go beyond health. These broader outcomes may be captured insufficiently by existing generic HRQOL instruments used to calculate QALYs [5, 6]. This issue is gaining attention, for example, in the area of elderly care, where broader measures like ICECAP-O, WOOP, and ASCOT have been developed [7‐9]. These measures capture broader life domains than health and are suitable for use in economic evaluations. In other areas, including mental health care and addiction-related treatments, broader preference-based outcome measures are also required but largely lacking [9‐14]. The issue of appropriate and comprehensive outcome measures, preference-based, and suitable for use in economic evaluations, is highly relevant in the context of mental health interventions, and particularly for systemic family interventions. These interventions are intended to have broad effects (e.g., related to substance use, family interactions, interaction with peers, and performance at school), that extend beyond the health domain. If not appropriately identified, measured, and valued, such broader effects may fall outside the scope of economic evaluations, risking mis-estimation of the benefits of systemic family interventions. Consequently, the results of economic evaluations may not reflect the actual value for money offered by these interventions and, potentially, result in non-optimal decisions concerning their reimbursement [10, 15, 16]. The results of this study aim to contribute to reducing this risk.

The relevance of this issue is emphasized by the fact that systemic family interventions for adolescents with problems of substance use and/or delinquency are increasingly subject of economic evaluations [17]. However, existing studies are limited in quality and comparability as settings, design, and outcome measures vary extensively [17]. The application of economic evaluations in the field of systemic family interventions is hampered by the lack of preference-based instruments that are validated, sensitive, and feasible to use and that capture all relevant benefits. Systemic family interventions are explicitly directed at improving interactions between the adolescent patient and surrounding systems, and are often used in the context of substance abuse and delinquency [18‐20]. Aims of such interventions are diverse and include improvements in family relations, peer interactions, performance at work or school, and reduction of substance use and delinquent activity [19, 21‐23]. In a meta-analysis evaluating the effectiveness of outpatient substance abuse treatments for adolescents, systemic family interventions were found to be effective in the treatment of substance abuse [24]. Given that these interventions typically are intensive and costly [15, 21, 26], economic evaluations are important, also to inform reimbursement or funding decisions. This requires validated, broad, multidimensional preference-based instruments that capture the relevant effects of such interventions.

A recent systematic review of the effectiveness literature on systemic family interventions identified existing instruments, which measure relevant benefits beyond health-related quality of life [14]. While no preference-based instruments were found, the Teen-Addiction Severity Index (T-ASI) [27] was identified as a multidimensional instrument capturing the main relevant life domains of adolescents affected by these interventions. Although preference scores for this instrument were lacking, it was considered potentially suitable for adaptation into a preference-based measure for use in economic evaluations of systemic family interventions alongside the use of common HRQOL instruments [14].

The original T-ASI is a relatively long semi-structured interview that measures symptoms of adolescent substance use based on seven domains and five levels of problem severity. The instrument is not a self-report instrument but completed by a therapist together with the patient. Some questions are directed at the patient while others ask the therapist to provide his or her judgment. In order to make the instrument suitable for use in economic evaluations, in which patients commonly report on their own functioning using a self-complete descriptive system, an abbreviated self-completion version of the T-ASI, the ASC T-ASI was created [16; see appendix A and B]. This abbreviated instrument was based on the main patient-reported questions from all domains of the T-ASI, reflecting the functioning of the patient as judged by him or herself. The ASC T-ASI is a broad outcome measure, suitable for self-completion and use in economic evaluations. The ASC T-ASI was subsequently validated, with favorable results [16]. However, since societal preference scores for the ASC T-ASI are lacking, this study set out to obtain such scores, using a discrete choice experiment (DCE).

The main study comprised 1500 respondents (after the exclusion of respondents who answered at least one of the control questions inconsistently and ‘speeders’). General respondent characteristics are displayed in Table 1.

Respondents’ distribution of age and sex was in line with the general adult population in the Netherlands. Mean age was 42 years (general Dutch population: 42 years) and the proportion of male respondents was 50.3% (general Dutch population: 50.2%) [37]. Respondents’ distribution of education level was less in line with the general adult population in the Netherlands, as respondents more frequently had a middle or higher education level. Completion time ranged from 4 min to nearly 24 min, with a mean completion time of 8 min and 13 s. This shorter completing time as compared to the pilot can be explained by the reduction in the number of choice sets and the addition of color-coding. A large proportion of the respondents stated not to have paid work (43.1%), 29.3% worked 36 or more hours a week and 27.6% worked part time with an average of 19.16 h/week (SD 9.063). Slightly more than half of respondents (50.8%) had children, of which 18.5% were between 12 and 21 years (an age group similar to the population that the discrete choice task referred to). Subjective health and health-related quality of life based on the EQ-5D and its Dutch tariffs were comparable to the values of the general Dutch population [38].

In this study, we obtained societal preference scores for the ASC T-ASI, contributing to the availability of a first, short, preference-based measure suitable for self-completion, and use in economic evaluations of systemic family interventions. Our primary aim was to obtain broad societal preference scores for the ASC T-ASI. We adopted a societal perspective, also given the broad scope of outcomes. The scope of this measure is deemed to be more in line with the goals of systemic interventions than currently available generic health-related quality of life measures, and hence enables a more comprehensive measurement of the effects of such interventions. The need for such measures in the context of substance abuse treatment was noted before [39]. The ASC T-ASI is an adaptation of the frequently used T-ASI [27], which may contribute to its acceptance, validity, and feasibility of implementation [16, 40]. We used a two stage-design, starting with an elaborate pilot study, followed by a large main study. Advantages of this approach were that adjustments to the design could be made in between the pilot and main study, enhancing the quality of the data obtained. We allowed interdependency of observations and heterogeneity in preferences in our analyses and the selected model fitted the panel data of the choice tasks and accounted for individual differences in choice behavior. The performed DCE yielded societal preference scores that showed logical orderings, and the different levels within each domain almost all were statistically significantly different from each other. The results indicated that the domains substance use, mental health, justice, and family were most important in our sample, representative of the Dutch population (aged 18–65 years) in terms of age, sex, and education. With these tariffs, the ASC T-ASI can be seen as a validated [16, 40], preference-based outcome measure with a scoring system ranging from 0 (worst state described with the instrument) and 1 (best state described with the instrument).

Before addressing some implications of this study, and discussing the use of the here derived preference scores, some limitations of this study need noting. First, one may argue that some of the included domains in the ASC T-ASI may not be relevant for all adolescents. For example, the domain ‘work’ may only be relevant for relatively old adolescents who work or would want to work [16]. Future research may investigate this issue further, for instance, by considering conditional questions or changes in the labeling of the domains or levels. Second, two parameters presented in Table 2 (family—‘large problems’ and work—‘large problems’) were not statistically significant at the 5% level. Note that the impact of the non-significant coefficients on the tariff was small (with values of 0.019 and 0.001, respectively). Merging the levels reduced the model fit. Hence, we chose to keep the separate levels. Third, implausible domain combinations and interactions between preferences for the attributes and levels of the ASC T-ASI were not explicitly accounted for in the study design [41]. In this first study deriving preference scores, we first focused on estimating main effects in order to allow establishing an ASC T-ASI tariff set for use in economic evaluations, in line with other tariffs (e.g., for EQ-5D and SF-6D instruments) which are usually additive. Furthermore, evidence on any interactions between preferences for ASC T-ASI attributes and levels was lacking, and accounting for all possible interaction effects—in addition to the main effects—would have resulted in a highly complex design. This, in turn, would potentially have resulted in increased cognitive burden for respondents or have required data collection in even larger pilot and main samples to maintain power, which was not feasible in this study. Interaction effects are important to explore in future research though. Fourth, the choice tasks were complex for respondents. After the pilot study we, therefore, decreased the number of choice tasks from ten to eight per respondent and applied color-coding to simplify the decision process and reduce cognitive demands to respondents. Moreover, respondents who answered the control questions inconsistently and ‘speeders’ were excluded from the analyses. Nonetheless, in the main study, a majority of the included respondents still considered the choice task to be (very) difficult. In total, 44.1% (n = 661) of respondents considered making a choice between the different states to be difficult, which may reflect the inherently difficult nature of the presented choices—also in relation to the Likert scale on which the attribute levels were presented in the choice tasks. Although we accounted for the uneven spacing between the levels (by means of dummy coding) in the analyses, we do not know to what extent respondents took this into account when making their choices and how this may have influenced our results [42]. The study design did not include an opt-out option [43], which may also have increased the difficulty and influenced our results. Fifth, potentially related to the previous point, we excluded a substantial number of respondents who answered one or both of the two control questions inconsistently. This was a strict rule, imposed in order to achieve the highest possible quality of data for the tariff set. Respondents who were excluded due to answering one control question inconsistently (n = 717) were significantly older (44.91 vs. 42.00 years; p = 0.000) and lower educated (p = 0.000) than included respondents. No difference in sex was observed. Sixth, duration of the ASC T-ASI states was not included as an attribute in the DCE, nor were additional time trade-off tasks used to anchor the tariff set on a ‘natural 0’. The latter means, like in other outcome measures like, e.g., the ICECAP [5, 6], the zero in in the obtained ASC T-ASI tariff does not equal the state of ‘dead’ but to the worst state defined by the scale. This also implies a different interpretation of changes on the instrument than in case of conventional QALY measures, as elaborated on below. The former, i.e., not specifying duration, implies that we could not observe discounting or duration effects in our study. Seventh, the current study was limited to the Dutch setting. Moreover, our sample was representative for the Dutch general population in terms of age and sex, but less so for education level. Furthermore, among our respondents, there was a high percentage of individuals without paid work. Respondents’ socio-demographic characteristics may have influenced our results. Future research may consider assessing the direction and size of this potential impact. Eighth, we observed quite some preference heterogeneity in the DCE. While the aim of the current study was to obtain overall preference scores rather than to differentiate between the scores of specific groups of respondents, it may be worth exploring this further in future studies.

Next to these limitations, the meaning and the interpretation of the here derived scores are distinct from those related to common HRQOL instruments for several reasons. First, conceptually, QALYs intend to measure health-related quality of life, whereas the ASC T-ASI aims to measure broader, and arguably less well-defined, ‘addiction-related quality of life’ in adolescents. This means it intends to capture a different concept and hence cannot be readily compared to or combined with QALY measures. Second, as already mentioned above, for generic health-related quality of life measures, like the EQ-5D, a preference-score or utility of 0 corresponds to the state ‘dead’ and hence represents a ‘natural zero’. This is not the case for the ASC T-ASI, where a score of 0 simply refers to the most severe problems in all domains of the instrument. This state could, in the more general sense of the word utility, still be associated with positive or negative utility. Combining ASC T-ASI scores with duration therefore requires a careful consideration and interpretation. This is similar to other recently developed broader outcome measures, like the ICECAP instruments [5, 6]. Third, QALY tariffs typically represent average valuations of health states obtained by asking respondents to imagine being in these health states themselves. Here, we asked adults to value states from a societal perspective, i.e., for an adolescent, which leads to fundamental differences. The observed scores reflect what people in the general public think is ‘best’ for the adolescent involved, rather than an indication of a preference to be in a certain state oneself. Hence, even the word ‘preference’ should be interpreted and understood in that context. This represents a crucial difference with other tariffs and common ‘utility scores’, which needs strong emphasis. It also emphasizes that the scores obtained here cannot be straightforward compared to, let alone aggregated with, QALYs. The approach adopted in this study shows similarities with valuation approaches in the context of child health (e.g., the valuation protocol of EQ-5D-Y-3L [34]). We requested (adult) respondents to select the better state ‘for the adolescent’, resulting in preferences that may be seen as somewhat ‘paternalistic’. Such preferences may be preferred over those of adolescents themselves (especially those experiencing these states), if one believes these preferences may be 'distorted' by underlying issues, such as alcohol and drug problems, or ill-informed or myopic due the age of adolescents. Moreover, such preferences could be influenced by mechanisms of coping and adaptation [44]. Nonetheless, future research on how the outcomes and the response patterns of valuation would differ when taking alternative perspectives or using alternative sources of valuation remains warranted, like recently done for child health [45]. Finally, we would also like to note that the respondents’ preferences may also be influenced by other elements not included in the ASC-T-ASI, such as the adolescent’s age and family circumstances. It is unclear whether respondents missed such information, whether they assumed a specific context, and whether the influence of such items would be significant. Future research could explore the potential influence of any systematic preferences that may be associated with ASC T-ASI states (beyond those directly related to its domains and levels) on the tariff set, and the potential implications this may have for policy.

Given the above, it is good to highlight that we intentionally opted for this valuation approach in the current context for several reasons. We set out to obtain societal preferences from the general population, in line with Dutch guidelines for economic evaluations in health care [1]. Given that the ASC T-ASI relates to adolescents, this implied, for almost all respondents, valuing not only hypothetical states but also for a person with different age and context than those of the respondents. This required additional instructions, as we did not want to obtain general preferences for ‘states of substance abuse’ but specifically in relation to the phase of adolescence. We framed the choices in terms of the best state for the adolescent, which may lead to somewhat ‘paternalistic’ (rather than more hedonistic) choices. This was done to stay close to the purpose of many interventions in this area. All these choices are inherently normative, and it is interesting to further investigate them and their influence on preferences in future studies. For instance, an alternative would have been to use preferences of adolescents actually being in these states. Besides practical issues of recruiting these adolescents, also normative issues regarding whether their preferences (including, for example, those related to substance abuse or school performance) would be most useful for societal decision-making. One could argue such preferences of adolescents actually experiencing these states could be influenced by coping and adaptation [45], but also by underlying problems like addiction and myopia (also due to the age of respondents). Other sources, like adolescents not experiencing these states and issues, or the general public like used in this study, all come with own limitations.

Future research could explore the important normative issue of ‘whose values count’ [44] in situations like these, but could also compare preferences of affected adolescents, adolescents without the specific problems described with the instrument, and those of the general public. Using preferences from non-affected adolescents may yield preferences that are more representative of those of the treated group. Moreover, arguably, such respondents might be more capable of imagining (what it means) being in the different states described with the ASC T-ASI than adults in the general population. However, whether their preferences would be (more) appropriate to use in societal decision-making remains unclear.

Future research may also consider the framing of the choice task. We chose the framing of asking which situation was ‘best for the adolescent’, reflecting potential treatment goals of the health system, which can be different from what the adolescent would prefer. The approach taken can, therefore, be viewed as being aligned with societal decision-making and collective financing of interventions, but this may come at the expense of not using (current or future) preferences of the treated adolescents.

In combination, these differences mark a fundamental distinction between the instrument presented here and the common HRQOL instruments. This also implies that using the ASC T-ASI leads to incomparability with conventional CUAs. Nonetheless, the ASC T-ASI can be used instead of, or (preferably at this stage) in addition, to generic HRQOL instruments as its use may be more informative and appropriate when performing economic evaluations of systemic family interventions where effects broader than health are expected. It also facilitates comparisons of benefits across such interventions. Indeed, the ASC T-ASI and the here presented tariffs can be used in several ways. It may be used as an add-on instrument in future cost-effectiveness studies and clinical trials with low burden to patients due to its brevity. Also, it can be used as a stand-alone self-completion instrument to weight changes in the situation of adolescents in the captured domains. Both options would provide valuable information for use in economic evaluations. When the ASC T-ASI is used in combination with other cost or benefit measures in economic evaluation, overlap and double counting need to be avoided. Such overlap could occur with common measures like the EQ-5D [16] or with cost components of economic evaluations. This, as well as the validity of the ASC T-ASI in different settings, needs to be investigated further in future research [16, 40]. Furthermore, though the ASC T-ASI is developed in the context of systemic family interventions, future studies may consider its application in a broader context of youth mental health interventions.

In conclusion, we performed a DCE to obtain societal preference scores for the ASC T-ASI facilitating its use in the context of economic evaluations of systemic family interventions in adolescents with problems with substance use and/or delinquency. To our knowledge, the ASC T-ASI is the first preference-based measure in adolescent mental health care for which societal preference scores have been obtained that capture benefits beyond those included in the QALY. As such, the results of this study may contribute to better reflecting of the value for money offered by such interventions and optimize decisions on their reimbursement. Many questions for further research were identified which exceed the scope of the current study. Nonetheless, the presented tariff may provide a first step in including relevant disease-specific aspects in economic evaluations of systemic family interventions.