Towards sustainable mental health promotion: trial-based health-economic evaluation of a positive psychology intervention versus usual care

The non-blinded randomized controlled trial was conducted in two parallel groups, with computerized randomization using Excel (1:1 allocation) and stratified for gender and education (low, medium, high) performed by the first author. For each group, a contact list was created in Qualtrics, making it possible to send personal emails to participants (e.g. including the result of the randomized assignment) without interference of the researcher. A sample size of 132 participants per condition was required to provide a statistical power of 80%, two-sided with α of 5% and compensating for a 25% drop-out rate, to detect a small to medium effect size (Cohen’s d of 0.40) on the main outcome i.e. mental well-being (a continuous measure) [16, 17]. Although the primary outcome of the current study was the binary measure of mental well-being, a similar sample size was needed when the power calculation was adjusted to a χ² test and intention-to-treat data (n = 126 per group). Participants aged 18 years or older were recruited from the general Dutch population in January 2014 via national advertisements calling for people who were motivated to actively work on their “well-being and resilience”. The participants were willing to invest an average time of 4 h per week for 9 weeks and had access to email and the Internet. Interested participants completed a contact form via a research website, and received an online informed consent procedure per email before they could access the online screening survey. Eligible participants were excluded when they were younger than 18 years of age, already possessed a flourishing mental health status as measured with the Mental Health Continuum-Short Form (MHC-SF); scoring 4 or 5 on at least one emotional well-being item together with a score of 4 or 5 on at least 6 of the 11 social and psychological well-being items [7, 18] or when they presented scores above 10 on at least one subscale of the Hospital Anxiety and Depression Scale (HADS), indicating moderate to severe anxiety or depressive symptoms [19]. Also, participants had to complete the screening and baseline questionnaire because randomization took place after baseline.

In total, 518 participants were interested in participating in the study, of which 243 participants had to be excluded [11]. The final sample of 275 participants were allocated to the intervention group (n = 137) or the wait-list control group (n = 138). The trial protocol was approved by the Ethics Committee of the University of Twente (no. 13212) and registered at The Netherlands Trial Register (NTR4297). The design of the study [20] and its main findings [11] are published elsewhere.

Participants in the intervention group received (1) the self-help book This is Your Life [21], (2) a 9-week time schedule for reading the book with recommended exercises, and (3) weekly email support from a personal counselor. The book consists of eight chapters containing psycho-education, theoretical background information, and a variety of evidence-based exercises from positive psychology, but also from mindfulness and acceptance and commitment therapy. The purpose of the book is to improve an individual’s capacity to savor positive emotions, discover and use character strengths, encourage flow and an optimistic attributional style, develop self-acceptance and compassion, learn to cope with adversity (resilience), and encourage to share and connect with others [20]. The chapter about discovering and using character strengths was spread out over 2 weeks in the time schedule. Participants had 8 to 12 weeks to complete the program. A full description of each chapter and recommended exercises can be found elsewhere [20].

Once a week, participants emailed their personal counselor with their experiences about the chapter and corresponding exercises. Five positive psychology students each guided 25 participants and the first author guided the remaining participants. The counselors were trained in providing email support during a study course plus a one-day workshop. In addition, they attended weekly supervision meetings. The email correspondence was aimed at increasing adherence. On average, participants reported that they had completed 6.4 (out of the 8) chapters (SD = 2.4) and had sent 6.4 extensive emails (SD = 3.62), indicating adequate adherence to the protocol [11].

The primary outcome was flourishing mental health and the secondary outcomes were anxiety and depressive symptoms. Self-reported data were obtained from online questionnaires at baseline and 6 months follow-up.

The 14-item MHC-SF measures mental well-being on a continuous scale but can also classify people into (1) flourishing mental health, (2) moderate mental health or (3) languishing mental health [18, 22], although we put the latter two categories together because there were few people with languishing mental health in the current sample at baseline (4.4%). In a cost-effectiveness study, hard currency (measured at the interval level) cannot be meaningfully related to health benefits that are measured at the (“elastic”) ordinal measurement scale. It is economically more meaningful to relate hard currency to a binary outcome such as treatment response, where treatment response is clearly defined as reliable change [23] or as a transition from one health state (e.g. languishing or moderate mental health) to another (e.g. flourishing mental health).

The first three items of the MHC-SF measure emotional well-being (i.e. happiness, interest, life-satisfaction), the next five items measure social well-being (i.e. social contribution, social integration, social actualization, social acceptance, social coherence) and the last six items tap into psychological well-being (i.e. self-acceptance, mastery, positive relations, personal growth, autonomy, purpose in life). Each item was scored on a 6-point scale from 0 (never) to 5 (almost always). Flourishing is theoretically operationalized as scoring 4 or 5 on at least one emotional well-being item together with a score of 4 or 5 on at least 6 of the 11 remaining items, which parallels the DSM-IV approach to diagnose a major depression [7, 18, 24]. The Dutch version of the MHC-SF has shown good psychometric properties [22] and showed good internal consistency in the current study (α = 0.88).

Anxiety and depressive symptoms were measured with the HADS-A and HADS-D respectively. Each subscale has 7 items with scores ranging from 0 to 3. Total summed scores range from 0 to 21, with higher scores indicating greater anxiety or depressive symptom severity. The HADS has shown good psychometric properties in the Dutch population [25, 26] and showed good internal consistency in the current sample (α = 0.76 for both subscales). To measure treatment response on these scales, Jacobson and Truax’ method [23] was applied to obtain the reliable change index to distinguish between treatment responders and non-responders. The reliable change index was calculated as x₂ – x₁ / S_diff, where x₂ is the post-test score and x₁ is the baseline score. S_diff is the standard error of difference between the pre- and post-test scores which is calculated as √(2(S_E)²), where the standard error of measurement (S_E) is calculated as SD√(1 – r_xx). r_xx is the test-retest reliability of the measure, which was 0.89 for the HADS-A and 0.86 for the HADS-D [26]. A treatment responder was estimated to be a pre-post change of at least 2.22 points on the anxiety subscale and 2.55 points on the depression subscale, taking the 95% criterion into account (z = 1.96).

The current study adopts a health sector perspective in accordance with national UK and US guidelines [27, 28]. Therefore, direct medical costs (health service use), direct non-medical costs (travel costs to health services) and intervention costs were included and not productivity losses. Resource use was measured with the Medical Consumption Questionnaire (MCQ) for three periods: 3 months before baseline (T0), baseline to 3 months follow-up (T1) and 3 to 6 months follow-up (T2) [29]. All costs are expressed in euros (€) for the reference year 2014. The main results are also expressed in US dollars ($). The general purchasing power parity (PPP) was used for conversion of the € to $ for 2014 (US$1.00 = NL€0.814) [30].

Table 1 displays an overview of health service units (contacts or hours) and their standard unit cost price as reported in the Dutch guidelines for health-economic evaluations [31]. Health service costs per participant were calculated by multiplying the utilized health service units of each participant in the past 3 months with the standard unit cost price of that service. Travel costs for each health service visit were calculated by multiplying the average distance to that service according to the Dutch guidelines (see Table 1) with the costs per km (€0.19). Parking costs were added to the travel costs which amounted to €3 per visit. Other costs of participants or its family members outside formal health care (e.g. informal care) which might have had a direct relation with an illness were not included in this health-economic evaluation because no severe (chronic) disorders were investigated and the sum of these costs would be limited.

Each participant in the intervention group received the self-help book This is Your Life which was valued at €25 in 2014 as found in a large and representative online bookstore in The Netherlands. Each participant also received personal email support. Although the current study used students as counselors, in real-life health care settings these will be replaced by the family doctor’s mental health nurse at €17 per consult [31]. We assume an average of nine email contacts, which puts the costs at €17*9 contacts = €153 per participant. Additional costs were incurred for recruitment, screening and training costs for the mental health nurse at €44 per participant in a real-world setting. In total, the estimated intervention costs were €222 ($273) per participant.

The analyses were repeated for three different scenarios to examine the robustness of the results. In scenario A, the intervention costs were based on the actual number of emails sent by the personal counselor to the participant (not the assumed maximum of 9 emails) and then multiplied by the costs for consulting a family’s doctor mental health nurse (€17). In scenario B, the intervention costs were raised by including the time investment of participants (valued at 3 h*9 weeks*€14) amounted to a total of €600 per participant. In scenario C, completers-only analyses were performed, using data of the 112 participants in the intervention condition and 125 participants in the control condition who completed all measurements at T2.

Participants were predominantly female (85.8%), higher educated (74.5%) and in paid employment (68.4%). Mean age was 47.8 years (SD = 10.9). The mean score of the total sample for mental well-being was 2.57 (SD = 0.63), for anxiety symptoms 7.28 (SD = 2.41) and for depressive symptoms 5.80 (SD = 2.47). There were no significant between-group differences regarding participant characteristics, main outcome measures and resource costs.

At 6 months, there were 42 participants (30.7%) with flourishing mental health in the intervention condition and 16 participants (11.6%) in the control condition (χ² = 15.01, df = 1, P = < 0.001), as has also been reported previously [11]. The incremental effect (i.e. the proportion of flourishers in the experimental group minus the proportion of flourishers in the control group) was 0.31–0.12 = 0.19. Similar effects were found for anxiety and depressive symptoms. For the HADS-A, 57 participants (41.6%) in the intervention group met the criteria for treatment response compared to 27 participants (19.6%) in the control group (χ² = 15.74, df = 1, P = < 0.001). For the HADS-D, these number of participants were 59 (43.1%) in the intervention group and 30 (21.7%) in the control group (χ² = 14.29, df = 1, P = < 0.001). The incremental effects were 0.42–0.20 = 0.22 for anxiety symptoms and 0.43–0.22 = 0.21 for depressive symptoms.

At baseline, total average annualized direct medical and direct non-medical costs were €581 (SD = €1190) in the intervention condition and €675 (SD = €1246) in the control condition (Table 2). At T1, these costs were €432 (SD = €983) and €658 (SD = €1224) respectively. At T2, these costs were €506 (SD = €1001) for the guided self-help intervention and €488 (SD = €1190) for usual care. These T2 annualized costs were used for the cost-effectiveness analyses, adding the intervention costs of €222 to the intervention condition. Therefore, the annualized incremental costs at the trial’s follow-up were €728–€488 = €240 ($295).

The bootstrapped mean ICER shows that additional costs of €2359 ($2899) had to be paid to improve one person from low or moderate well-being to flourishing mental health. The bootstrapped median ICER showed additional costs of €1245 ($1530; Table 3). The majority of the plotted ICERs (92%) occurred in the northeast quadrant, indicating that the intervention produced health gains at additional costs (Fig. 1). All other plotted ICERs occurred in the southeast quadrant (8%), which implies health gains at lower costs. When there is no WTP, there is a 12% probability that the guided self-help intervention is more cost-effective relative to usual care (Fig. 2). This probability increases to 93% with a WTP ceiling for a favorable treatment outcome of €10,000 ($12,290). Conversely, with a probability of 80% the WTP is approximately €8000 ($9832).

The bootstrapped mean and median ICERs for treatment response on anxiety symptoms were €2959 ($3637) and €1095 ($1346) respectively, and for depressive symptoms €2578 ($3168) and €1189 ($1461; Table 4). The distribution of the 2500 bootstrapped ICERs on the cost-effectiveness plane as well as the WTP curve show similar results as has been found for flourishing: 91 and 90% of the ICERs were plotted in the northeast quadrant for anxiety and depressive symptoms respectively. The probability of accepting the intervention in favor of usual care with no WTP was 13% for both outcomes. At a WTP of €10,000, the probability of accepting the intervention had risen to 92 and 93% respectively.

The results from the alternative scenarios show similar patterns for all three outcomes (see Table 3 for flourishing and Table 4 for anxiety and depression). Most mean and median bootstrapped ICERs were lowest in scenario A based on the actual intervention costs with mean ICERs between €1848 ($2271) and €2109 ($2592). Scenario B (intervention costs including the opportunity costs of the participants loss of leisure time) revealed the highest mean and median ICERs, while the completers-only analysis of scenario C was more comparable to scenario A. The percentage of ICERs on the cost-effectiveness planes for treatment response lies between 86% (scenario A) and 100% (scenario B). Furthermore, the probability of accepting the intervention over the control condition at no WTP lies between 4% (scenario B) and 18% (scenario A). In sum, these sensitivity analyses provide support for the robustness of the main analyses.

Main strengths of the study were its well-powered and randomized controlled design, the high adherence rates and results that appeared robust under sensitivity analyses. Also, as the self-selected sample of “well-being-seekers” recruited in the general Dutch population is congruent with usual recruitment strategies for self-help interventions in The Netherlands, the sample is representative for future applications of this intervention. However, we cannot generalize to the (unselected) general population because our sample overrepresented well-educated women in their late forties with paid jobs. An important limitation of the current study relates to the lack of an active control group wherein the self-help book could have been offered without email support, as was planned [20] but not feasible [11]. Prior studies demonstrated larger effects of guided CBT-based self-help than its unguided counterpart in improving mental well-being, anxiety and depression [13, 35, 36]. Hence, we cannot rule out the influence of adding email support to the PP-based bibliotherapy. Other limitations include the relatively short follow-up of 6 months and using this time-point for the extrapolation to an entire year (i.e. assuming a steady-state of the annualized costs and effects), the use of a wait-list control group (rather than usual care alone), unblinded participants, and the absence of assessing quality-adjusted life years (QALY’s) [27]. An advantage of using QALY’s is that a cost-utility analysis could have been added to the analyses to obtain the costs per QALY gained which is a generic health-related outcome that can be used across diseases and disorders. However, a strength of the current study was to use a specific health-related outcome that fitted well with the target group (people with languishing or moderate mental health) and the intervention aim (to establish the transition to flourishing mental health). In addition, the MHC-SF measuring mental well-being seems more sensitive to change [22] then the EQ-5D measuring QALY’s [37], although more research is needed to validate the theoretical cut-off scores for categorizing people into flourishing or not.

A valuable approach for public mental health seems to enhance flourishing mental health because this status has been related to reduced risk of developing anxiety and depressive symptoms and mortality [8, 9, 38, 39]. The results of the present study demonstrated that an early intervention based on PP principles has the potential to promote a flourishing mental health status and substantially reduce anxiety and depressive symptom severity at some additional costs. The intervention could offer good value for money when there is a WTP of around €8000 ($9832), which corresponds to a probability of 80% that the intervention is more cost-effective compared to usual care. Even when the intervention costs are tripled by including the time costs of the participants, the WTP for a probability of 80% falls around €8000 ($9832). The economic costs of the intervention would rise from €2359 ($2899) for one additional person to improve from suboptimal well-being to flourishing in the main analysis to €4534 ($5572) when intervention costs are tripled. However, when there is no WTP for a treatment response, there is only a 12–13% probability that the intervention is more cost-effective relative to usual care. Thus, it remains a challenge to further optimize the cost-effectiveness of PP-based self-help interventions. Perhaps this can be achieved by shortening and refining the current program and focus on its most efficacious processes: enhancing positive relations, self-compassion and optimism [40]. In addition, it remains a great challenge to reach people with a lower socio-economic status, also with PP-based interventions.

Future research should replicate our findings and examine longer-term costs and benefits of PP interventions to promote flourishing. In this regard, we recommend large-scale implementation of the guided bibliotherapy program in public mental health or primary care wherein the intervention costs and effects are carefully monitored for more than one year. It is also of utmost importance to conduct economic evaluations of a guided versus unguided PP-based intervention which could shed light on the added value of therapist involvement. Furthermore, a cost-effectiveness study wherein a self-help intervention based on CBT is compared with a self-help intervention based on PP would be interesting because a prior study found that both CBT and PP where efficacious in ameliorating mental well-being and depression, but that the PP-based intervention was significantly more preferred, thus, having an impact on a larger population [41]. Overall, a PP-based self-help intervention with some guidance over the Internet has the potential to reach large groups of people via public mental health services, primary care and as additional service in mental health institutions. Also, a PP-based self-help intervention might hold promise as a worthwhile alternative to the predominant CBT-based self-help interventions. We hope the current study inspires researchers to plan and conduct economic evaluations alongside their trials to address this gap in the mental health promotion literature [13, 42].