Exploring the potential cost-effectiveness of a vocational rehabilitation program for individuals with schizophrenia in a high-income welfare society

The objective of the current study was to explore the potential cost-effectiveness of the JUMP intervention compared to a treatment as usual (TAU) group in terms of mental health service costs and effectiveness.

The JUMP study was a multi-site VR program for adults with schizophrenia spectrum disorders conducted in six Norwegian counties. The program provided 10 months of standard VR services in competitive or sheltered workplaces, which involved assessments, writing job applications, and preparing for interviews. When required, participants also practiced skills, and had job tasks adapted to accommodate for difficulties at the workplace. In addition to the standard VR services the JUMP protocol included three add-ons: (1) there was a formalised collaboration between VR enterprises, NAV and the mental health services to ensure coordinated and ongoing support; (2) participants, collaborators, and in some cases employers received psychoeducation on common elements associated with schizophrenia spectrum disorders; and (3) either CR or use of CBT techniques provided by trained employment specialists twice a week for a 6 month period. Three counties were randomised to CR and three to CBT. Participants were given the intervention provided in their catchment area [16, 22, 24].

Participants were recruited from within the mental health services, NAV, and through self-referral in the six counties that were involved in the JUMP study. All participants provided written informed consent. Inclusion criteria were: age between 18 and 65; a diagnosis within the broad schizophrenia spectrum disorders (schizophrenia, schizoaffective disorder, psychotic disorder not otherwise specified and delusional disorder) [25]; sufficient understanding of the Norwegian language; and an IQ above 70. Individuals with neurological disorders, head trauma with more than ten minutes of unconsciousness, and medical conditions that interfered with cognitive function were excluded from the study. Also, individuals who displayed high risk of violent behaviour or severe suicidal ideation, and individuals with ongoing alcohol or substance abuse were not permitted to participate [16, 22, 26‐29].

A total of 148 participants were included in the JUMP study between August 2009 and March 2012, 84 and 64 respectively allocated to the CBT and CR interventions [16, 28] (Fig. 1).

The economic evaluation was carried out from the perspective of the health and social care system. This included calculating the cost of mental health services, community care services, social security services, medications and primary health care services, in addition to the costs of the CR and CBT interventions. In accordance with the Norwegian Directorate of Health’s guidelines for economic evaluations in the health sector [30], the number of months employed were converted into quality adjusted life years (QALYs) and served as the measure of effectiveness for the cost-effectiveness analysis. Costs were summarised for two 24-month periods: T0 = the last 2 years prior to inclusion in the study and T1 = from inclusion in the study to two-year follow-up. Intervention costs were included in T1.

We examined the intervention costs and health- and welfare costs for T0 and T1. The mental health service costs in the JUMP group were compared to a control group that received treatment as usual (TAU). Effectiveness was measured in terms of QALYs gained during T1 as compared to T0.

Data on services and costs were obtained from comprehensive health and welfare registers: The Norwegian Patient Register (NPR); The Norwegian Prescription Database (NorPD); The Norwegian Health Economics Administration (HELFO); The Individual-based Register of Care Services (IPLOS); and The Norwegian Labour and Welfare Administration’s (NAV) registers on social security benefits, sick leave payments, disability benefits, and competitive employment. The registers are described in detail elsewhere [7]. All data were collected based on the participants’ unique personal identification number. The cost-effectiveness analyses were not part of the original design of the JUMP study, thus a separate consent for extraction of register data after completion of the program was required. Participants were contacted during July – September 2014.

The intervention costs included costs of employment specialists, a coordinator from the mental health services at each of the six sites, and overhead costs. Each participant was allocated 6 h of the employment specialists’ time per week, including 2 h of CR or CBT in addition to VR services and collaboration with other involved parties.

The TAU group was drawn from the NPR. Upon approval of extracting data from the NPR, all JUMP participants (n = 148) were excluded from the draw based on personal identification numbers. Statisticians at the NPR drew a random sample of 100 individuals with a primary diagnosis of schizophrenia that was matched with the JUMP group on age and sex categories (Table 2). The data were annualised for the period 2010–2012. Only individuals in active treatment in the specialised mental health services (Hospitals, Community Mental Health Centres, and private Psychiatrists and Psychologists with a provider licence) are included in the NPR.

All costs were converted to 2015 Norwegian Kroner (NOK) based on the average consumer price index [31]. Costs are reported in €. For currency conversions we used The Central Bank of Norway’s average annual exchange rate for 2015 (8.95 NOK = 1 €) [32].

Employment status for JUMP participants was recorded weekly by employment specialists during the intervention period. Employment status between the end of the intervention period and two-year follow-up were obtained through interviews with the participants and confirmed against NAV’s employee register. Baseline (T0) employment data were obtained through NAV’s employee register. No employment data was available for the TAU group as we did not have approval to merge the NPR with the employment register for this sample. Thus we assumed a stable employment rate of 10.2% for this group based on the national employment rate identified for individuals with schizophrenia in past studies using data from 2012 [7, 8, 33‐35].

Cost-effectiveness studies commonly use quality-adjusted life-years (QALYs) as a generic outcome measure [36]. QALYs are estimated on the bases of health related quality of life measures. Quality of life measures were not part of the JUMP protocol, thus QALYs were calculated by multiplying months of employment and unemployment with quality of life tariff scores associated with paid employment (0.87) and unemployment (0.79) from the Schizophrenia Health Outcomes (SOHO) study [2]. These scores were based on EuroQol-5D (EQ-5D) scores in individuals who had received 3 years of continuous treatment for schizophrenia. Paid employment was defined as part-time or full-time employment in competitive workplaces. Twenty-one (30.4%) of the JUMP participants (n = 69) had paid employment during the two-year follow-up period, and sixteen (23.2%) were employed at the time of the two-year follow-up.

IBM SPSS Statistics version 21.0 [37] and Stata version 13.1 [38] were used for statistical analyses. All tests were two-tailed with a significance level of 5%. In order to examine group differences, baseline comparisons were conducted with Bootstrap Students t-tests or Chi-square tests. Generalized Linear models (GLM) with gamma family and identity link with baseline costs as a covariate were used to compare costs between groups during the two-year follow-up period. Cost data are often skewed as they are always > 0 and the variance is likely to increase with higher expected costs, and thus violate normality assumptions. The GLM approach provides robust estimates of the mean by accommodating for skewness in the data via 3 components: the random component (cost); the systematic component (treatment group); and the link function (relationship between the random and systematic components). In the gamma family the variance of cost is assumed to be proportional to the square of the mean, which gives a better fit than the normal distribution [39, 40]. To adjust for potential bias due to the lack of randomisation, we performed propensity score adjustments with age, sex, and baseline day treatment, inpatient and outpatient care as potential bias variables. The propensity score was included as a covariate in the GLM. In health economic evaluations, it is recommended to perform probabilistic sensitivity analysis to assess the uncertainty of the resulting cost-effectiveness. We performed probabilistic sensitivity analysis on resulting costs and effects and combined these to provide an estimate of the probability of JUMP being cost-effective compared to TAU.

Sixty-nine (46.6%) participants from the JUMP study consented to extraction of register data for this study. There were no significant differences between those who consented and those who declined on key variables (diagnosis, age, sex, units of DDD of antipsychotic medication, psychotic symptoms, education, previous work experience, employment outcome (Table 1) at two-year follow-up, thus we assume the results are fairly representative for the participants in the JUMP study.

All JUMP participants had primary diagnoses in the schizophrenia spectrum (87% schizophrenia) while subjects in the TAU group had schizophrenia as their primary diagnosis. There were no significant differences in sex or age (Table 2).

The mean cost of the JUMP intervention was € 9131 (SD 2123) per participant. The mean duration of the intervention was 26.52 weeks (SD 5.89).

There were significant reductions in inpatient services from T0 to T1 for both the JUMP participants (mean = € -90,944; 95CI -137,781, − 46,165; p < .012) and the TAU group (mean = € -78,116; 95% CI -126,941, − 33,275; p = .010). Combining inpatient and outpatient costs gives a mean reduction of € 87,809 in the JUMP group and € 76,386 in the TAU group (median: € -1335 and € 13,263 respectively). The reduction in the JUMP group was significant both for those who gained paid employment (n = 21; mean = € -80,776; 95% CI -140,112, − 21,467; p = .010) and for those who had work placement or sheltered work (n = 42; mean = € -90,885; 95% CI -153,873, − 27,897; p = .006). No significant changes were found for other health or welfare costs. Mean and median costs for each 24 - month period; T0 and T1 (inclusive of intervention costs) are detailed in Table 3.

In terms of utilisation of mental health services, the JUMP group had significantly more outpatient visits during both T0 and T1 compared to TAU. At T1 the JUMP group had significantly fewer days of hospitalisation than the TAU group (Table 4). Bootstrap paired samples t-tests revealed significant reductions in inpatient days for both JUMP (mean − 63.0; 95% CI -96.2, − 33.3; p = .006) and for TAU (mean − 53.8; 95% CI -87.5, − 23.1; p = .004). We found no significant differences in change scores between JUMP and TAU through bootstrap independent samples t-test (mean 9.2; 95% CI -39.9, 58.5; p = .71).

We performed a generalized linear regression with costs at T0, group (JUMP/TAU) and the propensity score as covariates using gamma family and identity link to examine group differences in total costs at T1 (Table 5). We tested the assumptions with a linktest which was non-significant, indicating that the assumptions are reasonable. Total mean costs for the JUMP group (inclusive of intervention costs and adjusted for baseline differences) were € 10,621 lower than for TAU (95% CI: -29,979, 8735; p = .282). Costs at T0 was a significant predictor of costs at T1 (Table 5).

When assessing whether a program is cost-effective it is useful to use the cost-effectiveness plane (Fig. 2). If the mean value of the new program is more effective and less costly than the reference program, it is cost-effective and is typically denoted as a dominant strategy. Likewise, if the new program is less effective and more costly it is dominated [41]. New programs in the two other quadrants of the plane are evaluated against a cost-effectiveness threshold value suggested by the Norwegian Directorate of Health which is € 62,000 per QALY gained [42].

In terms of exploring the cost effectiveness of an intervention the effect is usually estimated as a general measure of gained health related quality of life from the new intervention compared to existing treatment, which is converted into quality adjusted life years (QALY).

Dividing the aggregate difference in average specialised mental health service costs (€ 10,621) by the aggregated improvement in QALYs yields a negative incremental cost-effectiveness ratio, which places the JUMP group in the bottom right quadrant of the cost-effectiveness plane (more effective and less costly) (Fig. 2). Although the difference is not significant there is a trend towards the JUMP program being more cost-effective. This is also the case if using incremental months of work as the effect measure. JUMP generated more months of employment than TAU at a lower cost. Probabilistic analyses indicate an 85% probability that JUMP is cost-effective compared to TAU.

A major strength of this study was the use of comprehensive and compulsory health- and welfare registers, which provided detailed records of participants’ societal costs over a period of four years. The main limitation of the study is the lack of a randomised control group. Due to recruitment problems we were forced to abandon the original design of including a control group and thus drew a TAU group from the NPR for this study. Although the TAU group consisted of a random selection, there were large mean differences in resource utilisation between the TAU and JUMP groups at baseline. This may indicate that there were clinical differences between the two groups but as we do not have any clinical information about the TAU group, we are unable to determine if such differences were, in fact, present. By controlling for baseline differences, the effect of this difference is likely to have been reduced. Due to the lack of a control group receiving VR only, we were unable to disentangle the effects of the CR and CBT interventions. Also, no direct measure of health related quality of life was included in the design of the study; hence tariff scores from the literature were used. Although this method is commonly used in health economic evaluations where measures of health-related quality of life are not available, the validity of this estimation should be interpreted with caution as health related quality of life can be influenced by a number of factors [20]. The lack of employment and quality of life data for the control group is also a limitation that affects the validity of the results. Another limitation is that less than half of the participants in the JUMP study consented to extraction of register data. There were, however no significant differences on key variables between those who consented and those who did not. Finally, the high intervention costs of the JUMP study as well as Norway’s high treatment costs and strong welfare system affect the generalisability of our results.