Vocational Rehabilitation Program Evaluation: Comparison Group Challenges and the Role of Unmeasured Return-to-Work Expectations

Purpose Despite the importance and cost of workers’ compensation (WC)-based vocational rehabilitation (VR) programs, outcome evaluations are rare, in part due to the scarcity of suitable comparison groups. The aims of this study were to assess (1) the adequacy of a commonly recommended internal comparison group, i.e., workers who were eligible for but did not receive services, and (2) return-to-work (RTW) expectations as a potential source of bias. Methods In this prospective cohort study, we used WC claims data and worker-reported RTW expectations to compare workers who received vocational retraining services to eligible workers who did not receive such services. Workers were surveyed after retraining eligibility determination, prior to the initiation of retraining activities. VR progress and RTW wage outcomes were followed for 3 years. The magnitude of confounding contributed by RTW expectations and other covariates was quantified. Results Workers who were somewhat or very certain they would RTW had significantly better outcomes. RTW expectations played a strong confounding role, reducing the retraining plan effect estimate by about 23 %, while education and physical capacity each changed the effect estimate by <5 %. Conclusions RTW expectations predicted long-term RTW outcomes and can play a strong confounding role if unmeasured. We found that the internal comparison group approach, commonly recommended for VR program evaluation, is inappropriate for WC-based VR evaluations. Ultimately, there is no simple solution to the challenge of identifying a comparison group; however, measurement of RTW expectations, an easily-measured multi-dimensional construct, may be a useful addition to the VR evaluation toolbox.