Workplace Outcomes in Work-Disability Prevention Research: A Review with Recommendations for Future Research

Evaluating the outcome of any preventive intervention program is integral for program development and the future choice of initiatives. Work disability is costly for workplaces, families, and society at large with enormous expenditures every year [1, 2]. Workplaces invest sizeable amounts of resources to implement preventive interventions. Careful assessment is of central importance in the evaluation and comparison of interventions. To achieve a sound evaluation, relevant outcomes need to be identified and measured.

In this paper we address the question of how outcomes might best be assessed from a scientific as well as from a business perspective. As reviewed in earlier works [3], considerable effort has been made by the scientific community to develop instruments to measure work-disability and return-to-work outcomes; however, these tend to reflect the interests of scientists. The extent to which they resonate with employer groups is largely undocumented. Within the scientific community, there is emphasis on psychometrically vigorous instruments that assess outcomes such as symptoms and functions that are measured over periods of time. However, the workplace may have a different perspective. They may be interested in immediate results like the cost of the program and how much it disrupts production. It is our contention that in order to advance workplace disability prevention research, it is important to better understand the business community’s perspectives as well as the scientific.

Within both the business and scientific communities, outcomes are usually defined in relation to goals. Consequently, we define outcomes as the degree to which the goals of the work disability prevention (WDP) program are achieved. Because programs may have the goal of tackling certain risk factors (e.g., work limitations, workplace relationships and work engagements) measures of these are relevant as outcomes, with this being especially true for those that may be amenable to change. A further consideration is that WDP goals often focus on health and the ability to work productively. In many cases, both subjective and objective measures are available, and sometimes necessary, to evaluate goal attainment. Establishing clear goals is an important program feature as this enables the use of appropriate outcome measures. For the purpose of this paper, WDP outcomes measures should be understood as measures reflecting the effects of, formal or informal, work-disability policies and procedures addressing physical, social and/or psychological aspects of the workplace.

Given that is it is likely that science and business have different goals in evaluating outcomes, it is important to review measurement from both perspectives. To advance the understanding of currently available workplace WDP measures, identify disparities between employers and scientists and to pave the way for future research, this paper was written with the intention to:

With a goal toward improving future research of employer disability prevention strategies, the authors participated in an invited 3-day conference, “Improving Research of Employer Practices to Prevent Disability”, held October 14–16, 2015, in Hopkinton, MA, USA. Methods and general proceedings of the conference are described in the introductory article to this special issue [4]. The authors of the present article represented a sub-group tasked with examining workplace outcomes in work-disability prevention research. We were asked to address the question: “What are the principal workplace outcome measures in disability prevention research?” The overall purpose and design of the work disability symposium is described in the Introduction to this Special Issue [4].

We recognize that there are many stakeholders involved in work-disability prevention and that an integrated multidisciplinary partnership between the diverse groups (e.g., employers, workers, clinicians) is an effective approach to developing successful and efficient WDP strategies [5]. However, we were charged with focusing on measures addressing workplace features, including organizational policies and procedures, impacting and impacted by workers’ work-disability with a specific focus on the employers’ perspective. As such, WDP outcomes that are not explicitly workplace-related have not been included for discussion in this current work.

The understanding that WDP initiatives vary depending on where the individual is in the work-disability spectrum [6] provided a conceptual framework for our analysis. Based on the developmental conceptualization of return to work [6], we categorized work disability into four different types.

It should be noted that in suggesting this conceptualization, we feel it is important not to categorize the different work disability (WD) types as “phases” This is because we do not want to imply that people progress through each WD type. However, the categorization recognizes that people can move between WD types, and that interventions can help persons shift from one disability type to another. An illustration of this is included in Fig. 1 in which the arrows indicate movement: black = negative in terms of WD, white = positive in terms of work disability.

Prior to the conference, we conducted a narrative review of the scientific and gray (non-scientific business) literature reporting outcomes we felt relevant to workplace disability prevention research. The review was led by our chosen conceptual framework and professional experience in the domain. Our group was formed by the conference organizers based on our interest and experience in the field. We were instructed to address a particular topic area, and conduct a descriptive review, rather a comprehensive assessment of the literature. As such, the papers cited should be viewed as illustrative examples, and not an exhaustive representation of all works measuring workplace WDP outcomes. It should also be noted that when reporting our findings, unless it is specifically stated otherwise, citations refer to works from the scientific literature. Throughout the paper we comment regarding employers’ interest in workplace outcome measures that have been used in the scientific literature; however, it should be noted that not all workplace measures are likely to be of (equal) interest to this stakeholder group.

Once we had decided the outcomes on which we wanted to focus, and obtained input from conference participants, our group tasked itself with summarizing the various outcome groupings and making recommendations for future research. To help researchers in the field to better understand the potential strengths and weaknesses of the different WD measures, we conducted an analysis of the WD outcome groupings. To facilitate researchers and other stakeholders, such as employers, in their decision making regarding which RTW outcomes to employ in their studies, we assessed each of the groupings using six criteria. These were: (1) psychometric properties—the extent to which the measures for RTW outcomes have evidence of validity and reliability; (2) context independence—the meaning and interpretation of RTW outcomes is consistent across different systems and settings; (3) potential for trajectory/RTW process research—the capacity of the measures to be utilized for assessing changes in disability and RTW outcomes over time; (4) availability—the extent to which the data involved in the measures can be easily available; (5) cost—the amount of cost involved in capturing the data required in the measures; (6) employer interest—our subjective rating of the extent of employer interest in the RTW outcome.

Four of the authors of current paper reviewed the strengths and weaknesses of the WD measures groupings as part of the Hopkinton Conference program [4]. Groupings were rated according to a five-point rating scale: “+++” (yes—high), “++” (yes—medium), “+” (yes—low), “No” (no) and “NA” (not applicable). Each of the four group members presented their ratings verbally. If there was disagreement between ratings, this was discussed until consensus was achieved. The remaining authors reviewed the assigned ratings, and were in agreement with the assessments made. Unfortunately, it was beyond the scope of the current exercise to go into detail regarding how best to measure each of the identified constructs.

The results of our review revealed measures that included those that allow for the assessment of whether or not an intervention was successful in terms of helping a person stay at work, decreasing the amount of work absence, and returning workers to productivity. These are described below. Table 1 contains a summary of workplace outcome measures by WD type that have been referenced in the WDP literature, together with the results of our strength/weakness analysis.

Our review revealed workplace WDP outcomes that were many and varied, and we found both consistency and divergence as it related to scientist and employer interest. With that said, it should be noted that what we have presented is an overview and it is likely that the level of interest is not consistent across contexts. For example, outcomes of interest to employers may vary depending on factors such as condition etiology (occupational vs. non-occupational), who pays (employer, worker, or society), corporation size (large vs. small) and the worker’s skills (highly specialized vs. low skilled). In addition, interest is likely to be influenced by the role the employer representative plays within his or her organization. For example, human resources management may be more likely to be interested in policy compliance, whereas line managers are likely to be more focused on productivity and morale. When conducting WDP research, it is important to recognize these differences and incorporate them into study designs.

Although an attempt was made to address which measures have been assessed for reliability and validity, it was beyond the scope of this review to comprehensively assess levels of measure development. While outcomes are referenced in the literature, the degree to which measures have been developed varies greatly. It is our opinion that WDP research would greatly benefit from a dedicated effort aimed at developing a set of field-specific measures that can be applied depending on the aims of research, but would allow for cross context comparisons.

With regards to contrasting business and scientific approaches, we believe that there is a need to bring the views of employers and scientists together in order to achieve better outcome assessment. Based on our review, employers have had a primary interest in outcomes like the direct costs of the program, the extent to which the intervention will disrupt production, and the immediate benefits of the program for the workplace. On the other hand, scientists are likely concerned with the integrity of the intervention, the underlying mechanism involved, the process over longer periods of time, and the effects on the health of workers. Luckily, we observed some overlap in interest areas (see Table 1). This shared interest should provide fertile ground for workplace collaboration and engagement.

Our analysis of workplace WDP outcomes that appear in the literature indicates that WD Type 3 has the largest number and variety of outcomes; however, these outcomes are not necessarily of the greatest interest to employers, who appear more focused on WD Type 2 outcomes. Measures of WD Type 4 are few and do not appear to be of great interest to employers. This is, perhaps, not surprising as by this point, employers have had to deal with staffing and productivity issues, with WD costs shifting to other payers (e.g., welfare and social security systems). If we really want to focus on WD prevention, more effort needs to be put into measuring what is going on when people have WD Type 1. Because employers are likely less interested in this, we should focus on measures that are readily available within company registrations and work on monitoring and surveillance systems to observe trajectories. This would be helpful to detect unfavorable changes in Type 1 outcomes at an early stage.

While our presentation of workplace outcomes is organized by WD type, some outcomes can be used for one or more types of work disability. And sometimes it can be of interest to follow the development of an outcome over time at repeated time points. As has been indicated in Table 1, a few outcomes are appropriate for this and can, accordingly, be used as a basis for a trajectory. Trajectories can incorporate dynamic patterns for quantifiable elements over time. They can also identify distinct latent groups of subjects who tend to have a similar profile [82]. For example, work participation can be followed over time as work participation status (at work/off work) or proportion of time at work. Moreover, ordinal scales can be created to incorporate different grades of work participation (e.g., “not working,” “part-time working,” “full-time working”). Trajectory analysis can be done with rather small data samples. Similarly, RTW patterns have been examined in investigations focusing on recurrence of work disability [33, 83, 84] and patterns of employment following a work-related health condition [85, 86].

There are also measures that can be used for testing movement in the RTW process. As defined and elaborated upon in earlier works [6, 77], the RTW process can be described as being dynamic and bi-directional. It is said to be dynamic because the worker moves through different types of WD from when they begin experiencing health-related limitations until they achieve their final RTW status (see Fig. 1). Stages of RTW can be categorized as “off work due to health condition” (WD Type 2), “returned to work, with work limitations” (WD Type 3), and “at work, no work disability” (WD Type 0). Unsuccessful RTW process would result in workers “withdrawn from labor force due to health condition” (WD Type 4). When at work, workers can be with “no work disability” (WD Type 0) or “experiencing health-related work limitations” (WD Type 1). It is bi-directional because workers can, due to changes in the intrinsic factors such as physical or emotional health or extrinsic factors such as job or workplace demands, progress (e.g., from WD Types 3 to 0) or regress (e.g., from WD Type 3 to 4) through the RTW process. Difficulties with progressing in the RTW process may deter the workers from engagement in the workplace and, in the worst scenario, result in them being unable to move to a place where they are not experiencing work disability, hence withdrawn from the labor force. Within this context, WD outcomes are useful for expressing and predicting the movements of workers across the various stages of RTW. When testing RTW progress in terms of moving from being off work (WD Type 2) to returning to work with limitations (Type 3), it is useful to include an assessment of the time it takes for this occur (identified in the current review as “time off work” [33‐35]). In addition, one could also measure “RTW preparations” [48] and “supervisor/employee interactions” [37]. The measures are useful for describing and predicting RTW movements from return to work with limitations (Type 3) to at work with no disability (Type 0), including “proportion of time at work” [87] and “presenteeism” [12]. The workers’ movements between no disability (WD Type 0), working while experiencing health-related limitations (WD Type 1) can be assessed in terms of “productivity” [8‐10, 13], “presenteeism” [12], “work-related limitations and abilities” [16‐31, 88] and readiness of workers for increasing work hours or work duty (e.g., modified C-LASER [49]).

A critical step in conducting successful workplace WDP research is employer engagement. The role of the above-mentioned economic outcomes on the decision of employers to engage in programs directed at the prevention of work disability should not be overestimated. First, the entity making the investment is often different from the entity receiving the (positive) return. In the case of the trial by Lambeek et al., the health insurer needed to invest in a return-to-work program, whereas the employer received the benefits resulting from earlier return to work [89]. This is a major barrier for WDP implementation. The question who pays differs between countries. In the US for example, the employee will not receive salary in case of non-work-related sick leave, whereas in e.g., the Netherlands, the employer is responsible for paying the salary of the employee on sick leave for the first 2 years. This has major consequences for who will benefit from investment in a return-to-work program. Second, many (positive) aspects of work disability prevention programs cannot be caught in measures that can be included in economic evaluations. It is questionable whether return on investment should always be what is most important. Sometimes programs that do not make money, or even require an (acceptable) investment, are worth implementing, as they may contribute to positive processes that are difficult to include in economic evaluations. For example, employers may become more attractive for talented workers, or have a more positive image to society that may enhance profits. In addition, improved well-being likely has ripple effects in terms of improving productivity and decreasing presenteeism [90, 91].

The issue of cost versus benefit also applies to outcome measure usage. We note that outcomes that provide greater insight and that are, perhaps, more interesting to scientists, are often more difficult and costly to collect. We also note that subjective outcome data tends to be less available and more expensive to collect. In WDP research, subjective measures are commonly used to evaluate a wide range of variables including presenteeism, productivity, and quality of workplace accommodations. On the other hand, “objective” data is drawn from official records, e.g., from an insurance agency (number of compensated days), number of visits for health care covered by the workplace, or production output. While objective data is often thought of as having higher validity than self-reports, which are subject to bias and recall error, there can be problems with either source. Because of the errors in the administrative systems and the difficulties in putting the files together, self-reports may be just as accurate as the administrative data [92‐94]. Another problem with administrative data is that it usually does not cover an important aspect of evaluation, namely how the worker experiences the intervention and its consequences on health. As such, subjective data adds richness to understanding that cannot be achieved with objective data alone. Thus, depending on the research aims, there will be times when increased costs will need to be born to fully illuminate the impact of the study variables on WD outcome.

An additional consideration as it relates to evaluation of WDP initiatives is that a control group is not always readily available. As such, effect size can be difficult to determine. Demonstrating meaningful change is important for employer engagement. WDP researchers need to be creative in terms of evaluating their results. This could be various forms of benchmarking or other methodologies that allow for an interpretation of the effects of the program. An example comes from the RE-AIM framework, which advocates for the inclusion of measures of (1) the extent to which the intervention reaches the target population, (2) efficacy, settings, or institutions, (3), adoption by target staff, (4) implementation consistency, and (5) maintenance of intervention effects [95]. Not only would including such measures give stakeholders additional information upon which to judge the value of a workplace WDP initiative, it would also provide researchers with information regarding the likelihood of intervention success beyond the research setting.

Finally, we note that another key to program success is the timely reporting of results that are of interest to stakeholder groups [96]. Although researcher and employer interests will likely stress different aspects, it is our opinion that workplace WDP research would benefit from integration to achieve interim outcome evaluation and reporting opportunities. Ideally, this would involve multidimensional outcome assessment of a range of variables over extended periods of time. Doing this would benefit both researcher and employer groups. For scientists, this would facilitate an understanding of the immediate impact at work and the underlying organizational factors affecting implementation. For employers, it could provide a valuable understanding of the central factors involved and how they might be changed to benefit the worker and the workplace. In particular, since preventive interventions have a suspected impact over long periods of time, the true benefit of a program might need to be assessed accordingly.

Evaluating the outcome of any preventive intervention program is integral for program development and the future choice of initiatives. We found that there are differences in the way the business community approaches outcome evaluation as opposed to how scientists approach this question. By integrating the two perspectives, outcome evaluation could be significantly improved. This is vital since the development and implementation of WDP programs depend on being able to evaluate their strengths and weaknesses. A clear step forward would be for these two communities to agree upon a basic set of outcome measures which would facilitate both perspectives and a multilevel evaluation. In addition, seeking the input of other stakeholder groups would further illuminate key player’s perspectives and priorities. In the end, all parties have much to gain by coordinating and integrating outcome evaluation.