Background
Globally, non-communicable diseases (NCDs), including heart disease, stroke, cancer and diabetes, account for 71% of all deaths [
1]. Many NCD risk factors are preventable through health promotion initiatives to reduce tobacco use, unhealthy diets and alcohol use, and to promote physical activity in the population [
1]. Workplaces are a good setting for health promotion initiatives because of the potential reach: over 3.4 billion people make up the global labour force [
2]. In Australia, 12 million people, or 95% of the working-age population, are employed [
3] and, like other OECD countries, spend on average 36 h per week at work [
4].
Workplace health programs (WHPs) are coordinated and comprehensive health promotion strategies comprising of policies, environmental supports and activities in the workplace to engage workers in healthy behaviours and facilitate their wellbeing [
5]. WHPs differ from workplace health and safety programs in that the latter are injury-focussed while WHPs tend to focus on lifestyle-related NCD prevention [
5]. Benefits to employees of WHPs reported include decreased risk of NCDs and improved health behaviours (e.g., physical activity and nutrition) [
6‐
10]; while benefits to businesses have included improved market value [
11] and return on investment [
7,
12]. Evidence of the effectiveness of WHPs on productivity is mixed; one recent review [
13] was inconclusive while a meta-analysis found limited health and productivity benefits [
14].
Evaluating the implementation of WHPs is central to understanding the benefits and the factors which facilitate or inhibit their effectiveness and sustainability [
15]. Most evaluation research on WHPs has focused on measuring program outcomes, yet comprehensive evaluation should also capture the implementation process [
16,
17]. This is because WHPs are often complex, having multiple components, targeting multiple health behaviours, involving multiple levels of influence within an organisation or addressing multiple determinants [
18,
19]. The mechanisms for success of such programs depend on context and so evaluations need to examine contextual factors influencing implementation [
20]. Complexity is further increased when programs are implemented at-scale (e.g., state or nation-wide) across multiple workplace settings [
21]. Process evaluation becomes particularly important in this case, not only because one intervention may be implemented differently at multiple sites [
22] but also because multiple levels of implementation introduce additional layers of complexity [
23].
The various aspects of the program, the context it is delivered in, and the levels of implementation involved, contribute to an uncertainty about the impacts the program and how they manifest. This uncertainty produces what known in complex systems science (or systems thinking) as
emergence, and means that the specific outcomes and the way in which they occur may emerge during implementation process, and will be unknown a priori [
24]. Various frameworks for evaluating implementation processes in health promotion have been developed [
15,
19,
25,
26]. These emphasise a need to investigate the characteristics of, and interaction between, the intervention, the organisation and the implementer. Where there are multiple levels of implementation, actions at each level need to be included in the evaluation as variability in individual level outcomes may reflect contextual processes [
27] such as those occurring upstream at the policy level [
28,
29]. At the organisational level, Weiner suggests that implementation effectiveness is subject to (i) the adaption-fit between the program and the organisation; and (ii) employee acceptance of the program (both the end-user and those implementing it in the business) [
18]. Complex evaluation theory also emphasises the need for multiple evaluation methods, because neither quantitative nor qualitative approaches provide adequate insight into the implementation of complex programs [
20,
22,
30].
In practice, WHPs have tended to be limited in scope (e.g., comprise only an environmental support or address only one health behaviour) [
31], and implemented in one or a small number of organisations [
32]. Evidence of large-scale WHP implementation and evaluation is scarce: those that have been evaluated have concentrated on quantifying one or a limited number of health impacts at the employee level [
33‐
35]. While changes in health at the employee level should be the ultimate goal, there is often very little evaluation of the mechanisms of change in interventions with multiple levels of implementation or which are delivered at-scale. The purpose of this study was therefore to evaluate the state-wide implementation of a complex WHP in Australia and to assess its short-term impacts at the business level. Specifically, the aim of this study was to evaluate the
Get Healthy at Work (GHaW) program, a government-funded WHP initiative to reduce workers’ risk of chronic disease.
Discussion
One of the main challenges to the dissemination of health promotion programs in the workplace is the lack of evidence on the effectiveness of interventions within the real-world implementation context [
46]. This evaluation sought to assess the business-level impacts and factors supporting implementation of a state-wide comprehensive WHP which included a focus on health education, supportive environments, policies to integrate the program into the workplace, resources and implementation support [
31]. Implementation of WHPs at the business-level will determine individual worker-level outcomes. For this reason the evaluation focused specifically on the implementation process and the impact at the business-level. The evaluation research conducted with employees was used to gather information about the implementation process across individual businesses, rather than to evaluate worker-level outcomes, which would be unlikely to change over the short-term intervention period. This means we were concerned with assessing change at the organisational level, including organisational culture and climate [
47]. These need to be assessed prior to any impact evaluation of employee-level outcomes as such context-free evaluation will provide limited and potentially misleading conclusions about a program [
24]. The evaluation also helps to identify the obstacles and challenges involved in implementing a state-wide health promotion program which can be used to improve future program delivery.
Some positive impacts were observed which support GHaW as a vehicle for enabling NCD prevention across the workforce population. Specifically, survey findings reveal that while GHaW participants reported initially a much lower perception of their workplace as healthy or health promoting at registration, this improved significantly over the course of the evaluation to be well above control group perceptions (which remained unchanged). This suggests that interaction with the GHaW program had a positive impact on the business contacts’ perceptions of health promotion in their workplace. Poor perceptions of health promotion in their workplace at baseline may have been an essential reason why this group of businesses first registered for the program in the first place [
48‐
50]. The intervention group retained a higher willingness to participate in WHP activities. Together, these results suggest some level of organisational change occurred, an essential critical condition for effective implementation and success of workplace health promotion initiatives [
18,
51]. Improvements in workplace culture did not lead to improvements in perceived work productivity within the evaluation period. However it may be too early to gauge evidence of impact because of the lag in engagement: nearly a third of businesses had not yet started the program cycle by T3 and many others were still only at the initial pre-implementation stage of conducting brief health checks in the workplace. No specific organisational characteristic in the quantitative analyses explained why most businesses had not yet proceeded further along the program cycle.
Investigation of the implementation processes through the qualitative interviews with business contacts and service providers and the focus groups with employees revealed a number of factors contributing to the program outcomes observed; these include both business level factors and program and delivery factors. A recent synthesis of qualitative studies across all phases of WHP development identified six areas that facilitate or hinder WHP implementation. Three identified factors (which we also assessed) related to the intervention, the implementer and organisational level; the other three factors included the participant level, the planning phase, and evaluation methodology [
47]. Our qualitative analysis similarly identified organisational characteristics influencing program implementation, including business size and structures and experience with WHPs. For example, larger businesses were found to ‘pick and choose’ components of GHaW, like the brief health check, according to their workplace needs and may have used GHaW as part of a broader, internally-developed and pre-existing WHP.
Unfamiliarity with WHPs in general was a major determining factor for how smaller businesses interacted with the GHaW program and in their initial struggle to connect their desire to change their workplaces with the ability to navigate the steps involved. Small businesses make up 97.5% of businesses in Australia [
52] and are therefore an important group to equip to promote health. Smaller businesses, lacking corporate structure and personnel, must balance an interest in health promotion with core business priorities, and this was evident in the qualitative analysis. Despite financial incentives being a major facilitator for small businesses to adopt GHaW, changes may be required to develop these incentives to make it easier for businesses to implement WHPs alongside other business priorities. Organisational readiness for change and supportive senior leadership were also identified as influential factors in the qualitative data and elsewhere [
47]. Senior leadership support is frequently reported as a facilitator of WHP implementation and effectiveness within a workplace [
53]. Leadership commitment, measured quantitatively in this study, was not a statistically significant factor, possibly due to the size of this sample of participants who had implemented the WHP.
Government may play a central role in enabling prioritisation and incentivising health promotion in the workplace [
54]. We found that while small businesses are motivated to improve the health of their employees, they have limited tools and knowledge to implement WHPs. Government delivery and resource support were strong reasons businesses registered in the program. A lack of resources has often been reported as a barrier to WHP implementation [
47,
53]. GHaW’s financial incentive was particularly important to smaller businesses; however many had difficulty completing the brief health checks, a requirement for eligibility for receiving the financial benefit. A mismatch between what businesses desired from a WHP and how GHaW was offered led to variable implementation. Rojatz et al. [
47] also identified appropriateness of the intervention as central to business level implementation. We found that ‘wellness’ programs which include healthy eating and physical activity initiatives are preferred by businesses, and there was less interest in programs which specifically target alcohol consumption, active travel or smoking cessation, issues which may not encompass all employees. While GHaW can be tailored to a variety of employee health priority needs, the expressed preference points to allowing selection of a suite of priorities to suit business needs [
55].
Program delivery factors were also identified as potential barriers to implementation. Technical issues and delays in the operation of the online portal, complicated administrative processes and unclear communication were reasons given for delays in businesses receiving service provider support in the early stages of the program. This may be a key reason for why less than one third of businesses sampled had developed a WHP after 12 months. Quality improvements to processes, for example automation of some of the service provider administrative tasks that were occurring over the course of the evaluation, and clearer communication channels are likely to mitigate many of these issues for future participants [
48]. In order to improve long-term impact and sustainability of health promotion in workplaces, viability of the program as a business proposition for service providers may also need consideration. Partly this may be through better communication of the role of the service provider, yet it may also involve other strategies given the demand for service provider assistance to develop tailored WHPs.
Complex WHP programs like GHaW require multi-level evaluation methods to determine what worked, how it worked and what mechanisms enabled the program to work in this way [
56]. In one sense, it might appear that every group involved in the implementation attributes delay or failure to an external source. Yet the reality is that delivery of programs at-scale is often complicated and implementation is challenged by many discordant factors combining to influence outcomes in nonlinear and often unpredictable ways [
57]. Understanding this complexity is important for building the evidence to inform policy and practice [
23].
The complexity of WHP does not mean that the evaluation needs to be complex [
26], however there needs to be some understanding of the mechanisms or processes and context which produce the outcomes observed [
58,
59]. Essential in any complex program evaluation is an evaluation of the implementation process. Unfortunately, evaluation of the implementation process is still rarely applied to WHP programs in practice [
53]. Implementation context, in particular, is central to understanding outcomes, improving the reach and uptake of health promotion programs and ensuring their generalisability.
A major strength of the study was that it employed a mixed method approach, which integrated quantitative and qualitative data sources. The quasi-experimental longitudinal analysis allowed individual business responses assessed over time, adjusting for underlying trends in the control sample to account for baseline differences. Additionally, using qualitative data from different perspectives and points in the system (i.e., service providers, business key contacts and employees) further enriched the analysis of the quasi-experiment. Moreover, evaluations embedded into program delivery can facilitate continuous improvement of programs to optimise program engagement and outcomes. Ongoing program evaluation, which includes feedback loops in WHP design for ongoing refinement and course-correction, is an important practice for an effective program [
31]. The evaluation of the GHaW program [
36,
60] has led to a number of service delivery enhancements. Some of the modifications include simplification of the program for small businesses, increased funding for service providers to engage with businesses at early stages of the program, and regular electronic messaging to businesses to provide program updates. Redevelopment of the GHaW web portal is currently underway to address delivery issues raised by business contacts and service providers.
One of the major challenges of evaluating health promotion programs in practice is determining population impact. The reach of the GHaW program when it was first launched was limited if we were to consider just the initial sample in its first year as a proportion of all businesses in the State (which was less than 1%). A population dose of < 2% could be considered low impact, while a dose above 5% could be considered both significant and measureable [
61]. However these estimates are also arbitrary [
61], and in the case of WHPs, must also account for businesses size and other factors: larger businesses reach higher numbers of the workforce population, but smaller businesses often include the harder to reach workforce. [
46].
A limitation of the study was the rate of attrition in the evaluation sample; particularly at T3 amongst those who had reached the stage of adopting a WHP action plan. This attrition also highlights the challenges of evaluating WHPs in practice. Within the course of a year, businesses may expand or fold or change operating structures as revealed in the qualitative study and response rate of the survey. This can have a profound impact on evaluation. Tracking changes in the intervention group against a control sample, representative of the wider population of businesses, strengthens the study by accounting for secular trends which may be occurring over time [
44] and consequently the ability to make causal inferences about the program. This was a quasi-experiment. The rigour of the study design may have been strengthened were a natural experiment approach used to assess exposure to the GHaW program, i.e., before the intervention group was established, however this would add substantially to the cost and time demand.
The study relied on subjective measures of productivity and businesses’ interpretation of how far into the program cycle they progressed as a proxy for objective measures. Subjective responses were based on the opinions of the business contact who is likely to have a vested interest in the WHP and its impact. As noted earlier, this evaluation prioritised evaluating implementation at the business level over the employee level, assuming implementation at the business level leads to employee level impacts. Focus groups were used to develop an understanding of the impact as viewed at the employee level, however few businesses contacted felt they had the time or the logistical ability to enable employees to participate. The evidence obtained from these focus groups is therefore incomplete and should be treated with caution. This evaluation was conducted with businesses that were early adopters of the GHaW program and the findings therefore may not be representative of businesses in NSW. Nevertheless, the evaluation provides insights into the potential effectiveness of GHaW, as well as the complexities of implementing WHPs.