Design of a randomized controlled trial on the effects of Counseling of mental health problems by Occupational Physicians on return to work: the CO-OP-study

This study was conducted with the cooperation of the Dutch police force, which is an organization with a relatively high incidence of mental health problems. These problems are mainly work related as police work has inevitable risks and stress may develop as 'part of the job' [19]. The employer of the Dutch police, the Ministry of Internal Affairs, tried to provide an optimal care and was open to alternative effective treatments. Each Dutch police employee was insured by the insurance company, the DGVP. DGVP tried to provide optimal usual care by partly financing referrals by OPs of police employees with mental health problems to a commercial psychotherapeutic centre as part of a protocol. Therefore, these police departments and their occupational health care provided a representative study population.

This intervention was developed for the occupational health care setting with its typical case load of common mental disorders. Two police departments were chosen because they had contracts with the same private OHS, i.e. Commit. Consequently, uniformity in treatment was more secured. Commit is one of the largest OHSs in the country. The police departments, i.e. Zaanstreek-Waterland and Hollands Midden, were located in the South-West of the Netherlands. Hollands Midden comprises approximately 1700 employees; Zaanstreek-Waterland approximately 800, totalling a source population of 2500 police employees.

Because we wanted to prevent employees with chronic disability to participate in the study, each employee on sick leave due to mental health problems before the start of the study in 2002, was detected by the OHS. They received a treatment by a psychologist in secondary care funded by the DGVP.

Regularly, employees were registered on their first day of sick leave by the OHS (figure 1). Since January 2002, each employee on sick leave due to mental health problems was invited to meet with a case manager of the OHS within one week. This case manager informed the employee about the study and planned a consultation with an OP in the first two weeks of sick leave. To enhance recruitment one of the researchers (DB), who was allowed to check the registration system of the OHSs, informed the OP when a potential participant would come for consultation. Each employee who consulted an OP, and was still on sick leave due to mental health problems, was then asked by the OP to participate in the study. After an employee had signed informed consent during this consultation (T0), the OP unsealed a study envelope containing the allocated treatment for the patient, and sent the signed informed consent to the researcher (DR). In the same consultation the employee received the baseline questionnaires and was asked to return this questionnaire to the researcher after completion.

As the guideline focuses on all kinds of mental health problems, we aimed to include employees with a broad range of mental health problems consulting their OP. Employees were included if they met the following inclusion criteria:

Exclusion criteria were the same as stated in the OP-guideline:

The application of the exclusion criteria was dependent on the OPs expert judgement. To prevent selection bias, employees were not included of whom the period of sick leave started before 2002.

In this study the aim was to compare usual care of employees with mental health problems with an intervention. Usual care consisted of minimal involvement of the OP and access to treatment by a psychologist, as this represents daily practice. As the aim was to deliver the best optional care to our study population, optimal usual care was provided in this study. This consisted of the advice to OPs to refer to a psychologist, whose treatment was fully funded by the DGVP. The OP could refer to a psychologist working for a commercial multidisciplinary rehabilitation center, i.e. De Gezonde Zaak (DGZ), as this was part of an agreement with the OHS Commit. A patient was only referred if according to the expert judgement of the OP this made sense to the health condition of this person.

DGZ is one of the largest Dutch commercial psychotherapeutic intervention centers, which focuses on return to work of the employee. DGZ is located in different parts of the country. Besides physical therapists, around 100 psychologists are working for this organization. The psychologists are working according to cognitive behavioral principals. The standard therapy offered was based on protocols of the Dutch Institute for Work and Stress [20].

The intervention consisted of treatment by OPs according to the guideline of employees on sick leave due to mental health problems. The guideline promotes a more active role of the OP as case and care manager facilitating return to work of the employee. The guideline is based on an activating approach, time contingent process evaluation and cognitive behavioral principles. The latter mainly concern stress inoculation training and graded activity and aim to enhance the problem-solving capacity of patients in relation to their work environment.

The guideline focuses on four aspects of the management of mental health problems. First, an early and activating guidance by the OP is promoted, in which return to work is part of the recovery process, even if the mental health problems are not related to work. Second, a simplified classification of mental health problems is introduced, with only four categories: 1) adjustment disorder (distress, nervous breakdown, burnout), 2) depression, 3) anxiety, and 4) other psychiatric disorders. Third, the OP acts as case manager, who is stimulated to be a care manager by counselling employees with adjustment disorders and work-related problems. Fourth, the OP performs a time contingent process evaluation and intervenes when recovery stagnates.

OPs participating in the study received training in the guideline before the study started. During this training, consisting of a three-day course with 10–15 other OPs, knowledge about and practice in working with the guideline were educated and exchanged [21]. The course reflected the guideline by training OPs in multiple cognitive-behavioral prescriptive interventions, to stimulate the patients' acquisition of problem solving skills, and to structure the patients' daily activities. Information was given and OPs were trained to differentiate between adjustment disorder and depression, anxiety and other psychiatric disorders. Questionnaires were introduced, which can be helpful in making an accurate diagnosis. In addition, a graded activity treatment approach was introduced, which was based on a three stages model. This treatment approach resembles stress inoculation training, a highly effective form of cognitive behavioral treatment [16]. In the first stage, there is emphasis on information: understanding the origin and cause of the loss of control. Patients are also stimulated to do more non-demanding daily activities. In the second stage, patients are asked to draw up an inventory of stressors and to develop problem solving strategies for the causes of stress. In the third stage, patients put these problem solving strategies into practice and extend their activities to include more demanding ones. The patients' own responsibility and active role in the recovery process was emphasized and the same goes for the importance of an early start of the intervention aimed at the acquisition of coping skills and at regaining control. Problem solving activities according to a time contingent scheme were educated and practiced. The OPs were free to choose the specific tools for use in each phase of the process. The training course was given by four persons: an experienced OP/psychologist, a psychologist/therapist, an experienced general practitioner/researcher on emotional distress, and a psychiatrist.

Co-interventions cannot always be avoided. In case of post traumatic stress disorders, patients were also referred to a specialized trauma centre according to a special protocol of the DGVP. As the rehabilitation centre DGZ worked with multiple disciplines, it is possible that some patients in the control group received a combined intervention for mental and physical complaints by a psychologist and a physical therapist. In both the intervention and control groups co-interventions were registered in the medical files of the OHS and the database of the DGVP. These data can be used to adjust for co-interventions in the final multivariate analyses.

As this is an effectiveness trial, we tried to mimic a realistic situation in both treatment groups. Therefore, no activities were undertaken to improve the actual treatment compliance by the OP with the allocated treatment. Treatment compliance of the OPs was examined by measuring guideline adherence and assessing the proportion of referrals by the OP in both groups to the psychologist of DGZ. Patient compliance of the treatment was examined by registering no shows of patients during consultations with the OP were, as this may give information about the willingness of the patient to adhere to the treatment.

As randomization was done on patient level, OPs which were trained in the guideline treated all participants. Obviously this situation created a risk of treatment contamination between the groups. The trained OP treated an employee in the intervention group according to the guideline, as far as this happens in practice. The same OP treated an employee in the control group with minimal involvement and if applicable, direct referral to a psychologist. A cross-over learning effect may have happened in the control group, since the OP can adhere to the guideline in this group as well. The other way around, the OP may have referred an employee in the intervention group to a psychologist as well. The guideline promotes this in case of stagnation in recovery or in case of severe mental health problems of the employee. However, we tried to maximize the contrast by creating a situation in which referral to the psychologist in the control group was always granted by the insurance company (DGVP).

Return to work (RTW) was chosen as the primary outcome in this study [22]. A follow-up time of one year after inclusion was chosen, as effects of the intervention on return to work were expected to happen in this period. The RTW-outcomes are visualised in a time line in figure 2.

The primary outcome measure described in our study is full RTW: i.e. duration of sick leave due to mental health problems in calendar days from the first day of sick leave to full return to work in own or equal earnings (table 1). In addition, the net return to work to own or equal work was measured. This is the net duration of sick leave due to mental health problems in hours of full work absenteeism from the first day of sick leave to full return to work with own or equal earnings. The difference with full RTW is that the hours of partial return to work and % contract working hours (36 hours = 100%) are converted into the number of hours of full work absenteeism [23]. This may result in a more precise measure of RTW, when compared to full RTW. As figure 2 shows, net and full RTW consider the time period between T0 and T3, in which data collection by questionnaires took place (figure 1).

Another primary outcome variable that is part of the RTW process is first RTW: i.e. the duration of sick leave due to mental health problems in calendar days to first (partial or full) return to own or equal work. Other variables are related to recurrences of sick leave periods in the one year follow-up (T4)(table 1). These variables are the time in calendar days until the first recurrence of sick leave takes place and the number of and days during recurrences [21]. Total days of sick leave during follow-up is a primary outcome variables as well.

There was double registration of sick leave data, as both the employer and the OHS have their own registration system. The aim was to compare data of both systems, with the sick leave data of the employer as the 'golden standard'. This additional effort was done as reliable sick leave data are hard to get, since there is a known discrepancy with self-reported sick leave [24, 25].

Treatment satisfaction is a relevant outcome measure in occupational health care and therefore another primary outcome [26]. Higher patient satisfaction is related to better patient compliance and can improve the quality of OHSs. To date, most researchers assume that patient satisfaction is best defined as a patient's evaluation of aspects of a health care service based on the fulfilment of patient expectations. Since patients, employers and health care providers (OPs) are all involved stakeholders in the RTW process, it is important to measure the treatment satisfaction of all of these stakeholders.

Patient and employer satisfaction were measured using a short version of the Patient Satisfaction with Occupational Health Professionals Questionnaire [26]. This questionnaire was designed specifically for measuring satisfaction with occupational health care. It was designed in previous research on the quality of rehabilitation of cancer survivors [27] and transferred to occupational health care in another study on employees with mental health problems [18]. The 13 items of this questionnaire refer to (a) satisfaction in general (2 items), (b) interpersonal approach (4 items), (c) communication manner (2 items), (d) professional knowledge (5 items), and (e) total satisfaction of the treatment by the OP (13 items). Respondents answered on thirteen statements on a 5-point Likert scale: 'totally disagree – disagree – no opinion – agree – totally agree' (table 2). Because a higher score indicates more treatment satisfaction, item 3,4,9,10 and 12 will be recoded. The patient satisfaction questionnaire was adapted to the situation of the employer, to measure the treatment satisfaction of the supervisor (table 3).

To measure treatment satisfaction of the OPs, OPs filled in an evaluation questionnaire for each employee treated. This questionnaire consisted of 6 items, the first 4 referring to possible barriers in the RTW process and the last 2 items referring to the treatment success of their OHS (table 4).

Cost-effectiveness of the intervention is a secondary outcome and was evaluated from the employers and the health care insurance company's perspective (expenditures for the employer and insurance company, respectively), as they are responsible for covering the costs of sick leave and treatment [28]. Direct costs of health care treatment are (table 1): (a) consultations of OPs and other OHS-professionals, (b) consultations of the psychologists from DGZ, (c) consultations of general practitioners, (d) consultations of a psychiatrist and/or psychologist and/or alternative therapist not participating in the study, and (e) medication related to the treatment of mental health problems [29].

Indirect costs are not related to health care, but are costs as a consequence of absence from work because of sickness: sick leave, disability and or death of productive persons. Costs of lost productivity caused by (partial) sick leave due to mental health problems were calculated from the net number of days of sick leave and lost earnings, as provided by the employer. Since our study took place in occupational health care and since most costs were caused by sick leave, extra efforts were made to gather reliable data on sick leave [30].

To measure mental health complaints at baseline in this study, the Depression Anxiety Stress Scales (DASS) were used [33]. The structure of the DASS seems to support the view that both anxiety disorders and depression need to be distinguished from adjustment disorders in spite of their communality. The psychometric properties of this instrument appear to be sound enough to be applied to both healthy and psychiatric populations. Therefore, the psychometric properties of the DASS are suitable for use in an occupational health care setting. Moreover, convergent and divergent validity have been shown to be satisfactory [34].

The employees participating in this study filled in a self-report questionnaire that comprises the DASS-42, which takes 7 minutes to complete. The DASS-42 consists of 42 symptoms divided into three subscales of 14 items: depression scale, anxiety scale, and stress scale. Participants rated at baseline the extent to which they had experienced each symptom over the previous week on a four point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time).

Based on the results of their study on employees with mental health problems in occupational health care, Nieuwenhuijsen et al. [34] developed cut-off scores to divide the DASS-rates into four categories: stress, depression, anxiety, and depression/anxiety. The cut-off scores are > 12 on symptoms of depression and > 5 on symptoms of anxiety.

The HADS is a 14-item screening scale that measures the presence of anxiety and depressive states [35]. It contains two 7-item subscales: a depression subscale and an anxiety subscale, each item being scored on a four point Likert scale (0–3) that applies to the previous week. The HADS has been developed as a screen for detecting depressive and anxiety disorders in hospitalised patients. Items referring to symptoms that may have a physical cause (for example, weight loss or insomnia) are not included in the scale. Because a higher rate indicates more mental health symptoms, item 1,3,5,6,8,10,11 and 13 will be recoded.

The HADS is easily administered as a self-report measure as it usually takes 3–5 minutes to complete. A total score (out of a possible 21) for each subscale is then calculated. Zigmond et al. [35] recommended cut-off points with scores less than eight on either of the two subscales to be non-cases and scores between eight and ten as borderline cases.

The aim of this study is to examine the effectiveness of the management by Dutch OPs, under the expectation that (training in) the guideline will lead to additional skills for the OP and consequently to positive outcomes. To explore our hypothesis that the guideline leads to additional skills and outcomes, we examined the performance by the OP according to the guideline (guideline adherence). Guideline adherence by the OP was checked by means of an audit of the medical files. Guideline adherence was defined as the total score on ten validated performance indicators for the treatment of each participant by the OP (table 5)[11, 18, 36]. For each performance indicator, we used validated criteria. If a criterion was not met, the case was assigned 1 for that performance indicator. If all applicable criteria for a performance indicator were met, the resulting score was 0 for that case. The medical files of all the participants were assessed on if they met the criteria of the different indicators (0=adequate care; 1=deviant care). In this way an average performance rate was obtained for each performance indicator. Furthermore, a total score of all performance indicators was calculated (guideline adherence). Guideline adherence was dichotomized into adequate adherence and deviant adherence. Adherence was considered deviant if three or more performance indicators were assigned a score of 1, and adequate if less than three performance indicators had a score of 1.

Additionally, this audit gave us information about treatment compliance by the OP (guideline adherence) in both groups. In this way contamination between the study groups was studied as well. Adherence in the intervention group was considered compliant if there was adequate adherence. Guideline adherence in the control group was considered compliant if there was deviant adherence.

The performance indicators will be assessed on their criteria by three independent researchers, resulting in a dichotomised score on guideline adherence for each employee (adequate versus deviant).

The study population, Dutch police employees, has a higher risk of getting into stressful situations than regular workers [19]. This is caused by a relatively high workload and emotional pressure, and to a certain extent this reflects that police employees have other occupational risks than the regular worker population.

Additionally, the Dutch police workforce underwent two big reorganizations in the last ten years. This resulted in some negative consequences as problems with the internal communication, especially between employee and employer [39]. Because of the Volendam fire in January 2001, the police department Zaanstreek-Waterland had been exposed to extra professional risks on mental health problems and traumas one year before the start of the study [40]. As the study population will not be fully representative of the general working population, external validity of study results may be limited and caution has to be taken in generalizing the results.

Unfortunately, we cannot rule out selection bias. This may have occurred as the OPs were asked to select employees to participate in the study. Because the treatment of the OP depends on the randomization, an OP could have been tempted to forfeit the randomization procedure. As mentioned earlier, a check was made to detect this possible selection bias, and eventual irregularities would or will be noticed. Also, selection bias may have been introduced because participants completed the mental health symptoms questionnaires (DASS and HADS) after the consultation of inclusion. The consultation with their OP might have changed their point of view on their mental health problems and consequently, their response behaviour.

Still, these disadvantages regarding the ability to generalize our study results do not outweigh the advantages of this study population. This project was developed for the occupational health care setting with its typical case load of stress-related mental disorders, and not for specialized care, in which patients have more clearly defined mental disorders. The police is an organization with a relatively high incidence of common mental health problems and is therefore an interesting and representative target population. In addition, the police is a homogenous population that was treated in a confident manner, has a uniform sick leave registration, is connected to one insurance company, and has a well-defined 'usual care'.

In this pragmatic study design we examine the effectiveness of occupational health care by OPs who are trained in a practice guideline, compared to usual care. The best situation would be to randomize on physician level and patient level. Due to the limited number of participating OPs this was not possible, hence randomization was done on patient level. Consequently, there was a risk of treatment contamination between the groups. However, contrast between the treatment groups was maximized by free access to the psychologist in usual care as this was granted by the insurance company (DGVP). Treatment adherence will be examined by per-protocol analyses of guideline adherence by the OP. Furthermore, the current method allows us to consider the pragmatic effectiveness and to avoid much interference with daily practice of consulting hours.

In our cost-effectiveness evaluation we will not consider productivity loss due to sick leave prior and after the episode of sick leave due to mental health problems as proposed by Brouwer et al. [41]. Considering productivity loss prior and after the episode of sick leave can lead to an increase in estimated production losses of about 16%. In this study productivity loss during the RTW process has been measured.

Productivity loss is not only influenced by the cause of sick leave, but also by the type of work. Some jobs can only be performed in case of full functioning. Police employees, for instance, are called off sick leave only when they can perform all necessary tasks. In all other cases they are still on sick leave, for instance if they have a restriction in work on the street or wearing a gun. Since the start of our study in 2002 better methods in calculating costs have become available [30]. The availability of instruments for the measurement of productivity losses in recent years will give a better estimate of costs in this study.