Main Findings
Our objective was to perform a process evaluation of the Care for Work intervention to assess whether the intervention was implemented as planned. We furthermore investigated the satisfaction of the patients with the intervention program. Overall, the implementation of the participatory workplace intervention was adequate. The implementation of integrated care was less successful. We will compare our findings with findings from other studies, although in the field of rheumatology, there are to our knowledge no process evaluations available on comparable interventions.
We were not able to determine the actual reach of our intervention. The rheumatologists did not have information about the work status of invited patients, as a consequence, the letter was also sent to patients within the specified age group, but without a paid job. This might explain the high number of patients who did not send back the reply care or send back the reply card indicating that they were not willing to participate (40 and 38 % out of all invited patients, respectively). Because we do not know the percentage of patients in our invited sample who had a paid job, we do not know how many of the invited patients actually belong to the intended target audience of our intervention, and we cannot calculate the actual reach. Work disability rates among patients with RA differ tremendously between studies [
28,
29], we were therefore also not able to make an estimation of the number of patients with a paid job in our invited sample. About half of the patients we were able to contact by phone, eventually did not participate. This percentage is comparable to another study in which a similar intervention was offered to workers with low back pain [
30].
The intervention was delivered to a lesser extent than was intended. Only 81.3 % of patients took part in the initial intake and started the intervention, compared to 92.5 % in a comparable study [
30]. The other intervention components could only take place when the intake was carried out, so because almost 20 % of patients did not participate in the intake, they did not start the intervention. The evaluations by the care manager were delivered in a few cases only. The workplace intervention was delivered to 85.3 % of patients who had started the intervention, and the evaluation was offered to more than half of the participants. In most cases, the supervisor was present during the workplace visit, which was a very important part of our intervention. In another intervention study which also consisted of, amongst others, a meeting between the worker and supervisor, it was found that this meeting only took place for 10 % of the participants [
31]. Eventually, the mean dose delivered for integrated care was 46.7 %, and for the workplace intervention 80.6 %. The low percentage for dose delivered was mostly due to the low delivery of the intake and the evaluations by the care manager.
We found that 69.5 % of the solutions proposed during the workplace intervention were implemented. This percentage is comparable to another study in which the participatory workplace intervention was evaluated (72 %) [
30]. In the comparable study, the intervention was offered to workers on sick leave. We therefore suspected the percentage of implemented solutions to be lower in our study, since one could argue that the need to implement solutions is higher and more urgent when a worker is on sick leave. Our study results show however, that the percentage of implemented solutions is comparable in a study sample of workers who are not on sick leave. It has been proposed before that the implementation of solutions might also be related to whether the workers suffers from a chronic disorder or not. The Lambeek study on chronic low back pain had an implementation rate for the solutions of 72 % [
30], while two other studies on (sub) acute low back pain had implementation rates of the solutions of only 50 % [
22,
32].
The fidelity score for integrated care was high, which means that of the intervention components that were delivered, the quality was good. The fidelity score for the participatory workplace intervention was a bit lower, but still 68.7 %, which is satisfactory. The communication of the care manager with other caregivers involved was executed poorly. Previous research has also emphasized that communication between an occupational physician and other care givers is poor [
33]. Given that this RCT was carried out in a controlled environment, the communication efforts executed are very poor. Thereby, the integration of the care offered to our participants failed, and hence the linkage of all care givers towards one treatment goal was not achieved. It has been described by previous studies that it is difficult to enhance interprofessional collaboration [
34,
35].
Despite the previously described shortcomings in the implementation of the intervention, patients were satisfied with the intervention. They felt taken seriously by the care manager and occupational therapist, and they rated them with high marks. The issue of trust in an occupational physician has been documented before [
36]. Although the care manager who delivered the intervention in our study was not linked to the employers of our patients, and hence was independent, our patients still had concerns about trust and confidentiality of the care manager.
Strengths and Limitations
This process evaluation provides insight into the implementation of an intervention program, consisting of integrated care and a participatory workplace intervention. We collected data for this process evaluation from both the patients, as well as the intervention providers. The components of this process evaluation were collected by means of self-reported data; patients filled out a questionnaire, and intervention providers wrote reports, and hence no objective data was collected. Furthermore, in this process evaluation, only quantitative data was collected. Qualitative data could add a more context-specific insight into the implementation, which would help to interpret our findings [
37].
Because we were not able to calculate the actual reach of our intervention, we cannot fully determine the representativeness of our study sample. In RCT studies, the study sample is generally not representative of the target group which might lead to bias, since typically, motivated patients participate in research projects. In our sample, especially men might be underrepresented. We based our fidelity scoring on medical records kept by the care manager and occupational therapist; we were not there during consultations. Therefore, we cannot rule out bias. Furthermore, we asked the care manager about their communication efforts by sending them a questionnaire. This might have led to socially desirable answers, since they were aware what their efforts should have been according to the protocol. Furthermore, there was one care manager who performed most intakes and evaluations. Therefore, a lot of the results depended on the skills of this specific care manager. The possibility or recall bias is negligible in our study design, since we asked participants about their experiences with the intervention shortly after the intervention.
We have not collected data about the reasons why our intervention, and especially integrated care, was not delivered as planned in the study protocol. In future research it is very important to study these issues, to overcome them. We found that communication between members of the multidisciplinary team was limited. To implement an intervention consisting of integrated care, it is important to find out why there was only little communication. If there are practical reasons for this, these barriers have to be addressed in order to improve dose delivered by the intervention providers.
Implications
This study shows that a process evaluation can provide essential information about the implementation of an intervention, which is vital if an intervention is to be implemented in practice. We made use of quantitative data. For future process evaluations, we recommend to use both quantitative and qualitative data. Qualitative data can add insight information and lead to an explanation of findings. Given that our intervention was not delivered to the extent we aimed for, we recommend to look critically to the intervention protocol with the intervention providers. Our intervention was not delivered to the extent we aimed for. Especially for the integrated care component, a large number of participants have not received the evaluations. It is important to discuss with the intervention providers why the evaluations were delivered only seldom, and why the intake was delivered to only 81.3 % of the participants. Whether this occurred by for example administrative issues, it should be addressed, and consequent adaptations to the protocol are needed. Furthermore, we were not able to integrate care towards our patients. There was only little communication between members of the multidisciplinary team. Previous research has also shown that communication between medical specialists is difficult [
33]. For medical specialists working under pressure, communication with other medical specialists might be very difficult to establish. If an intervention is to be implemented aiming to integrate care, opportunities for communication should be embedded in the daily practice of medical specialists involved, such as specific planned time points for conference calls.