Process evaluation of a tailored work-related support intervention for patients diagnosed with gastrointestinal cancer

This process evaluation was part of the GIRONA study (Gastro Intestinal cancer patients Receiving Occupational support Near and After diagnosis), which included a multicentre Randomised Controlled Trial (RCT). The full study design of the RCT was published previously [27].

For the process evaluation, we collected qualitative and quantitative data on the process indicators using (1) GIRONA study follow-up questionnaires completed by the patients, (2) checklists completed by the healthcare professionals during the intervention meetings, (3) research logbooks completed by the research team during the GIRONA study and (4) a questionnaire on the substantive process outcomes after the intervention was performed/received, completed by both the patients and the healthcare professionals.

Patients in the GIRONA study were diagnosed with GI cancer to be treated with curative intent, aged 18–63 years, had paid work at time of diagnosis and were on sick leave. The healthcare professionals who participated in the GIRONA study included oncological GI nurses from the participating hospitals and oncological occupational physicians (OOP). These OOPs had undergone an accredited, specialised training programme that included a theoretical module and a traineeship, which prepared them to support patients diagnosed with cancer. These OOPs were not affiliated to the hospital, in which the patient received treatment, nor to the patient’s own workplace.

Prior to the start of the GIRONA study, the oncology nurses attended a training session, during which they learned about the study’s objectives and activities, certain processes of good clinical practice research (e.g. the informed consent procedure), the problems faced by patients diagnosed with cancer regarding work/RTW and basic knowledge about the Improved Gatekeeper Act, which takes effect when an employee/patient is granted sick leave. The training session lasted about 2–2.5 h; it was given by the researcher [AZ] at the participating hospitals.

The OOPs were informed about the GIRONA study’s objectives and activities.

Patients were asked to participate at the hospital where they were receiving treatment. The oncologist or oncological nurse checked whether the patient was eligible for this study during the first visit to the hospital. A brief explanation was given and the patient was asked whether one of the researchers of the project team could contact them by telephone.

Patients who agreed to participate signed an informed consent form and subsequently received the baseline questionnaire. When the questionnaire was returned, randomisation took place and patients were informed about the outcome of the randomisation process. If they were randomly allocated to the intervention group, the decision diagram was filled out by the one of the researchers of the project team, based on the data from the baseline questionnaire. The healthcare professional was informed so she/he could make an appointment for the first meeting.

To outline the process evaluation, we adapted the model of Linnan and Steckler [21] and used the following key process evaluation components: recruitment, context, reach, dose delivered, dose received and fidelity.

The data on reach, dose delivered, dose received and fidelity were described using SPSS (IBM 24.0 statistics) descriptives, namely numbers and percentages supplemented with mean and standard deviation, if appropriate.

Twenty-five Dutch hospitals were contacted by email and phone, followed by a face-to-face meeting to explain the study and discuss participation. Ten hospitals declined to participate. Reasons given for non-participation included ‘limited time’, ‘no nurses/research nurses available’, ‘shortage of medical staff’ and ‘interference with other ongoing research’. The patient inclusion period was extended twice, in agreement with the local investigators. Because the date of onset varied between the hospitals, the extension time also varied between the hospitals. Patients’ reactions to the timing of their introduction to the study (i.e. at the moment they were diagnosed) were of qualitative character and ranged from ‘At such a moment any help is welcome’ to ‘Too early, as you are more inclined to participate in anything because you are very uncertain about everything at that moment’.

In total, 15 Dutch hospitals (one of which has two sites) participated, located in North and South Holland, Utrecht, North- Brabant, Limburg, Gelderland, Flevoland and Groningen. Academic medical centres and peripheral hospitals were included. Due to the local medical ethical approvals of the participating hospitals, not all started including patients at the same moment. Patients were included between July 2015 and May 2017. One hospital did not include any patients, which was added last to perform the RCT. No patients covered the inclusion criteria of this study during the time left of the inclusion period.

In total, 33 nurses attended a training session; of these nurses, 13 (39%) actually held work-related support meetings. All nurses who participated were specialised in gastrointestinal cancer; their job functions ranged from oncological nurse, nurse practitioner, research nurse to stoma care nurse. Seven OOPs, distributed over the 15 hospitals, participated in the study. Five (71%) of them held work-related support meetings.

Thirty-eight patients with GI cancer participated diagnosed with the following: small intestine (N = 1), colon (N = 30), rectal (N = 6) and anal (N = 1). The average age was 54 ± 8.1 years. Twenty-three of the patients (61%) were male (Table 1). At baseline, 42 patients were randomised to the intervention group. However, four of these patients were not included for further analysis. Reasons were entering palliative trajectory before intervention meeting was planned (N = 1) and first meeting was not planned before start of treatment (N = 3). These four patients did not wish to pursue the intervention meetings.

Based on the findings from two hospitals, which recorded the eligible patients for the study correctly and comprehensively during the whole study period, we used this information to describe the reach.

In these two hospitals, we included a total of 18 patients. Sixteen other eligible patients did not participate. Reasons for non-participation included the following: already participating in another study, treatment had already started, having no work-related problems or support for sickness absence was well organised at work. Therefore, the reach of participating patients was 47%.

The nurses were asked if they had an opinion on the fact why fewer patients were included than expected (approximately 20 patients per hospital). Some of the reasons given were as follows: ‘population screening (colorectal cancer) mainly affects patients older than 60 years’, ‘lack of time to include patients’, ‘timing of inclusion is not the moment when patients are in the process of returning to work, and patients who do not expect any problems in advance, because they have a good relationship with their employer, often reject study participation’, ‘some mistrust, afraid to burden the employer and then perhaps experience consequences of that’ and ‘patients had already arranged things well at work’.

Thirty-eight patients were included, namely 20 (53%) who had been referred to support type A and 18 (47%) who had been referred to support type B. For 35 patients (89%) (two of the 38 patients were no-shows, and for one, the meeting took place after treatment and thus was considered the second meeting), the first meeting was held face to face (Table 2). The average duration of the first meeting between patient and nurse was 30 min (range of 15–60 min). The first meetings with the OOPs lasted for an average of 55 min (range 30–90 min) (Table 3).

To tailor the support to the patient, patients and healthcare professionals could discuss after the first meeting whether another type of support in the intervention would be more appropriate. One patient was referred from support type B to support type A.

Thirty-two patients had a second meeting; 18 patients received (56%) the second meeting face to face (Table 2). According to protocol, the decision diagram had to be completed after the second meeting. The researchers of the project team received the checklist completed by the healthcare professional after 30 of the 32-s meetings (94%) (18 support type A/12 support type B). Accordingly, 19 (11 A/8 B) decision diagram forms were used. One patient was referred after the second meeting from support type A to support type B. Reasons for not returning the decision diagram form included the use of the wrong form during the second meeting (N = 1 in the support type B group), the follow-up meeting was cancelled due to the nurse taking maternity leave (N = 1 in the support type A group) and no follow-up meeting was needed (N = 3, all in the support type B group).

Seventeen patients had a third meeting of whom 11 patients received (65%) the third meeting face to face (Table 2). For 19 patients, a third meeting was not indicated, according to mutual agreement between healthcare professional and patient. The OOP needed more minutes for all their meetings compared with the nurses (Table 3).

According to protocol, at least the first two meetings should have been held, unless it was indicated that no second follow-up support was needed. Of the 38 intervention group patients, 33 (87%) had the two first meetings; three of the 38 patients did not need the second meeting.

The protocol also stated that the first meeting should be held before treatment was started. For 32% of the patients receiving support type A and 13% of those receiving support type B, this meeting was according to protocol. The timing of the other two meetings was realised according to protocol, with ≥ 80% for both types of healthcare professionals in both meetings (Table 4).

In order to conduct the intervention correctly, it was of importance to go through all the meeting items (ESM Appendices 1 and 2 for an extensive description of these items) on the checklist. The nurses had to complete 11 items in the first meeting, 14 in the second (or 12 if a third meeting was not necessary) and 10 items in the third. The OOP had to complete 13 items in the first, 15 (or 13) items in the second and 11 items in the third meeting (Table 5).

The process evaluation questionnaire was completed by 23 (61%) patients, six nurses who held no intervention support meetings, eight nurses who provided support and four OOPs. From the patients’ responses to the open-ended questions, we concluded that the intervention was mainly regarded as positive. From 22 responses, 11 patients experienced the intervention as useful. Fourteen (64%) patients answered the question about the general satisfaction with scores 1 and 2 (1 = very satisfied, 5 = very dissatisfied). Nevertheless, two (9%) were very dissatisfied with the intervention, stating that there had been no work-related support. The items that were discussed during the intervention meeting were scored as good by 18 of 22 patients (Table 7 in the supplementary file).

Nurses scored the training they had received before the study as either good (71%) or sufficient (29%). In response to the open questions, however, two said that they would have liked a refresher course.

The OOPs were unanimous that the time available for the first meeting was too short to provide this kind of support, while most (75%) of the nurses considered the duration to be just right (Table 8 in the supplementary file).

All nurses and OOPs stated that everything was clear before the study started and that it was easy to follow the study protocol except for one OOP, who had logistical problems.

The total overall scores for fidelity were 78% for the nurses and 44% for the OOPs. The duration of the three meetings held by the OOPs was scored as not sufficient (Table 8 in the ESM), while the nurses completed all three meetings within the pre-defined average of 30 min. For most nurses and OOPs (84% and 94%, respectively), the first meetings were held face to face. Giving the first meeting before treatment was started was a challenge for both nurses and OOPs: the nurses and the OOPs managed to hold the first meeting before the start of the treatment in only 21% and 13% of cases, respectively (Table 6).

In our study we had an inclusion rate of 47% (process evaluation component ‘reach’), which can be considered a low percentage. Low or high inclusion rates have not be defined in the literature; however, similar ‘low’ rates were reported in other studies, all of them under 55% [23, 29‐31]. A study by Tamminga et al. [23], regarding a hospital-based intervention that was targeted at patients diagnosed with cancer, reported an inclusion rate similar to that in our study. Another hospital-based study involving occupational counselling combined with physical exercise for patients diagnosed with cancer [25] reported a higher rate of participating patients (77%). Nevertheless, the study by Velthuis et al. [31], which also included an exercise programme intervention, reported a low inclusion rate of 40%. Although the same types of patients (e.g. diagnosed with cancer to be treated with curative intent) were included, the inclusion rates varied between the abovementioned studies. A possible reason for the low inclusion rate mentioned in the study by Velthuis et al. [31] is that patients declined participation before randomisation, because detailed information about the intervention was given after this step. In our study, however, we provided detailed information before randomisation, so this cannot explain our low inclusion rate. Unfortunately, we were not able to give the reasons of the non-participants. This caused by the fact that patients, by law, were not obliged to give a reason. However, these reasons would have been of added value to understand the process for those who possibly already received work-related support. Interestingly, in the study by Leensen et al. [25], the majority of the patients were diagnosed with breast cancer (N = 78, 84%) compared with a minor group of colorectal cancer (N = 8, 9%). In our study, we only included patients with gastrointestinal cancer and despite the population-based colorectal screening programme, we did not include as many as we expected. De Boer et al. [18] concluded that in this group of patients, work-related problems are already acknowledged at the moment of diagnosis. Their survey study included 333 patients who were diagnosed with GI cancer in one 7-month period at one hospital. Of those patients, 95 (28%) were employed or self-employed at the moment of diagnosis. However, in our study, we maintained other inclusion criteria, including an age of 18–63 years. In the Netherlands, the incidence rate of GI cancer in both men and women increases at the age of 64 [1], whereas we only included people up to the age of 63, which could have caused the low inclusion rate. Another reason for the low inclusion rate could be that in our study, patients were asked to participate at the moment they were diagnosed with cancer. Not all eligible patients were informed about the study by the nurses or physicians, possibly because they had limited consultation time to inform patients. Another reason could be that it is considered unethical to ask patients to participate at the moment of diagnosis. As Hubbard et al. [32] noted, the priorities in the clinic limit this kind of research. Along with Tamminga [23], and in line with the responses of the nurses in our study, we think that participation in an intervention will improve when the additional steps entailed by research requirements, e.g. information about the study, reflection period to take part in the study and contact about the other research procedures including informed consent and questionnaires, are no longer necessary.

We believe that this intervention is of value, which was also noted by healthcare professionals and patients. Thus, despite the low inclusion rate, the data derived from this study reflect the importance, feasibility and usefulness of the tailored work-related support. However, further research is needed.

Another process evaluation component to discuss is fidelity. Other intervention trials that concerned work support showed considerably higher fidelity scores than found in our study. As shown in Table 8 in the ESM, the total score for the nurses reached a score of 78%, which is somewhat comparable with the study by Tamminga [23], who assessed that on average, the nurses performed 85% of the study items. The total fidelity score for the OOPs in our study was considerably lower, namely 44%.

In the study by Leensen et al. [25], the total average fidelity score was 85%, which also included OOPs who performed some items. Conversely, in our study, of the nine (100%) fidelity score items, three were about the duration of each meeting, which per protocol was set at 30 min. All three items were scored as insufficient (33%) which is the main reason for the fidelity score for the OOPs. The nurses reported the pre-defined 30 min as acceptable: they knew their patients, because they had followed the patient journey from the start of diagnosis, so less time was needed to explore the patients’ history. In addition, the cap of 30 min for the meeting was feasible for the nurses besides their other daily activities in clinical practice. However, the OOPs needed more time to ‘get to know’ their patients (medical diagnoses, treatment and work-related outcomes/problems). Second, the longer time needed by the OOPs could be a result of the complexity of the work-related problems for which the patients were referred to them in the first place. This fidelity score was pre-established; however, on reflection, 30 min should have been the minimum number of minutes for a meeting. Although the 44% fidelity score shows that the OOPs did not follow the protocol, they did spend enough time with the patients discussing complex work-related problems, which the patients appreciated. It would therefore be appropriate to adjust the time available/needed for the meetings in the tailored intervention for the OOPs. Furthermore, research is needed to establish for which meeting (i.e. before the start of treatment or after treatment) needs more time.

The GIRONA intervention initiates tailored work-related support starting before treatment. It is known from earlier research that patients at the moment of diagnosis experience work-related problems [18]. The OOPs in our study consider it important to start early with work-related support, which is also underlined in other studies [33‐36]. In our intervention, the first meeting was to be before the start of treatment, because it would inform the patients about what kind of effects the treatment could have on their work capacity, as also suggested by McGrath et al. [36]. Conversely, from the findings of this process evaluation, the question is whether it is the right moment (i.e. before treatment is started) to start work-related support. As presented in this study, the support was given before treatment in only 32% of the cases in the support type A group and 13% of the cases in the support type B group. So, time from diagnosis to start of treatment varies, but it is possible that patients start their treatment within 3 weeks. Based on the results of this process evaluation, it is a logistical challenge to provide the support before treatment. Patients’ thoughts about the timing were asked for in the process evaluation questionnaire and the majority scored this as ‘right timing’ (Table 6). However, these results must be interpreted with caution, because the patients answered the question about the timing of the meeting for the moment they actually received their work-related support. As presented, in most cases, this was after treatment. The patients were satisfied, but on the other hand, we have no information concerning patients’ satisfaction when they had the first meeting before treatment as per protocol.

A strength of this study is that we derived data from various sources and participants: GIRONA questionnaires with self-reported outcomes of patients, checklists completed by healthcare professionals during the intervention meetings, research logbooks completed by the research team and questionnaires on process outcomes after the intervention was performed, completed by both the patients and the healthcare professionals. This resulted in a full perspective of the process evaluation of all participating stakeholders. In addition, the study was analysed in accordance with the process evaluation key components of the Linnan and Steckler framework [21], a model that is frequently used to define and systematically report a process evaluation. We used the six most relevant key components; the seventh—implementation—was not included in our study because it is a combination score of reach, dose delivered, dose received and fidelity, which was not the main focus of our study.

A limitation of the study is that the patients who received the work-related support were spread over 15 hospitals, which in itself is not a problem if the protocol is followed. However, due to the few patients per hospital who were randomised to receive work-related support, nurses did not have many patients with whom they had to hold the work-related support meetings. Considering that, and as a nurse indicated in our study, the nurses were not familiar with providing this kind of work-related support. It is therefore possible that the meetings were not conducted as intended, resulting in the lower completion percentage of discussed items during the meetings (Table 5). At the same time, the nurses stated that they were interested in supporting patients with work-related problems and, as one indicated, a brief refresher training just before the start of the study could have supported nurses in their new role.

Another limitation of this study is the moment of the process questionnaire at the end of the follow-up. This questionnaire was sent after all participants ended their follow-up, so for some patients, this took some longer time than others. This could potentially result in ‘recall bias’, as in that some patients were not able to accurately recall the details of the intervention.

Effectiveness outcome studies are well known to be executed. However, a process evaluation, just as important, helps to realise if an intervention is successful [21, 37]. An implication for future research is to include a process evaluation in the study project, to increase the value of an effectiveness study and place it in perspective.

The tailored component in our study requires improvement. Further research is needed to establish the optimal moment to provide support, and especially what is achievable in clinical practice. As we identified in this study, some patients received work-related support but did not need it, while others did not receive support but reported that they needed it. This could be linked with the timing of the work-related support given and the fact that patients do not realise that problems/work problems can arise after a certain time period, as a result of disease- and treatment-related side effects [36, 38, 39]. However, some patients simply did not need any work-related support, because proper support was provided at their workplace.

Moreover, it is necessary to define and develop the roles of healthcare providers in work-related support, as also previously mentioned by Bains et al. [40]. As a result of this study, we know that oncological nurses have an important role in identifying those who need work-related support in an early phase of the diagnosis with cancer. However, to optimise that, we first have to support oncological nurses in their work-related support role with training as a starting point, as also underlined by Stergiou-Kita [34]. To integrate the work-related support in the clinical setting, it is of importance that this kind of support achieves lasting acceptance by the ones providing it. However, intervention implementation and evaluation in general are a complex phenomenon [41, 42]. Finally, there remains a challenging hurdle in clinical practice: obtaining the time and financial support (work-related support should be recognised within the official reimbursement system) for both nurses and OOPs. For this study, ‘research’ time was allocated and a financial concession was financed from the research grant.