The e-IMR was tested in an exploratory multi-centre cluster randomized controlled trial. According to the Medical Research Council guidance [
9], an exploratory trial evaluated an intervention before testing it in a confirmative trial. In this study, a cluster was a subdivision of a mental health institute. The cluster randomization prevented contamination between the intervention and control group participants. Data were collected at baseline, halfway and endpoint. The inclusion period was between January and October 2015. Data collecting lasted until October 2016.
Eligible clusters delivered the IMR-programme as a whole package with an experienced trainer-couple meaning that at least one trainer completed the IMR-total-training organized by the Dutch IMR-network and executed at least the first five modules of the IMR-programme before starting the IMR-programme in the trial.
Intervention
On top of this care as usual, participants in the intervention group had the opportunity to use the e-IMR intervention [
8]. The e-IMR intervention started with a ‘welcome page’ explaining the use of e-IMR and leading participants to the 11 modules. The e-IMR intervention included the same fill-in forms as in the hard-copy version of the IMR-programme. E-IMR added illustrative videos showing peer testimonials to encourage participants to talk more freely about themselves and to take steps in their recovery process. E-IMR also added problem-solving forms at the end of each module, registration of successful coping strategies, and a symptom-monitoring page.
The e-IMR was introduced to the trainers and participants of the intervention group by the first researcher in the second group session. Individuals who did not provide informed consent were allowed to join the e-IMR without participating in the research. The trainer-couples were supported in learning how to support participants in the use of e-IMR; how to install e-IMR on a computer in the session room and how to use e-IMR during the sessions.
In e-IMR, the registration forms of successful coping strategies and the symptom-monitoring page were introduced after the second module ‘practical facts about mental illnesses’. Weekly emails with a link to the e-IMR platform led the participants to the symptom-monitoring page. After closing each module, one of the trainers gave feedback to the participants via the platform and guided the participants to the next module.
Outcome measures
At baseline, independent demographic and clinical characteristics were recorded. At all three time points, six dependent outcome domains were gathered.
At T0 the following participant characteristics were collected: age, gender, physical comorbidities, treatment history, cultural background, social economic status, education level, computer/Internet availability and use. At T0, the participant’s case manager provided their diagnostic classification according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition.
The participant’s ability to manage their illness was measured with the consumer version of the
Illness Management & Recovery Scales (IMRS), consisting of 15 items [
13]. The response anchors, on a five-point Likert scale (1–5) vary depending on the item. The IMRS total-up score ranged between 15 and 75. The IMRS’ Cronbach’s alpha is .55–.83 [
14‐
17].
The participants’ self-management ability, which refers to the individual’s knowledge, skill and confidence for managing his/her own health and healthcare, was measured with the
Patients Activation Measure (PAM-13) [
18], consisting of 13 items. The response anchors on a five-point scale, vary from not applicable (0), ‘strongly disagree’ (1) to ‘strongly agree’ (4). The term ‘doctor’ in the items five and six was explained as their mental health clinician, which includes a nurse and/or case manager. Raw scores were transformed into standardized activation scores ranging between 0 and 100. The PAM-13’s Cronbach’s alpha is .84–.88 [
19‐
22].
The Mental Health Recovery Measure (MHRM) assessed the participants’ progress in their recovery process. The MHRM consists of 30 items with response anchors, on a five-point scale, varying from ‘strongly disagree’ (0) to ‘strongly agree’ (4), and ‘neutral’ (2) in between [
23]. The MHRM total-up scores ranged between 0 and 120. The MHRM’s Cronbach’s alpha is .93 [
24].
The participants estimated the level of burden of symptoms they experienced using the
Brief Symptom Inventory (BSI), consisting of 53 items [
25]. The response anchors, on a five-point scale, vary from ‘not at all’ (0) to ‘extremely’ (4). The mean BSI scores ranged between 0 and 4. A negative time trend for the BSI means a reduced level of burden. The BSI’s Cronbach’s alpha is .96 [
26].
The participants’ subjective satisfaction with life was measured with the
Manchester Short Assessment of quality of life (MANSA), consisting of 12 items [
27]. The response anchors on a seven-point scale vary from ‘couldn’t be worse’ (1) to ‘couldn’t be better’ (7). The mean MANSA score ranged between 1 and 7. The MANSA’s Cronbach’s alpha is .81 [
28].
The participants’ general health status was measured with the
Rand 36-item Health Survey (Rand-36), consisting of eight subscales: physical functioning (Rand-PF), social functioning (Rand-SF), role limitations due to a physical (Rand-RLPP) and an emotional problem (Rand-RLEP), mental health (Rand-MH), vitality (Rand-V), pain (Rand-P), and general health perception (Rand-GHP) [
29]. The response anchors vary between yes/no to Likert scales with three, five, and six options. Raw scores of all the concepts were transformed into scores ranging between 0 and 100. The Cronbach’s alpha of Rand-36’s eight concepts are .71 and .92 [
30].
The extent of participants’ activity on the e-IMR platform was determined by counting the number of completed modules and number of log-ins. An e-IMR user is identified by having completed at least module one or having logged in at least five times. Users were regarded as having had the opportunity to benefit from the e-IMR.
As in other studies on IMR [
31], participants who attended the face-to-face IMR programme sessions less than 50% were considered to be non-completers. In our study, this resembles stopping the IMR programme before T
1.
Statistical methods
The Statistical Package for the Social Sciences®.23 [
32] was used to carry out the analyses. Mixed model multilevel regression analyses were used to examine the main effects on the outcome measures, taking into account clustering of participants and repeated measures. This method automatically uses the ‘missing at random’ assumption to handle missing data. Random effects on cluster, trainer-couple, and individual participants nested within the cluster were included in the model. Model 1 included fixed main effects for time trend and group. The analyses were executed according to the intention-to-treat principle to prevent bias caused by the loss of participants [
33] and to reflect the normal practice [
34] of high attrition rates in treatments of people with SMI [
7] and e-health [
5].
Post hoc analyses of effect differences were performed to control for covariates. We considered the covariate gender to be a potential confounder because of its known differences in exposure and reactions to stress and health [
35]. The covariate was included in model 2, controlling for confounding time trend effects.
In model 3, covariates were included that were expected to interact with the effect differences.
Non-completion of the face-to-face IMR-programme sessions was expected to interact with the effects because being a non-completer is correlated to lower functioning; for instance, lower social functioning [
36] and higher emergency room visits and hospitalization [
37]. In addition, we searched for correlations in T
0 scores between the groups of completers and non-completers.
Because of the known low adherence-rate to Web-based interventions [
5], additional subgroup analyses were performed within the intervention group to investigate whether actual use compared to non-use of the e-IMR leads to outcome differences. Thus in model 4, two groups of e-IMR users and non-users were included according to the aforementioned adherence measurement.