Figures
Abstract
Objective
Illness Management and Recovery (IMR) is a psychosocial intervention with a recovery-oriented approach. The program has been evaluated in different settings; however evidence for the effects of IMR is still deficient. The aim of this trial was to investigate the benefits and harms of the IMR program compared with treatment as usual in Danish patients with schizophrenia or bipolar disorder.
Method
The trial was designed as a randomized, assessor-blinded, multi-center, clinical trial investigating the IMR program compared with usual treatment. 198 people diagnosed with schizophrenia or bipolar disorder participated. The primary outcome was the Global Assessment of Functioning (GAF-F) at the end of intervention and the secondary and explorative outcomes included severity of symptoms and service utilization.
Citation: Dalum HS, Waldemar AK, Korsbek L, Hjorthøj C, Mikkelsen JH, Thomsen K, et al. (2018) Illness management and recovery: Clinical outcomes of a randomized clinical trial in community mental health centers. PLoS ONE 13(4): e0194027. https://doi.org/10.1371/journal.pone.0194027
Editor: Ethan Moitra, Brown University, UNITED STATES
Received: July 14, 2016; Accepted: February 23, 2018; Published: April 5, 2018
Copyright: © 2018 Dalum et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are not publicly available as permission to release data was not gained by participants. This restriction has been imposed by the Danish Data Protection Agency, however this data can be requested by contacting: videnscenterfordataanmeldelser.rigshospitalet@regionh.dk.
Funding: This RTC was founded by the Danish Health Fund who supported the research team (the Ph.d.-project), Mental Health Centre Frederiksberg and Mental Health Centre Ballerup who supported the interventions and the research. And finally Capital Region of Denmark supported HSD in pH.d.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Illness Management and Recovery (IMR) is a curriculum-based rehabilitation program developed to help people with severe mental illness improve their illness self-management and to obtain a decreasing severity of symptoms, increase level of functioning, and achieve prolonged remission [1]. Previously, five randomized trials, investigating IMR have been conducted with varying sizes and design, among others four randomized on an individual level [2–5], whereas one used a cluster randomization design [6].
The findings from four of these trials indicate that IMR is effective in terms of clinical recovery measures. Participants receiving IMR showed significant improvements in psychiatric symptoms, that is positive symptoms, negative symptoms, symptoms of depression [2,4], in coping with symptoms [3], in psychosocial functioning [4], reduced hospital use over time [5], and less suicidal ideation [2] compared with participants who receive treatment as usual. The latest published trial investigating IMR versus an active control group reported that IMR was not significantly different from the control group on symptoms, medication adherence, or service utilization, but there were very low rates of retention in both the IMR and control groups over the study period [6]. The results from these trials should, however, be interpreted with caution. All trials lack one or more desirable aspects of controlled clinical trials, such as power and sample size calculation, outcome assessment by blinded assessors, or systematic approach to handling missing data, resulting in an increased risk of bias. Consequently, the intention with this trial was to add knowledge to the existing evidence of the IMR program by meeting the highest standards of methods in a randomized clinical trial.
The aim of this randomized trial was to investigate the effects of the IMR program compared with treatment as usual in Danish patients with schizophrenia or bipolar disorder. The following hypothesis was tested: Participants in the IMR program would score higher than participants receiving treatment as usual on the Global Assessment of Function scale (GAF-F) at the end of the 9 month intervention period. In this paper, the results on the clinical outcomes: functioning, symptom severity, social functioning, drug/alcohol abuse, and service utilization will be presented. Results regarding self-perceived recovery have been presented elsewhere [7].
Methods
Design
The trial was designed as a randomized, assessor-blinded, multi-center, clinical trial of the IMR program compared with treatment as usual in community mental health centers in the Capital Region of Denmark with a post-intervention assessment and a one-year follow-up; see the trial protocol [8].
Recruitment
Participants were recruited from three community mental health centers (CMHC) in the Capital Region of Denmark. To increase the number of participants enrolled in the study an additional community mental health center was added to the original two. The project was planned in collaboration with the heads of management at the first two of the three CMHC, all of which had shown commitment to implement IMR.
Participants
Eligible participants were at least 18 years old, receiving services from one of the CMHCs, diagnosed according to the ICD-10 criteria with schizophrenia or bipolar disorder, able to speak and understand Danish, and provided written informed consent. The Present State Examination [9] was used to verify the diagnosis prior entering the trial. Patients were excluded if they had a guardian or a forensic arrangement, or met the criteria for ICD-10 diagnosis of dementia or mental retardation, had an active substance use disorder, lived in a community residential home as treatment as usual is different for this group, or were involved in psychoeducation at the time of inclusion.
Randomization
After the baseline assessment, participants were randomly allocated to IMR plus treatment as usual or treatment as usual alone. To secure the concealment of the allocation sequence, the randomization was central and telephone-based through an administrative office unrelated to the research team. The allocation sequence was stratified by diagnosis and CMHC. The allocation sequence was computer-generated using permuted blocks varying in sizes of 6, 8 and 10. Randomization was done one at a time and on average the patients in the intervention group waited 87 days (SD 77) before the first group session was held and 62% of the patients waiting under 90 days before the first group session took place.
Interventions
The illness management and recovery program.
Patients randomized to the experimental intervention were offered group-based IMR plus treatment as usual (see below). IMR is a curriculum-based program organized in 11 modules which consist of: recovery strategies; practical facts about mental illness; the stress-vulnerability model; building social support; using medication effectively; drug and alcohol use; reducing relapses; healthy lifestyle; coping with stress; coping with problems and symptoms; and getting your needs met in the mental health system. IMR was provided in a group format with weekly one-hour sessions over a period of nine months. Ten patients were assigned to each group which was facilitated by two or three practitioners. IMR was conducted using a closed enrollment group format, such that once the group was started new participants were not enrolled into it (IMR has also been conducted using open group formats). Strategies such as motivational phone calls and in-hospital-sessions were employed to motivate participants to attend and to be engaged in the IMR program. A further description of the IMR program can be found elsewhere [8, 10].
Treatment as usual.
Participants randomized to the control group received treatment as usual only, which included individually adapted interdisciplinary treatment at the CMHC or in the patient’s own home. Treatment included medication, case management, group therapy (i.e. cognitive behavioral therapy), as well as unstandardized psycho-education. Staff members across the three CMHCs had comparable levels of education and years of experience working with people with severe mental illness. Every patient had a case manager who together with the patient planned the individualized treatment. The patient met with case manager at the CMHC or at home or attended other activities at the community mental health center approximately once a week.
Outcomes
Baseline assessments were conducted from February 2011 till December 2012 and follow-up assessments from March 2012 till December 2013. The primary outcome was global functioning post-intervention assessed by the Global Assessment of Functioning (GAF-F). The GAF scale can be divided into two scales GAF-F and GAF-S (one that focuses on functioning and one that focuses on symptoms) [11–13]. In this trial the focus of the primary outcome was functioning which is why the GAF-F scale was used. The secondary outcomes were symptom severity on Positive and Negative Syndrome Scale (PANSS) [14] and social functioning on Personal and Social Performance Scale (PSP) [15]. The PANSS has been validated in Danish [14] but not the PSP scale. Exploratory outcomes were symptoms on the GAF-S, depression on the Hamilton Rating Scale for Depression (HAM-6) [16, 17], mania on Young Mania Rating Scale (YMRS) [18], substance abuse (assessed by the case manager), and service utilization based on hospital records.
Assessment of implementation
The IMR Fidelity Scale [10] was used to assess fidelity to the IMR in all the groups. A multiple data approach was used including interviews, observation of the IMR group, an audit of the patient service records as well as audits of the IMR notes of progress. The fidelity assessments were made half-way through the program (after 4 months) and at the end of the intervention for each IMR group (9 months).
Power and sample size
Prior to recruitment a sample size of 200 participants was estimated sufficient for detecting a true difference in the IMR and control group of at least 6 points on the GAF-F. The few previous studies using IMR or elements of IMR, where the effectiveness has been assessed by using the total GAF score, have showed a difference from 6 to 10 points [19, 20]. Based on this knowledge we conservatively estimated the true difference in the experimental and control group means to be 6 points on the GAF-F score. The sample size calculation was furthermore based on a power of 80%, an alpha of 5% and a standard deviation (SD) of 15 points [21, 22]. For the secondary outcomes it was determined that the sample size of 200 participants was sufficient to test minimal clinically relevant differences with an alpha of 5% and a power of 80%. See the trial protocol for more information on power and sample size calculation [8].
Blinding
Post intervention assessments were conducted by assessors who were blind to treatment allocation. Two different assessors performed the assessments, and therefore the assessors rated together to reach agreement before rating patients individually to ensure inter-rater reliability. Analysis of these data showed an acceptable inter-rater reliability (S1 Text). Participants as well as the staff were strongly inculcated not to disclose the allocation status of the participant at the follow up assessment. The statistical analyses were conducted with the two intervention groups coded as A and B, and the Steering Committee drew the conclusions with the blind still intact. In order to secure this blinding the analyses exploring the interactions between degree of participation in IMR and outcomes were carried out after all the other analyses were completed.
Statistical analysis
The data analysis was based on the intention-to-treat principle. The analysis of difference between the two groups was conducted using analysis of covariance for the continuous primary and secondary outcome measures (GAF-F, GAF-S, PSP, and PANSS). This was estimated as marginal means adjusted for the baseline value of the variable in question [23]. Independent sample t-tests were conducted for the variables concerning substance use, service utilization, that is number and length of hospital admission and number of emergency service visits together with number of visit to the community mental health center (use of treatment as usual) and harm and adverse effect. As service utilization and harms and adverse effects were measured during the study period no baseline mean existed. As missing data is a potential source of bias a strategy of conducting an analysis using multiple imputations were decided beforehand [8]. An analysis of missing data was conducted for the primary and secondary variables GAF-F, PSP, and PANSS and it showed that up to 23% of all observations were incomplete. Therefore, multiple imputation with chained equations under the assumption of data missing at random (MAR) was conducted to enable intention-to-treat analyses. Post-treatment values were imputed for GAF-F, PSP, and PANSS, using baseline values of all three variables, sex, diagnosis, age, community mental health center and intervention group as covariates. The automatic procedure was used and 100 imputations estimated. Per-protocol analysis was performed to see if group attendance influenced the result as proposed in one trial [24]. An analysis using the continuous variable of group attendance for the participants randomized to IMR was made as well as an analysis using a dichotomous variable categorizing attendance into 0–10 sessions or ≥11 sessions. Subgroup analyses tested whether diagnosis or sex interacted with the primary and secondary outcomes. The level of significance for all statistical tests was 0.05. The IBM SPSS Statistics version 19 for Windows was used for statistical analysis and STATA /SE version 13.1 was used for multiple imputations.
Ethical considerations and information to participants
Patients were recruited from the involved community mental health centers in the Capital Region of Denmark by their case manager supported by a local IMR supervisor. The patients were informed about the on-going trial by a poster in the waiting room and verbally by their case manager. If they were interested in participating they received more information about the IMR program and what their role would be in the controlled trial both verbally and written. Interested participants provided consent verbally and in writing. The case managers were informed about the trial and how to provide information to the patients by the researchers, the local IMR supervisor and their management. The trial was approved by the Ethics Committee in the Capital Region of Denmark (H-1-2010-134) January 20th 2011, before recruiting patients, reported to the Danish Data Protection Agency (RHP-2011-09) during 2011, and registered on www.clinicaltrials.gov (NCT01361698). The authors confirm that all ongoing and related trials for this intervention are registered.
Results
Fig 1 shows the CONSORT diagram for the participants through the trial. 202 participants were included and randomized but four of them were excluded immediately after randomization: Two participants from the control group withdrew their informed consent, one participant from the control group did not meet the criteria for diagnosis after all, and one participant was assigned as the only participant in an IMR group, that was never conducted and therefore the research group decided to exclude this individual from the trial instead. Therefore, 198 participants entered the trial, 99 participants in each intervention group.
The baseline characteristics of the 198 participants are listed in Table 1, showing an equally distributed in the two groups. A total of 26 participants from the IMR group and 11 from the control group did not participate in the follow-up assessments, a statistically significant difference (χ2 = 7.48, df = 1, p = 0.006). The reason for drop-out was that most patients in both groups simply did not want to continue participating with a higher percentage giving this answer in the intervention group compared to the control group (55% vs. 45%).
Exposure to IMR in the intervention group
The exposure to IMR in the intervention group is shown in Table 2. The exposure rate was 57.6%, as 57 of the 99 participants attended 10 or more IMR sessions. Among all the participants mean number of sessions were 16.4 (SD 13.3), compared to a mean of 26.1 (SD 8.1) among the 58, who participated in 10+ sessions. To explore if there were any differences between the non-exposed participants (attending 0–10 sessions) and the exposed participants (attending 10+ sessions) we did some post-hoc analyses to see, if there was a different in the time they had to wait from completion of baseline assessment to beginning the first IMR group as well as differences in baseline characteristics. There were no significant different in the mean waiting-time for the two groups (S1 Table). Furthermore, there were no significant differences in sex, age, housing, employment status, education, living status, diagnoses and substance abuse, but a significant difference in numbers in the two groups when we compared the three CMHCs (S2 Table).
Intention-to-treat analyses
Multiple imputations were used to analyze data according to intention-to-treat principle (Table 3). IMR was not significantly different from treatment as usual regarding level of functioning assessed by GAF-F, mean difference: +2.5 points (95% confidence interval (CI): -1.4 points to +6.4points, t = 1.25, df = 1, p = 0.21). For the secondary outcomes, there were no group differences regarding PSP, mean difference: +3.7points (95% CI: -0.8points to +8.1points, t = 1.62, df = 1, p = 0.11), or PANSS, mean difference: -2.7points (95% CI: -8.3points to2.8 points, t = —0.4498, df = 1, p = 0.33).
Complete case analyses
When analyzing data as complete cases, which are all cases where we had observed data on, similar results were found for GAF-F, PSP, and PANSS as in the intention-to-treat analyses, please see Table 2. No differences between the two intervention groups were seen in any of the explorative assessments of GAF-S, HAM-D, and Young Mania Rating Scale.
Service utilization
Treatment as usual that is the number of visits to CMHC (treatment i.e. meetings with the case manager, meetings with a psychologist or a psychiatrist, or times participating in group activities) was the same for both groups, see Table 3. IMR participants had a mean of 24.1 visits (SD = 23.8) and control group participants had a mean of 24.5 visits (SD = 20.2), respectively. This number covered a wide range where some participants in both groups had no visits and some had more than 130 visits during the follow-up period. Exposure to IMR in the intervention group is shown in Table 2.
Participants in the IMR group had a mean number of days in the psychiatric hospital of 13.5 (SD = 71.1) whereas participants in the control group had a mean number of 12.5 days (SD = 46.3), there was no statistically significant difference. There were no significant differences in the number of times the psychiatric emergency service was used or in number of hospital admissions between the two groups.
Per-protocol subgroup analysis
The results of the three per-protocol subgroup analyses are listed in Table 4. There was no association between a higher number of sessions attended and the end of intervention GAF-F score (F = 5.7, df = 1, p = 0.49). Subgroup analyses showed no differences in the effect of IMR according to diagnosis or sex regarding the outcomes of GAF-F, PSP or PANSS.
Harms and adverse events
In Table 3 is an overview of harms and adverse effects. There were no significant differences between the two groups in suicide or deaths. Participants in IMR did not differ from participants in the control group in terms of misuse of alcohol or drugs (F = 1.46, df = 1, p = 0.57). Three participants died in the period of follow-up, one of suicide from the control group and two died of natural causes, one from each group. No relation between participating in the IMR trial and the deaths was detected. No further harms or life-threatening conditions were reported during the trial.
Implementation
Staff specially trained in IMR fidelity assessment from one participating community mental health center was conducting the assessments at the other community mental health centers’ groups and vice versa. The IMR Fidelity mean score across the three participating community mental health centers assessed half-way at 4 months was 4.2 (SD 0.3) indicating good fidelity and the mean score for the end assessment was 4.9 (0.2) indicating high fidelity. A total of 10 IMR groups started and 9 of them completed. One group ended before time because the participants could not come to sessions because of personal reasons (e.g., serious hospitalization, got a job in the day hours, recently had a baby).
Discussion
This trial investigated the benefits and harms of the IMR program compared with treatment as usual. The trial was designed as an “add-on” study, and therefore patients in the intervention group received IMR as well as “treatment as usual”. Thereby potential difference between the two groups can only be explained by the effect of the IMR program. We found no effect on functioning, symptoms, substance abuse or service utilization. Two of the five earlier studies on IMR also examined the effect of IMR on functioning. One found no effect [5], but the other had a positive result [4]. Furthermore two of the five studies examined the effect on symptoms. Levitt et al. had a conflicting result, as they found a significant effect on a clinicians measured scale, mainly because of a reduction of the level of depression and anxiety, but no effect on a self-report scale [4], whereas Färdig had a positive result [2]. Two studies of the five studies measured IMR effect on substance abuse, and our result is consistent with these earlier results, as they found no effect [4, 6]. Finally, four of the five studies measured the effect on service utilization, three with a result in line with ours, as they found no effect [2, 4, 5], but the last study had a positive result [6]. As the results of all five earlier trials have to be addressed with caution due to a high risk of bias, one might conclude in line with our study that IMR have no effect on functioning, symptoms, substance abuse or service utilization. However the present study also has some limitation, i.e. not as high an exposure to IMR as wanted. Therefore, in order to get a robust answer to whether IMR is an effective program or not, more research is needed, among others there is a need for collecting the evidence from randomized trials of IMR in a systematic review with a meta-analysis, which is under preparation [25].
Strengths and limitations
This trial has several strengths. The trial was conducted with adequate generation of allocation sequence; adequate allocation concealment; adequate blinding wherever possible; adequate reporting of all relevant outcomes; intention-to-treat analyses; and no for-profit bias [26–28]. The Illness Management and Recovery Fidelity Score indicated that the implementation of the program was satisfactory. A strength of this trial is that prior beginning recruitment a sample size calculation was made. Furthermore, only patients with a validated diagnosis of schizophrenia or bipolar disorder were included in the trial. In the randomization process an external partner conducted the randomization and assured the concealment of sequence and allocation. It is also strength of the trial that most outcome assessments were conducted blinded, so that the knowledge of group status did not influenced the process. The external validity is high because the included participants represent the majority of patients with severe mental illnesses getting treatment in a community.
However, this trial also has some limitations. It is a limitation that 23 percent of the observed data were incomplete because the attrition rate was high. The attrition from research was higher in the IMR group. Our results indicate that the difference in attrition rate mainly is because more participants in the intervention group simply just did not want to participate in the follow-up interview. As 50% of the participants who withdraw from the research in the intervention group had attended zero or only one IMR-group session, one can argue that the main reason for the higher attrition rate in the intervention group is that these patients were not motivated to participate in the study all together not the IMR-intervention or the research interview. To address attrition from research, intention-to-treat analyses were done with multiple imputation. This is the least biased way to deal with missing data [29]. We may also have been too optimistic when planning the present trial as we based the number of participants needed to include on a 6 points difference on the GAF-F scale. Due to this our sample size may be too small, whereby we do not have the power to state if the observed difference of 2.4 is in fact a true difference, but it is most likely not of ‘clinical’ relevance. A study performed after we have started our study, indicate that the minimum clinically important difference for GAF is 4 points [30]. Furthermore, the waiting-time from randomization to the first IMR group session was on average 87 days and this can have affected the results, since it is important to provide psychosocial treatment when a participant is interested and motivated for it, and this may not be the case in the present study as there was a significant delay from the time a participant said yes to the treatment until they started actually started. Therefore we did a post-hoc analysis to see if the ones attending from 0–10 sessions waited longer than the ones attending 10+ sessions had a longer waiting time, but this was not the case, indicating that the waiting time didn’t affect the motivation for participating in the IMR program. Moreover, only 57.6% were fully exposed to the IMR program and this may have influenced the outcome measure. Therefore we did a subgroup analyses to see if the degree of exposure to IMR affected the primary and secondary outcomes. This was not the case. Finally, it is also a limitation that the secondary outcome Personal and Social Performance scale was not validated in Danish. However, the scale has been validated in a number of other countries and seems reliable and valid.
Overall IMR did not differ from treatment as usual in any of the analyzed outcomes. This means that IMR added to treatment as usual in a Danish context does not result in better or worse outcomes for the participants. A possible explanation is that the participants already received a sufficient high quality treatment as part of the usual treatment and perhaps mental health services and the intensity of it in Denmark is somewhat bigger compared to other countries, and that IMR in that context does not add anything further.
Another possible explanation can be found in the theoretic framework for IMR. According to the theory of the IMR program clinical recovery outcomes such as functioning and symptoms remission are considered to be distal outcomes, therefore a longer follow-up period is needed to address the effectiveness of the IMR program properly. Improvements in functioning is supposed to be found after the participant has adapted the new illness management skills into his/hers life. Perhaps the reason why this trial did not find any difference between IMR and the treatment as usual is that the period of follow-up was too brief. Without setting an exact time frame for the IMR theory maybe this trial assessed a distal outcome in a proximal short-time perspective (post treatment). A longer follow-up assessment could perhaps show if IMR is effective according to global assessment of functioning and the other distal clinical recovery outcomes. The trial is therefore also designed to have a longer follow-up and a 12-month after post treatment follow-up is completed currently.
Conclusion
IMR had no significant effect on the primary outcome GAF-F at the end of intervention (difference of 2.5 points in favor of IMR, P = 0.21). The 95% confidence interval (-1.4 point to +6.4points) makes the possibility of the hypothesized difference of 6 points favoring IMR unlikely. IMR was not significantly different from treatment as usual at end of intervention in terms of clinical outcomes such as level of functioning, symptoms severity, or service utilization.
This randomized trial contributes to the evidence base of IMR by providing a methodological solid base for its conclusions; however the trial has some important limitations. More research is needed to get a firm answer on the effectiveness of the IMR.
Supporting information
S1 Table. Waiting time before starting Illness management and recovery among participants in the intervention group.
https://doi.org/10.1371/journal.pone.0194027.s001
(DOCX)
S2 Table. Baseline characteristic of the participants in the intervention group attending up to 10 sessions and 10 and more sessions.
https://doi.org/10.1371/journal.pone.0194027.s002
(DOCX)
Acknowledgments
This trial was founded by the Danish Health Fund, Mental Health Center Ballerup, Mental Health Center Frederiksberg and the Capital Region of Denmark. We are grateful for the participation of the patients and the assistance of the clinical staff in each community mental health centers, and for the help from Ann-Lis Gregersen for text revision.
References
- 1. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The illness management and recovery program: Rationale, development, and preliminary findings. Schizophr Bull. 2006; 32: S32–43. pmid:16899534
- 2. Färdig R, Lewander T, Melin L, Folke F, Fredriksson A. A randomized controlled trial of the illness management and recovery program for persons with schizophrenia. Psychiatr Serv. 2011; 62:606–612. pmid:21632728
- 3. Hasson-Ohayon I, Roe D, Kravetz S. A randomized controlled trial of the effectiveness of the illness management and recovery program. Psychiatr Serv. 2007;58:1461–1466. pmid:17978257
- 4. Levitt AJ, Mueser KT, Degenova J, Lorenzo J, Bradford-Watt D, Barbosa A, et al. Randomized controlled trial of illness management and recovery in multiple-unit supportive housing. Psychiatr Serv. 2009; 60:1629–1636. pmid:19952153
- 5. Salyers MP, McGuire AB, Kukla M, Fukui S, Lysaker PH, Mueser KT. A randomized controlled trial of illness management and recovery with an active control group. Psychiatr Serv. 2014;65: 1–7.
- 6. Salyers MP, McGuire AB, Rollins AL, Bond GR, Mueser KT, Macy VR. Integrating assertive community treatment and illness management and recovery for consumers with severe mental illness. Community Ment Health J. 2010;46:319–329. pmid:20077006
- 7. Dalum HS, Waldemar AK, Korsbek L, Hjorthøj CR, Mikkelsen JH, Thomsen K et al. Participants' and staffs' evaluation of the Illness Management and Recovery program: a randomized clinical trial. J Ment Health. 2016 Nov 12:1–8. [Epub ahead of print].
- 8. Dalum HS, Korsbek L, Hagel Mikkelsen J, Thomsen K, Kistrup K, Olander M et al. Illness management and recovery (IMR) in Danish community mental health centres. Trials. 2011 Aug 17;12:195. pmid:21849024
- 9. Cooper JE, Copeland JRM, Brown GW, Harris T, Gourlay Aj.Further studies on interviewer training and inter-rater reliability of the Present State Examination (PSE). Psychol Med. 1977; 7:517–523. pmid:905468
- 10.
Gingerich S, Mueser KT. Illness management and recovery. In: Drake RE, Merrens MR, Lynde DW, editors. Evidence-Based Mental Health Practice: A Textbook. New York, NY: Norton; 2005. pp. 395–424.
- 11. Pedersen G, Hagtvet KA, Karterud S. Generalizability studies of the Global Assessment of Functioning-Split version. Compr Psychiatry. 2007;48:88–94. pmid:17145287
- 12. Pedersen G, Karterud S. The symptom and fusnction dimensions of the Global Assessment of Functioning (GAF) scale. Compr Psychiatry. 2012;53:292–298. pmid:21632038
- 13. Piersma HL, Boes JL. The GAF and psychiatric outcome: A descriptive report. Community Ment Health J. 1997;33:35–41. pmid:9061261
- 14. Kay SR, Opler LA, Lindenmayer JP. The Positive and Negative Syndrome Scale (PANSS): Rationale and standardisation. Br J Psychiatry. 1989; 155: 59–65.
- 15. Burns T, Patrick D. Social functioning as an outcome measure in schizophrenia studies. Acta Psychiatr Scand. 2007; 116: 403–418. pmid:17941964
- 16. Lecrubier Y, Bech P. The Ham D6 is more homogenous and as sensitive as the Ham D17. Eur Psychiatry. 2007;22:252–255. pmid:17344030
- 17. Hooper CL, Bakish D. An examination of the sensitivity of the six-item Hamilton Rating Scale for Depression in a sample of patients suffering from major depressive disorder. J Psychiatry Neurosci. 2000;25:178–184. pmid:10740991
- 18. Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale for mania: Reliability, validity and sensitivity. Br J Psychiatry. 1978;133:429–435. pmid:728692
- 19. Mueser KT, Corrigan PW, Hilton DW, Tanzman B, Schaub A, Gingerich S et al. Illness Management and Recovery. A Review of the Research. Psychiatr Serv 2002; 53: 1272–1284. pmid:12364675
- 20. Haddock G, Barrowclough C, Tarrier N, Morning J, O’Brien R, Schofield N et al. Cognitive behavioural therapy and motivational intervention for schizophrenia and substance misuse: 18-month outcomes of a randomised controlled trial. Br J Psychiatry. 2003; 183:418–426. pmid:14594917
- 21. Dupont WD, Plummer WD. Power and sample size calculations for studies involving linear regression. Control Clin Trials. 1998;19:589–601. pmid:9875838
- 22. Merinder LB, Viuff AG, Laugesen HD, Clemmensen K, Misfelt S, Espensen B. Patient and relative education in community psychiatry: A randomized controlled trial regarding its effectiveness. Soc Psychiatry Psychiatr Epidemiol. 1999;34:287–294. pmid:10422481
- 23. Vickers AJ, Altman DG. Statistics notes: Analysing controlled trials with baseline and follow up measurements. BMJ. 2001;323:1123–1124. pmid:11701584
- 24. McGuire AB, Bonfils KA, Kukla M, Myers L, Salyers MP. Measuring participation in an evidence-based practice: Illness management and recovery group attendance. Psychiatry Res. 2013;210:684–689. pmid:24011850
- 25. Korsbek L, Dalum H S, Lindschou J, Eplov L F. Illness management and recovery programme for people with severe mental illness. Cochrane Database of Systematic Reviews. 2014.
- 26.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011.
- 27. Bero L. Industry Sponsorship and Research Outcome. JAMA Intern Med. 2013:580.
- 28. Savović J, Jones HE, Altman DG, Harris RJ, Juni P, Pildal J et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: Combined analysis of meta-epidemiological studies. Health Technol Assess. 2012;16:1–81.
- 29. Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367:1355–1360. pmid:23034025
- 30. Amri I, Millier A, Toumi M. PMH2- Minimum Clinically Important Difference in the Global Assessment Functioning in Patients with Schizophrenia. Value Health. 2014;17:A765–A766.