Skip to main content
Erschienen in: Social Psychiatry and Psychiatric Epidemiology 2/2014

01.02.2014 | Original Paper

Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales

verfasst von: Alberto Parabiaghi, Hans E. Kortrijk, Cornelis L. Mulder

Erschienen in: Social Psychiatry and Psychiatric Epidemiology | Ausgabe 2/2014

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Using the reliable and clinically significant change approach, we aimed to identify meaningful outcome indicators for the Health of the Nation Outcome Scales (HoNOS) and to combine them in a single model. We applied these indicators to the 1-year outcome of two large samples of people attending community mental health services in Italy (cohort 1) and the Netherlands (cohort 2).

Methods

Data were drawn from two studies on routine outcome assessment. The criteria for meaningful outcome were defined on both study cohorts and both language versions of the scale. The model combined (a) two criteria for adequate change (at least 4 or 8 points change), (b) two cut-offs for clinically significant change (a total score of 10 was the threshold between mild and moderate illness, 13 between moderate and severe illness), and (c) a method for classifying stable subjects in three degrees of severity (stable in mild, moderate or severe illness). Results were compared with those given by the effect size (ES) and analysis of variance and covariance (ANOVA and ANCOVA).

Results

For the proposed approach the outcome of cohort 1 was better than cohort 2, with 65–67 % of its subjects showing a positive outcome compared to only 45–46 %. The other reference methods (ES and ANOVA), however, showed a greater improvement for cohort 2. ANCOVA indicated that the differences were due to regression to the mean (RTM) which showed opposite effects across the two cohorts.

Conclusions

The proposed approach proved valuable and generalizable for interpreting outcome on HoNOS, scarcely influenced by the RTM effect. Its introduction could benefit outcome evaluation and management.
Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
1
The population mean should be a known true mean in a population. In the present study, the mean of the aggregate sample (i.e. both sites) was used as an estimate of μ.
 
Literatur
1.
Zurück zum Zitat Department of Health (2001) Mental health information strategy. Department of Health, London Department of Health (2001) Mental health information strategy. Department of Health, London
2.
Zurück zum Zitat Commonwealth Department of Health and Ageing (2002) National outcome and casemix collection: overview of clinical measures and data items. Commonwealth Department of Health and Ageing, Canberra Commonwealth Department of Health and Ageing (2002) National outcome and casemix collection: overview of clinical measures and data items. Commonwealth Department of Health and Ageing, Canberra
3.
Zurück zum Zitat Ministry of Health (2005) National mental health information strategy 2005–2010. Ministry of Health, Wellington Ministry of Health (2005) National mental health information strategy 2005–2010. Ministry of Health, Wellington
4.
Zurück zum Zitat Trauer T (2010) Outcome measurement around the world. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 13–101 Trauer T (2010) Outcome measurement around the world. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 13–101
5.
Zurück zum Zitat Norman GR, Sridhar FG, Guyatt GH (2001) Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care 39:1039–1047PubMedCrossRef Norman GR, Sridhar FG, Guyatt GH (2001) Relation of distribution- and anchor-based approaches in interpretation of changes in health-related quality of life. Med Care 39:1039–1047PubMedCrossRef
6.
Zurück zum Zitat Burgess P, Pirkis J (2008) Key performance indicators for Australian Public Mental Health Services—potential contributions of MH-NOCC data: developing indicators of effectiveness Version 2.0, Brisbane, Queensland Burgess P, Pirkis J (2008) Key performance indicators for Australian Public Mental Health Services—potential contributions of MH-NOCC data: developing indicators of effectiveness Version 2.0, Brisbane, Queensland
7.
Zurück zum Zitat Eisen SV, Ranganathan G, Seal P, Spiro A 3rd (2007) Measuring clinically meaningful change following mental health treatment. J Behav Health Serv Res 34:272–289PubMedCrossRef Eisen SV, Ranganathan G, Seal P, Spiro A 3rd (2007) Measuring clinically meaningful change following mental health treatment. J Behav Health Serv Res 34:272–289PubMedCrossRef
8.
Zurück zum Zitat Trauer T (2010) Assessment of change in outcome measurement. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 206–218 Trauer T (2010) Assessment of change in outcome measurement. In: Trauer T (ed) Outcome measurement in mental health. Cambridge University Press, Cambridge, pp 206–218
9.
Zurück zum Zitat Speer DC (1999) What is the role of two-wave designs in clinical research? Comment on Hageman and Arrindell. Behav Res Ther 37:1203–1210PubMedCrossRef Speer DC (1999) What is the role of two-wave designs in clinical research? Comment on Hageman and Arrindell. Behav Res Ther 37:1203–1210PubMedCrossRef
10.
Zurück zum Zitat Cella D, Bullinger M, Scott C, Barofsky I et al (2002) Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin Proc 77:384–392PubMedCrossRef Cella D, Bullinger M, Scott C, Barofsky I et al (2002) Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin Proc 77:384–392PubMedCrossRef
11.
Zurück zum Zitat Lydick E, Epstein RS (1993) Interpretation of quality of life changes. Qual Life Res 2:221–226PubMedCrossRef Lydick E, Epstein RS (1993) Interpretation of quality of life changes. Qual Life Res 2:221–226PubMedCrossRef
12.
Zurück zum Zitat Crosby RD, Kolotkin RL, Williams GR (2003) Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 56:395–407PubMedCrossRef Crosby RD, Kolotkin RL, Williams GR (2003) Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 56:395–407PubMedCrossRef
13.
Zurück zum Zitat Jacobson NS, Follette WC, Revenstorf D, Baucom DH et al (1984) Variability in outcome and clinical significance of behavioral marital therapy: a reanalysis of outcome data. J Consult Clin Psychol 52:497–504PubMedCrossRef Jacobson NS, Follette WC, Revenstorf D, Baucom DH et al (1984) Variability in outcome and clinical significance of behavioral marital therapy: a reanalysis of outcome data. J Consult Clin Psychol 52:497–504PubMedCrossRef
14.
Zurück zum Zitat Christensen L, Mendoza JL (1986) A method of assessing change in a single subject: an alteration of the RC index. Behav Therapy 17:305–308CrossRef Christensen L, Mendoza JL (1986) A method of assessing change in a single subject: an alteration of the RC index. Behav Therapy 17:305–308CrossRef
15.
Zurück zum Zitat Jacobson NS, Truax P (1991) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59:12–19PubMedCrossRef Jacobson NS, Truax P (1991) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59:12–19PubMedCrossRef
16.
Zurück zum Zitat Hageman WJ, Arrindell WA (1993) A further refinement of the reliable change (RC) index by improving the pre-post difference score: introducing RCID. Behav Res Ther 31:693–700PubMedCrossRef Hageman WJ, Arrindell WA (1993) A further refinement of the reliable change (RC) index by improving the pre-post difference score: introducing RCID. Behav Res Ther 31:693–700PubMedCrossRef
17.
Zurück zum Zitat Evans C, Margison F, Barkham M (1998) The contribution of reliable and clinically significant change methods to evidence-based mental health. Evid Based Ment Health 1:70–72CrossRef Evans C, Margison F, Barkham M (1998) The contribution of reliable and clinically significant change methods to evidence-based mental health. Evid Based Ment Health 1:70–72CrossRef
18.
Zurück zum Zitat Hageman WJ, Arrindell WA (1999) Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behav Res Ther 37:1169–1193PubMedCrossRef Hageman WJ, Arrindell WA (1999) Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behav Res Ther 37:1169–1193PubMedCrossRef
19.
Zurück zum Zitat Hsu LM (1999) A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behav Res Ther 37:1195–1202PubMedCrossRef Hsu LM (1999) A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behav Res Ther 37:1195–1202PubMedCrossRef
20.
Zurück zum Zitat Hageman WJ, Arrindell WA (1999) Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer. Behav Res Ther 37:1219–1233PubMedCrossRef Hageman WJ, Arrindell WA (1999) Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer. Behav Res Ther 37:1219–1233PubMedCrossRef
21.
Zurück zum Zitat McGlinchey JB, Atkins DC, Jacobson NS (2002) Clinical significance methods: which one to use and how useful are they? Behav Ther 33:529–550CrossRef McGlinchey JB, Atkins DC, Jacobson NS (2002) Clinical significance methods: which one to use and how useful are they? Behav Ther 33:529–550CrossRef
22.
Zurück zum Zitat Burgess P, Pirkis J, Coombs T (2009) Modelling candidate effectiveness indicators for mental health services. Aust N Z J Psychiatry 43:531–538PubMedCrossRef Burgess P, Pirkis J, Coombs T (2009) Modelling candidate effectiveness indicators for mental health services. Aust N Z J Psychiatry 43:531–538PubMedCrossRef
23.
Zurück zum Zitat Cohen J (1969) Statistical power analysis for the behavioural sciences. Academic Press, London Cohen J (1969) Statistical power analysis for the behavioural sciences. Academic Press, London
24.
Zurück zum Zitat Kazis LE, Anderson JJ, Meenan RF (1989) Effect sizes for interpreting changes in health status. Med Care 27(Suppl 3):178–189CrossRef Kazis LE, Anderson JJ, Meenan RF (1989) Effect sizes for interpreting changes in health status. Med Care 27(Suppl 3):178–189CrossRef
25.
Zurück zum Zitat McHorney CA, Tarlov AR (1995) Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 4:293–307PubMedCrossRef McHorney CA, Tarlov AR (1995) Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 4:293–307PubMedCrossRef
26.
Zurück zum Zitat Grundy CT, Lambert MJ, Grundy EM (1996) Assessing clinical significance: application to the Hamilton Rating Scale for depression. J Ment Health 5:25–33CrossRef Grundy CT, Lambert MJ, Grundy EM (1996) Assessing clinical significance: application to the Hamilton Rating Scale for depression. J Ment Health 5:25–33CrossRef
27.
Zurück zum Zitat Hafkenscheid A (2000) Psychometric measures of individual change: an empirical comparison with the Brief Psychiatric Rating Scale (BPRS). Acta Psychiatr Scand 101:235–242PubMedCrossRef Hafkenscheid A (2000) Psychometric measures of individual change: an empirical comparison with the Brief Psychiatric Rating Scale (BPRS). Acta Psychiatr Scand 101:235–242PubMedCrossRef
28.
Zurück zum Zitat Audin K, Margison FR, Clark JM, Barkham M (2001) Value of HoNOS in assessing patient change in NHS psychotherapy and psychological treatment services. Br J Psychiatry 178:561–566PubMedCrossRef Audin K, Margison FR, Clark JM, Barkham M (2001) Value of HoNOS in assessing patient change in NHS psychotherapy and psychological treatment services. Br J Psychiatry 178:561–566PubMedCrossRef
29.
Zurück zum Zitat Matthey S (2004) Calculating clinically significant change in postnatal depression studies using the Edinburgh Postnatal Depression Scale. J Affect Disord 78:269–272PubMedCrossRef Matthey S (2004) Calculating clinically significant change in postnatal depression studies using the Edinburgh Postnatal Depression Scale. J Affect Disord 78:269–272PubMedCrossRef
30.
Zurück zum Zitat Parabiaghi A, Barbato A, D’Avanzo B, Erlicher A et al (2005) Assessing reliable and clinically significant change on Health of the Nation Outcome Scales: method for displaying longitudinal data. Aust N Z J Psychiatry 39:719–725PubMedCrossRef Parabiaghi A, Barbato A, D’Avanzo B, Erlicher A et al (2005) Assessing reliable and clinically significant change on Health of the Nation Outcome Scales: method for displaying longitudinal data. Aust N Z J Psychiatry 39:719–725PubMedCrossRef
31.
Zurück zum Zitat Barbato A, Parabiaghi A, Panicali F, Battino N et al (2011) Do patients improve after short psychiatric admission? A cohort study in Italy. Nord J Psychiatry 65:251–258PubMedCrossRef Barbato A, Parabiaghi A, Panicali F, Battino N et al (2011) Do patients improve after short psychiatric admission? A cohort study in Italy. Nord J Psychiatry 65:251–258PubMedCrossRef
32.
Zurück zum Zitat Parabiaghi A, Rapisarda F, D’Avanzo B, Erlicher A et al (2011) Measuring clinical change in routine mental health care: differences between first time and longer term service users. Aust N Z J Psychiatry 45:558–568PubMedCrossRef Parabiaghi A, Rapisarda F, D’Avanzo B, Erlicher A et al (2011) Measuring clinical change in routine mental health care: differences between first time and longer term service users. Aust N Z J Psychiatry 45:558–568PubMedCrossRef
33.
Zurück zum Zitat Bauer S, Lambert MJ, Nielsen SL (2003) Clinical significance methods: a comparison of statistical techniques. J Pers Assess 82:60–70CrossRef Bauer S, Lambert MJ, Nielsen SL (2003) Clinical significance methods: a comparison of statistical techniques. J Pers Assess 82:60–70CrossRef
34.
Zurück zum Zitat Speer DC (1992) Clinically significant change: jacobson & Truax (1991) revisited. J Consult Clin Psych 60:402–408CrossRef Speer DC (1992) Clinically significant change: jacobson & Truax (1991) revisited. J Consult Clin Psych 60:402–408CrossRef
35.
Zurück zum Zitat Kortrijk HE, Staring AB, van Baars AW, Mulder CL (2010) Involuntary admission may support treatment outcome and motivation in patients receiving assertive community treatment. Soc Psychiatry Psychiatr Epidemiol 45:245–252PubMedCrossRef Kortrijk HE, Staring AB, van Baars AW, Mulder CL (2010) Involuntary admission may support treatment outcome and motivation in patients receiving assertive community treatment. Soc Psychiatry Psychiatr Epidemiol 45:245–252PubMedCrossRef
36.
Zurück zum Zitat Kortrijk HE, Mulder CL, Drukker M, Wiersma D, Duivenvoorden HJ (2012) Duration of assertive community treatment and the interpretation of routine outcome data. Aust N Z J Psychiatry 46:240–248PubMedCrossRef Kortrijk HE, Mulder CL, Drukker M, Wiersma D, Duivenvoorden HJ (2012) Duration of assertive community treatment and the interpretation of routine outcome data. Aust N Z J Psychiatry 46:240–248PubMedCrossRef
37.
Zurück zum Zitat Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry 172:11–18PubMedCrossRef Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A (1998) Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry 172:11–18PubMedCrossRef
38.
Zurück zum Zitat Pirkis JE, Burgess PM, Kirk PK, Dodson S, Coombs TJ, Williamson MK (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health Qual Life Outcomes 3:76PubMedCentralPubMedCrossRef Pirkis JE, Burgess PM, Kirk PK, Dodson S, Coombs TJ, Williamson MK (2005) A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures. Health Qual Life Outcomes 3:76PubMedCentralPubMedCrossRef
39.
Zurück zum Zitat Rossi R, Blaco R, Castelli C, Civenti G et al (1999) Il costo dei pazienti psichiatrici per classi di gravita. Epidemiol Psichiatr Soc 7:198–208CrossRef Rossi R, Blaco R, Castelli C, Civenti G et al (1999) Il costo dei pazienti psichiatrici per classi di gravita. Epidemiol Psichiatr Soc 7:198–208CrossRef
40.
Zurück zum Zitat Lora A, Bai G, Bianchi S, Bolongaro G et al (2001) La versione italiana della HoNOS (Health of the Nation Outcome Scales), una scala per la valutazione della gravita e dell’esito nei servizi di salute mentale. Epidemiol Psichiatr Soc 10:198–212CrossRef Lora A, Bai G, Bianchi S, Bolongaro G et al (2001) La versione italiana della HoNOS (Health of the Nation Outcome Scales), una scala per la valutazione della gravita e dell’esito nei servizi di salute mentale. Epidemiol Psichiatr Soc 10:198–212CrossRef
41.
Zurück zum Zitat Mulder CL, Staring ABP, Loos J, Buwalda V et al (2004) De Health of the Nations Outcome Scales (HoNOS) als instrument voor routine outcome assessment. Tijdschr Psychiatr 46:273–285 Mulder CL, Staring ABP, Loos J, Buwalda V et al (2004) De Health of the Nations Outcome Scales (HoNOS) als instrument voor routine outcome assessment. Tijdschr Psychiatr 46:273–285
42.
Zurück zum Zitat Lelliott P (1999) Definition of severe mental illness. In: Charlwood P, Mason A, Goldacre M, Cleary R, Wilkinson E (eds) Health outcome indicators: severe mental illness. Report of a working group to the Department of Health. National Centre for Health Outcomes Development, Oxford, pp 87–93 Lelliott P (1999) Definition of severe mental illness. In: Charlwood P, Mason A, Goldacre M, Cleary R, Wilkinson E (eds) Health outcome indicators: severe mental illness. Report of a working group to the Department of Health. National Centre for Health Outcomes Development, Oxford, pp 87–93
43.
Zurück zum Zitat Barnett AG, Van Der Pols JC, Dobson AJ (2005) Regression to the mean: what it is and how to deal with it. Int J Epidemiol 34:215–220PubMedCrossRef Barnett AG, Van Der Pols JC, Dobson AJ (2005) Regression to the mean: what it is and how to deal with it. Int J Epidemiol 34:215–220PubMedCrossRef
44.
Zurück zum Zitat Gardner MJ, Heady JA (1973) Some effects of within-person variability. J Chronic Dis 26:781–795CrossRef Gardner MJ, Heady JA (1973) Some effects of within-person variability. J Chronic Dis 26:781–795CrossRef
45.
Zurück zum Zitat Davis CE (1976) The effect of regression to the mean in epidemiologic and clinical studies. Am J Epidemiol 104:493–498PubMed Davis CE (1976) The effect of regression to the mean in epidemiologic and clinical studies. Am J Epidemiol 104:493–498PubMed
47.
Zurück zum Zitat Harmon C, Hawkins EJ, Lambert MJ, Slade K, Whipple JS (2005) Improving outcomes for poorly responding clients: the use of clinical support tools and feedback to clients. J Clin Psychol 61:175–185PubMedCrossRef Harmon C, Hawkins EJ, Lambert MJ, Slade K, Whipple JS (2005) Improving outcomes for poorly responding clients: the use of clinical support tools and feedback to clients. J Clin Psychol 61:175–185PubMedCrossRef
48.
Zurück zum Zitat Drukker M, Bak M, à Campo J, Driessen G, van Os J, Delespaul P (2010) The cumulative needs for care monitor: a unique monitoring system in the south of the Netherlands. Soc Psychiatry Psychiatr Epidemiol 45:475–485PubMedCentralPubMedCrossRef Drukker M, Bak M, à Campo J, Driessen G, van Os J, Delespaul P (2010) The cumulative needs for care monitor: a unique monitoring system in the south of the Netherlands. Soc Psychiatry Psychiatr Epidemiol 45:475–485PubMedCentralPubMedCrossRef
Metadaten
Titel
Defining multiple criteria for meaningful outcome in routine outcome measurement using the Health of the Nation Outcome Scales
verfasst von
Alberto Parabiaghi
Hans E. Kortrijk
Cornelis L. Mulder
Publikationsdatum
01.02.2014
Verlag
Springer Berlin Heidelberg
Erschienen in
Social Psychiatry and Psychiatric Epidemiology / Ausgabe 2/2014
Print ISSN: 0933-7954
Elektronische ISSN: 1433-9285
DOI
https://doi.org/10.1007/s00127-013-0750-7

Weitere Artikel der Ausgabe 2/2014

Social Psychiatry and Psychiatric Epidemiology 2/2014 Zur Ausgabe

Update Psychiatrie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.