Hostname: page-component-7c8c6479df-ph5wq Total loading time: 0 Render date: 2024-03-27T16:25:17.969Z Has data issue: false hasContentIssue false

Measuring patient-reported outcomes in psychosis: conceptual and methodological review

Published online by Cambridge University Press:  02 January 2018

Ulrich Reininghaus*
Affiliation:
King's College London, Institute of Psychiatry
Stefan Priebe
Affiliation:
Queen Mary University of London, Unit for Social and Community Psychiatry, Barts and the London School of Medicine, UK
*
Dr Ulrich Reininghaus, Section of Social Psychiatry, PO Box 33, Health Service and Population Research Department, Institute of Psychiatry, De Crespigny Park, London SE5 8AF, UK. Email: ulrich.reininghaus@kcl.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background

There are calls to use patient-reported outcomes (PROs) routinely across mental health services. However, the use of PROs in patients with psychosis has been questioned.

Aims

To examine the concepts and measures of four widely used PROs: treatment satisfaction, subjective quality of life, needs for care and the quality of the therapeutic relationship.

Method

We conducted a literature search of academic databases on concepts, characteristics and psychometric properties of the four PROs in patients with psychosis.

Results

Although numerous concepts and measures have been published, evidence on the methodological quality of existing PROs is limited. Measures designed to assess distinct PROs showed a considerable conceptual, operational and empirical overlap, and some of them also included specific aspects. The impact of symptoms and cognitive deficits appears unlikely to be of clinical significance.

Conclusions

The popularity of PROs has not been matched with progress in their conceptualisation and measurement. Based on current evidence, some recommendations can be made. Distinct and short measures with clinical relevance and sufficient psychometric properties should be preferred. Future research should optimise the validity and measurement precision of PROs, while reducing assessment burden.

Type
Review Article
Copyright
Copyright © Royal College of Psychiatrists, 2012 

Since the 1960s, patient-reported outcomes (PROs) have become increasingly popular in the care of patients with psychosis. There is no universally accepted terminology and definition of such outcomes. In the literature the terms ‘PROs’, ‘patient-reported outcome measures’ (PROMs), ‘patient-based outcomes’, ‘patient-driven outcomes’, ‘self-rated outcomes’ and ‘subjective evaluation criteria’ have been used interchangeably. Reference Priebe, Gruyters, Heinze, Hoffmann and Jakel13 In recent years the term ‘PRO’ appears to be most widely used. Reference Hansson, Bjorkman and Priebe2 The US Food and Drug Administration (FDA) defined PROs as:

‘any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else’ (p. 2). 4

Treatment satisfaction, subjective quality of life (SQoL), needs and the quality of the therapeutic relationship can be considered as four historically rooted, commonly used and important PRO concepts in the care of patients with psychosis. Reference Hansson, Bjorkman and Priebe2,Reference McCabe, Saidi and Priebe5,Reference Kilian and Angermeyer6 Whereas the list of PROs has increased steadily, their popularity has gained momentum over the past decade, partly through their intuitive appeal for stakeholder groups. 3Reference McCabe, Saidi and Priebe5 In the UK a recent National Health Service (NHS) White Paper announced plans for new outcome assessments, in which PROs were to be used to measure the effectiveness of services. 3 Using PROs in the monitoring of outcomes of individual patients and services can also feed into the patient–clinician communication, reflective practice, quality management and service development. Reference Slade, McCrone, Kuipers, Leese, Cahill and Parabiaghi7,Reference Priebe, McCabe, Bullenkamp, Hansson, Lauber and Martinez-Leal8 However, the selection of appropriate concepts and measures often remains difficult. Further, some authors have questioned the use of PROs in patients with psychosis owing to conceptual and methodological shortcomings, Reference Atkinson, Zibin and Chuang9 with some proposing to discard them entirely. Reference Epstein, Hall, Tognetti, Son and Conant10 Against this background, this review aimed to examine the concepts and measures of four widely used PROs – treatment satisfaction, SQoL, needs for care and the quality of the therapeutic relationship – in the evaluation of care of patients with psychosis.

Method

A review of the conceptual and methodological literature on the four PROs in the care of patients with psychosis was conducted. We searched the literature systematically and also followed the recommendations for conceptual and methodological reviews to search widely in disparate sources and allow for overlap in the various stages (literature search, analysis and writing). Reference Lilford, Richardson, Stevens, Fitzpatrick, Edwards and Rock11,Reference Morgan, Burns, Fitzpatrick, Pinfold and Priebe12

Search strategy and selection criteria

A search of the academic databases EMBASE, Medline and PsycINFO was performed to identify papers that, first, reported the characteristics and psychometric properties of PRO measures to assess treatment satisfaction, SQoL, needs for care and the therapeutic relationship in the care of patients with psychosis, and second, provided definitions of concepts intended to be assessed by at least one of the identified measures. The term ‘PRO’ was used in accordance with the FDA definition given earlier. The literature search combined three groups of keywords in each database:

  1. (a) schizophr*, psychosis OR psychoses;

  2. (b) quality of life, subjective quality of life, treatment satisfaction, patient satisfaction, need*, therapeutic relationship, therapeutic alliance, helping alliance OR working alliance;

  3. (c) psychometric*, validity, reliability OR responsiveness.

Titles and abstracts were screened and papers retrieved to assess their relevance. Reference lists of relevant papers were inspected for additional papers. References that cited previously identified papers were searched using the ‘cited by’ option in the electronic database Web of Science. In addition to the search of academic databases, informal networks were used to identify papers.

Data extraction and synthesis

As the conceptual and methodological literature on PROs in the evaluation of treatments for psychosis is vast and disparate, a quantitative synthesis appeared neither appropriate nor feasible. The findings are presented descriptively. Although PRO measures can be distinguished according to various characteristics, we focused on the following ones: concept purported to be measured, number and content of domains, estimated completion time, response options and type (generic, condition- or disease-specific, treatment-specific and utility measures). 4,Reference Fitzpatrick, Davey, Buxton and Jones13 Numerous psychometric properties for evaluating PROs have been proposed in the literature. Reference Mokkink, Terwee, Patrick, Alonso, Stratford and Knol14 We distinguished between reliability (i.e. internal consistency, reliability, scale information), validity (i.e. content validity, including face validity, and construct validity, including structural, convergent, discriminant, cross-cultural, concurrent and predictive validity) and responsiveness. Reference Mokkink, Terwee, Patrick, Alonso, Stratford and Knol14 Given the lack of consensus on how these psychometric properties are best evaluated and findings synthesised, 4,Reference Fitzpatrick, Davey, Buxton and Jones13 we used a simple, dichotomous rating of whether or not a psychometric property had been examined for a given instrument.

Results

The results of the search strategy are summarised in Fig. 1. The search initially yielded a total of 2181 items (813 duplicates). Titles and abstracts were screened for 1368 references. Based on title and abstract sifts, 1238 references were excluded because they did not focus on the four PROs or psychosis. The number of potentially relevant references increased from 130 to 224 when additional items were added. Of these, 49 references were excluded for different reasons. Hence, from the 2181 initially identified references, only 175 were included in our review.

FIG. 1 Results of database search. PRO, patient-reported outcome.

Concepts and definitions

Definitions of concepts to be assessed by the identified PRO measures of treatment satisfaction, SQoL, needs for care and the therapeutic relationship are summarised in online Table DS1. Measures of treatment satisfaction for which a definition of the concept to be measured was provided all purported to assess the multidimensional satisfaction concept of a personal evaluation of healthcare services and providers as proposed by Ware et al and Ruggeri et al. Reference Ware, Snyder, Wright and Davies15,Reference Ruggeri, Dall'Agnola, Agostini and Bisoffi16 The identified SQoL measures were intended to assess a range of concepts (Table DS1). The only measure of needs that provided a definition of the concept to be measured, the Camberwell Assessment of Need (CAN), Reference Phelan, Slade, Thornicroft, Dunn, Holloway and Wykes17 purported to assess a supply and perceived need concept. Pantheoretical, Reference Bordin18 Rogerian, Reference Priebe and Gruyters19 systemic Reference Pinsof and Catherall20 and psychoanalytic Reference Freud and Strachey21,Reference Sterba22 concepts were intended to be assessed by the identified measures of the therapeutic relationship. For each of the four PROs, no single universally accepted definition could be identified. Nevertheless, there were attempts to identify a common conceptual basis. Lauer noted that:

‘There is agreement that quality of life is a multi-dimensional phenomenon and construct, aiming at a holistic or global perspective of individuals in their biopsychosocial nature’ (p. 19). Reference Lauer, Priebe, Oliver and Kaiser23

Similarly, Ware et al emphasised that treatment satisfaction is most widely measured as a multidomain concept. Reference Ware, Snyder, Wright and Davies15 However, this may imply a risk of providing non-specific or overinclusive definitions. Several PRO concepts other than the one to be measured may meet very broad definitions; for example, Stevens & Gabbay defined needs as ‘the ability to benefit in some way from health care’ (p. 21). Reference Stevens and Gabbay24 Others have found a lack of clarity of the precise nature of some PRO concepts:

‘In psychiatry, there is as yet no clearly defined concept of the therapeutic alliance’ (Catty: p. 265). Reference Catty25

A tendency was found to use terms from different theoretical backgrounds and traditions with at least slightly different connotations synonymously. For example, the term ‘therapeutic relationship’ has been used interchangeably with the terms ‘therapeutic alliance’, ‘helping alliance’ or ‘working alliance’, each of which has emerged from different lines of research. Reference Catty25 Similarly, ‘treatment satisfaction’ has been used synonymously with ‘patient satisfaction’, ‘service satisfaction’ and ‘satisfaction with care’, to name a few. Reference Ware, Snyder, Wright and Davies15,Reference Ruggeri, Dall'Agnola, Agostini and Bisoffi16 This may lead to a lack of clarity as to precisely which conceptualisation of PROs is being referred to. Reference Catty, Winfield and Clement26 Several definitions of PRO concepts were found to overlap with others (Table DS1). However, some definitions of PRO concepts did not, and contained specific elements: this applied to definitions of SQoL, Reference Zautra and Goodhard27,Reference Clare, Corney and Cairns28 needs for care, 29 and the therapeutic relationship. Reference Bordin18,Reference Freud and Strachey21,Reference Sterba22 Overall, definitions of PRO concepts were found to vary in the extent to which they included overlapping and specific aspects.

Characteristics of PRO measures

Findings on characteristics and psychometric properties of PRO measures to assess treatment satisfaction, SQoL, needs for care and the therapeutic relationship are summarised respectively in online Tables DS2–DS5. For several measures the concept that the measure was intended to assess was not provided. Most measures were generic in nature and used Likert scales. Short versions have been developed for several measures, based on conceptual and practical rather than empirical considerations. A number of measures were found to be long and time-consuming to administer: several had more than 30 items and a completion time greater than 20 min.

Several PRO measures were intended to assess multidomain concepts, with items being grouped within domains, and domains within more general PRO concepts. An overlap in the content of domains was observed across measures that were intended to assess different PROs. Specifically, the domains of measures to assess SQoL are similar and in part even identical to domains included in measures of needs. This applies likewise to measures of treatment satisfaction and the therapeutic relationship. The content of domains of treatment satisfaction and needs for care measures, and the content of treatment satisfaction and SQoL measures, show substantial overlap (Tables DS2–DS5).

Psychometric properties of PRO measures

The evaluation of the reviewed measures often included only limited information on psychometric properties in patients with psychosis (Tables DS2–DS5). The methods used to assess structural validity were largely not appropriate for ordinal data, as required for the predominantly used Likert scales. Reference Gibbons, Bock, Hedeker, Weiss, Segawa and Bhaumik30 Only for two measures, the Quality of Life Interview (QoLI) and EuroQoL-5D, Reference Lehman3134 was there evidence on structural validity based on confirmatory factor analysis for ordinal data or item response modelling. Reference Gibbons, Bock, Hedeker, Weiss, Segawa and Bhaumik30,Reference Prieto, Novick, Sacristan and Edgell35,Reference Uttaro and Lehman36 For most measures there was no evidence on their measurement precision throughout the full range of scores; only for the QoLI was this psychometric property examined. Reference Uttaro and Lehman36 For some measures no evidence on their internal consistency, test–retest reliability and scale information as well as content, structural, discriminant, convergent, concurrent, predictive or cross-cultural validity was found in the included studies.

Empirical overlap of PRO measures

Only a few studies assessed more than one outcome at a time. They consistently suggest low discriminant validity due to an empirical overlap of measures designed to assess different outcomes. The outcomes were substantially correlated, Reference De Weert-van Oene, Havenaar and Schrijvers37Reference Slade, Leese, Cahill, Thornicroft and Kuipers41 and a single general factor explained more than half of the variance in SQoL, needs for care and treatment satisfaction scores. Reference Hansson, Bjorkman and Priebe2,Reference Priebe, Kaiser, Huxley, Roder-Wanner and Rudolf42,Reference Fakhoury, Kaiser, Roeder-Wanner and Priebe43 The general factor has been interpreted as a general appraisal tendency of patients for positive or negative ratings across measures designed to assess different PRO concepts. Reference Priebe, Kaiser, Huxley, Roder-Wanner and Rudolf42 However, this general appraisal tendency left about half of the variance unexplained, which is potentially concept-specific. A recent study suggested a bifactor model which confirms the importance of a general appraisal tendency, but also shows the relevance of concept-specific aspects. The latter provide distinct information that is independent from both the general appraisal tendency and other concepts. Reference Reininghaus, McCabe, Burns, Croudace and Priebe44

Association with psychiatric symptoms and cognitive deficits

There was also evidence from several studies that less favourable SQoL is related to higher levels of psychopathologic disorder including positive, negative and depressive symptoms. Reference Norholm and Bech45Reference Vatne and Bjorkly56 For the association of psychiatric symptoms and treatment satisfaction, Katsakou & Priebe reported an inverse relationship between psychiatric symptoms and level of treatment satisfaction, Reference Katsakou and Priebe57 which is in line with other studies. Reference Katsakou, Bowers, Amos, Morriss, Rose and Wykes58 There are also a number of studies suggesting that patients with more severe psychotic symptoms have more unmet and total needs for care. Reference Grinshpoon and Ponizovsky59Reference Ochoa, Haro, Autonell, Pendas, Teba and Marquez61 However, a more recent pooled analysis of individual patient-level data obtained from 16 studies found that symptom levels were less strongly associated with SQoL in schizophrenia than in other mental disorders. Reference Priebe, Reininghaus, McCabe, Burns, Eklund and Hansson62 A pooled analysis of associations between changes of symptoms and SQoL ratings over time identified an explained variance of only 5.5%. Reference Priebe, McCabe, Junghan, Kallert, Ruggeri and Slade63

With respect to cognitive deficits, evidence on associations with PROs remains inconsistent. Fujii et al found that better cognitive performance was associated with lower SQoL ratings in a prospective study of patients with severe and enduring psychosis, Reference Fujii, Wylie and Nathan64 which is consistent with other studies. Reference Corrigan and Buican50,Reference Addington and Addington65Reference Wegener, Redoblado-Hodge, Lucas, Fitzgerald, Harris and Brennan70 However, Galletly et al, Ritsner and Sota found the opposite. Reference Galletly, Clark, McFarlane and Weber71Reference Sota73 Deficits in executive functioning, attention, memory and motor skills were associated with lower SQoL. One recent study on bias of PRO ratings by psychiatric symptoms and cognitive deficits at the item level identified no effect of cognitive deficits on the responses to single items and no effect of symptoms on the responses of only two single items. The study concluded that the magnitude of any response bias through symptoms or cognitive deficits, if present, is small and unlikely to be of clinical significance. Reference Reininghaus, McCabe, Burns, Croudace and Priebe74

Discussion

Our review examining concepts and measures of four established PROs in the evaluation of treatments for psychosis generated at least three important findings. First, despite the increasing popularity of PROs with numerous concepts and measures, evidence of their methodological quality remains limited. Second, there is a considerable conceptual, operational and empirical overlap across measures designed to assess different PROs, although some concepts and measures also included aspects specific to individual PROs. Last, the influence of (or bias by) cognitive deficits and psychiatric symptoms appears limited and unlikely to be of clinical significance.

Limitations

The review has several limitations. The findings may be biased, as important references on concepts, characteristics and psychometric properties of PRO measures may have been missed. Concepts that might be relevant for one of the four PROs, but were not captured in an existing measure, were not included. The review was selective in examining concepts and measures of only four PROs and only a limited number of psychometric properties. Although Mokkink et al achieved a degree of consensus on the terminology and definitions of psychometric properties and provided guidance on data synthesis for reviews of the methodological quality of studies investigating psychometric properties of PROs, Reference Mokkink, Terwee, Patrick, Alonso, Stratford and Knol14 there is no consensus on how to synthesise findings on psychometric properties per se. We classified PROs according to whether or not they assessed specific psychometric properties. Given the absence of a consensus, this did not include ratings of the extent to which these psychometric properties were met. Finally, given the nature of conceptual and methodological reviews, Reference Lilford, Richardson, Stevens, Fitzpatrick, Edwards and Rock11,Reference Mokkink, Terwee, Patrick, Alonso, Stratford and Knol14 there may have been a subjective bias of the authors in the analysis and interpretation of the literature.

Methodological quality of PROs

Over the past decades numerous concepts and measures of PROs have emerged. Reference Cramer, Rosenheck, Xu, Thomas, Henderson and Charney75Reference Van Nieuwenhuizen, Schene, Boevink and Wolf77 In contrast, our review found only limited evidence of their methodological quality. Several measures were not linked to specific concepts. A number of measures were long and time-consuming to administer. This may imply undue assessment burden on patients with psychosis as well as increased assessment costs. For most measures there was no evidence on their measurement precision throughout the full range of scores, as has been established by a few studies for observer-rated outcome measures in mental health, Reference Uher, Farmer, Maier, Rietschel, Hauser and Marusic78 and, on a larger scale, for PROs in other medical disorders. Reference Cella, Riley, Stone, Rothrock, Reeve and Yount79 The methods used to assess structural validity were largely not appropriate for ordinal data. Reference Flora and Curran80 Only a few of the reviewed studies conducted analyses based on confirmatory factor analysis for ordinal data or item response modelling. Reference Flora and Curran80,Reference Embretson and Reise81 There are several implications of treating ordinal data as continuous, including attenuated relationships among PRO items in the presence of floor or ceiling effects, presence of pseudofactors and incorrect parameter estimates. Reference Brown82 These may challenge findings on the structural validity of PRO measures. In other words, measures using Likert scales, which have not been examined with psychometric methods appropriate for ordinal data, may be impaired in their ability to summarise patients’ item responses into scores that adequately reflect their dimensional structure. This is, however, central for the use of PROs in the evaluation of care, as such scores provide the basis on which value is assigned to treatments.

Conceptual, operational and empirical overlap

The conceptual, operational and empirical overlap of PROs has several implications for the validity of existing PRO measures. Campbell & Fiske, in their seminal work on discriminant and convergent validity, stated:

‘One cannot define without implying distinctions, and the verification of these distinctions is an important part of the validational process’ (p. 84). Reference Campbell and Fiske83

The verification of distinctions appears to be a part of the validational process that has been neglected by most of the research into PROs. New concepts were often proposed without assessing whether they were sufficiently distinct from existing concepts to warrant them being measured separately. This review suggests that an insufficient distinction between PROs at the conceptual level has led to a considerable overlap in the content of specific domains. This implies that, both at a conceptual and operational level, the requirements for establishing discriminant validity were not sufficiently considered when developing PROs. Empirically this may limit the ability of established measures to capture variance specific to the given concept. Indeed, this points towards substantial empirical overlap across measures. Although such overlap may reflect real associations between different PROs (e.g. one PRO influencing another), it still impairs the ability of each PRO measure to capture distinct information and, in psychometric terms, their discriminant validity. Reference Campbell and Fiske83 However, some concepts and operationalisations included aspects that were specific to one or more PROs. Recent evidence suggests that PROs may reflect both a general appraisal tendency that uniformly influences all PRO ratings in a positive or negative direction and components that are specific for each PRO. The specific information is independent of the general appraisal tendency. Maximising the specific information may be a challenge for future scale improvements.

Influence of cognitive deficits and psychiatric symptoms

In contrast to the concerns of some authors that the validity of existing PRO measures might be impaired owing to the influence of psychiatric symptoms and cognitive deficits, Reference Atkinson, Zibin and Chuang9,Reference Epstein, Hall, Tognetti, Son and Conant10 findings from our review suggested that the influence of (or bias by) cognitive deficits and psychiatric symptoms is very limited. The identified associations of PROs with symptoms and deficits do not compromise their validity as independent outcome criteria. However, all the evidence was taken from patients who consented to participate in research and were seen as capable of providing reasonable responses. Patients with high symptom levels may have been excluded from such studies, by clinicians or researchers. There is no evidence of a possible threshold of general or specific symptoms above which PROs might yield less reliable results.

Routine use of PROs

The conceptualisation and measurement of PROs in patients with psychosis are of practical relevance. These measures have an intuitive appeal for various stakeholder groups and there are calls to use them routinely across mental health services. 3 Even though evidence on the methodological quality of PROs is limited overall, there are at least five recommendations that can be made about the routine use of PROs in the evaluation of treatments for psychosis.

  1. (a) It should be carefully considered which PRO is relevant to the aim and approach of the given service, and what the implications of its results would be for service delivery and development.

  2. (b) The use of several PRO measures should be avoided unless they address clearly distinct domains.

  3. (c) Measures with evidence of good psychometric properties should be preferred; the evidence on psychometric properties is limited for most measures. Overall, measures using satisfaction-based concepts (e.g. assessing satisfaction with life domains or with treatment) have been more rigorously studied than others.

  4. (d) In the absence of evidence showing that longer measures have superior properties, shorter measures should be prioritised to minimise the burden and costs of measurement. However, longer measures tend to be more reliable, and there can be a trade-off between brevity and psychometric quality.

  5. (e) The influence of symptoms and cognitive deficits is unlikely to affect findings in small samples (although even a small explained variance may be relevant for research in large samples).

Future research

Despite the popularity of PROs for measuring the quality of routine mental healthcare, there are a number of conceptual and methodological shortcomings. Although according to our main findings this includes considerable conceptual, operational and empirical overlap across measures designed to assess different PROs, the influence of cognitive deficits and psychiatric symptoms appears limited. There is a need for more rigorous research to identify short measures that assess distinct PROs independent from overlap with highest possible precision. New methods such as item response modelling, item banking and computerised adaptive testing may help move this forward. Reference Reininghaus, McCabe, Burns, Croudace and Priebe74,Reference Cella, Riley, Stone, Rothrock, Reeve and Yount79 Although such methods have been infrequently used in psychiatric studies, Reference Uher, Farmer, Maier, Rietschel, Hauser and Marusic78 they have led to progress in measuring PROs in other medical conditions. A prominent example is the Patient-Reported Outcomes Measurement Information System (PROMIS). Reference Cella, Riley, Stone, Rothrock, Reeve and Yount79 Computerised adaptive testing iteratively selects the item providing the highest precision for a given patient until a desired level of precision is achieved. This minimises the number of items each patient has to complete, Reference Rose, Bjorner, Becker, Fries and Ware84 and can be implemented on handheld electronic devices. Ideally, conceptual and methodological work should be linked in future research to advance the measurement of PROs in patients with psychosis, so that concepts can both inform research and be refined on the basis of empirical data.

Funding

This work was supported by a research training fellowship funded by the UK National Institute of Health Research to U.R. The report is independent research and the views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Resaerch or the Department of Health.

Footnotes

Declaration of interest

None.

References

1 Priebe, S, Gruyters, T, Heinze, M, Hoffmann, C, Jakel, A. Subjective evaluation criteria in psychiatric care – methods of assessment for research and general practice [in German]. Psych Prax 1995; 22: 140–4.Google ScholarPubMed
2 Hansson, L, Bjorkman, T, Priebe, S. Are important patient-rated outcomes in community mental health care explained by only one factor? Acta Psychiatr Scand 2007; 116: 113–8.Google Scholar
3 Department of Health. Equity and Excellence: Liberating the NHS. Department of Health, 2010.Google Scholar
4 US Food & Drug Administration. Guidance for Industry. Patient-reported Outcome Measures: Use in Medical Product Development to Support Labeling Claim. FDA, 2009.Google Scholar
5 McCabe, R, Saidi, M, Priebe, S. Patient-reported outcomes in schizophrenia. Br J Psychiatry 2007; 191 (suppl 50): s218.Google Scholar
6 Kilian, R, Angermeyer, M. Quality of life in psychiatry as an ethical duty: from the clinical to the societal perspective. Psychopathology 1999; 32: 127–34.Google Scholar
7 Slade, M, McCrone, P, Kuipers, E, Leese, M, Cahill, S, Parabiaghi, A, et al. Use of standardised outcome measures in adult mental health services. Randomised controlled trial. Br J Psychiatry 2006; 189: 330–6.CrossRefGoogle ScholarPubMed
8 Priebe, S, McCabe, R, Bullenkamp, J, Hansson, L, Lauber, C, Martinez-Leal, R, et al. Structured patient–clinician communication and 1-year outcome in community mental healthcare. Cluster randomised controlled trial. Br J Psychiatry 2007; 191: 420–6.Google Scholar
9 Atkinson, M, Zibin, S, Chuang, H. Characterizing quality of life among patients with chronic mental illness: a critical examination of the self-report methodology. Am J Psychiatry 1997; 154: 99105.Google Scholar
10 Epstein, A, Hall, J, Tognetti, J, Son, L, Conant, L. Using proxies to evaluate quality of life: can they provide valid information about patients' health status and satisfaction with medical care? Med Care 1989; 27 (suppl 3): s918.Google Scholar
11 Lilford, R, Richardson, A, Stevens, A, Fitzpatrick, R, Edwards, S, Rock, F, et al. Issues in methodological research: perspectives from researchers and commissioners. Health Technol Assess 2001; 5: 157.CrossRefGoogle ScholarPubMed
12 Morgan, C, Burns, T, Fitzpatrick, R, Pinfold, V, Priebe, S. Social exclusion and mental health. Conceptual and methodological review. Br J Psychiatry 2007; 191: 477–83.Google Scholar
13 Fitzpatrick, R, Davey, C, Buxton, M, Jones, D. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998; 2.Google Scholar
14 Mokkink, L, Terwee, C, Patrick, D, Alonso, J, Stratford, P, Knol, D, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63: 737–45.Google Scholar
15 Ware, J, Snyder, M, Wright, W, Davies, A. Defining and measuring patient satisfaction with medical care. Eval Progr Plan 1983; 6: 247–63.Google Scholar
16 Ruggeri, M, Dall'Agnola, R, Agostini, C, Bisoffi, G. Acceptability, sensitivity and content validity of the VECS and VSSS in measuring expectations and satisfaction in psychiatric patients and their relatives. Soc Psychiatry Psychiatr Epidemiol 1994; 29: 265–76.Google Scholar
17 Phelan, M, Slade, M, Thornicroft, G, Dunn, G, Holloway, F, Wykes, T, et al. The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. Br J Psychiatry 1995; 167: 589–95.CrossRefGoogle ScholarPubMed
18 Bordin, E. The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy 1976; 16: 252–60.Google Scholar
19 Priebe, S, Gruyters, T. The role of the helping alliance in psychiatric community care. A prospective study. J Nerv Ment Dis 1993; 181: 552–7.Google Scholar
20 Pinsof, W, Catherall, D. The integrative psychotherapy alliance: family, couple and individual therapy scales. J Marital Fam Therapy 1986; 12: 137–51.Google Scholar
21 Freud, S. On beginning the treatment. Reprinted (1953–1974) in the Standard Edition of the Complete Psychological Works of Sigmund Freud (trans & ed Strachey, J), vol 12. Hogarth Press.Google Scholar
22 Sterba, R. The fate of the ego in analytic therapy. Int J Psychoanalysis 1934; 115: 117–26.Google Scholar
23 Lauer, G. Concepts of quality of life in mental health care. In Quality of Life in Mental Health Care (eds Priebe, S, Oliver, J, Kaiser, W): 1934. Wrightson, 1999.Google Scholar
24 Stevens, A, Gabbay, J. Needs assessment needs assessment. Health Trends 1991; 23: 20–3.Google Scholar
25 Catty, J. ‘The vehicle of success’: theoretical and empirical perspectives on the therapeutic alliance in psychotherapy and psychiatry. Psychol Psychother 2004; 77: 255–72.Google Scholar
26 Catty, J, Winfield, H, Clement, S. The therapeutic relationship in secondary mental health care: a conceptual review of measures. Acta Psychiatr Scand 2007; 116: 238–52.Google Scholar
27 Zautra, A, Goodhard, D. Quality of life indicators: a review of the literature. Commun Ment Health Rev 1979; 4: 310.Google Scholar
28 Clare, A, Corney, R, Cairns, V. Social adjustment: the design and use of an instrument for social work and social work research. Br J Soc Work 1984; 14: 323–36.Google Scholar
29 Department of Health Social Services Inspectorate. Care Management and Assessment: Summary of Practice Guidance. HMSO, 1991.Google Scholar
30 Gibbons, R, Bock, D, Hedeker, D, Weiss, D, Segawa, E, Bhaumik, D, et al. Full-information bifactor analysis for graded response data. Appl Psychol Meas 2007; 31: 419.CrossRefGoogle Scholar
31 Lehman, A. A quality of life interview for the chronically mentally ill. Eval Progr Plan 1988; 11: 5162.Google Scholar
32 Lehman, A, Ward, N, Linn, L. Chronic mental patients: the quality of life issue. Am J Psychiatry 1982; 139: 1271–6.Google Scholar
33 Brooks, R. EuroQol: the current state of play. Health Policy 1996; 37: 5372.CrossRefGoogle ScholarPubMed
34 EuroQol Group. EuroQol – a new facility for the measurement of health-related quality of life. Health Policy 1990; 16: 199206.Google Scholar
35 Prieto, L, Novick, D, Sacristan, J, Edgell, G. A Rasch model analysis to test the cross-cultural validity of the EuroQoL-5D in the Schizophrenia Outpatient Health Outcomes study. Acta Psychiatr Scand 2003; 107: 24–9.CrossRefGoogle Scholar
36 Uttaro, T, Lehman, A. Graded response modeling of the Quality of Life Interview. Eval Progr Plan 1999; 22: 4152.Google Scholar
37 De Weert-van Oene, G, Havenaar, J, Schrijvers, A. Self-assessment of need for help in patients undergoing psychiatric treatment. Psychiatry Res 2009; 167: 221–30.Google Scholar
38 Lasalvia, A, Bonetto, C, Tansella, M, Stefani, B, Ruggeri, M. Does staff–patient agreement on needs for care predict a better mental health outcome? A 4-year follow-up in a community service. Psychol Med 2008; 38: 123–33.CrossRefGoogle Scholar
39 McCabe, R, Roder-Wanner, U, Hoffmann, K, Priebe, S. Therapeutic relationships and quality of life: association of two subjective constructs in schizophrenia patients. Int J Soc Psychiatry 1999; 45: 276–83.Google Scholar
40 Priebe, S, McCabe, R. The therapeutic relationship in psychiatric settings. Acta Psychiatr Scand Suppl 2006; 429: 6972.Google Scholar
41 Slade, M, Leese, M, Cahill, S, Thornicroft, G, Kuipers, E. Patient-rated mental health needs and quality of life improvement. Br J Psychiatry 2005; 187: 256–61.Google Scholar
42 Priebe, S, Kaiser, W, Huxley, P, Roder-Wanner, U, Rudolf, H. Do different subjective evaluation criteria reflect distinct constructs? J Nerv Ment Dis 1998; 186: 385–92.Google Scholar
43 Fakhoury, W, Kaiser, W, Roeder-Wanner, U, Priebe, S. Subjective evaluation: is there more than one criterion? Schizophr Bull 2002; 28: 319–27.Google Scholar
44 Reininghaus, U, McCabe, R, Burns, T, Croudace, T, Priebe, S. Measuring patients' views: a bi-factor model of distinct patient-reported outcomes in psychosis. Psychol Med 2011; 41: 277–89.Google Scholar
45 Norholm, V, Bech, P. Quality of life in schizophrenic patients: association with depressive symptoms. Nord J Psychiatry 2006; 60: 32–7.CrossRefGoogle ScholarPubMed
46 Oersel, S, Akdemir, A, Dag, I. The sensitivity of quality-of-life scale WHOQOL-100 to psychopathological measures in schizophrenia. Compr Psychiatry 2004; 45: 5761.Google Scholar
47 Wilkinson, G, Hesdon, B, Wild, D, Cookson, R, Farina, C, Sharma, V, et al. Self-report quality of life measure for people with schizophrenia: the SQLS. Br J Psychiatry 2000; 177: 42–6.Google Scholar
48 Endicott, J, Nee, J, Harrison, W, Blumenthal, R. Quality of Life Enjoyment and Satisfaction Questionnaire: a new measure. Psychopharmacol Bull 1993; 29: 321–6.Google ScholarPubMed
49 Ritsner, M, Kurs, R, Ratner, Y, Gibel, A. Condensed version of the Quality of Life Scale for schizophrenia for use in outcome studies. Psychiatry Res 2005; 135: 6575.Google Scholar
50 Corrigan, P, Buican, B. The construct validity of subjective quality of life for the severely mentally ill. J Nerv Ment Dis 1995; 183: 281–5.Google Scholar
51 Eack, S, Newhill, C. Psychiatric symptoms and quality of life in schizophrenia. Schizophr Bull 2007; 33: 1225–37.Google Scholar
52 Mechanic, D, McAlpine, D, Rosenfield, S, Davis, D. Effects of illness attribution and depression on the quality of life among persons with serious mental illness. Soc Sci Med 1994; 39: 155–64.Google Scholar
53 Pukrop, R, Schlaak, V, Moller-Leimkuhler, A, Albus, M, Czernik, A, Klosterkotter, J, et al. Reliability and validity of quality of life assessed by the Short-Form 36 and the Modular System for Quality of Life in patients with schizophrenia and patients with depression. Psychiatry Res 2003; 119: 6379.CrossRefGoogle ScholarPubMed
54 Price, M, Hill, C, Williams, V, Morlock, R, Leeuwenkamp, O, Patterson, T. Measures of functional status and quality-of-life in schizophrenia. Curr Psychiatry Rev 2008; 4: 2838.Google Scholar
55 Sim, K, Mahendran, R, Siris, S, Heckers, S, Chong, S. Subjective quality of life in first episode schizophrenia spectrum disorders with comorbid depression. Psychiatry Res 2004; 129: 141–7.CrossRefGoogle ScholarPubMed
56 Vatne, S, Bjorkly, S. Empirical evidence for using subjective quality of life as an outcome variable in clinical studies: a meta-analysis of correlates and predictors in persons with a major mental disorder living in the community. Clin Psychology Rev 2008; 28: 869–89.Google Scholar
57 Katsakou, C, Priebe, S. Outcomes of involuntary hospital admission – a review. Acta Psychiatr Scand 2006; 114: 232–41.Google Scholar
58 Katsakou, C, Bowers, L, Amos, T, Morriss, R, Rose, D, Wykes, T, et al. Coercion and treatment satisfaction among involuntary patients. Psychiatr Serv 2010; 61: 286–92.Google Scholar
59 Grinshpoon, A, Ponizovsky, A. The relationships between need profiles, clinical symptoms, functioning and the well-being of inpatients with severe mental disorders. J Eval Clin Pract 2008; 14: 218–25.CrossRefGoogle ScholarPubMed
60 Mojtabai, R, Fochtmann, L, Chang, S, Kotov, R, Craig, T, Bromet, E. Unmet need for mental health care in schizophrenia: an overview of literature and new data from a first-admission study. Schizophr Bull 2009; 35: 679–95.Google Scholar
61 Ochoa, S, Haro, J, Autonell, J, Pendas, A, Teba, F, Marquez, M. Met and unmet needs of schizophrenia patients in a Spanish sample. Schizophr Bull 2003; 29: 201–10.Google Scholar
62 Priebe, S, Reininghaus, U, McCabe, R, Burns, T, Eklund, M, Hansson, L, et al. Factors influencing subjective quality of life in patients with schizophrenia and other mental disorders: a pooled analysis. Schizophr Res 2010; 121: 251–8.Google Scholar
63 Priebe, S, McCabe, R, Junghan, U, Kallert, T, Ruggeri, M, Slade, M, et al. Association between symptoms and quality of life in patients with schizophrenia: a pooled analysis of changes over time. Schizophr Res 2011; 133: 1721.Google Scholar
64 Fujii, D, Wylie, A, Nathan, J. Neurocognition and long-term prediction of quality of life in outpatients with severe and persistent mental illness. Schizophr Res 2004; 69: 6773.Google Scholar
65 Addington, J, Addington, D. Neurocognitive and social functioning in schizophrenia: a 2.5 year follow-up study. Schizophr Res 2000; 44: 4756.Google Scholar
66 Brekke, J, Kohrt, B, Green, M. Neuropsychological functioning as a moderator of the relationship between psychosocial functioning and the subjective experience of self and life in schizophrenia. Schizophr Bull 2001; 27: 697708.Google Scholar
67 Green, M, Kern, R, Heaton, R. Longitudinal studies of cognition and functional outcome in schizophrenia: implications for MATRICS. Schizophr Res 2004; 72: 4151.Google Scholar
68 Skantze, K, Malm, U, Dencker, SJ, May, PR, Corrigan, P. Comparison of quality of life with standard of living in schizophrenic out-patients. Br J Psychiatry 1992; 161: 797801.Google Scholar
69 Smith, D, Mar, C, Turoff, B. The structure of schizophrenic symptoms: a meta-analytic confirmatory factor analysis. Schizophr Res 1998; 31: 5770.Google Scholar
70 Wegener, S, Redoblado-Hodge, M, Lucas, S, Fitzgerald, D, Harris, A, Brennan, J. Relative contributions of psychiatric symptoms and neuropsychological functioning to quality of life in first-episode psychosis. Aust N Z J Psychiatry 2005; 39: 487–92.Google Scholar
71 Galletly, C, Clark, C, McFarlane, A, Weber, D. Relationships between changes in symptom ratings, neurophysiological test performance and quality of life in schizophrenic patients treated with clozapine. Psychiatry Res 1997; 72: 161–6.CrossRefGoogle ScholarPubMed
72 Ritsner, M. Comparison of instruments for measuring the quality of life impairment syndrome in severe mental disorders. In Quality of Life Impairment in Schizophrenia, Mood and Anxiety Disorders: New Perspectives on Research and Treatment (eds Ritsner, M, Awad, A): 133–42. Springer, 2007.Google Scholar
73 Sota, T. Outcome in schizophrenia: are cognitive variables predictors of rehospitalization and quality of life? Diss Abstr Int 2000; 60: 4253.Google Scholar
74 Reininghaus, U, McCabe, R, Burns, T, Croudace, T, Priebe, S. The validity of subjective quality of life measures in psychotic patients with severe psychopathology and cognitive deficits: an item response model analysis. Qual Life Res 2012; 21: 237–46.Google Scholar
75 Cramer, J, Rosenheck, R, Xu, W, Thomas, J, Henderson, W, Charney, D. Quality of life in schizophrenia: a comparison of instruments. Schizophr Bull 2000; 26: 659–66.Google Scholar
76 Lehman, A. Measures of quality of life among persons with severe and persistent mental disorders. Soc Psychiatry Psychiatr Epidemiol 1996; 31: 7888.Google Scholar
77 Van Nieuwenhuizen, C, Schene, A, Boevink, W, Wolf, J. Measuring the quality of life of clients with severe mental illness. Psychiatr Rehab J 1997; 20: 3341.Google Scholar
78 Uher, R, Farmer, A, Maier, W, Rietschel, M, Hauser, J, Marusic, A, et al. Measuring depression: comparison and integration of three scales in the GENDEP study. Psychol Med 2008; 38: 289300.Google Scholar
79 Cella, D, Riley, W, Stone, A, Rothrock, N, Reeve, B, Yount, S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol 2010; 63: 1179–94.Google Scholar
80 Flora, D, Curran, P. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol Methods 2004; 9: 466–91.Google Scholar
81 Embretson, S, Reise, S. Item Response Theory for Psychologists. Erlbaum, 2000.Google Scholar
82 Brown, T. Confirmatory Factor Analysis for Applied Research. Guilford, 2006.Google Scholar
83 Campbell, D, Fiske, D. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 1959; 56: 81105.Google Scholar
84 Rose, M, Bjorner, J, Becker, J, Fries, J, Ware, J. Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). J Clin Epidemiol 2008; 61: 1733.CrossRefGoogle ScholarPubMed
Figure 0

FIG. 1 Results of database search. PRO, patient-reported outcome.

Supplementary material: PDF

Reininghaus and Priebe supplementary material

Supplementary Material

Download Reininghaus and Priebe supplementary material(PDF)
PDF 404.8 KB
Submit a response

eLetters

No eLetters have been published for this article.