Validating a Brief Version of the Mental Health Recovery Measure for Individuals With Schizophrenia

Recovery from mental illness has been defined as a personal process of change through which individuals improve their health and well-being, live a self-directed life, and strive to reach their maximum potential (1). For more than a decade, U.S. policy has emphasized the need for transformation to recovery-oriented service delivery in mental health care systems (2,3). In recent years, the Veterans Health Administration (VHA) and numerous state mental health systems have begun to transform care to reflect principles of recovery, which include hope, empowerment, respect, and responsibility, and provide recovery-oriented services, such as peer support and employment services (1,4). Despite consensus that recovery is expected for individuals with mental illness and that health care transformation is needed to support their recovery, little guidance has been provided on the changes necessary in mental health care to reach these expectations. Along with care transformation, brief validated measures that can assess individual recovery are needed (5,6).

Although some recovery assessment measures exist, most consist of multiple domains that have not been well established as separate recovery constructs, are too cumbersome for routine practice, or are unpublished (7). One such measure is the Mental Health Recovery Measure (MHRM) (8), a 30-item scale that assesses perceived recovery for individuals with serious mental illness. Of the 30 MHRM items, 24 are categorized into one of six conceptual subscales that were derived from a grounded-theory recovery model based on qualitative interviews with mental health consumers (9): overcoming stuckness, self-empowerment, learning and self-redefinition, basic functioning, overall well-being, and new potentials. The scale developers later added four items that constitute a seventh MHRM subscale, advocacy/enrichment, as well as two items that assess spirituality (10).

In the MHRM developers’ samples (10,11), the measure has been found to have good internal consistency and test-retest reliability; has correlated positively with measures of empowerment, resilience, and community living; has correlated negatively with symptom distress (11); and has demonstrated an ability to detect change among individuals after completion of an evidence-based, recovery-promoting intervention (10,12). At the time of preparing this report, however, MHRM development data, including evaluation of the original measure’s factor structure, had not been published in a peer-reviewed journal.

This study aimed to examine the construct validity and reliability of the 30-item MHRM in a large, multisite sample of veterans with schizophrenia and to establish a brief, ten-item version of the MHRM that would be psychometrically comparable with the original measure and more easily administered in routine care settings. To accomplish these aims, this study examined baseline data from a recently completed multisite, clinic-level controlled trial, Enhancing Quality of Care in Psychosis (EQUIP), which evaluated effectiveness and implementation of evidence-based care for schizophrenia at mental health clinics in the VHA. Construct validity was assessed through factor analysis of the MHRM and correlational analyses of the 30- and ten-item versions of the MHRM with other recovery-relevant outcome measures. Reliability was assessed by calculating the internal consistency of the full and brief versions of the MHRM.

Methods

Sample selection

The study involved secondary analysis of data that were collected in the EQUIP trial. Details of the study method can be found elsewhere (13–15), but in brief, individuals were recruited from eight VHA medical centers across four regions of the country. Eligible individuals were at least 18 years old, had a diagnosis of schizophrenia or schizoaffective disorder, and had at least two mental health clinic visits within a six-month eligibility period. From the overall population of eligible patients, a random sample was identified at each site. Probability of inclusion was based on the overall eligible population, desired sample size, and expected nonparticipation.

Participants were recruited through clinician referrals, review of clinic rosters, and flyers posted in participating clinics. A partial Health Insurance Portability and Accountability Act waiver was obtained to allow review of patients’ charts in order to determine eligibility before approaching them at their clinical appointment to discuss the study.

Procedures

Eligible veterans were approached in person at clinic visits. Of 1,964 patients who were eligible, 530 were not approached, 633 refused to participate, and 801 consented to be enrolled (41%). All enrolled patients completed a baseline assessment between January 2008 and May 2009. At baseline, 797 patients completed the MHRM. Of those, data from two participants with extreme scores were considered to be invalid and were excluded. Thus a total of 795 individuals were included in the analyses. The study was approved by the institutional review boards of all eight VHA sites. The baseline assessment occurred before any intervention occurred in the EQUIP trial; therefore, intervention and control site data were examined together.

Instruments

The baseline assessment included demographic and clinical characteristics and perceived level of mental health recovery, quality of life, and satisfaction with mental health services. Psychiatric diagnosis was confirmed, and measures of psychiatric symptom severity and occupational, social, and symptomatic functioning were obtained. Research assistants administered the baseline assessment and were trained to a high level of reliability, and quality assurance checks were completed during the study.

Demographic characteristics

Information was obtained for age, sex, race, ethnicity, education, marital status, and employment status of patients.

Clinical characteristics

Psychiatric diagnosis of schizophrenia or schizoaffective disorder was confirmed with an abbreviated version of the Structured Clinical Interview for DSM-IV (16). Psychiatric symptoms were rated with the Brief Psychiatric Rating Scale (BPRS) (17,18). The BPRS is an 18-item, clinician-rated scale designed to assess change in severity of psychopathology, with an emphasis on symptoms of psychotic illnesses. BPRS items are rated on a Likert scale ranging from 1, not present, to 7, extremely severe. Three BPRS subscale scores (thought disturbance, anergia, and disorganization), as well as scores on the BPRS depressive mood item, were used as symptom measures. Possible scores on the thought disturbance and anergia subscales range from 4 to 28, and possible scores on the disorganization subscale range from 3 to 21, with higher scores for all three subscales indicating greater symptom severity. Full scores on the BPRS affect subscale, which includes the depressive mood item, could not be calculated because four of the five items that constitute that subscale (19) were not included in the baseline assessment.

Patient occupational, social, and symptomatic functioning during the week before baseline assessment were rated on the Mental Illness Research, Education and Clinical Centers version of the Global Assessment of Functioning scale (MIRECC GAF) (20). Possible scores on each of the three MIRECC GAF subscales range from 0, no information, to 100, fully functional. The MIRECC GAF has good interrater reliability, as well as good concurrent and predictive validity.

Mental health recovery.

The MHRM is a 30-item, self-report measure designed to assess perceived mental health recovery among individuals with serious mental illness (8,10,11). Items are rated on a 5-point scale ranging from 0, strongly disagree, to 4, strongly agree, for a total score range of 0–120. Higher MHRM scores indicate higher self-reported levels of mental health recovery. In addition to a total score, subscale scores can be computed by summing individual item scores within eight conceptual domains: overcoming stuckness (items 1–4), self-empowerment (items 5–8), learning and self-redefinition (items 9–12), basic functioning (items 13–16), overall well-being (items 17–20), new potentials (items 21–24), spirituality (items 25 and 26), and advocacy/enrichment (items 27–30).

In the test developers’ original normative sample of adults (N=279) with serious mental illness from community- and consumer-operated mental health settings, the mean±SD MHRM total score was 80±20, and internal consistency was good (Cronbach’s α=.93) (11). In the test developers’ more recent MHRM data collected from a larger but similar sample (N=671), mean±SD score and internal consistency results were similar (78.0±21.7, Cronbach’s α=.95) (11). The MHRM has demonstrated ability to detect reliable change among adults with serious mental illness after participation in a group intervention designed to promote personal recovery and empowerment (12).

Quality of life.

The Lehman Quality of Life Interview–Short Version (TL-30) is a reliable and valid self-report measure designed to assess several quality-of-life domains among individuals with serious mental illness (21,22). It has been used extensively in clinical and research settings. Respondents rate TL-30 items on a Likert scale ranging from 1, terrible, to 7, delighted, with higher scores indicating greater satisfaction with life quality. Scores on the TL-30 item “How do you feel about your life in general?” were used as a measure of subjective overall quality of life, and scores on the subjective TL-30 items were used as measures of satisfaction with leisure (“How do you feel about the amount of fun you have?”), general health (“How do you feel about your health in general?”), daily activities (“How do you feel about how you spend your time?”), friendships (“How do you feel about the amount of friendship in your life?”), finances (“How do you feel about how well off you are financially?”), family relationships (“How do you feel about the way things are in general between you and your family?”), and privacy (“How do you feel about the privacy you have where you live?”).

Satisfaction with mental health services.

The Client Satisfaction Questionnaire (CSQ-8) is an eight-item, patient self-report measure designed to assess general satisfaction with health and human services (23). Items are rated on a 1-to-4 scale, for a total score range of 8–32, with higher total scores indicating higher satisfaction with mental health services. The CSQ-8 has been used in various research and clinical settings, including mental health clinics, and demonstrates high internal consistency (Cronbach’s α=.92–.93) and consumer acceptance (24).

Statistical analyses

Exploratory factor analysis (EFA) with maximum-likelihood estimation was used to examine interpretable underlying constructs or factors of the 30-item MHRM. The number of factors to retain in a solution was based on eigenvalues and the scree test. To minimize administration burden associated with the MHRM, we sought to reduce the total number of scale items from 30 to ten, making an effort to retain the most homogeneous subset of items from the MHRM.

Means, standard deviations, and internal consistency coefficients (Cronbach’s alpha) were calculated for both the 30-item MHRM and the MHRM-10. Two-tailed Pearson correlations (r) were computed to examine the relationships of the MHRM and MHRM-10 with subjective measures of quality of life (TL-30 items) and satisfaction with mental health services (CSQ-8 total score), as well as with interviewer-rated measures of symptom severity (BPRS depressive mood item score and thought disturbance, anergia, and disorganization subscale scores) and occupational, social, and symptomatic functioning (MIRECC GAF subscale scores). All data analysis procedures were conducted with IBM SPSS Statistics for Windows, version 20.0 (25).

Results

Sample characteristics

Table 1 shows participants’ characteristics. The average participant was 54 years old, male, African American or Caucasian, not presently married, and had some education after high school.

Exploratory factor analysis

EFA procedures resulted in five factors with an eigenvalue greater than 1.0, which collectively explained 55.6% of the variance. Visual scree plot inspection, however, showed that a one-factor solution was a more appropriate option. Moreover, the four additional factors uniquely contributed to only 3.8%−5.5% of the overall variance of the measure. Hence, EFA procedures were repeated to force a one-factor solution, which explained 37.3% of the variability in the MHRM. Table 2 shows all 30 items and their factor loadings. Factor loadings above .40, which is a common cutoff for item inclusion in a factor score, indicate a moderate relationship between the item and latent recovery, whereas factor loadings above .70 indicate a strong relationship. All items but three loaded above .40 on the single factor, with 77% of the items loading above .50.

We chose the ten items with the highest factor loadings for our reduced scale (the MHRM-10), because these items most strongly represent the underlying recovery construct. The factor loadings on the ten selected items were all very high and of similar magnitude. The MHRM-10 included three items each from the MHRM developers’ suggested domains of overall well-being, new potentials, and learning and self-redefinition, as well as one item from the advocacy/enrichment domain.

Internal consistency

Participants’ total scores on the MHRM-10 were obtained by summing their ten individual item scores. Table 3 shows the mean±SD scores and the internal reliability coefficients for the MHRM and MHRM-10. Overall internal consistency of the MHRM-10 was good and comparable with that of the MHRM in this sample, as well as with reliability estimates observed in the MHRM developers’ two nonveteran normative samples (11). Although EFA results of this study suggested that the MHRM is best interpreted as a total score, reliability estimates for the MHRM developers’ eight suggested conceptual domains are presented (Table 3) for dissemination purposes.

Correlations

Table 4 shows results of correlation (Pearson r) analyses. Statistical correlations with measures of quality of life, satisfaction with care, symptoms, and functioning were very similar for the MHRM and the MHRM-10. Strong positive relationships were shown for both measures overall and with several domains of quality of life, and a strong negative relationship was observed between the instruments and depressive mood. Moderate positive associations were revealed between both recovery measures and subjective quality of friendships. Weak, but statistically significant, positive correlations were demonstrated between the recovery measures and subjective quality of finances, family relationships, and privacy. There were negligible, but similar and significant, relationships between the recovery measures and measures of satisfaction with care, thought disturbance, and functioning.

Discussion

To our knowledge, this is the first study to examine the factor structure and reliability of the 30-item MHRM in a large group of veterans with diagnosed schizophrenia and to assess the measure’s relation to both subjective and interviewer-rated recovery outcomes. The results suggest that a brief, ten-item version of the MHRM demonstrated psychometric properties equivalent to the full measure; because of its brevity the MHRM-10 could be more practical for routine and broad use in mental health care settings.

The study demonstrated that a single recovery factor adequately summarized the information in the MHRM. This finding lends support for the MHRM as a valid self-report assessment of overall mental health recovery and indicates that the subscales or conceptual domains originally suggested by the MHRM developers (8–10) may not be uniquely significant in measuring recovery because they did not emerge as empirically distinct factors in our study. These results are not surprising given that the MHRM subscales were qualitatively derived and are consistent with the MHRM developers’ focus on using the total score as a holistic assessment of recovery (11). With MHRM items loading strongly on one recovery factor in this study, our brief version of the measure consists of the ten items that loaded most strongly on that single factor. Both the MHRM and MHRM-10 demonstrated excellent internal consistency.

Our results demonstrated that the MHRM and MHRM-10 were moderately to strongly associated with measures of overall quality of life and quality of leisure, general health, daily activities, friendships, and depressive mood. This suggests that meaningful and enjoyable activities, social peer relationships, and overall sense of health may be important factors in individual recovery. Interestingly, these factors seem to be in concert with recovery-supporting dimensions recently identified by the Substance Abuse and Mental Health Services Administration, including health, purpose, and community (1). The weak correlations between recovery and quality of finances, family relationships, and privacy indicate that such variables, which often entail little to no individual choice among persons with serious mental illness, may not be particularly vital to personal recovery.

The negligible to nonexistent relationships of self-reported recovery with interviewer-rated psychotic symptoms (specifically, thought disturbance, anergia, and disorganization) and functioning (GAF scores) suggest that the MHRM and MHRM-10 are measuring something other than traditional, illness-based, recovery outcomes for individuals with schizophrenia. This is consistent with recent reports that subjective and objective measures of recovery may be somewhat independent or may measure divergent recovery constructs (26,27). Interviewer-rated depressive mood was an exception to this among our findings, in that it was moderately negatively related to subjective recovery. It is possible that the recovery factor measured by the MHRM and MHRM-10 is somewhat convergent with the concept of hope, given that hope has been noted as a crucial factor in consumer-based recovery definitions (1) and hopelessness is a hallmark of depression.

The negligible relationship between recovery and service satisfaction may indicate that consumers’ self-perceived level of recovery is relatively independent of their contentment with mental health services. In an era of patient-centered health care that places a premium on patient satisfaction (28), our results underscore the importance of measuring personal recovery distinctly from patient satisfaction or service-based outcomes.

A key limitation of the study was our inability to examine the sensitivity of the MHRM and MHRM-10 to change. Although Bullock and colleagues (12) demonstrated the MHRM’s ability to detect change in a sample of individuals with serious mental illness after completion of a recovery intervention, replication of the study findings is needed, as is evaluation of the MHRM-10’s ability to reliably detect change at various points along the recovery process, including pre- and postintervention. Similarly, future studies exploring the measures’ test-retest reliability and criterion validity would be beneficial. In addition, the items selected for the MHRM-10 were based on an a priori decision to reduce the MHRM to the ten most homogeneous items. An alternative item reduction approach might result in selection of a different set of items for a brief measure. Last, our sample consisted predominantly of middle-age, male veterans with schizophrenia who were enrolled in mental health services, which could limit generalizability of findings. Studies exploring the use of the MHRM-10 in a more demographically heterogeneous sample, as well as among other specific diagnostic groups (including bipolar or depressive disorders) would be of value.

Conclusions

This investigation provided initial support for the use of the MHRM-10 as a valid and reliable assessment of perceived recovery among individuals with schizophrenia. The MHRM-10 is brief and therefore likely to be easily integrated into routine care. Future research is needed, however, to examine additional psychometric properties of the measure, particularly its sensitivity to detect change in personal recovery.