Subjective assessments of comorbidity correlate with quality of life health outcomes: Initial validation of a comorbidity assessment instrument

The goal of caring for persons with chronic medical conditions is frequently to maximize quality of life (QOL) rather than to 'cure' illness. Therefore interventions to improve processes of care for this population often assess QOL outcomes such as physical functioning, overall health status, and emotional well being. These outcomes are, by definition, subjective. The values assigned to these outcomes are most meaningful to the patients themselves. However, these subjective outcomes have been shown to correlate with mortality, health care utilization, job loss, and many other more 'quantifiable' outcomes [1‐3].

The outcomes of a chronic condition may be affected by coexisting (comorbid) chronic conditions as well as the index condition of interest and analyses must adjust for this effect of comorbidity. Multiple instruments have been developed and validated to quantify comorbidity for purposes of statistical adjustment and clinical decision making. The majority of these use medical record review or administrative data as sources of information; observation during clinical encounters and self report have also been used for this purpose. These instruments have primarily been validated against 'objective' health outcomes such as mortality, length of stay, and cost of care [4‐13]. We are aware of two such instruments that have been validated against QOL outcomes [5, 14]. In addition, many of these instruments were designed for use in hospitalized patients or populations characterized by specific illnesses.

Self-reported information about comorbidity and the burden it imposes can provide information about the concurrent impact of multiple disease states on QOL outcomes. Self-reported comorbidity information is also efficient in studies in which other information, such as QOL outcomes, is collected by survey. Instruments designed to assess comorbidity by self-report have reported significant correlations between comorbidity score and utilization, QOL, mortality and hospitalization [15‐20].

It is important to incorporate assessment of disease severity into comorbidity measurement [6]. Some self-report instruments incorporate various weighting systems for this purpose and two of these have been validated in hospitalized populations [15, 18]. We have developed a self-report instrument that incorporates disease severity by quantifying the respondent's subjective 'disease burden' which we define as the number of self-identified comorbid conditions weighted by the degree to which each condition limits daily activity. We hypothesized that a subjective measure of comorbidity such as this may be more strongly correlated with QOL outcomes than measures of comorbidity previously validated against other, more objective, health outcomes.

Our goals in this investigation were to validate this newly-developed instrument against a presumed 'gold standard' of chart review, and to conduct an initial comparison of this instrument with other well known measures of comorbidity (chart review of number of conditions, the Charlson Comorbidity Index and the RxRisk score) by correlating these measures with selected QOL outcomes.

After two consecutive single mailings, 157 individuals completed the survey. The response rate of 28% was obtained without the use of strategies typically employed to increased response, such as multiple mailings or more active follow-up. Characteristics of respondents are noted in Table 1. Mean age was 75, health status ranged from excellent to poor and respondents reported an average of 5.9 chronic conditions. Respondents did not differ from non-respondents with regard to age, gender, number of chronic conditions (as estimated by the initial screen with the RxRisk instrument), or duration of HMO membership.

One hundred fifty-one respondents reported at least one of the conditions and 6 reported none. In analyses by condition, median sensitivity of patient report of a condition relative to a 'gold standard' of chart review was 75% (range 35% to 100%) and median specificity was 92% (range 61% to 100%). In analyses by respondent, sensitivities (agreement on number of conditions positive relative to chart review) ranged from 14% (n = 1) to 100% (n = 53); the median was 83%. Sensitivities were not calculated for the ten respondents who did not agree with the chart on any conditions, including those who agreed with their medical record that they had none of the conditions (n = 2). Specificities by respondent ranged from 59% (n = 1) to 100% (n = 34); the median was 91%. Sensitivity and specificity of self-report of each condition relative to chart review are reported in Table 2. (Not included on the table are results for 2 of the original 25 conditions: liver disease and alcoholism. Two respondents and one separate chart reported alcohol abuse, and no respondents or charts reported liver disease.)

In order to assess the relative contributions of self reported diseases and disease count by chart review to the outcomes of general health status and physical functioning, we entered these two variables into limited logistic regression models. In these models containing only these two variables, the predictive accuracy of the model (as measured by the c-statistic) was not significantly different using each of the two variables, implying comparable contributions of either measure. C-statistics for overall health status ranged from 0.521 ("other rheumatic disease) to 0.669 ("osteoarthritis"); and for physical functioning ranged from 0.515 ("other rheumatic disease") to 0.679 ("overweight").

QOL outcomes of interest correlated most strongly with self-reported disease burden, followed by number of conditions by chart review, self-reported number of conditions, the Charlson index score and the RxRisk score. Although all measures of comorbidity except the RxRisk score showed comparable p values (p <= 0.001) for the outcomes of health status and physical functioning, the correlations for disease burden were significantly stronger than those for self-reported number of conditions or Charson comorbidity score for these outcomes. Table 3 lists these correlations for our primary outcomes of interest – overall health status and physical functioning – and our secondary outcomes of positive screen for depression and self-efficacy.

It is important to incorporate assessment of comorbidity into studies involving QOL outcomes for persons with chronic medical conditions, as coexisting conditions may substantially affect outcomes of interest such as physical functioning, overall health status, depression and self-efficacy. In our study population, patients with multiple chronic medical conditions accurately reported a majority of common comorbid conditions relative to chart review. In addition, they were aware of most of their own diagnoses. Furthermore, self-reported disease burden correlated well with QOL outcomes, and correlated more strongly than did the two other measures of comorbidity that we used for comparison. This is consistent with our hypothesis that, for investigations using QOL outcomes, it is most appropriate to adjust for comorbidity using a subjective measure of comorbidity.

Previous investigations that have compared self-report with administrative data reported 59–79%, 72–73%, and 78–83% agreement on diagnoses of hypercholesterolemia, diabetes, and hypertension respectively; and 56% and 69% agreement on stroke and myocardial infarction [30, 34]. In our investigation we expanded the number of conditions for comparison to 23 and additionally assessed respondents' tendencies to accurately report all of their own conditions. Certain diagnoses were reported with high levels of sensitivity and specificity, while others were not.

A sensitivity greater than specificity may be due to either 'over-reporting' by participants or 'under-reporting' in the chart. Examples from our list included asthma, back pain, overweight and hard-of-hearing. We suspect that, for the first case, some participants reported COPD as asthma. For the remaining cases, we suspect that the conditions were under-reported in the chart – either because they had not been brought to medical attention or because they had not been assessed as isolated problems in the context of medical visits during the period covered by the chart review.

Sensitivity was substantially less than specificity for angina, nerve conditions, cancer and kidney disease. Although there may be a tendency to under-report chronic conditions, and respondents are more likely to report conditions with more severe symptoms [17, 35]; we re-reviewed charts of persons with these diagnoses to see if we could determine the cause of the discrepancies. From these repeat chart reviews, we concluded that these discrepancies were due to wording based more on symptoms than diagnosis (angina), under-reporting of conditions with stable or few symptoms (renal and neurological), and possible perceptions of cure or remission after acute treatment (cancer). In addition we analyzed the demographic and health characteristics (from Table 1) of respondents for each of these four conditions to see if any demographic or disease characteristics were likely to predict a low agreement with chart review and found no patterns.

In our assessments of sensitivity and specificity, we assumed that the presence of a diagnosis in the chart was a 'gold standard' – an assumption that may not be entirely accurate. We suspect that diagnoses for which there are obvious medical treatments – especially medications – are more likely to be recorded in the chart. Chart diagnoses may be less accurate for conditions for which a person is less likely to seek (or for which a provider is less likely to offer) specifically biomedical solutions.

We found a high correlation between our measure of disease burden and our QOL outcomes of interest, as compared to lower correlations between two other comorbidity indices and these same outcomes. However, the correlations between the other comorbidity indices and health status and physical functioning were also significant and have been noted previously [36]. The correlations between the Charlson and RxRisk scores and our secondary outcomes of interest (depression screen and self-efficacy) were not significant. Based on the pattern of these associations, we suggest that assessment of comorbidity is a function of the outcome of interest, the population studied, and the different (subjective versus objective) aspects of comorbidity measured by each instrument. The effect of comorbidities on QOL outcomes may be most accurately assessed when subjective measures are used to adjust for comorbidity. In contrast, for situations in which mortality, for example, is the outcome of interest, comorbidity should be assessed using instruments that have been developed for that purpose. These suggestions are consistent with the notion that 'complete' measurement of all health states requires both self-reported and objectively reported measures [37].

It is certainly possible that one comorbidity measure may work for many situations. Other self-report instruments have been shown to predict mortality and hospitalization in addition to QOL [15, 16, 18]. We are also aware of at least two investigations in which comorbidity measured by chart review correlated with QOL outcomes [5, 14]. The two instruments with which we compared our own instrument use different methodologies and were originally developed to assess comorbidity in studies investigating the objective outcomes of mortality and cost of care respectively [4, 7]. The Charlson index has been subsequently validated against length of stay, post operative complications, discharge to nursing home, disability, hospital readmission and hospital charges [6, 8, 38‐40]. The RxRisk score has subsequently been adapted and validated against administrative data on diagnoses and disease burden in certain populations [4, 32]. Our investigation adds to the growing body of knowledge on measuring comorbidity by highlighting the different results that may be obtained when using different methodologies to adjust for comorbidity in studies assessing QOL outcomes.

We did not incorporate additional measures of comorbidity, such as those that use administrative data into our analysis [8, 12, 13]. Previous comparative studies suggest that chart-review-based measures may be slightly more accurate than administrative data-based comorbidity measures in predicting objective outcomes such as mortality and length of hospital stay [6, 38, 41]. Further investigation is necessary to assess association of comorbidity measured by administrative data with QOL outcomes.

As with any initial validation effort, the generalizability of our conclusions is limited by the characteristics of the population studied – a relatively small HMO population aged 65 years or older. It is possible that this population is relatively 'well-educated' regarding the number and type of their medical conditions. If so, some of the sensitivities we report may be at the upper end of the spectrum that may be anticipated from self-report. In addition, we terminated the sampling process when we attained a sample size sufficient to test our primary hypothesis, without maximizing response rate. Thus, the findings in this sample may not represent the associations of a broader population. Although respondents did not differ significantly from non-respondents on RxRisk comorbidity score, more motivated or knowledgeable participants may have been more likely to respond promptly to our survey. Correlations and sensitivities could be lower when examined in a less motivated population or those with a lower knowledge base. Specifically, self-report may be less reliable in the geriatric sub-population that may suffer from cognitive impairment. Additional validation studies will be required in order to assess the usefulness of this instrument in other populations and for different QOL and other outcomes. We anticipate that these changes will strengthen our results for sensitivity in comparison to chart review and that they will not change the overall correlations with our outcomes of interest.

Disease burden (as we defined it) may in itself constitute a substantial portion of any patient's assessment of health status and physical functioning. Our incorporation of perceived limitation into a disease count may be similar to other investigations that have coupled a simple disease count with a health status measure such as the SF-36^® and found that doing so strengthened the relationship between comorbidity and utilization and mortality [16, 19]. However, models that attempt to explain the relationship between symptom burden, overall quality of life and physical functioning note that these outcomes are also affected by environmental characteristics, individual personality, expectations, values, and social and psychological supports [42, 43]. What we refer to as disease burden explains part, but not all, of our QOL outcomes as is illustrated by the values of our c-statistics. To the extent that investigations that use QOL outcomes concentrate on participants with one index condition and need to adjust for comorbidities, a subjective measure of disease burden using self-report may be an accurate way to account for the effect of other coexisting conditions with regard to that outcome.

Finally, depression is both an important potential comorbidity for anyone with chronic illness as well as an equally important component of the QOL outcome of emotional well being. We chose to treat it as the latter. As depression severity independently contributes to general QOL over and above other coexisting chronic illness, we suspect that including depression on our list of conditions would have increased the strength of correlations between self-reported disease burden and general health status [44, 45].

Assessing comorbidity is relevant to investigations of populations with multiple medical conditions and should be incorporated into the associated analyses. Not only is self-report likely to give a reasonable estimate of comorbidity, for investigations using QOL outcomes, self-reported disease burden (or other subjective assessments of comorbidity) may provide a more accurate comorbidity adjustment than measures that have been validated against other outcomes. If this finding is confirmed by additional investigation, subjective measures of comorbidity that incorporate disease severity should be added to QOL assessments for populations with high rates of comorbidity.