Longitudinal construct validity of the minimum data set health status index

This study utilized a longitudinal, population-based, retrospective cohort of adult, home care patients in the Hamilton-Niagara-Haldimand-Brant region of Ontario, Canada. This health region is one of the largest health regions in Ontario with a population of over 1.5 million persons spread across urban, suburban, and rural locales. This health region contains over 10 municipalities with wide variations in population density, socioeconomic status, and access to tertiary care centers. Community-dwelling seniors receiving publicly-funded home care in Ontario are periodically assessed with the home care variant of the RAI-MDS, the RAI-MDS Home Care, as part of their routine care.

This study involved the retrospective analysis of routinely-collected health administrative data from secondary sources; there was no primary data collection. RAI-MDS assessments that are performed as part of routine care for home care patients in Ontario are stored in the Client and Health Information System (CHRIS), which is the health administrative database used by Ontario’s publicly-funded home care agencies [11]. CHRIS additionally contains administrative data on home care episodes and billed services that can be individually linked to the clinical assessment records. CHRIS is regularly checked for validity and has been used extensively in research [12, 13]. We utilized CHRIS records stored at the Health Data Library at McMaster University to create the retrospective cohort for this study. Ethics approval was obtained from the Hamilton Integrated Research Ethics Board.

An anonymized regional version of CHRIS was used to create a retrospective cohort of home care patients 19 years of age and older in the Hamilton-Niagara-Haldimand-Brant region with at least two RAI-MDS Home Care assessments. Administrative home care records were individually linked to clinical assessment records to identify adult patients who had a baseline assessment between January 1st, 2010 and December 31st, 2014, and a follow-up assessment between 3 and 18 months after the baseline assessment but before discharge from care. All patients 19 years of age and older with a valid baseline and follow-up assessment were included in the study.

All measures in the study were calculated from the RAI-MDS Home Care assessment. The RAI-MDS Home Care is a comprehensive clinical assessment that contains over 200 items in the domains of function, health, social support, and service use [3]. The internationally-developed assessment was created to identify multidimensional needs, aid in the development of appropriate care plans, and establish outcomes for tracking purposes. The RAI-MDS Home Care is used for clinical home care assessment in most Canadian provinces and territories, many U.S. states, and numerous countries in Europe and Asia/Pacific Rim. RAI-MDS Home Care assessments are typically performed in-person by registered nurses and take approximately 45 minutes to complete. RAI-MDS Home Care assessment data have previously been used in clinical and epidemiological research [14]. Average inter-assessor reliability weighed kappa scores for RAI-MDS Home Care items have been estimated at 0.74 for items that are shared with the RAI MDS 2.0 and 0.70 for stand-alone home care items [15]. Pearson correlation coefficients between functional and cognitive scales from the RAI-MDS Home Care assessment and gold standard scales have been calculated to be between 0.74 and 0.81 [16]. A comparison of the RAI-MDS Home Care assessment with the International Classification of Functioning, Disability, and Health has shown substantial overlap in content [17].

The primary measure of interest in this study was change in MDS-HSI between baseline and follow-up assessment. The MDS-HSI is a preference-based HRQOL measure based on the Health Utilities Index Mark 2 classification system and defined using items from the RAI-MDS Home Care assessment. The MDS-HSI is calculated using 31 elements of the RAI-MDS Home Care assessment to map a subject onto six health domains of the Health Utilities Index Mark 2 classification system (sensation, mobility, emotion, cognition, self-care, and pain). Published community Health Utilities Index Mark 2 preference weights are applied to produce scores [5]. Previous studies have supported the validity of the MDS-HSI in cross-sectional settings, finding an intraclass correlation coefficient (ICC) between the MDS-HSI and Health Utilities Index Mark 2 of 0.46 in a home care population, increasing to 0.63 with the 10% most discrepant scores removed [5, 8]. The MDS-HSI ranges from − 0.03 to 1, with 0 representing dead and 1 perfect health. As is true with the HUI2, MDS-HSI scores less than zero represent states worse than dead. A difference of 0.03 or more indicates a clearly clinically important difference [18, 19].

The measures used in this study to assess the convergent and known-groups validity of the MDS-HSI included six validated, ordinal indices of the health domains of function, cognition, health stability, pain, mood, and communication that are calculated from RAI-MDS Home Care assessment. The ADL Hierarchy Scale measures impairment in self-performance of the activities of daily living (ADL) tasks of dressing, eating, bathing and personal hygiene [20]. The ADL Hierarchy Scale ranges from 0 (Independent) to 6 (Total Dependence). The IADL Difficulty Scale measures difficulty in performing the instrumental activities of daily living (IADL) tasks of meal preparation, housework, and telephone use [21]. The IADL Difficulty Scale ranges from 0 (no difficulty in any of the three tasks) to 6 (great difficulty in all of the tasks). The Cognitive Performance Scale (CPS) measures impairment in memory and cognitive skills [22]. CPS ranges from 0 (intact cognition) to 6 (very severe impairment). The Changes in Health, End-Stage Disease, Signs, and Symptoms Scale (CHESS) measures health instability and risk of death [23]. The CHESS scale ranges from 0 (no health instability) to 5 (highly unstable health). The Pain Scale measures the frequency and intensity of pain and ranges from 1 (no pain) to 4 (excruciating pain) [24]. The Depression Rating Scale (DRS) measures the presence and severity of mood symptoms and ranges from 0 to 14 [25]. The Communication Scale measures impairment in expression or comprehension and ranges from 0 to 8 [26]. Neither the DRS nor the Communication Scale has descriptions for specific values of the index. Also used to assess known-groups validity were 15 disease symptoms recorded on the RAI-MDS Home Care assessment: angina, constipation, dizziness, edema, dyspnea, delusions, hallucinations, dysrhythmia, poor self-reported health, flare-up of chronic condition, disruptive pain, weight loss, one or fewer meals per day, insufficient fluid intake, and declining food/liquid consumption. Age and sex were additional measures used as covariates in the linear regression analysis.

Longitudinal construct validity was examined using convergent and known-groups approaches. Convergent validity is an aspect of construct validity that can be generated by demonstrating that the measure of interest correlates with other measures as expected [27]. In a longitudinal setting, convergent validity can be investigated by examining the correlation between the change in the measure of interest and changes in other measures with which the measure of interest is expected to be associated. In this study, we assessed convergent validity by examining the correlations between the change in the MDS-HSI and the changes in six validated indices of individual health domains. Known-groups validity is another aspect of construct validity and can be generated by demonstrating that differences exist in the measure of interest between groups that are expected to score differently [27]. In a longitudinal context, known-groups validity can be investigated by examining average change in the measure of interest in groups that are known to have experienced a change in the underlying construct. In this study, we used changes in the six health domain indices and symptoms to define patients that were known to have experienced a clinically important change in their health status and examined whether the MDS-HSI indicated that there had been a clinically important change in HRQOL.

Convergent validity was assessed using correlational methods [28‐30]. The association between the MDS-HSI change scores and changes in the health domain indices were examined with Spearman rank correlations (ρ). The degree of the association was categorized according to Cohen: negligible (ρ < 0.1), small (0.1 ≤ ρ < 0.3), moderate (0.3 ≤ ρ < 0.5) and large (ρ ≥ 0.5) [31]. As the health domain indices represent single domains of health and the MDS-HSI is a multidimensional measure of HRQOL, small to moderate correlations were hypothesized. A negligible correlation would suggest that the MDS-HSI is not sensitive to change in the domain, while a large correlation would suggest that change in the domain may be inappropriately dominating change in the MDS-HSI. Given that the health domain indices measure increasing impairment on their individual domains, the correlations were hypothesized to be negative.

Known-groups validity was assessed using linear regression to estimate the mean change in MDS-HSI associated with a clinically important change in each of the health domains or symptoms while controlling for age and sex [30]. A clinically important change in any of the health domain indices or symptoms represents a change in the patient condition which would generally be expected to be associated with a clinically important change in HRQOL. Multivariable linear regression models were fit for each index and symptom with change in MDS-HSI as the dependent variable and change in the index or symptom as an independent variable. One-unit changes were taken to be clinically important for all indices except the depression rating scale, where, by convention, a two-unit change is considered clinically important. Minimal clinically important difference ratios were calculated by dividing the regression estimates of mean change in MSI-HSI by 0.03. All models included age and sex as covariates with age treated as a linear continuous variable.

Linear regression was selected as the analysis method as the distribution of the change in MDS-HSI was expected to be reasonably close to the normal distribution (Appendix). Models were checked for multicollinearity and violations of model assumptions by examining variance inflation factors and residual and diagnostic plots. Cases with missing data were removed from the analysis. All analyses were performed using SAS 9.4 (SAS Institute, Cary NC).

Several sensitivity analyses were undertaken to investigate the robustness of results. These included restricting the follow-up assessment window to be between 6 and 12 months after the initial assessment rather than between 3 and 18 months, stratifying analyses by sex rather than adjusting for sex, using Pearson correlations rather than Spearman correlations, altering the formulation of the age covariate, and excluding age and sex from the models.

There were 62,857 adult home care patients in the Hamilton-Niagara-Haldimand-Brant region with a baseline RAI-MDS Home Care assessment between January 1st, 2010 and December 31st, 2014 (Fig. 1). Of those patients, 25,187 received a follow-up RAI-MDS Home Care assessment between 90 and 540 days after the baseline assessment without being discharged from care between the assessments. Five cases (0.02%) had missing data and were excluded from the analysis, reducing the final cohort to 25,182.

The characteristics of the study cohort can be found in Table 1. The participants tended to be elderly (78 ± 14 years) and nearly two-thirds were female (65.6%). Just over one-third (35.1%) lived alone. Significant impairments in the activities of daily living (45.4%) and cognitive performance (60.9%) were common, as well as mood symptoms (39.0%) and a recent history of falls (41.5%). A large majority of patients (83.6%) had multiple chronic conditions. The mean baseline MDS-HSI was 0.50, falling to 0.48 at follow-up. Between the baseline and follow-up assessments, 9872 (39.2%) patients experienced a decline in MDS-HSI, 8767 (34.8%) had no change, and 6543 (26.0%) improved.

Spearman rank correlation coefficients (ρ) between the MDS-HSI change scores and health domain change scores are presented in Table 2. MDS-HSI change scores were significantly correlated with changes in each of the health domain indices, ranging from − 0.48 [− 0.49,-0.47] for the pain scale to − 0.21 [− 0.23,-0.20] for the communication scale. All correlations were negative and in the small to moderate range.

Results from the regression analysis can be found in Table 3. The mean change in MDS-HSI associated with clinically important changes in the health domain indices exceeded the minimal clinically important difference of 0.03 in each case. Minimal clinically important difference ratios varied from 1.27 for the communication scale to 3.04 for the pain scale.

Of the 15 symptoms examined, changes in 13 of them were associated with a clinically important change in the MDS-HSI. The greatest change was observed with the resolution or onset of pain intense enough to disrupt activities (minimal clinically important difference ratio 4.68), and the smallest change was associated with resolution or onset of dysrhythmia (minimal clinically important difference ratio 0.52). No variance inflation factors over 5 were observed and examination of residual and diagnostic plots did not indicate any substantial departures from model assumptions.

Restricting the population to only those with a valid follow-up assessment between 6 and 12 months after the initial assessment had no meaningful impact on the results (Tables 4 and 5). Stratifying results by sex produced very similar results for both males and females. Alternative formulations of age as a cubic smoothing spline or the inclusion of higher-order age polynomials had no significant impact on the model. The overall exclusion of age and sex from the models likewise had no significant impact.

The limitations of this study include lack of access to an external gold standard criterion and uncertainty of the generalizability of the results outside of home care settings. Additional research should be conducted to continue to investigate the validity of the MDS-HSI, in particular, the comparability of results between different care settings.