Evidence supporting the use of a subjective staff evaluation to assess the benefit of rehabilitation in hemodialysis patients undergoing inpatient rehabilitation

The goal of rehabilitation is to optimize an individual’s functioning despite disease, injury or other health conditions. Specific activity goals are determined in the context of the individual’s lifestyle and resources, as well as with regard to how they interact with their physical, attitudinal and social environment [1]. Thus when evaluating whether a period of rehabilitation has gone well, it is important to include the individual expectations for their own lifestyle together with their ability to complete functional tasks. To capture patient-specific goals into the evaluation of rehabilitation outcomes, we established a process where goals were identified and reassessed, subjectively, by the clinicians at both admission and discharge. This subjective-staff evaluation was applied to all admissions in a unit providing inpatient rehabilitation care to patients maintained chronically on hemodialysis (HD).

The objective of this study was to evaluate if the subjective staff evaluation was validated by objective measures of rehabilitation success (determined from the Functional Independence Measure (FIM®) score [2]) and by changes in patient-reported quality of life scores (measured using the KDQoL [3]).

Data were obtained from an ongoing prospective cohort study of patients undergoing rehabilitation and hemodialysis (HD) at the Toronto Rehabilitation Institute, University Health Network, Toronto. All admissions to the hemodialysis rehabilitation service between April 2013 and August 2016 were included. Patient characteristics were determined from the prospectively-maintained clinical database and included age, sex, time on renal replacement therapy, cause of renal failure, reason for rehabilitation, comorbidity condition, hemoglobin, albumin, serum creatinine, and depressive symptoms. The study was approved by the University Health Network research ethics board. All patients received a thorough explanation of the study and only patients able and willing to provide written consent were included.

At the time of admission, realistic goals were set by the rehabilitation clinicians and the patient collectively, and documented in the chart (Appendix). At the time of discharge, each goal was evaluated, and compared to the patient’s function at that time. The team collectively assessed the proportion of patient-specific rehabilitation goals that had been met into three groups: < 30% (subjective evaluation ‘not successful’); 30–69% (subjective evaluation ‘partial success’); ≥70% (subjective evaluation ‘success’). To ensure team consensus, a minimum of three different health care disciplines had to be involved in the discharge evaluation (occupational therapy/ physiotherapy/ recreational therapy/ rehabilitation nurse/ unit physicians).

The objective change in physical function, attributable to rehabilitation, was quantified using the change in FIM® [2] score over the admission period. Admission and discharge FIM® scores are collected routinely in the minimal dataset for centers providing active inpatient rehabilitation in Ontario. The score is a validated measure of independence in core daily activity, and comprises 13 motor items and 5 cognitive items, with each item worth 7 points (1 - complete assistance, to 7 - complete independence). A maximal score of 126 is consistent with independence in all activities of daily living, while 18 is the lowest score possible.

A subset of patients also completed the KDQoL-36 [3] at both admission and discharge. The KDQoL-36 has 36 items that include all items within the ShortForm-12 (SF-12) generic quality of life scale, plus 24 disease-specific kidney-related items. KDQoL results are often further subdivided into physical component scores (PCS) and mental component scores (MCS).

A total of 249 patients underwent rehabilitation and dialysis over the 35 month study period. Of these 11 had an incomplete FIM at baseline and 37 had an incomplete FIM at discharge (Fig. 1) resulting in a total of 201 participants with complete FIM results. Patients who had a complete follow up FIM® were mostly male (57%, n = 115), with a mean age of 71.5 years (interquartile range, 64.9–78.9) (Table 1). One hundred forty- three of the patients underwent rehabilitation for acute hospitalization, 59 were admitted for rehabilitation after an acute fall. The median length of stay was 41 days (IQR 31–47). More than 80% of patients succeeded in ≥70% of the goals identified at admission (n = 163, 81.1%). Of the remaining patients 30 (14.9%) and 8 (4.0%), achieved 30–69% of goals or < 30% of goals respectively. The median FIM® score at the time of admission was 77 (n = 238, IQR 68–89) and at discharge 110 (n = 201, IQR 97–117)). Ninety-nine patients did not give consent to participation in the quality of life assessment. As only one patient with complete KDQOL data fell in the Not Successful group, data for those with Partial Success and Not Successful were condensed into one group for all analyses involving KDQOL data.

The subjective staff evaluation corresponded well with discharge FIM® scores (Success: 111 [IQR 104–118], Partial Success: 93 [IQR 82–109], Not Successful: 86 [71–102], Fig. 2; P < 0.001). At the time of discharge, patients with a subjective staff evaluation “success” had higher quality of life scores than those in the other groups although this was largely due to changes in physical component scores (median PCS scores of 37.4 [IQR 31.0–44.7] compared to 28.8 [IQR 22.4–39.2] for those in the combined Partial Success & Not Successful group). MCS scores at discharge were similar across groups 55.2 [IQR 51.2–60.2] and 58.4 [IQR 50.1–63.1] respectively.

The subjective staff evaluation correlated only weakly with absolute change in FIM® scores (Success: 28 [IQR 20–34], Partial Success: 24 [IQR 18–31], Not Successful: 16 [IQR 11–34], Fig. 3) and in PCS scores (Success: 8 [IQR 0–14], Partial Success & Not Successful: 1 [IQR -4 – 7]) and MCS scores (Success: 10 [IQR -2 – 17], Partial Success & Not Successful: 5 [IQR -3 – 12] (Fig. 4). However when defined using a threshold for improvement, more patients reported substantial improvements in quality of life (defined as > 5 point change in PCS scores over time) in those with Success than the combined Partial Success & Not Successful group (for PCS change n = 44, 60% vs. n = 4, 40% respectively). A similar trend was not seen with change in baseline MCS change (n = 38, 51% vs n = 6, 60% respectively).

The data presented in this study suggest that the routine use of a staff subjective evaluation is a valid quick assessment tool that corresponds with objective measures of physical function and patient-reported quality of life. The assessment is rapid and easily applied and has the added advantage of being specific to an individual’s needs and goals. We therefore suggest it may be an appropriate method to assess the success of our rehabilitation program although acknowledge that further work is required to assess if it can be used to estimate prognosis and long term outcomes. Similar methods have been proposed in other areas of rehabilitation such as paediatric rehabilitation, neurorehabilitation and in those using prosthetics for example [4, 5]. However in a geriatric dialysis population, the use of traditional goal attainment scaling methods may be more challenging due to the multiple dimensions of goals, and the challenge of concomitant medical instability.

We chose to study a small group of hemodialysis patients undergoing rehabilitation as part of a project to assess the value of subjective team evaluation. Although dialysis patients have a unique and often complex pattern of comorbidity, the nature of their impairment and disability is similar to the general rehabilitation population. Consistent with this expectation, recent studies show the magnitude of improvement with rehabilitation, in patients recently started onto dialysis, to be similar to that of non-dialysis patients [6‐8]. The study population has similar characteristics to other studies of dialysis patients suggesting low risk of selection bias and that our findings may be widely generalizable [9‐21]. For example, Forrest et al., reported that the median change in FIM scores, in 40 HD patients, with a mean age of 69 years, undergoing inpatient rehabilitation was similar to our population (a change from mean baseline scores of 80 to 104 by discharge) [11]. Our multidisciplinary team is highly experienced at obtaining detailed information about the patient’s characteristics, physical function, social factors and environmental factors, and therefore at setting realistic rehab goals for each patient. As a result our findings may not be generalisable to units with fewer health professionals or a less experienced team. The high burden of depression, and its modest responsiveness to treatment is increasingly recognized in the dialysis literature. We were therefore not surprised to find little correlation with physical outcomes and MCS scores, however we also acknowledge that our findings may in part relate to the small study sample, and to the nature of the KDQoL assessment of mental health wellbeing.

The data are limited by several factors. We did not include a validation cohort or long-term follow up data. A total of 12 patients did not have FIM data at baseline or follow up for unclear reasons. Another 37 did not have discharge FIM scores, mostly because these patients became acutely unwell and were transferred emergently to another acute care institution. Thus it is possible that our results are biased towards the inclusion only of those doing well with rehabilitation. Similarly a substantial number of patients did not complete both admission and discharge quality of life questionnaires limiting generalizability. Our study is also of relatively small size, and may not be generalizable across units with smaller teams or fewer health disciplines.

We propose that the use of a simple staff subjective evaluation is a valid clinical tool that can be easily used to assess improvement in functional and physical aspects after rehabilitation. As little to no change was seen in the mental health scores on the KDQoL, it remains unclear whether the evaluation tool is able to capture changes in emotional well-being, were they to have occurred. Further studies are needed to determine whether this subjective evaluation can be extended to other patient subgroups.