Comprehensive rehabilitation outcome measurement scale (CROMS): development and preliminary validation of an interdisciplinary measure for rehabilitation outcomes

Measurement and documentation of health outcomes are critical factors in health care systems and they play a vital role in evidence-based practice [3, 24, 39]. Appropriate measurement tools are imperative to deliver client-centric rehabilitation care, and accountable and ethical professional practice [48]. Progress achieved from any set of interventions must be measured and documented using appropriate tools for assessment. These tools must take into account various domains of functioning that are meaningful to the patient and also relevant to individual domains of expertise of team members.

Rehabilitation may be defined as the multi and interdisciplinary management of a person's functioning and health [34, 60, 68]. The interdisciplinary team consists of professionals including physiotherapists (PT), occupational therapists (OT), speech and language pathologists (SLP), clinical psychologists (CP), medical social workers (MSW), registered nurses (RN), medical doctors (MD) working in concert with the patient and family. Each individual has a distinct yet cooperative role in rehabilitating a patient from the day of admission to community re-entry, achieving the optimum level of functional independence [42, 70]. It is imperative to measure and document the patient progress using appropriate tools to help plan interventions and necessary care deserved by the patient [43, 74, 81].

Multiple outcome measurement tools have been developed and used across various rehabilitation disciplines to objectively quantify the patient status and progression [33]. However, many of which are discipline-specific and may hinder inter-disciplinary understanding and move away from the patient-first philosophy of rehabilitation. Moreover, they may aid confusion and reduce the consistency of practice even within a single institution [18, 19, 33, 38]. Functional Independence Measure (FIM) [18], Barthel index (BI- 5 and 10 items) [26], Modified Rankin Scale (MRS) [53, 79] are some of the more widely used such tools [46, 57, 62, 64]. Each of these scales has significant advantages and disadvantages, and also no particular scale is recommended for all situations and patient groups [26]. FIM is one of the commonly employed tools in functional assessment to monitor patient progress throughout the rehabilitation process [1, 22]. However, FIM requires paid professional training, and the software-based tool requires a subscription (https://​www.​uow.​edu.​au/​ahsri/​aroc/​fim-weefim/​workshops/​). This imposes restricted access to the tool, especially for rehabilitation professionals from low and middle-income countries (LMIC).

The International Classification of Functioning, Disability, and Health (ICF) provides a comprehensive frame of reference that allows for direct examination of the relevance and comprehensiveness of activity and function [47]. Within the ICF categorization system, the 'activity' dimension represents a person's perspective on functioning and is described as 'difficulties an individual may have while performing tasks [61]. This dimension is comprised of areas pertaining to domestic life, self-care, mobility, activities of daily living (including instrumentally aided activities), and responsibility for one’s health. Many of the existing functional scales have been framed before the ICF was introduced. The tasks involved in domestic life and the way of performance of activities of daily living and self-care may vary across cultures and may consist of domains that are not necessarily part of the hegemonic constructs included in traditional scales like FIM. The FIM for instance takes into account bathing in settings common in Western societies (tub or shower) which are far different from the activities involved in bathing in many LMIC (using a bucket and dipper to take water and our over oneself/ use a waterbody to immerse oneself for bathing). Similarly, FIM assumes that toileting is done on a sitting type commode with indoor plumbing which can be different in many Asian countries. Many Asian countries still utilize squat toilets and many rural homes do not have indoor plumbing and they will have to walk to an “outhouse”. Functional independence and limitations are proven to be associated with social welfare, ethnicity, and culture [8]. Additionally, there are differences in how people interpret measurement scales, and the most relevant measure may vary between populations depending on their age, literacy level, and cultural background [50].

The currently used functional measurement scales are developed for Western countries considering the specific functional requirements and the overall health care systems in these countries. These scales are limited in utility for conditions in LMICs due to various factors, including the differences in social structure and requirements, accessibility, and health care systems. Moreover, evidence from upper-middle and high-income countries also suggest the inadequacy of commonly used tools due to the limited range of domains, item definitions, scoring, and psychometrics limiting the use of a single outcome tool for various patient populations and conditions [16, 27, 63]. Existing tools are notable in the absence of certain domains like hygiene, outdoor ambulation, nursing care which form important aspects of functional independence in persons with chronic illness, aging, or disability, requiring the rehabilitation professional to resort to other tools to measure these constructs separately. Commuting multiple outcome tools to assess various domains could reduce the confidence in overall scoring and thus affect the management planning of the patient [41, 73]. Moreover, generic outcome measures are generally either patient-reported or therapist assessed and rarely encompass both aspects. In short, the need for a comprehensive rehabilitation outcome tool is justifiable. Although the scale has been constructed with populations in LMIC in mind its utility in specific situations in the Global North is plausible.

Current rehabilitation assessment in India relies heavily on diagnostic classification, objective measures like imaging and laboratory findings and does not follow the ICF framework of health comprehensively. Function is assessed by informal modification of existing tools developed in western countries or by descriptive methods. Informal modifications are not standardized and cannot be used to measure improvement or research purposes. Likewise, descriptive assessment is difficult to compare across patients, populations and centers and in research. These are other reasons that the need for a culture appropriate scale to measure function was deemed necessary.

Therefore, the objectives of this study were to develop a tool to measure functioning relevant for a person with disability in India and similar countries. This scale attempted to overcome the difficulties of cultural relevance, cost, and comprehensiveness in currently used scales [40]. Additionally, the authors attempted to develop a freely available scale with scoring criteria that are self-explanatory and that can be learned from a user manual, thus avoiding the need for additional training and training-related costs. The CROMs has scales that can be administered by both health care professional and patients or caregivers.

To fulfill the objectives, the study was undertaken in two major sections. Section 1 consisted of the development of the professional reported scale. This was conducted in four phases using a modified Delphi approach and based on previous literature in tool development [25, 52, 54‐56, 66]. In Section 2, we developed the patient-reported questionnaire and this was done in three phases. Both tools are meant as assessment tools or functional status.

The study protocol was reviewed and approved by the institutional review board (JSSCPT_IRB_06/06/2019). Participants (patients and caregivers) included in all the phases were selected if they had a good command over the respective language, did not have any altered mental status as judged by a staff member.

Participants for all phases were recruited through convenience sampling. This study was conducted in a physical medicine and rehabilitation center with interdisciplinary professionals such as Physiotherapist (PT), Occupational Therapist (OT), Speech-Language Pathologist (SLP), Clinical Psychologist (CP), Rehabilitation Nurse (RN), General Physician (GP) and Medical Social Worker (MSW). The kind of patients commonly seen at the center are Traumatic brain injury, Stroke, Spinal cord injury, Amputation, Cerebral Palsy, Chronic pain disorders.

Phases I, II, III, V and VI were analyzed descriptively.

In phase IV, the data were tested to ensure that they met the requirements for Principal Components Analysis (PCA) using the Kaiser–Meyer–Olkin measure of Sampling Adequacy and Bartlett’s test of sphericity. Items that did not have a factor loading of 0.4 were eliminated from PCA. For the PCA, varimax rotation was chosen as this is an exploratory analysis. The number of components was determined using the PCA and the number of components to retain was determined using scree plot with parallel analysis. Thirty-two items were calculated for item analysis. The internal consistency of the factorially derived scale was assessed by calculating Cronbach’s alpha. The internal consistency was interpreted as excellent (0.9 ≤ α), good (0.8 ≤ α < 0.9), and acceptable (0.7 ≤ α < 0.8) based on the derived Cronbach’s alpha value [14, 67, 69].

Relationships between CROM subscales and FIM were computed using ICC (ICC estimates and their 95% confident intervals were calculated based on the two-way mixed-effects model, with single rater performing both CROM and FIM) [31]. An α-level of 0.05 was used to determine statistical significance.

In phase VII, an agreement between the therapist-reported scale and patient-reported scale was analyzed using ICC (ICC estimates and their 95% confident intervals were calculated based on one-way random effects model, considering absolute agreement, and multiple raters [31]. ICC values less than 0.5 are considered as poor reliability, values between 0.5 and 0.75 are considered as moderate reliability, values between 0.75 and 0.9 are considered as good reliability and values greater than 0.90 indicate excellent reliability [31, 51].

Demographic characteristics of the samples at various phases are given in Table 1.

The sample size in various phases of the study is given in Table 2.

The development of this tool has focused on assessing the functional independence of the patient over time. This conceptual model ensures documenting patients’ ability to participate partially or fully in life situations that require various functions. Functional independence has been suggested to be an important aspect of quality of life. The CROM is a functional scale and does not consider quality of life but the relationship between the two is a consideration. [2, 6, 11]. We tried to include various indicators of the patient’s rehabilitation status and the amount of caregiving provided by professionals or family members demonstrating the level of transition from interdependence to independence throughout rehabilitation from admission to community re-entry using the functional outcome measures. This transition is necessary to be captured by functional scales [9, 17, 32, 65, 80].

Comprehensive Rehabilitation Outcome Measurement Scale (CROMS) was developed as an interdisciplinary venture of PTs, OTs, SLTs, CP, MSW, MD, and RNs by compiling specific components to be measured and recorded during the rehabilitation process. This scale attempted to overcome the difficulties of cultural compatibility, cost, and comprehensiveness in currently used scales [7, 12, 40]. The rationale for some of the additional areas included in CROMS is as follows.

In low and middle-income countries, a significant part of the workplace or the community consists of uneven terrain to do their daily functional and vocational work [23]. The cumulative experience of professionals who have worked with these patients guided the identification of the missing components from existing scales. Adding this item in the domain of locomotion was unanimously approved.

Health management was a domain that was deemed necessary to add to the scale. This domain consisted of clothing hygiene, hand washing, oral hygiene, and awareness of personal hygiene. This was added as these areas are also considered as functional tasks. Poor attention to hygiene can cause social isolation and deterioration in health.

The team members recommended considering an individual’s skin inspection, changing position in a timely manner, and identifying objects or surfaces that can breach the skin. Skin integrity is important in functional independence as it has been reported that lapses can lead to physical, psychological, and economic burden to the patients and their families [37].

The item generation team identified this item as an important aspect of functioning. Attention to a healthy diet and hydration were therefore added to the scale [10, 13, 78].

Most patients will have regular medication to be taken at scheduled intervals. The team involved in item generation believed that the patient’s/ caretakers must be independent in the appropriate management of prescribed medications and hence this item was added to the scale. This fact has been reported in literature as well [35, 77].

All the members were of the opinion that the addition of these items was essential. It is clinically correlated that if the patient or caretakers do not take the necessary steps to identify the safety parameters, it can lead to a negative effect on the patient [71].

Another domain that was identified by the team members is scoring of functioning once the patient is outside the sheltered care. It was evident from the previous studies that there is a significant change in the patient’s functional status in the community due to the mismatch of capacity and actual performance [36, 58, 58, 59, 59]. So adding an item of reintegration was considered.

In the attitude of the individual, his or her attitude towards his or her disability and attitude of family or friends are important determinants of reintegration [29]. Due to a change in the individual's functional capacity after the trauma or disease, the person may have to compromise in certain areas. This can have an effect on house, place of work, vehicle used, and other areas of functioning.

After a disability, the patient and caregivers need to adjust to new ways of performing work. It is noted that the majority of the patients will be unable to return to the same job especially those engaged in manual labor. Therefore, a plan for new employment which was acceptable to the patient was considered to be necessary. So it was decided to add this item as ‘work planning’.

The team also noted that there should be an item on readjustment to the community. It is noted that an individual often needed to learn new skills to adjust to the new reality. The carry-over of the newly learned skills from the institution to the community is an important aspect of re-entry. Therefore, this was considered as an item.

Items such as hydration, menstrual hygiene, floor transfer, and money management after consensus. The hydration was added as a part of nutrition, and they have combined as hydration and nutrition. The menstrual hygiene was a separate entity, and it was removed later as it is a part of general hygiene. Floor transfer was removed because the transfer from upper to lower position was already added to avoid duplication. Money management was removed as it may not be appropriate for all the patients (patients from rural areas and older adults may not deal with money management).

Without any further training, a vast majority (> 80%) of the population in our pilot study expressed satisfaction with the scale's clarity, applicability, and relevance (Fig. 2). This shows that the CROM can be administered with the help of a training manual without any specific training. Additionally, the types of patients in the pilot study consisted of neurological, orthopedic, and other disabilities. Hence, it can be cautiously suggested that the scale applies to varying types of diseases. The concurrent validity of the newly developed tool was performed by correlating with the relevant components of FIM, as the scale has considered the background of FIM. The good correlation achieved; was expected as the general scoring and items were similar between CROMS and FIM. The good concurrent validity may be taken as justification for the ability of this scale to capture the relevant constructs [44].

In the second section of the study, the patient-reported questionnaire was developed in two languages as the objective of the study is to have a comprehensive measure consisting of both the patient and therapist reported scores. A large number (> 80%) of the patients reported that questions were relevant. From the previous studies, it is evident that if a tool that is generated suggests more than 60% of content validity, it can be considered as a good tool [21].

At the end of item generation, we had classified the items into seven domains. This was confirmed after PCA. Toileting was removed due to poor correlation with other items on the scale. We hypothesise that this could be because of the overlap with toilet transfers, bladder and bowel management. Comparison with existing scales was not possible for all domains as we could not find comparable tools for several items. Further validation must be undertaken in the future for these items and new domains (health management, reintegration, work planning). We included a domain “other specific” to include any items that may be specific to a condition of a patient such as return to work in a modified manner and the biomechanical and attitude adjustments required, working in areas not accessible to wheelchair like rice paddies, where the person would have to transfer to a particular form of mobility device like a wheeled platform close to the ground. We did not analyze this item as the responses were heterogeneous. We anticipate that over time, this particular domain may be useful for item generation for specific groups of the population.

This scale considers only performance and does not consider the quality of function. Poor quality can result in increased time required for performance and greater errors. Thus, poor quality of functional performance may force patients to rely on caregivers increasing dependent. Work planning is not a homologous entity and requires varying levels of functions. Hence the score on this item must be interpreted with caution. We suggest that this score be used as a starting point to explore reasons for difficulty with work. The same is the case with reintegration which may be varying in nature over time. These items must be taken only as a guide and not as a definitive score.

The scale was developed using very exhaustive methods over four years and several iterations with a variety of health care professionals and we consider this to be a major strength of this study. The patient-reported component is a necessary addition and responses received for these may be useful in the future for setting goals. Another strength is the addition of domains related to “Activity and Participation” aspects as defined in ICF. These domains are frequently the main goals of rehabilitation in low and middle-income countries where accessible environments may not be commonly encountered [30, 49]. Personal choice or autonomy has been considered as the cornerstone of functional performance in this scale, for example while scoring dressing and grooming. In rural areas of India changing the type of clothing and grooming (like short hair) are often considered as an unacceptable modification and hence may be considered as a “disability”. These aspects have been taken into consideration in the scoring rubric.

The development of the scoring rubric was decided during the course of several meetings. Items were also removed after consensus. These are swallowing, money management, house cleaning and were removed as they were considered as part of other items like eating, problem-solving, and work reintegration.

This scale is a functional assessment tool which is thought to measure patients rehabilitation status and not a screening or imaging or laboratory investigations or tool. The scale is useful only for outcome evaluation for rehabilitation professionals and for goals setting and is in no way intended as a tool to measure or diagnose health conditions. Scores must be correlated with relevant investigations and clinical examination to form a basis for intervention.

Correlation between patient-reported and therapist reported showed variability in communication and cognition. This was expected due to different expectations of patients. It has been documented that Indians tend to have expectations of complete recovery despite information from medical professionals to the contrary.

We have done preliminary factor analysis on the patient-reported scale and it showed similar factorization as the therapist-reported counterpart. However, this finding is not conclusive due to the small number of data that we were able to use from the patient-reported responses. Many caregivers were used to help the patients, and were not able to dissociate from caregiving, to enable their wards to perform functions independently and thus complete the tool appropriately. One of the possible reasons could be that the caregivers were predominantly female spouses and due to cultural reasons they were bound to care for their husbands or family elders and had limited authority over their wards [72].

Most of the responses received were from caregivers rather than patients. A large number of responses received from caregivers were non-usable as many items were left unfilled or noted as not relevant. The usable responses were from caregivers of patients with neurological disabilities. That may be one of the reasons for the correlation achieved between patient-reported and therapist reported. The sample size of 30 usable responses was adequate as this is a preliminary study [5]. The homogeneity inpatient presentations limit the generalisation of the patient-reported component to individuals with neurological disabilities (Additional file 1).

This scale was conceived using the personal autonomy model of function and hence those items considered necessary by the patient alone can be considered for scoring. This is different from existing scales which have defined commonly used methods of performing activities for scoring. Some of these definitions do not match the ways that functions are performed in the Global South. Hence we believe that this scale is more versatile in its reach.