The Risk Instrument for Screening in the Community (RISC): a new instrument for predicting risk of adverse outcomes in community dwelling older adults

The number and proportion of older adults in the European Union is growing in the face of limited healthcare resources [1]. This rise is associated with an increased prevalence of functional decline and frailty, in community dwelling older adults [2], creating risk of adverse healthcare outcomes [3]. Identifying those likely to develop adverse outcomes is important, to allocate existing resources more effectively. Rational decision-making in healthcare requires reliable and valid quantitative ways of expressing risk, that balance the potential costs and benefits of different management strategies [4]. While risk assessment, utilizing risk prediction models, is increasing [5], quantitative health impact assessment remains relatively rare in healthcare [4].

The risk of adverse outcomes in community dwelling older adults predisposes to the development of frailty and functional decline [6] and is associated with multiple conditions including depression, cognitive impairment, medical comorbidities and physical inactivity [7‐9], factors that can be grouped into three main domains: mental state, activities of daily living (ADL) and medical state. Another important factor is the ability of social networks to manage an individual’s needs [10]. There are several different approaches to identify older adults at risk of adverse outcomes. Some act as rapid screens, others as short surrogates for comprehensive geriatric assessment (CGA) [11]. While many focus on risk of specific outcomes, such as hospital readmission [5], others target frailty [12]. A wide selection of frailty screening instruments are currently available, each with varying psychometric properties [12]. No single approach is regarded as ‘gold-standard’. It is suggested that the combination of a short screen followed by more detailed triage and assessment of high-risk individuals, may be most effective [13]. Although the stratification of risk scores is associated with clinically meaningful gradients for some outcomes [5], most have poor predictive ability [14], particularly at an individual level, possibly reflecting a failure to incorporate important personalised data [5].

In Ireland, Public Health Nurses (PHN) provide the core nursing and midwifery services in the community. They work as part of the primary healthcare multidisciplinary team, with the majority assigned to geographical areas. They take a population health approach to assessment of needs at individual, family and community level [15]. PHNs visit people in their home and are ideally placed to screen, both opportunistically and proactively [16], and deliver medical [17], psychological and social [18] interventions in community settings. In some countries, people with chronic illnesses are more likely to have their condition managed by PHNs than other healthcare professionals [19]. There are high levels of frailty related risk factors in patients under PHN follow-up [20]. Despite this, to our knowledge, no short global risk-screening instrument, designed specifically for community healthcare workers, has been developed. We created the Risk Instrument for Screening in the Community (RISC), to identify those at greatest risk of institutionalisation, hospitalisation and death [21, 22]. The purpose of this study was to determine the accuracy and predictive ability of the RISC, scored by PHNs, to another subjective global assessment and frailty scale, the Clinical Frailty Scale (CFS).

The median age of the 803 patients reviewed was 80 years, interquartile range (IQR +/−10). There were 516 females (64 %) and 287 males (36 %). PHNs perception of frailty, without using a standardized frailty instrument, described 335/803 (42 %) patients as frail. The CFS score was available for 784 patients, median score was 5 (+/−2). RISC scores were available for 782 patients (97 %). The majority was scored as minimum-risk (RISC score 1 or 2/5) for the three outcomes of interest (institutionalisation, hospitalisation and death). These are shown in Table 1, which also lists the characteristics of patients according to each outcome. Detailed baseline demographic data were published previously [21].

From the total sample assessed, follow-up data were available for 801. Two patients moved location and could not be followed. The incidence of institutionalisation at 1-year was 82/801 (10.2 %). The incidence rate for hospitalisation (at least one) was 142/801 (17.7 %), while the overall mortality rate was 125/801 (5.6 %). Those institutionalised were significantly older, and more cognitively and functionally impaired than those who were not. Age, gender, cognition and social isolation did not influence whether patients were hospitalised. Those alive at 1 year were more likely to be female. At 1-year, there were no significant differences in the burden of co-morbidities between those institutionalised, hospitalised or dead and those who were not. PHNs perception of frailty had a small correlation with risk of death (r = 0.21), but not institutionalisation (r = 0.07) or hospitalisation, (r = 0.04). The CFS only correlated with death, albeit weakly, (r = 0.23).

Of patients predicted at baseline to be at maximum-risk of institutionalisation, using the RISC, 30/96 (31.3 %) were admitted to LTC in the first year compared to 49/686 (7.1 %) of those scored minimum-risk, p < 0.001. Of those scoring maximum-risk for hospitalisation, 72/283 (25.4 %) were admitted versus 66/499 (13.2 %) of minimum-risk patients, p < 0.001. The mortality rate of those scoring maximum on the RISC was 54/161 (33.5 %) compared to 67/621 (10.8 %) in the minimum-risk group. Of those scored as frail on the CFS (score of ≥5/9), 60/422 (14.2 %), 85/422 (20.1 %), and 91/422 (21.6 %), were institutionalised, hospitalised or dead at 1 year respectively. When patients deemed pre-frail (CFS score of 4/9) were included, these percentages reduced to 11.8, 19.1 and 17.9 % respectively. Table 2 presents the results of the binary logistic regression models created for each of the three adverse outcomes, assessed with the addition of the CFS in model 2 and the RISC in model 3. The inclusion of the RISC, but not the CFS, significantly increased the predictive power of model one to predict institutionalisation, OR of 1.43 (p = 0.01) versus OR of 1.03 (p = 0.87) respectively. Similar results were seen for risk of hospitalisation and death. Kaplan-Meier curves (see Fig. 1), adjusted for baseline demographics (age, gender, living alone), compared with Cox regression, showed that the those classified as maximum-risk had a significantly greater time to event than minimum-risk for institutionalisation (OR 1.76, 95 % CI: 1.06–2.94, p = 0.03), and death (OR 1.7, 95 % CI: 1.15–2.52, p = 0.009), but not hospitalisation (OR 0.95, 95 % CI: 0.67–1.36, p = 0.79).

Figure 2 shows ROC curves demonstrating the accuracy of the RISC and the CFS in predicting each outcome. The RISC and CFS had comparable accuracy at predicting institutionalisation (AUC of 0.70 [95 % CI: 0.62–0.76] versus 0.63 [95 % CI: 0.57–0.67] respectively), hospitalisation (AUC of 0.61 [95 % CI: 0.55–0.66] versus 0.55 [95 % CI: 0.50–0.61]), and death (AUC of 0.70 [95 % CI: 0.64–0.75] versus 0.67 [95 % CI: 0.61–0.72]). These differences did not reach statistical significance. Table 3 compares the sensitivity, specificity, PPV and NPV for the CFS, RISC and the PHN perception of frailty. The predictive validity of both instruments was compared using an optimal cut-off score of ≥2 for the RISC and ≥5 for the CFS, based upon the sensitivity and specificity, calculated a priori, from the ROC curves. This cut-off for the CFS is the same as the established frailty cut-off [30]. The CFS had higher sensitivity for most of the three outcomes but lower specificity. The RISC was most sensitive for hospitalization (70 %), the CFS most sensitive for institutionalization (76 %). The PHNs’ perception of frailty had lower sensitivity and specificity for most of the three outcomes. Increasing the cut-off score for the RISC to ≥3, corresponding to the cut-off for maximum-risk, increased the specificity of the RISC, but reduced its sensitivity.

Comparing the predictive ability of global RISC scores, for other outcomes apart from their indications, found that a significantly larger percentage of patients classified as maximum-risk on the global RISC score for one outcome (e.g. institutionalisation) also experienced another outcome (e.g. hospitalised or dead at 1 year), compared to those classified as minimum-risk, see Table 1. The RISC score for institutionalisation was significantly more accurate at identifying institutionalisation than hospitalisation (p < 0.001) or death (p = 0.03). The RISC score for hospitalisation was significantly more accurate at predicting death (AUC of 0.70, p = 0.03), while the RISC score for death was significantly more accurate at identifying death than institutionalization (p = 0.05) and hospitalization (p < 0.001).

This study compares the ability of a short global subjective assessment of risk, the RISC, to identify adverse outcomes with the CFS, a validated measure of frailty, and PHNs perception of frailty in a sample of community dwelling older adults in Cork, Ireland. In this sample there was a high incidence of all three adverse outcomes, reflective of the frail nature of patients under PHN follow-up. While there were some significant differences between those institutionalised, hospitalised or dead, at one year, and those who were not, traditional markers of risk including age, gender, and living alone, were inconsistently associated with 1-year outcomes. Regression modeling showed that the RISC increased the predictive accuracy of a model that included common patient variables and assessment scores, while the addition of the CFS had no significant effect. The results suggest that the RISC had comparable accuracy to the CFS in predicting institutionalisation, hospitalisation and death. Although AUC scores suggested that both tests were relatively poor at differentiating patients and differences were not statistically significant, the RISC identified those at greatest risk of all three adverse healthcare outcomes in a clinically meaningful way. Patients at maximum-risk were approximately four times more likely to be institutionalised, twice as likely to be hospitalised and three times more likely to die at 1 year follow-up than those in the minimum-risk category. The differences were all statistically significant. Similarly, those classified as maximum-risk had a significantly shorter median time to institutionalisation and death. While the global RISC scores correctly classified more patients as maximum-risk for each corresponding outcome, there was crossover, suggesting that a global subjective assessment of risk is a general marker for increasing susceptibility to all adverse events. Only the RISC score for death was unable to significantly separate individuals at minimum and maximum risk of hospitalisation. Indeed, the RISC score for death was significantly more accurate in identifying risk of hospitalisation than the RISC score for hospitalisation. This supports data suggesting that hospital admission is difficult to predict and most instruments have poor accuracy in identifying hospitalization and readmission [5].

The CFS was less useful in stratifying patients according to all three outcomes. Most frailty instruments are designed exclusively to predict frailty. The CFS focuses primarily on activity levels and ability to perform ADLs. The RISC incorporates mental state, ADLs and medical problems, in the context of the caregiver network. In this respect, it is a holistic measure, incorporating more domains and contextualising problems to create an individualised measure of risk. Given that frailty is a state of increased vulnerability [32], the RISC may act as a surrogate measure of this vulnerability, operationalising frailty as a risk of three important adverse outcomes: death, hospitalisation and institutionalisation within the context of the caregiver networks’ ability to manage a patients’ care. Furthermore the CFS, because of its lower specificity, <50 % for each of the three outcomes, was less efficient in identifying and triaging older adults. In this study 22 % of patients were deemed pre-frail and 54 % frail, using the CFS, resulting in the need to triage larger numbers for further comprehensive assessment and management. A much smaller number of patients were identified as maximum-risk using the RISC [21]. In addition, the PHNs perception of frailty, albeit a crude measure of frailty (frail, yes or no), performed well compared to both the RISC and CFS, suggesting that simple qualitative judgements made by healthcare professionals, with detailed knowledge of their population, may be sufficient. Simple subjective assessments have been used successfully in other studies. For example the “surprise” question is an independent predictor of 1-year mortality [33] and is validated used in different clinical settings [34, 35].

This paper has several limitations. The data collection was based upon a retrospective review of PHN records, some of which were incomplete. The retrospective nature of the chart review also limited the variables that could be included in this analysis such that instruments like the AMTS [36] and Charlson Co-morbidity Index [37] are criticized for their poor accuracy. Like all screening instruments they suggest the need for further assessment rather than a specific diagnosis. These instruments and others such as the BI are continuous variables that may have affected the regression analysis. However, each continuous variable was explored and a somewhat linear association was found with each of the adverse healthcare outcomes and there is no accepted way to categorise these variables in community samples. Further study should explore the transformation of such variables into categorical data using statistical techniques such as cluster analysis. This could also provide a stratification of risk according to age and functional levels. The prevalence of frailty was high, which affects the ability to interpret the PPV and NPV analysis. The method of sampling may also have led to a large degree of selection bias in that patients under PHN follow-up are at higher risk of adverse outcomes. The study was conducted by patient’s PHN, each of whom were trained before scoring the RISC. However, the reliability and validity of the CFS, scored by PHNs was not examined, which may have led to bias. In addition, none of the measures used are considered ‘gold-standard’. Inclusion of objective observer-rated instruments [38, 39] could have reduced potential bias relating to the PHNs perception of frailty. The strengths of this paper include the comprehensive nature of the PHN records, PHNs knowledge of their patients and the inclusion of a large cross-sectional and representative community sample of patients under PHN follow-up. The RISC is currently being validated in several other countries including Australia [40], Northern Ireland, Portugal and Spain. Given the paucity of risk prediction instruments in community settings [41], future studies should compare the RISC and its subtests [42] to comprehensive instruments such as the InterRAI home assessment [43]. Studies should also investigate if assessments like the RISC, scored by healthcare providers working in the community, could be used to triage patients deemed most at risk and pre-frail-frail for CGA.

In summary, the RISC predicted adverse outcomes in a community cohort of older adults, such that those at maximum-risk were significantly more likely to be institutionalizsed, hospitalised or die at follow-up, than those at minimum-risk. The RISC, a short (2–5 min) risk prediction screen, was better than a more traditional frailty scale (the CFS) in predicting outcomes. This study suggests the potential of a short global subjective risk assessment, scored by trained healthcare workers, familiar with their patients, as an alternative to short frailty measures in the prediction of adverse healthcare outcomes. Given the limitations of the study and potential for bias, further research is needed to confirm the external validity of the RISC in different settings and to compare it with more detailed assessment instruments.