Discussions
We hypothesized that ED revisit risk can be forecasted from the statistical learning of ED discharged subjects’ comprehensive longitudinal clinical histories. Utilizing the population based HIE facilitated the development and prospective testing of the ED revisit risk stratification algorithm presented here whereby each ED discharge triggered an analysis of subsequent revisit risk. Designed for real time use by care providers and managers to forecast a future ED revisit, our EMR based predictive method was prospectively validated with a reasonable level of sensitivity and specificity. After calculating the ED revisit risk scores, this information is then made available to clinicians and care-givers at the point of care to support both individual patient and population based decision-making. Moreover, high-risk patients with similar longitudinal clinical patterns can be sub-grouped for targeted post-discharge intervention in real time.
Variance analysis and two rounds of decision tree modeling process were carried out sequentially for feature selection. 152 out of 14,680 features were chosen for the final ensemble model development. Among these features, age, length of hospitalized stay, previous ED or inpatient histories, and chronic conditions were also the predictors of ED utilization found by other studies [
9‐
11,
31]. There are 6-variable risk assessment tools that have been successfully validated and widely applied in ED settings, including Identification of Seniors At Risk (ISAR) [
19,
32], Triage Risk Screening Tool (TRST) [
12,
33], and Silver Code [
18,
34]. However, these tools were developed for senior patients who had increasing risk of adverse outcomes post ED discharge. Our model, on the other hand, is a generalized tool targeting at statewide population at all age groups. Comparatively, we performed Silver Code and LACE index analysis [
35] on our prospective cohort. Both tools however had poor performance with c-statistics of 0.61 and 0.57, respectively. Therefore, we concluded that our EMR-based model had better predicative results across statewide population in Maine.
Although our model achieved a prospective c-statistics of 0.73, which was higher than that of other risk assessment tools like Identification of Seniors At Risk (ISAR) [
19,
36] and Triage Risk Screening Tool (TRST) [
12,
14], our model would be of less utility if the analytical goal was set to achieve binary classification. In our study context, this ED risk scoring metric aimed to stratify patients in all-age, all-disease, and all-payor groups. The effectiveness of our risk stratification of ED revisit was supported by a “time to event” analysis (Additional file
10) on low-, intermediate-, and high-risk patient subgroups. Patients in higher risk categories returned to the ED earlier (prospective time to event analysis:
p < 0.001) over the post discharge 6-month period.
Beyond identifying at risk ED discharges for potentially preventive services, a deeper understanding of both the unique and common attributes of various sub-groups may further facilitate overall management and the prevention of un-wanted ED utilization [
10,
11,
31,
37]. Moreover, to be clinically useful, the risk stratification model should be iterative and facilitate exploration of the potential benefit (PPV) or burden (false positive rate) (business case) of managing sub-populations of high-risk patients. Accordingly, we sought to determine whether unique patterns of resource utilization or clusters of patient sub-populations existed among the considerable heterogeneity of the high-risk patient population when considered together. We demonstrated that among the high-risk group patients, their associated demographics, chronic conditions and varying patterns of resource consumption do not occur in isolation.
Our hypothesis was that the identified high-risk patients can be further divided into subgroups with unique clinical patterns. Thus, the providers and care managers would be empowered to device stratified care management plans to allow personalized care to reduce ED utilization. Cluster analysis revealed six clinically relevant subgroups among the high-risk patient population that were confirmed as durable upon prospective testing. These subgroups have unique patterns of demographics, disease severities, comorbidities and resource consumption, suggesting new opportunities to provide stratified care management among these groups. For example, cluster #4 and #6 had senior patients with co-occurring histories of the most diverse chronic conditions and linked to the highest utilization of clinical tests and prescriptions, which could be addressed through more targeted care management strategies. As shown in Table
2, Cluster 4# and 6# are two subgroups sharing similar characteristics in some chronic diseases. The average total lab test, average total radiology, and total chronic disease counts per person of Cluster 4# (514.73, 60, and 15.26) in prior 1 year however were higher than Cluster 6# (351.06, 38.77, and 12.2), which contributed to higher future 6-month ED utilization. We noted a decreased prevalence of the co-occurring chronic conditions in four other cluster groups of relatively younger adults with much less resource consumption. 24.8 % of cluster #1 subjects, who were not associated with any chronic disease history, may benefit from targeted care to keep them out of the emergency room (e.g. provision of a primary care physician or access to an outpatient clinic), although more analysis is needed to understand the risk drivers within this group. Currently, many existing care management strategies are directed toward single conditions. Our ED risk stratification model provides novel opportunities to experiment with new strategies of coordinated care targeting a combination of conditions across different age and demographic groups that we speculate may lead to greater case management efficacy. In addition, our analysis may facilitate targeted optimization in specific resource utilization for those patients with repeat healthcare visits, e.g. frequent radiographs and laboratory tests.
Our study analyzed the ED return risks with a focus on patient factors. However, it is plausible that some ED revisits can be due more to geographic ED resource accessibility factors. Therefore, we examined the geographic factors in relation to the ED revisit rates (summarized in Fig.
5). The heatmap graphic representation of the “hot” areas where high-volume of ED revisit rates correlated strongly with the local ED facility distribution, while less ED return rates were found in rural areas. Such findings were similar to that of previous study on ED use patterns of older adults [
38]. Geographic analysis can help provide a comprehensive guidance to field ED care givers in regard to the patient geographic location, ED resource allocation, and targeted intervention delivery.
Senior patients are usually with higher rate of ED visits and end with poor outcomes, resulting repeated and frequent ED revisits. Our clustering analysis of high-risk patients identified 3 clusters with the majority at age 50+ (69.16 % of Cluster 2#, 75.26 % of Cluster 4#, and 94.04 % of Cluster 6#), compared with the rest 3 clusters made up by younger adults (less than 35: 52.86 % of Cluster 1#, 40.83 % of Cluster 3#, and 51.96 of Cluster 5#). Observations such as these suggest that a one size fits all approach to case management targeting the avoidance of ED return is likely insufficient as each of these sub-groups has unique characteristics demanding targeted post-discharge strategies. It requires providers and care managers to apply stratified care management plans accordingly to reduce the risk of ED revisit. For example, post-discharge plans including follow-up calls, clinic visits, and medications need to be designed more carefully for the elderly who are more vulnerable to poor outcomes and ED return. It is intriguing to speculate that our clustering analysis could be used for a more personalized or precise approach to prevention of unnecessary revisit that would be amenable to ongoing adjustment and adaptation according to ongoing success and failures to prevent revisit. Our risk assessment has been successfully deployed within the HIE and is made available on a real time basis. The operational advantage of the presented tool will allow post-discharge plans to be carefully designed. Accordingly, real time operational solutions such as that presented here is a necessary step in addressing the issue of repeated ED utilization contributed by older adults.
Our model and associated application were designed to track the evolving nature of post ED discharge risk of revisit, in a longitudinal manner, across all payers, all diseases and all age groups. With our ED risk model, tactics for modifying care management programs can be driven and measured against the analytical risk assessment derived from the HIE records, with knowledge of high risk population distribution among the chronic conditions and physical locations. After our initial success in ED risk modeling, we will, as a next step, focus to develop hypotheses on what factors determine the probability of a return ED visit (main outcome) or cost (secondary outcome). However, while HIE data represents an ideal source of community-wide/regional patient data, operational HIEs are not present in all States. The predictive model and patient clustering method can be applied to any clinical data set including the clinical EMRs directly as well as private HIEs within hospital networks.
The ED 6 month revisit model was deployed as part of the dashboard system, which is currently in production in Maine HIE. The platform allows real time risk profiling of all Maine HIE patients to support patient targeted care and population management. Applying analytical tools on EMR and HIE data, including the ED revisit risk model and the high-risk patient clustering method, will help health care providers effectively leverage their EMR to better understand ED service delivery while providing opportunities for improved healthcare delivery for the patients.
Competing interests
KGS, EW and XBL are co-founders and equity holders of HBI Solutions, Inc., which is currently developing predictive analytics solutions in health care. The other authors declare that they have no competing interests. From the Departments of Pediatrics, Surgery, and Statistics, Stanford University School of Medicine, Stanford, California, ZH, AYS, CF, JJ, YonW, YifZ, KGS, XBL conducted this research as part of a personal outside consulting arrangement with HBI Solutions, Inc. The research and research results are not, in any way, associated with Stanford University.
Authors’ contributions
All authors have read and approved the final version of the manuscript, and agreed to be accountable for all aspects of the work. Study concept and design: BJ, XBL, ZH, DSC, STA, TR, FS, KGS, EW. Acquisition of data: BJ, ZH, CF, YinZ, JJ, CZ, DD, XBL, KGS, EW. Analysis and interpretation of data: ZH, YifZ, CZ, BJ, SH, YueW, YonW, YJ, AYS, XBL, KGS, EW. Drafting of the manuscript: SH, BJ, ZH, KGS, AYS, YueW, YJ, YonW, XBL. Critical revision of the manuscript for important intellectual content: XBL, SH, YifZ, CZ, CF, JJ, YinZ, DD, DSC, TR, KGS, AYS, STA, FS, EW.