Identifying the relative importance of predictors of survival in out of hospital cardiac arrest: a machine learning study

The data are collected from the Swedish Registry of Cardiopulmonary Resuscitation (SRCR).

Stromsoe et al. have provided a detailed description of SRCR and a validation of the reported data [19]. In short, reporting is comprised of two steps; initial reporting by the Emergency Medical Services (EMS) crews, and follow-up, which is mostly performed by a healthcare provider in the hospital. Online data entry by the EMS has been utilized since late 2007. All EMS units in Sweden report to the registry. We therefore included all 45,067 cases of OHCA recorded during 2008 to 2016, where CPR had been attempted. Information regarding 30-day survival was retrieved from the Cause of Death Registry. Data linkage is virtually complete due to unique personal identification numbers, which are assigned to all Swedes from birth or immigration. We took a closer look into the five strongest and the five weakest factors in predicting survival. Time to CPR and bystander CPR were discussed together and regarded as “early CPR”.

To employ a data-driven approach, we included all 16 variables available in the Swedish Registry of Cardiopulmonary Resuscitation (SRCR). Some of these variables have previously been reported to be of importance for the chance to survive a cardiac arrest [20]. All variables that are collected in the registry were included. Initial rhythm was categorized as either shockable (ventricular fibrillation [VF] or pulseless ventricular tachycardia [VT]) or non-shockable (pulseless electrical activity (PEA) or asystole). The following time intervals (in minutes) were assessed: time from cardiac arrest (CA) to CPR; time from CA to the 911 call; time from 911 call to Emergency Medical Service (EMS) dispatch; and time from EMS dispatch to EMS arrival at the patients’ side. Place of CA was defined as home, public or non-public places. Non-public places include facilities outside hospital such as nursing homes, dental and primary healthcare facilities, ambulances, but also private work offices. Public places include all other places outside hospital and outside home such as train stations, shopping centers etc. Patients were further divided into those who had OHCA at home or not at home. Whether the CA was witnessed or not, whether CPR was attempted before EMS arrival, whether patients were defibrillated before EMS arrival, and whether patients were defibrillated by the EMS personnel were also dichotomous variables. Defibrillation before ambulance arrival was carried out using AEDs which are widely available in Sweden. Hence, defibrillation before ambulance arrival is in our study synonymous with the use of AEDs. Finally, the calendar year was a categorical variable, ranging from 2008 to 2016. Cause of CA had nine categories: heart disease, drug overdose, accident, pulmonary disease, suffocation, suicide, drowning, or other as reported by the EMS personnel. Thus, the etiology of cardiac arrest was categorized according to the initial assessment of the EMS personnel. Patients were divided into two groups i.e. cardiac etiology or no cardiac etiology.

We used random forest, a machine learning algorithm that has become a standard tool in medical research, to examine the relative importance of 16 clinical predictors assessed during the pre-hospital phase of OHCA [21]. Binary classification models were developed in order to compute the individual importance of the predictors in the models. Since random forest is highly resistant to overfitting, we used 3000 trees for each model, although we observed no material improvement in accuracy beyond 2800 trees. Missing data was imputed using the MICE (Multiple Imputation by Chained Equations) algorithm. We did not impute the following variables: defibrillation (yes/no), time to defibrillation, defibrillation before EMS arrival since these variables refer to a subset of patients, i.e. those who were found in ventricular fibrillation.

We have validated our findings using gradient boosting, and there were no material differences compared with random forest. We performed 10-fold cross validation and obtained an accuracy of 82.1% of the final model.

To elucidate the associations between key predictors (initial rhythm, cause of CA, age, time to CPR, time to EMS arrival) and 30-days survival we used partial dependence plots (PDP). These plots are low-dimensional graphical renderings of the prediction function, making it possible to visualize the association between the predictors and survival, while accounting for all the predictors in the model. PDP is an effective means of explaining the output from machine learning models [22].

A detailed discussion on variable importance is provided by van der Laan [23]. Briefly, variable importance is a measure of how important each predictor is. The most commonly used importance measure is the permutation accuracy importance. This measure is obtained by randomly permuting each predictor, which disrupts the association between the predictor and the response variable. The new predictions are made with the permuted variable and the remaining non-permuted variables; if the permuted variable was important, the prediction accuracy decreases. Hence, variable importance is measured as the difference in prediction accuracy before and after permuting the variable. These machine learning methods elaborate prediction models and, in passing, automatically quantify the relative importance of all variables included in the model, thus eliminating bias. We used the permutation accuracy importance developed by Strobl et al., which better handles differences in the scales of the predictors (e.g. continuous vs factor variables) as well as correlations between them [24, 25]. The unit for importance is arbitrary. It was normalized to 100 in order to facilitate interpretation. It is derived by quantifying the improvement or loss in predictive accuracy by means of permutation. Data were analyzed using R (R Foundation for Statistical Computing, v 3.6.1).

The mean age was 68.3 years and women constituted 33.1% of the cohort. A shockable initial rhythm was seen in 21.7% of patients and 5.2% of all patients were defibrillated before EMS arrival. Sixty-five percent of the cases were witnessed. CPR before arrival of EMS was attempted in over half of the total study population, with the vast majority being performed by laymen. The majority of OHCAs occurred at home and two thirds had a cardiac etiology. The median times from collapse to CPR and defibrillation were 3.0 and 15.0 min, respectively. In total, 34 % were defibrillated during the pre-hospital course. Detailed characteristics of the study population and delay times are seen in Table 1.

The relative importance of all predictors is presented in Fig. 1a to c. Overall, the top five factors to predict survival in order of importance were: initial rhythm, age, early CPR (time to CPR and CPR before arrival of EMS), time from EMS dispatch until EMS arrival (EMS response time), and place of CA. The largest difference in importance was noted between initial rhythm and the remaining predictors. The five factors that were of least importance were: sex, time during the day when OHCA took place, time from emergency call to EMS dispatch (procedure time at the dispatch center), region where and calendar year when OHCA took place (Fig. 1a).

Among patients who were found in a shockable rhythm (Fig. 1b), the five most important predictors were: time to defibrillation, age, defibrillation (yes/no), place of cardiac arrest, and time from CA to CPR. The largest difference in importance was noted between time from CA to defibrillation and age. The five least important predictors were the same as for the overall population.

For patients who were found in a non-shockable rhythm, the most important predictors were age, defibrillation before EMS arrival, time to EMS arrival, place of CA and cause of CA. We did not note any particularly strong predictor; importance declined gradually from the most important to the least important predictor (Fig. 1c).

Overall, survival was highest among the youngest patients (Fig. 2a–c). However, the shape of the association curve differed for shockable and non-shockable rhythm, such that survival was relatively stable between 0 and 30 years of age among patients with shockable rhythm, while survival dropped rapidly in the same age range for those with non-shockable rhythm.

The association between time from collapse until start of CPR and 30-days survival was uniform across all three groups (Fig. 2d–f). Overall, survival is reduced by half during the first 10 min from collapse (Fig. 2d). A similar pattern was noted for those who were found in a shockable rhythm (Fig. 2e), and for those who were found in a non-shockable rhythm (Fig. 2f), albeit less pronounced for the latter.

The same pattern was noted for the association between time from EMS dispatch until EMS arrival i.e. EMS response time and 30-days survival.

Figure 3 displays the interaction between cause of arrest and initial rhythm and 30-days survival. It is evident that overdose and drowning correlated with better survival across all rhythms, although PEA and asystole consistently confer very low probabilities of survival.

Although machine learning algorithms are capable of handling vast number of predictors, the 16 predictors used here are widely regarded as the most relevant predictors (and this was indicated by the accuracy achieved by our model). However, having access to more variables would presumably elucidate additional interesting findings.

Although we used models specifically developed to minimize issues with collinearity, differences in scales, etc., we cannot rule out that variable importance was affected by such factors. Although we limited our analysis to random forest, we did fit gradient boosted trees but noted no material difference in model accuracy.

This is, to our knowledge, one of the first studies of its kind to utilize machine learning to examine the relative importance of various factors in predicting survival after OHCA. Recently, machine learning has been proposed as an appropriate technique to predict outcome after OHCA [36]. More studies, including more variables, are warranted to further delineate the importance of various factors in OHCA.

Despite advances in the treatment of OHCA during the last decades, still only a minority will survive the event. Therefore, it is important to develop decision support tools for the EMS staff which can be helpful already before arrival in hospital to make decisions on whether it is meaningful to continue resuscitation or not. The type of data that we report in this study may form the basis for such a tool. Furthermore, our results give indications on which of the links in the chain of survival are particularly important to strengthen even further.