Improving Prediction of Favourable Outcome After 6 Months in Patients with Severe Traumatic Brain Injury Using Physiological Cerebral Parameters in a Multivariable Logistic Regression Model

Patients from the ICU of the University Medical Centre Groningen (UMCG) and Maastricht University Medical Centre (MUMC), both in the Netherlands, were retrospectively analysed. Data were included from two centres to increase the number of patients. In both the centres, medical ethical committees approved anonymized physiological, diagnostic, clinical and outcome data collection. The need for informed consent was waived in Groningen. In Maastricht, informed consent was obtained from the closest relative. Inclusion occurred between May 2012 and March 2015 in the UMCG and between April 2017 and January 2019 in the MUMC. Inclusion criteria were (1) severe TBI and (2) ICP/CPP monitoring. Exclusion criteria were (1) moribund at admission, (2) pregnancy, (3) monitoring started > 24 h after trauma, (4) loss to follow-up or (5) incomplete baseline data for the extended CRASH score.

Outcome after 6 months was obtained by consultation over the phone by a clinician. The outcome of patients was scored on the five-point Glasgow Outcome Scale (GOS). The GOS was subsequently dichotomized to unfavourable (dead, vegetative state or severe disability [GOS 1–3]) or favourable (mild disability or full recovery [GOS 4–5]) outcome. Mortality after 14 days was not evaluated as this was not standardly collected. The patients were treated by the same neuro-intensivist in both hospitals (MJ Aries).

Electrocardiogram, arterial blood pressure (ABP) and ICP were recorded at 250 Hz using ICM + software (www.​icmplus.​neurosurg.​cam.​ac.​uk) running on a bedside computer. In Groningen, an external ventricular drain was used with an electronic (ICP) sensor in the tip for intraventricular pressure monitoring and optional cerebral spinal fluid drainage capacity (Neurovent, RAUMEDIC AG, Helmbrechts, Germany). In Maastricht, a parenchymal sensor was used. ABP was zeroed at heart level in Groningen. In Maastricht, ABP was zeroed at heart level up to 2018, after which ABP was measured at the brain level. Corresponding HR, ABP and ICP were down-sampled to 1 sample/min. PRx was calculated as the moving Pearson correlation of 30 consecutive 10-s non-overlapping moving window averages of ABP and ICP, updated every minute, resulting in one averaged correlation value per minute. The PRx index indicates the intactness of the cerebrovascular reactivity. With an intact cerebrovascular reactivity, a slow increase in ABP will be counteracted by cerebral vasoconstriction, leading to a decrease in cerebral volume and subsequent in ICP. If the cerebrovascular reactivity is impaired, the cerebral volume and ICP will rise as a result of an increase in ABP. Therefore, the correlation coefficient will be negative or around zero in the case of an intact cerebrovascular reactivity, and positive when the cerebrovascular reactivity is impaired. Similar correlation-based parameters calculate the correlation coefficient in the same way, but by using different input and output parameters. These correlation-based parameters are the correlation coefficient between: ABP and pulse amplitude of ICP (AMP) (pressure amplitude index [PAx]), which describes the cerebrovascular reactivity; moving correlation coefficient between AMP and CPP (RAC), which includes information about the cerebral compensatory reserve and cerebrovascular reactivity; moving correlation coefficient between AMP and ICP (RAP), which describes the intracranial compliance. For more in-depth information about these dynamic parameters, we refer to recent literature [16, 18]. The extended CRASH risk score was calculated using basic clinical parameters obtained at admission [4].

MATLAB (2018A, The MathWorks, Natick, MA, USA) was used for all analyses. Outliers were replaced in HR, MAP and ICP with the filloutliers function. The algorithm replaced data points more than six median absolute deviations above or below the median by a linearly interpolated value using the previous and the next data point. No outliers were removed in parameters describing the trends in cerebral compliance, compensatory reserve and autoregulation, as individual data points are known to be noisy [19].

We investigated which of four data periods after the start of neuromonitoring were most informative to predict outcome. These periods contained data from 0–6 h, 0–12 h, 0–18 h and 0–24 h after the start of monitoring. Missing data were not replaced. If a period contained less data than 50% of the total length or less than 3 h additional to the previous period, the period was excluded. We chose to exclude only time periods and not the entire patient, in order to represent the clinical setting. In total, 15 parameters for each period were selected representing different physiological domains: (1) average ICP, ABP and HR; (2) average PRx, PAx, RAP and RAC; (3) the slope of a linear line fitted on the PRx, PAx and RAC (as an indicator how these parameters progress over time); (4) the amount of impairment of autoregulatory parameters, defined as the area of the signal above a set threshold divided by the total amount of samples: for the PRx (threshold  > 0.35), PAx (> 0.25), RAC (> − 0.05) and ICP ( > 2 mmHg) [12]; and (5) the individual unfavourable outcome risk score (CRASH, %). This resulted in 15 values per time segment.

A logistic regression outcome model was trained based on the above parameters, called the ‘combined’ model. To train the model with those parameters that are most predictive of outcome, the algorithm will first rank the parameters on their respective predictive power (Fig. 1).

To rank the parameters, leave-one-out cross-validation (LOOCV) [20] was combined with forward feature selection (FFS) (Fig. 1a) [21]. LOOCV splits the data set 45 times (called folds) in a training set containing all but one and a test set containing the remaining patient. For each fold, the training set was used to determine the optimal order of parameters by FFS, from best to worst. The CRASH risk score was fixed to be the first ranked parameter. This results in an optimal order of parameters for each patient when not including that patient in the FFS. To define the overall optimal order of parameters, the kth optimal parameter is selected as the most occurring parameter at position k from all 45 LOOCV models. This is continued until all parameters are included, resulting in an overall ranking of the 15 parameters.

To train the model, the data set is again divided by LOOCV (Fig. 1b) and training and testing are performed for each fold. Starting with the neuromonitoring-derived parameter that was ranked highest as single input, a logistic regression model is trained on the training set and tested on the test set for each split. The output of the test set is the probability that the patient has an unfavourable outcome. As this is repeated for each fold, the probability of an unfavourable outcome is predicted for each patient. Thereafter, a model that included the two highest ranked parameters was trained and tested, again resulting in an individual probability of unfavourable outcome. This is repeated until all parameters are added. The number of parameters with the highest area under the curve (AUC) in the corresponding ROC curve is selected as the best model for that time period. A common problem in machine learning is overfitting, which occurs when the number of parameters is too large compared to the sample size. Because the number of patients is limited, a maximum of six parameters were included. The selection of parameters and training of the model was performed for each time period.

The accuracy of the model, expressed in correctly predicted patients, is based on the optimal cut-off value determined by the Youden’s index [22]. The performance of the CRASH model and the different combined models will be compared using the ROC curves, the AUC values and the prediction accuracy. We decided not to statistically compare AUC curves due to the limited sample size, and hence, results should be interpreted at a qualitative level. Model calibration is assessed by visualization of the predicted probabilities versus the actual outcome.

The CRASH model performs adequate for all periods (AUC 0.75–0.76, accuracy 75.0–75.6%) (Table 1, Fig. 2). Slight deviations in CRASH performance are due to the fact that not all time periods had an equal number of subjects. The combined model shows a high AUC and accuracy for each time period, especially in the early monitoring period (0–6 h, AUC 0.90, accuracy 86.7%). The parameters included per time segment are shown in Table 2. The logistic regression model coefficients and thresholds for favourable vs unfavourable prediction are given in Supplemental Material 2, Table S2–S5. The calibration curves indicate a systematic underestimation for the lower predicted probabilities, whilst a systematic overestimation for the higher predicted probabilities is observed (Fig. 3). However, as the data set used in this study is fairly small and the magnitude of error is not severe, calibration is deemed acceptable [23].

Both the CRASH and the combined models classified a few more patients to have a favourable than an unfavourable outcome (Table 3). The number of misclassified subjects was comparable over all GOS outcomes. The percentage of patients who were classified to have a favourable outcome but actually died ranged from 7.7% (1 out of 13) to 46.2% (6 out of 13). The percentage of patients who were predicted to have an unfavourable outcome but actually had a GOS score of 5 ranged from 0 to 26.7% (4 out of 15). The combined model using data of 0–6 h showed the highest AUC, the least mortality misclassifications and the second-best good recovery misclassifications.

This study has several limitations. First, the sample size of this study is small. Therefore, we did not have the possibility to use a separate test set. Instead, we performed feature selection in a separate cross-validation before training the model. Selection of parameters by FFS may vary between subjects in the case of correlation between parameters, as the information between parameters is quite similar. As some parameters are correlated in this study, the order of parameters in different patients as selected by FFS varied. Therefore, variability in feature selection and subsequent low performance would be seen if FFS was performed in the cross-validation splits together with prediction. Although the current method might introduce a slight bias, it ensures that each model uses the same features. To improve training and ensure correct generalization, we recommend increasing the sample size and validate the found models on an external data set. Second, the parameters used show correlation, especially parameters describing autoregulation such as PRx and PAx. The feature selection algorithm used in this method does not take correlation into account. Although this is partly solved due to the selection of parameters that perform best most often, it is recommended to use algorithms that are capable of handling correlation in parameters, such as lasso logistic regression. To test the influence of correlation between parameters, model performance was evaluated once without PRx and once without PAx. AUC was the highest using both PRx and PAx in three out of the four time segments and equal in the remaining one time segment (data not shown). Therefore, although the correlation is present, including both parameters improves prediction accuracy. We hypothesize that both might contain different cerebral hemodynamic information. Third, selection of the 14 monitoring-based parameters was based on availability and proven relationship to outcome in the literature. The parameters in this study are mainly perfusion related, whilst brain oxygenation or metabolism is not considered. Adding the latter to the model, for instance using near-infrared spectroscopy or parenchymal brain tissue oxygenation, may further increase the predictive value. Fourth, it is possible that events occur after the first 24 h, such as a deterioration or improvement in the measured physiological parameters, complications or independent issues, such as unrelated death after discharge. Future models should consider incorporating such long-term deviations in addition to the early-phase parameters. Fifth, the combined model included ICU-admitted severe TBI patients, whilst the CRASH model was trained on all TBI patients with a GCS lower than 14 [4]. Therefore, the CRASH model is not optimized for our specific data set. If the initial CRASH model accuracy would be higher, the addition of early neuromonitoring data may result in even higher prediction accuracies than currently found. In future studies, we will include the model created on the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT score) in TBI database [5], which may result in additional info and thus better outcome prediction. Sixth, the model did not directly take the influence of treatment into account. In future work, the (intracranial hypertension) treatment intensity level is worth adding as a separate parameter to the model. Last, this study used the time from the start of neuromonitoring to divide data in time segments. However, the start of neuromonitoring (data collection) may be postponed due to for example operator availability or delayed ICU arrival due to urgent (life-saving) surgery. Dividing data according to time after trauma and adding parameters describing time and procedures from trauma to ICU admission/data collection may improve outcome prediction.

Prognostication in ICU patients can be improved by physiological parameters. Meiring et al. [7] showed that common physiological parameters such as HR and MAP and treatment given can predict mortality on the ICU on subsequent days. Other applications are prediction of delayed cerebral ischaemia after subarachnoid haemorrhage [9], prediction of favourable neurological outcome among children on the ICU with critical illness [8] or prediction of impending sepsis in neonates [28]. Although it also has been attempted to use physiological parameters to predict outcome 6 to 12 months after TBI, data used are solely measured before admission or incorporate the whole ICU admission period, hampering (early) clinical assistance [4, 5, 29‐31].