Standardized visual EEG features predict outcome in patients with acute consciousness impairment of various etiologies

We performed a post hoc analysis of data prospectively acquired during the multicentric study CERTA (Continuous EEG Randomized Trial in Adults; NCT03129438). Details of the study have been published elsewhere [32, 33]. In short, patients > 18 with disorder of consciousness of any etiology (defined as GCS ≤ 11 or FOUR ≤ 12) hospitalized on the Intensive or Intermediate Care Units of four Swiss hospitals (Lausanne University Hospital (CHUV), Sion Hospital, Bern University Hospital (Inselspital), Basel University Hospital) for whom an EEG was requested for medical reasons were included. Exclusion criteria of the original study were clinical or electroencephalographic signs for seizures in the 36 h or status epilepticus in the 96 h preceding randomization, a palliative situation or documented refusal to participate to a clinical study. After inclusion, patients were randomized to undergo either continuous EEG monitoring for 30–48 h, or two standard EEGs (20–30 min) within the same timeframe. The CERTA study and the post hoc analysis of EEG data were approved by the local ethic commissions (Project-ID 2017-00268).

Video-EEGs were performed with a NicoletOne system (Viasys Neurocare, Madison WI, USA) using 21 or 23 electrodes placed according to the international 10:20-system. In neurosurgical patients, a reduced montage with 11 electrodes could be used [34]. For the present study, we considered only the first EEG (in case two standard EEGs were performed) or the first interpretation documented (in case of continuous EEG). EEG traces were interpreted during hospitalization by board-certified electroencephalographers with additional certification in ACNS Standard Critical Care Terminology [31] (AOR, RZ, VA, SR, KS, FZ).

At recruitment time, 14 non-mutually exclusive etiologies of ACI were registered. For the present study, we retrospectively formed four mutually exclusive etiologic groups, namely (1) Stroke: ischemic stroke, hemorrhagic stroke, non-traumatic subarachnoid hemorrhage; (2) TBI/NS: traumatic brain injury (TBI), subdural hematoma or other non-vascular non-traumatic neurosurgical conditions such as postoperative coma after brain tumor resection; (3) MIII: metabolic disturbance, intoxication, infection, inflammation; (4) HIE: hypoxic–ischemic encephalopathy after cardiac arrest from cardiac or non-cardiac origin. In cases where the patient had diagnoses belonging to two of the above categories, the "most severe" diagnosis would be considered (e.g. a patient with metabolic disturbance after HIE would be attributed to the etiologic group HIE). Two patients with both subarachnoid hemorrhage and HIE were not attributed to any diagnostic group (due to "equal severity"). The above-mentioned categorization was performed prior to data analysis. The rational for grouping traumatic and non-traumatic non-vascular neurosurgical diagnoses was the presence of brain lesions and possible skull defect (causing a breach rhythm in the EEG) in both categories. However, since outcome prediction in critically ill patients with traumatic brain injury is of particular interest, we also considered in the present study a subgroup with "pure" TBI patients (a subset of the TBI/NS subgroup described above).

As was the case in the CERTA study, we considered as primary outcome the mortality at 6 months, which was prospectively collected. We also considered a secondary outcome based on the best cerebral performance category (CPC) value reached within 6 months [35], dichotomized into a favorable (CPC 1 or 2) or unfavorable outcome (CPC 3–5).

Electroencephalographic and clinical features were used to predict the outcome (Table 1). EEG features were based on the ACNS standard terminology [31] and consisted of: background continuity, background amplitude, background predominant frequency, background reactivity, background symmetry, presence of stage Non-REM 2 sleep transients, presence of sporadic (i.e. non-rhythmic or periodic) epileptiform discharges and finally presence of rhythmic or periodic patterns. Five demographic and clinical features were considered, namely age, gender, Glasgow coma scale (GCS) at inclusion time (just prior to EEG recording), C-reactive protein (CRP) value during the first EEG and etiology of coma postulated within 1 week after inclusion.

We used random forest (RF) classifiers [36] to predict clinical outcome. A RF consists of multiple decision trees that have been devised successively using the training set. For classification, the decision of all the trees is aggregated to reach a global decision. We chose RF because of their natural ability to handle all types of variables, continuous, ordinal and nominal, without any preceding reduction or transformation of the predictor space. In addition, RF models have a relatively low risk of overfitting, and they allow to easily counteract unbalanced data sets by setting class weights [37]. Finally, it has been shown that good performances are achieved without hyperparameter tuning (that is, using standard parameters) [38]. RF also provides a way to estimate the relative importance of the respective features (by evaluating their role as splitting variables in the decision trees composing the random forest), even though a direct computation of the predictive value (such as odds ratios) of each feature is not possible.

We first considered all patients, regardless of the underlying ACI etiology ("general classifier"). For this task, patients were divided into a stratified training set (2/3 of patients, N = 242) and a test set (1/3, N = 122). Each classifier was sequentially trained using EEG features alone, clinical features alone and finally a combination of EEG and clinical features.

We then trained RF classifiers independently on each etiology subgroup ("specific classifiers"). Because of the limited number of subjects in each subgroup, we omitted a separate test set and used instead fivefold cross-validation on all available data.

For each classification task, 500 trees with a maximal depth of \(\left\lfloor {m/2} \right\rfloor\), where m is the number of incorporated features, were trained using the AdaBoost algorithm [39] with a learning rate of 0.1. Because RF, as most classification algorithms, tends to be biased toward the majority class, we used a weighted approach to penalize misclassifying the minority class [40], whereby class weights were 1/relative frequencies. All parameters were defined prior to analysis and were not optimized on data.

We assessed the performance of the RF classifiers with areas under the receiving operating characteristics (ROC) curve (AUC). The 95% confidence intervals of ROC curves and corresponding AUC values were determined via 3000 bootstrapped replicas. For the general classifier, we also computed accuracy, sensitivity, specificity, positive predictive value and negative predictive value with their binomial 95% confidence intervals. Statistical significance of differences between AUCs was assessed with a Z test [41]

RF classifiers were implemented in MATLAB version 2015b (MathWorks) using the function fitensemble from the Statistics and Machine Learning Toolbox. Unless stated otherwise, parameters were set to their default values.

A total of 364 patients (34% women, mean age 64 ± 15 years) for whom the outcome at 6 month was known were included, of which 187 (51%) survived, and 139 (38%) had a favorable outcome (for details about the inclusion and drop-out see [32]). The detailed number of patients in each etiological category is shown in Table 2.

The detailed performances of the general classifiers (trained on patients with all etiologies) are presented in Table 3, whereas ROC curves are shown in Fig. 1. Using EEG features, the area under the ROC curve was 0.812 for predicting survival and 0.790 for predicting favorable outcome. The prediction using the EEG features alone was more accurate than with clinical features alone for both outcomes (p = 0.008 for survival, p = 0.031 for favorable outcome). A combination of both sets of features did not improve the performance compared to EEG features alone (p = 0.984 for survival, p = 0.887 for favorable outcome).

Our relatively large test set allowed for a subanalysis of the performance of the general classifier depending on the etiology, whereby only EEG features were used (Table 4). The prediction performance was better for the subgroups of patients with TBI/NS (including the subgroup with pure TBI) or HIE than for patients with stroke or with metabolic, inflammatory, infectious origin of consciousness impairment. The between group difference of AUC values was statistically significant for predicting survival (p < 0.05 for HIE-Stroke, HIE-MIII, TBI/NS-Stroke, TBI/NS-MIII).

Figure 2 illustrates the relative importance of the different features in the general classifiers. EEG background reactivity was the most important electroencephalographic feature for both outcomes. The most important clinical features were Glasgow coma score (closely followed by age) for predicting survival, and the age for predicting a favorable vs. unfavorable outcome. Of note, reactivity remained the main EEG feature when both electroencephalographic and clinical features were used.

Specific classifiers were then trained independently on subgroups of patients with related etiologic category based on EEG features alone and using fivefold cross-validation. The performances of the specific classifiers were systematically lower than those of the general classifier on patients from the same etiology (Table 4). The drop in performance was statistically significant for the TBI/NS and HIE subgroups for predicting survival (p = 0.004 and p = 0.037, respectively) and for the MIII subgroup for predicting a favorable outcome (p = 0.010).

The relative importance of the single EEG features in the different etiological groups is presented in Fig. 3 for predicting survival and in Fig. 4 for predicting favorable outcome. Of note, EEG background reactivity was the most important feature only in the subgroups of TBI/neurosurgery patients, as well as in the subgroup of patients for which the etiology was not known.

Training the classifiers on subgroups of patients with similar etiologies led to poorer performances. However, it is important to note that in the present study, using more homogeneous subgroups also meant using less data: the training set for the general classifiers comprised 242 patients (1/3 of all patients), whereas for the specific classifiers the size of the training set was between 38 (for MIII) and 88 (for HIE) for each cross-validation run (4/5 of patients with a given etiology). It is known that a smaller training set increases the risk of overfitting and leads to poorer performance [47]. Overfitting could explain part of the large variation in relative feature importance observed when changing the outcome and etiologies in the specific classifiers (Figs. 3, 4), whereas the unique combination of features from the general model was better for practically all subgroups. Our results suggest thus that using a larger training set, even at the cost of reducing the homogeneity of the group, can ultimately increase the performance of an EEG-based classifier. This is an important observation in an era where computer-assisted medicine is growing, but available data are often insufficient for modern machine learning methods—due to the cost of data labeling, and because data sharing between organizations is limited in practice [48]. Whether the same principle applies to other EEG-based clinical settings (such as the detection of epileptiform activity in various types of epilepsies, for instance) remains to be confirmed.

When considering all patients together, EEG background reactivity (EEG-R) was the most important EEG feature, both for predicting survival and favorable functional outcome. EEG-R has been extensively used for prognostication in several etiologies [19, 42] and is thought to require functioning spino-thalamic and thalamo-cortical afferences [42]; it has been shown to correlate with neuron-specific enolase (a biomarker for neural lesion) [49] and topography of MRI lesions [50] in patients with HIE. Interestingly, EEG-R was no longer the most important feature when classifiers were trained specifically on subgroups of patients with similar etiology, except for the TBI/NS group (for which we can postulate that a known marker for the integrity of afferent pathways is a good predictor). Reactivity was also the most important feature in the subgroup of patients for whom the diagnosis was unknown at the time of the EEG, possibly due to the fact that this group was probably also very heterogeneous.

It might seem surprising that reactivity was not the most important feature for HIE, since it has been shown in numerous studies to be a good predictor for this condition. This can be due to the correlation of EEG-R with amplitude and continuity (both important features in this subgroup): since a suppressed background is usually not reactive, EEG-R is unlikely to be chosen as splitting variable in a sub-branch of a decision tree under a splitting node describing a flat line. We also note the importance of periodic or rhythmic pattern (Main term 2). Usually, the appearance or modification of periodic pattern or rhythmic spike waves following stimulus [51] is not considered background reactivity and can complicate the detection of true background reactivity. In summary, EEG-R could appear as being less important than it would if used in isolation due to its correlation with other variables and how this affects the RF algorithm.

EEG-R was not one of the most relevant features for the MIII subgroup. As already mentioned, the MIII group contains a larger proportion of patients with non-primarily cerebral cause for consciousness impairment. For these patients, probing the integrity of afferent pathways is less informative, in particular considering the fact that reactivity is often present, and thus less discriminative.

EEG and clinical data have been prospectively acquired in the course of a large multicentric study. EEG feature scoring has been performed by experienced encephalographers with by additional certification in ACNS terminology, which has been validated and offers good interrater agreement [52]. However, our study has several limitations. First, the decision to perform an EEG was taken by treating physicians based on clinical criteria. As such, there is a selection bias, since patients for whom a decision to withdraw life supporting therapy was already made, or patients who were quickly improving, were not included. Also, the timing of EEG after onset of consciousness impairment was not consistent between all patients. However, patient selection and timing of EEG correspond to "real-world" situations where an EEG is considered relevant by the treating physician. Of note, most HIE patients from the hospitals in Lausanne, Bern and Sion were recorded early after cardiac arrest (during targeted temperature management), since they were also included in an observational registry [6]. It is difficult to estimate how these selection biases influence the performance of the classifiers. It is possible, for instance, that a model trained on more systematically acquired data would lead to better results. By design of the CERTA study, no patient with known seizures in the previous 36 h or status epilepticus in the previous 90 h was included, which constitutes another selection bias. This bias is the reason why we did not include the presence of an electroencephalographic seizure as feature in our models. In case of known seizures, however, the response to anti-seizure medication is the main prognostic feature, and not the other ACNS criteria. As previously published [32], about 10% of patients in the present cohort (4.4% in those undergoing routine EEG, 15.7% in the group with continuous EEG) had ictal or interictal epileptiform activity during at least one EEG recording; while these patients were not excluded from the analysis, these numbers correspond to a recent meta-analysis of available cohort studies [53] and thus in our view reinforce generalizability of our findings. Also, we did not account for potential preexisting disability, which could affect the best CPC.

Self-fulfilling prophecy is a potential risk in all prognostication studies, especially in retrospective studies in which the decision for withdrawal of life supporting treatment (WLST) was left to the treating physician and not explicitly documented. However, EEG plays a decisive role in WLST decision mainly in patients with HIE. The majority of patients with HIE were recruited in Lausanne and Bern, where according to current guidelines, the first EEG (the one used for analysis in the present study) is usually not considered for decision to WLST [6, 54]. In patients with other etiologies, EEG is not a primary tool for clinical prognostication; therefore, the risk of self-fulfilling prophecy is probably limited.

Finally, the choice of clinical features was limited by the data registered during the CERTA study. Due to the lack of neuroradiological data, the complementarity of neuroradiological and electroencephalographic features could not be investigated. Also, we decided to consider a single blood test; the results might have been better with more blood markers, or after selection of the clinical values with a univariate or multivariate assessment. Of note, CRP levels have been shown to be correlated with the outcome in several etiologies, in particular in sepsis [55] and intracerebral hemorrhage [56].