Background
Continuous electroencephalography (cEEG) is increasingly utilized in hospitalized patients with acute brain injury or altered mental status to detect seizures and other seizure-like patterns that can worsen outcomes [1]. In the United States, there has been a 10-fold increase in the use of cEEG in acute inpatient setting, particularly in critical care [2, 3]. Detection of seizures and other seizure-like patterns on cEEG frequently results in anti-seizure medication (ASM) treatment escalation [4‐8]. However, there is limited data on whether cEEG-guided ASM escalation improves outcomes [9]. At the same time, cEEG is resource intensive with limited availability in smaller health care facilities, being utilized more frequently in larger, urban and academic centers [2, 3]. Epidemiologic studies and observational studies using large administrative datasets can provide insights into the comparative effectiveness and cost effectiveness of cEEG utilization in acutely ill patients, and guide policies and protocols that can improve access to cEEG for patients where indicated (e.g., identifying patients that may benefit most from transfer to centers performing cEEG), develop cEEG utilization quality measures, generate evidence for rigorous randomized trials on cEEG guided anti-seizure treatment, and ultimately improve outcomes.
Prior work examining administrative datasets has shown that cEEG monitoring in hospitalized critically ill patients is associated with lower in-hospital mortality [2, 3]. However, a limitation of prior studies that have used administrative datasets is the lack of validated codes differentiating elective epilepsy monitoring unit (EMU) admissions from acute inpatient hospitalization with cEEG utilization. Acute inpatient cEEG and EMU EEG have the same International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes. As a result, prior work has excluded all patients that were elective admissions or were not mechanically ventilated to define patient cohorts that underwent acute inpatient continuous EEG monitoring, resulting in potential selection bias. The aim of this study is to develop hospital administrative data-based models to identify acute inpatient admissions with cEEG monitoring.
Anzeige
Methods
Study cohort
In this study, we conducted a retrospective analysis of adult patients (≥ 18 years old) admitted to a single center between January 1st 2016 and April 30th 2022. The research protocol was approved by the Mass General Brigham (MGB) Institutional Review Board and a waiver of informed consent was obtained. The selection of patients for our cohort was performed considering the aim of the study in identifying acute inpatient admissions with cEEG monitoring. Figure 1 shows the patient selection flow chart. Patients were included if they underwent cEEG monitoring (either in the EMU or as part of an acute inpatient hospitalization). Our institution is a Level 4 Epilepsy Center approved by the National Association of Epilepsy Centers. Our EMU is also accredited by the American Board of Registration of Electroencephalographic and Evoked Potential Technologists. The EMU has 11 acquisition units (5 adult beds, 4 pediatric and 2 portable units). The average EMU volume is 174/year with approximately 16% diagnostic, 68% Phase 1 and 14% Phase 2. The average volume of acute inpatient cEEG at our center is approximately 1600/year.
Study outcome variables
Our study outcome consisted of a binary variable indicating whether an inpatient admission with a cEEG procedure was performed in the acute inpatient hospital setting (cEEG) or in the EMU setting (EMU). From here on “cEEG” will refer to acute inpatient admissions with continuous EEG monitoring, and “EMU” will refer to epilepsy monitoring unit admissions. Reference standard for cEEG vs. EMU was determined using the local hospital Natus EEG database that contains all the hospital EEG data.
Study covariates
The study covariates for the hospital admissions in our study cohort are presented in Table A2 from the Additional File. Diagnoses and procedures were defined using ICD and CPT codes and are presented in Table A1 from the Additional File. The binary covariates considered were indication (‘1’ for presence and ‘0’ for absence) of daily laboratory values acquired, inpatient medications ordered, procedures performed, type of admission – elective, emergency and urgent, primary and secondary diagnoses of traumatic brain injury (TBI), stroke and epilepsy, seizures or convulsions, death at discharge, discharged to home or self-care and female sex. The numerical covariates consisted of the number of distinct procedures, number of distinct medications, days of hospital length of stay (LOS) and age at admission. For modeling covariates, we used procedures and medications that are more likely to be used in inpatient and high illness acuity settings to distinguish from the epilepsy monitoring unit setting [10‐14]. Numerical covariates were normalized using the min-max normalization [15] where the minimum and maximum reference values for each covariate were calculated from a training set. The data splitting into training and testing sets is detailed in the following section. Regarding outliers preprocessing, we identified one outlier for hospital LOS, which we imputed with the median LOS.
The procedures (Table A1 from the Additional File) considered were the following: abdomen/pelvis computerized tomography (CT) scan, arterial line, chest X-ray, head CT scan, lumbar puncture, magnetic resonance imaging (MRI), mechanical ventilation, transthoracic echocardiogram, and tube feed orders. The number of procedures consisted of the sum of the distinct procedures performed during the hospital stay, varying in the range between zero and nine.
Anzeige
The set of inpatient medications considered were the following: cefepime, ceftriaxone, dexmedetomidine, dobutamine, dopamine, enoxaparin, epinephrine, heparin, midazolam, nicardipine, norepinephrine, phenylephrine, piperacillin, piperacillin/tazobactam, propofol, vancomycin and vasopressin. The number of medications consisted of the sum of the distinct inpatient medications ordered during the hospital stay, varying in the range between zero and seventeen.
Modeling design and evaluation
We performed a random sampling of hospital admissions in our cohort to create training (70%) and hold-out testing (30%) sets with distinct patients. With the training set we developed an extreme gradient boosting model (XGBoost) [16] and performed hyperparameter tuning in 100 iterations of 10-fold cross validation. The hyperparameter tunning methodology is described in Additional File section A.2. We selected a threshold for binary classification on the training data that achieved a positive predictive value (PPV) yielding a balance between false positives and false negative predictions. We assessed both the positive and negative predictive values (PPV and NPV, respectively). We evaluated model performance using the area under the precision recall-curve (AUPRC) [17], showing the trade-off between PPV and sensitivity, also called true positive rate or recall, for different thresholds. We also evaluated the area under the receiver operating characteristic (AUROC), which quantifies the tradeoff between sensitivity and false positive rate (also known as 1- specificity), across different decision thresholds [18]. Given the imbalance in our dataset, we present the macro average [19] performance for the classification, and the performance for each class (EMU vs. cEEG). A macro-average calculates performance metrics independently for both classes and then takes the average, giving both classes equal weight [19]. We performed 1000 bootstrapping iterations to calculate 95% confidence intervals (CI) in the hold-out test set, an external and independent test set not used for model training or validation. We assessed covariate importance using SHapley Additive exPlanations (SHAP) [20], which estimates the contribution of each feature to the model’s predictions.
Results
Cohort characteristics
Our cohort comprised 9,523 patients and a total of 10,783 hospital admissions, after applying inclusion and exclusion criteria (Fig. 1). The average age of the cohort was 59 years (standard deviation (SD) 18.2), with the majority being males (53.8%), White (75.5%) and non-Hispanic (82.7%) (Table 1). The majority of admissions (91.5%) were acute inpatient hospitalizations (i.e., cEEG rather than EMU). Demographic characteristics were approximately the same at the hospital admission level (Table A2 from the Additional File; counting all admissions separately) as those at the patient level (Table 1; counting each patient only once).
Table 1
Main characteristics of the study cohort
Characteristic | Study cohort (n = 9523, N = 10,783) |
|---|---|
Age(a), (years, mean (SD)) | 59 (18.2) |
Female sex, n (%) | 4402 (46.2) |
Race, n (%) | |
White | 7182 (75.5) |
Black or African American | 818 (8.6) |
Asian | 329 (3.6) |
Other (b) | 1173 (12.3) |
Ethnicity, n (%) | |
Non-Hispanic | 7879 (82.7) |
Hispanic | 701 (7.4) |
Unknown | 943 (9.9) |
Hospital admissions, N (%) | |
EMU | 912 (8.5) |
cEEG | 9871 (91.5) |
Type of admission, N (%) | |
Emergency | 6902 (64.0) |
Urgent | 2146 (20.0) |
Elective | 1729 (16.0) |
Discharge disposition, N (%) | |
Deceased | 1897 (17.6) |
Home or Self Care | 2812 (26.1) |
Diagnosis, N (%) | |
TBI | 698 (6.5) |
Stroke | 1507 (14.0) |
Epilepsy, seizures or convulsions | 6876 (63.8) |
Daily laboratory values acquired, N (%) | 10,127 (93.9) |
Top procedures, N (%) | |
Chest X-ray | 7052 (65.4) |
Head CT scan | 6546 (60.7) |
MRI | 5869 (54.4) |
Top medications, N (%) | |
Enoxaparin | 7137 (66.2) |
Propofol | 6067 (56.3) |
Vancomycin | 5777 (53.6) |
Medications(b), (number, median [IQR]) | |
Procedures(b), (number, median [IQR]) | |
LOS (days, median [IQR]) |
Fig. 1
Study cohort inclusion and exclusion criteria. Legend: N – number of visits; n – number of patients; cEEG – acute inpatient hospitalizations admissions class; EMU – epilepsy monitoring unit admissions class
×
![]()
Modeling results
The XGboost model was trained with all study covariates described in the Methods section (Table A2 from the Additional File). Model performance evaluated on the testing set is presented in Table 2. The hyperparameters selected during training in 10-fold cross validation are presented in Table A3 from the Additional File. We experimented different thresholds for fixed values of PPV between 95% and 98% (Table A4 from the Additional File). When setting PPV to 98%, the binary threshold was 0.80 and yielded a balanced sensitivity and specificity. The model achieved a macro AUROC of 0.92 (95% CI [0.91–0.94]) and AUPRC of 0.99 [0.98–0.99]. The AUROC and AUPRC curves are presented in Figure A1 from the Additional File. There were 130 (4%) misclassifications of acute inpatient cEEG incorrectly classified as EMU admissions, and 73 (26.6%) misclassifications of EMU admissions incorrectly classified as acute inpatient cEEG, as presented in Fig. 2. When analyzing misclassifications, 130 cEEG admissions incorrectly classified as EMU, we observed that 79% (N = 102) of admissions were elective and 90% (N = 117) discharged to home or self-care. For the EMU admissions incorrectly classified as acute inpatient cEEG we observed that 71% (N = 52) were emergency and 93% of the admissions (N = 68) had daily laboratory values acquired, a higher proportion when compared with that of the EMU class (59%, Table A2 from the Additional File).
Table 2
Modeling performance [95% confidence intervals] of the extreme gradient boosting model evaluated in test
Classes | AUROC | AUPRC | Sensitivity | PPV | NPV | Specificity |
|---|---|---|---|---|---|---|
Macro average | 0.92 [0.91–0.94] | 0.98 [0.98–0.99] | 0.78 [0.75–0.81] | 0.83 [0.79–0.85] | 0.78 [0.79–0.85] | 0.78 [0.75–0.81] |
EMU | 0.92 [0.91–0.94] | 0.66 [0.60–0.72] | 0.73 [0.68–0.79] | 0.61 [0.55–0.65] | 0.97 [0.97–0.98] | 0.96 [0.95–0.96] |
cEEG | 0.92 [0.91–0.94] | 0.99 [0.98–0.99] | 0.96 [0.95–0.96] | 0.97 [0.97–0.98] | 0.61 [0.55–0.65] | 0.73 [0.68–0.79] |
Fig. 2
Confusion matrices normalized by (a) sensitivity, (b) positive predictive value, (c) without normalization. Legend: cEEG – acute inpatient hospitalizations admissions class; EMU – epilepsy monitoring unit admissions class
×
![]()
Anzeige
Since most EMU admissions in our cohort are elective (72.4%) and discharged to home or self-care (82.7%), (Table A2 from the Additional File), we trained a model excluding both the admission type covariates (emergency, urgent and elective) and discharge disposition to home or self-care and evaluated the performance in test (AUROC and AUPRC presented in Figure A2 from the Additional File). The overall macro average model performance (Table A5 from the Additional File) was similar to that of the model using all covariates (Table 2) with an AUROC of 0.90 (95% CI [0.88–0.92]) and AUPRC of 0.99 [0.98–0.99]. However, this model showed a higher number of misclassifications, especially false positives, mainly due to errors in classifying the EMU class.
Covariates importance
We analyzed the importance of the covariates in the design of the XGBoost model. The average magnitude of the SHAP values for the 20 top features is presented in Fig. 3, and the SHAP raw values are presented in Figure A3 from the Additional File.
Elective admission was the most important covariate for the model to classify an admission as EMU (left side in Figure A3 from the Additional File). We sought to understand the type of admissions distribution for each class, since EMU admissions are frequently elective. For both train and test sets, 72% of EMU and 11% of acute inpatients (cEEG class) admissions were elective, respectively. Furthermore, we assessed if the EMU non-elective admissions were correlated with the COVID-19 pandemic. According to a study [21], in the setting of the COVID-19 pandemic, urgent and emergent EMU admissions were required due to increased seizure or event frequency. We confirmed that the non-elective EMU admissions spanned all years of our study period with the following number of admissions per year: 39, 46, 48, 41, 31, 38, 9, from 2016 to 2022, respectively. Since not all EMU admissions were elective, it was important to combine this covariate with others to develop the classification model.
Emergency admissions, orders of medications such as heparin, vasopressin, cefepime or epinephrine, daily laboratory values acquired, mechanical ventilation, transthoracic echocardiogram and chest X-ray and also diagnosis of stroke were important predictors of acute inpatients hospitalizations (cEEG).
Anzeige
The EMU class was associated with younger age at admission (blue color in Figure A3 from the Additional File) when compared to the cEEG class (mostly pink in Figure A3 from the Additional File). The average age (SD) for EMU admissions was 42 (18) while for the cEEG class it was 60 (18), a difference of approximately 20 years (Table A2 from the Additional File). The EMU admissions class was also associated with lower number of procedures and medications, when compared to the cEEG class. EMU admissions were also associated with being discharged home, having a diagnosis of epilepsy or seizures, orders of enoxaparin and ceftriaxone. Patients receiving ceftriaxone alone were likely intracranial monitoring admissions.
Fig. 3
Average magnitude of the Shapley Additive exPlanations (SHAP) values for the model top 20 features. Legend: Positive SHAP values further from zero represent higher impact for the acute inpatient admissions class (cEEG) while negative SHAP represent higher impact for the epilepsy monitoring unit admissions (EMU) class
×
![]()
Discussion
Our model using hospital administrative and billing data distinguishes continuous EEG performed in acute inpatient setting from the EMU setting. The model can enable identification of acute inpatient cEEG from administrative datasets with higher accuracy, and therefore be used for comprehensive comparative effectiveness and cost effectiveness analysis. Such large epidemiologic studies can then provide further guidance for randomized trials of continuous EEG guided anti-seizure treatment in the acute setting, and refinement of continuous EEG guidelines and protocols, particularly for resource limited settings.
There has been limited prior work in the development and validation of administrative models for accurate identification of continuous EEG in the hospital setting from administrative datasets. One prior study evaluated ICD based models for accurate identification of EMU admissions from administrative datasets [22]. The authors examined three queries, with varying use of admission and primary diagnosis ICD codes in 351 admissions. They found that queries combining ICD and CPT codes for continuous EEG, with ICD codes for epilepsy, seizure, or seizure mimic codes as the admitting diagnosis had a sensitivity of 96.3%, specificity of 100.0%, positive predictive value of 98.3%, and negative predictive value of 100.0%. Models combining ICD/CPT codes for continuous EEG and ICD codes for epilepsy and seizures as principal diagnosis had sensitivity of 94.9%, specificity of 100.0%, and positive and negative predictive values of 98.8%, and 100.0% respectively. However, these queries only included elective admissions, with focus only on EMU admissions and therefore cannot be applied for identification of acute inpatient continuous EEG utilization. Additionally, our work demonstrates that a subset of EMU admissions are emergent or urgent (up to 28% in our cohort), and a subset of acute inpatient cEEG admissions are elective (up to 11% in our cohort). Urgent admissions to EMU are indicated when there are significant clinical risks for patients. These include rapid medication switches in patients experiencing adverse effects or high seizure frequency and cannot be safely titrated outpatient, seizure frequency exacerbation with unsuccessful outpatient efforts at controlling seizures, concern for non-epileptic spells occurring at a high frequency, and differentiating between new acute symptoms versus medication side effects [23]. While there is no robust data on frequency of urgent admissions to EMUs, cohort studies have shown medication adjustments account for approximately 20–30% of EMU admissions and referrals [24, 25]. Our model reduces the misclassification rate based on admission status (4% cEEGs misclassified as EMU vs. 11% using a priori stratification on the elective vs. non elective admission status). We did not see a significant change in the misclassification of EMU (27% of EMU admissions misclassified as acute inpatient cEEG vs. 28% using a priori stratification on the elective vs. non elective admission status). While elective vs. emergent and urgent admissions, continues to be the most important predictors in our model, combining them with additional ICD diagnosis, procedure and medication codes can enable identification of acute inpatient cEEGs without a priori exclusion of patients.
Anzeige
Two prior epidemiologic studies have examined the impact of cEEG utilization in critically ill patients using the Nationwide Inpatient Sample [2, 3]. Both studies found that cEEG utilization is associated with lower in-hospital mortality, and is not associated with increased costs when compared with routine (brief) EEG [2, 3]. However, to ensure exclusion of EMU admission, the studies excluded all elective admissions. Additionally, to define a cohort of critically ill patients they only included patients that received mechanical ventilation. However, epidemiologic studies have shown that more than half of patients admitted to intensive care units do not receive mechanical ventilation [26, 27]. Moreover, our data demonstrates that approximately 60% of patients undergoing cEEG were not on mechanical ventilation. Therefore, a priori exclusion of patients not on mechanical ventilation potentially results in an exclusion of a large proportion of patients that are critically ill and could have undergone cEEG, resulting in potential sampling bias in the prior studies. Our model can eliminate the need for upfront inclusion and exclusion or filtering criteria based on admission status, and use of specific medical procedures such as mechanical ventilation, enabling identification of a broader more complete cohort of admissions with inpatient continuous EEG utilization from administrative datasets.
The main limitation of the study is that it is a single center study, therefore may not be generalizable. While the billing and procedure codes, medications and admission data we used in our models may not overlap with all claims datasets, the variables used are routinely available in institutional electronic health and administrative/billing data, as well as in several critical care and population based datasets (e.g. MIMIC, Premier, Nationwide inpatient sample) [28‐31]. Other covariates could have been included in the model, such as free text clinical notes, including EEG reports, and discharging providers taxonomy codes, which we propose as future work, along with validation in other administrative datasets.
Conclusions
The model developed in this study can identify continuous EEG performed in the acute inpatient setting from continuous EEG performed in the EMU setting and reduces the number of misclassifications. This model will allow the identification of continuous EEG monitoring admission in larger cohorts, thereby contributing to the scale of research of EEG utilization.
Acknowledgements
During this research, Dr. Westover was supported by the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; and grants from the National Institutes of Health (NIH) (R01NS102190, R01NS102574, R01NS107291, RF1AG064312, RF1NS120947, R01AG073410, R01HL161253), and National Science Foundation (2014431). Dr. Sahar F. Zafar is a clinical neurophysiologist for Corticare, unrelated to this work and was supported by the NIH (K23NS114201). There are no conflicts of interest.
Declarations
Ethics approval and consent to participate
We confirm that all methods were carried out in accordance with relevant guidelines and regulation. The need for informed consent was waived by the ethics committee/Institutional Review Board of Massachusetts General Brigham, because of the retrospective nature of the study. The research protocol was approved by the Mass General Brigham (MGB) Institutional Review Board.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.