Background
Continuous electroencephalography (cEEG) is increasingly utilized in hospitalized patients with acute brain injury or altered mental status to detect seizures and other seizure-like patterns that can worsen outcomes [
1]. In the United States, there has been a 10-fold increase in the use of cEEG in acute inpatient setting, particularly in critical care [
2,
3]. Detection of seizures and other seizure-like patterns on cEEG frequently results in anti-seizure medication (ASM) treatment escalation [
4‐
8]. However, there is limited data on whether cEEG-guided ASM escalation improves outcomes [
9]. At the same time, cEEG is resource intensive with limited availability in smaller health care facilities, being utilized more frequently in larger, urban and academic centers [
2,
3]. Epidemiologic studies and observational studies using large administrative datasets can provide insights into the comparative effectiveness and cost effectiveness of cEEG utilization in acutely ill patients, and guide policies and protocols that can improve access to cEEG for patients where indicated (e.g., identifying patients that may benefit most from transfer to centers performing cEEG), develop cEEG utilization quality measures, generate evidence for rigorous randomized trials on cEEG guided anti-seizure treatment, and ultimately improve outcomes.
Prior work examining administrative datasets has shown that cEEG monitoring in hospitalized critically ill patients is associated with lower in-hospital mortality [
2,
3]. However, a limitation of prior studies that have used administrative datasets is the lack of validated codes differentiating elective epilepsy monitoring unit (EMU) admissions from acute inpatient hospitalization with cEEG utilization. Acute inpatient cEEG and EMU EEG have the same International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes. As a result, prior work has excluded all patients that were elective admissions or were not mechanically ventilated to define patient cohorts that underwent acute inpatient continuous EEG monitoring, resulting in potential selection bias. The aim of this study is to develop hospital administrative data-based models to identify acute inpatient admissions with cEEG monitoring.
Methods
Study cohort
In this study, we conducted a retrospective analysis of adult patients (≥ 18 years old) admitted to a single center between January 1st 2016 and April 30th 2022. The research protocol was approved by the Mass General Brigham (MGB) Institutional Review Board and a waiver of informed consent was obtained. The selection of patients for our cohort was performed considering the aim of the study in identifying acute inpatient admissions with cEEG monitoring. Figure
1 shows the patient selection flow chart. Patients were included if they underwent cEEG monitoring (either in the EMU or as part of an acute inpatient hospitalization). Our institution is a Level 4 Epilepsy Center approved by the National Association of Epilepsy Centers. Our EMU is also accredited by the American Board of Registration of Electroencephalographic and Evoked Potential Technologists. The EMU has 11 acquisition units (5 adult beds, 4 pediatric and 2 portable units). The average EMU volume is 174/year with approximately 16% diagnostic, 68% Phase 1 and 14% Phase 2. The average volume of acute inpatient cEEG at our center is approximately 1600/year.
Study outcome variables
Our study outcome consisted of a binary variable indicating whether an inpatient admission with a cEEG procedure was performed in the acute inpatient hospital setting (cEEG) or in the EMU setting (EMU). From here on “cEEG” will refer to acute inpatient admissions with continuous EEG monitoring, and “EMU” will refer to epilepsy monitoring unit admissions. Reference standard for cEEG vs. EMU was determined using the local hospital Natus EEG database that contains all the hospital EEG data.
Study covariates
The study covariates for the hospital admissions in our study cohort are presented in Table
A2 from the Additional File. Diagnoses and procedures were defined using ICD and CPT codes and are presented in Table
A1 from the Additional File. The binary covariates considered were indication (‘1’ for presence and ‘0’ for absence) of daily laboratory values acquired, inpatient medications ordered, procedures performed, type of admission – elective, emergency and urgent, primary and secondary diagnoses of traumatic brain injury (TBI), stroke and epilepsy, seizures or convulsions, death at discharge, discharged to home or self-care and female sex. The numerical covariates consisted of the number of distinct procedures, number of distinct medications, days of hospital length of stay (LOS) and age at admission. For modeling covariates, we used procedures and medications that are more likely to be used in inpatient and high illness acuity settings to distinguish from the epilepsy monitoring unit setting [
10‐
14]. Numerical covariates were normalized using the min-max normalization [
15] where the minimum and maximum reference values for each covariate were calculated from a training set. The data splitting into training and testing sets is detailed in the following section. Regarding outliers preprocessing, we identified one outlier for hospital LOS, which we imputed with the median LOS.
The procedures (Table
A1 from the Additional File) considered were the following: abdomen/pelvis computerized tomography (CT) scan, arterial line, chest X-ray, head CT scan, lumbar puncture, magnetic resonance imaging (MRI), mechanical ventilation, transthoracic echocardiogram, and tube feed orders. The number of procedures consisted of the sum of the distinct procedures performed during the hospital stay, varying in the range between zero and nine.
The set of inpatient medications considered were the following: cefepime, ceftriaxone, dexmedetomidine, dobutamine, dopamine, enoxaparin, epinephrine, heparin, midazolam, nicardipine, norepinephrine, phenylephrine, piperacillin, piperacillin/tazobactam, propofol, vancomycin and vasopressin. The number of medications consisted of the sum of the distinct inpatient medications ordered during the hospital stay, varying in the range between zero and seventeen.
Modeling design and evaluation
We performed a random sampling of hospital admissions in our cohort to create training (70%) and hold-out testing (30%) sets with distinct patients. With the training set we developed an extreme gradient boosting model (XGBoost) [
16] and performed hyperparameter tuning in 100 iterations of 10-fold cross validation. The hyperparameter tunning methodology is described in Additional File section A.2. We selected a threshold for binary classification on the training data that achieved a positive predictive value (PPV) yielding a balance between false positives and false negative predictions. We assessed both the positive and negative predictive values (PPV and NPV, respectively). We evaluated model performance using the area under the precision recall-curve (AUPRC) [
17], showing the trade-off between PPV and sensitivity, also called true positive rate or recall, for different thresholds. We also evaluated the area under the receiver operating characteristic (AUROC), which quantifies the tradeoff between sensitivity and false positive rate (also known as 1- specificity), across different decision thresholds [
18]. Given the imbalance in our dataset, we present the macro average [
19] performance for the classification, and the performance for each class (EMU vs. cEEG). A macro-average calculates performance metrics independently for both classes and then takes the average, giving both classes equal weight [
19]. We performed 1000 bootstrapping iterations to calculate 95% confidence intervals (CI) in the hold-out test set, an external and independent test set not used for model training or validation. We assessed covariate importance using SHapley Additive exPlanations (SHAP) [
20], which estimates the contribution of each feature to the model’s predictions.
Discussion
Our model using hospital administrative and billing data distinguishes continuous EEG performed in acute inpatient setting from the EMU setting. The model can enable identification of acute inpatient cEEG from administrative datasets with higher accuracy, and therefore be used for comprehensive comparative effectiveness and cost effectiveness analysis. Such large epidemiologic studies can then provide further guidance for randomized trials of continuous EEG guided anti-seizure treatment in the acute setting, and refinement of continuous EEG guidelines and protocols, particularly for resource limited settings.
There has been limited prior work in the development and validation of administrative models for accurate identification of continuous EEG in the hospital setting from administrative datasets. One prior study evaluated ICD based models for accurate identification of EMU admissions from administrative datasets [
22]. The authors examined three queries, with varying use of admission and primary diagnosis ICD codes in 351 admissions. They found that queries combining ICD and CPT codes for continuous EEG, with ICD codes for epilepsy, seizure, or seizure mimic codes as the admitting diagnosis had a sensitivity of 96.3%, specificity of 100.0%, positive predictive value of 98.3%, and negative predictive value of 100.0%. Models combining ICD/CPT codes for continuous EEG and ICD codes for epilepsy and seizures as principal diagnosis had sensitivity of 94.9%, specificity of 100.0%, and positive and negative predictive values of 98.8%, and 100.0% respectively. However, these queries only included elective admissions, with focus only on EMU admissions and therefore cannot be applied for identification of acute inpatient continuous EEG utilization. Additionally, our work demonstrates that a subset of EMU admissions are emergent or urgent (up to 28% in our cohort), and a subset of acute inpatient cEEG admissions are elective (up to 11% in our cohort). Urgent admissions to EMU are indicated when there are significant clinical risks for patients. These include rapid medication switches in patients experiencing adverse effects or high seizure frequency and cannot be safely titrated outpatient, seizure frequency exacerbation with unsuccessful outpatient efforts at controlling seizures, concern for non-epileptic spells occurring at a high frequency, and differentiating between new acute symptoms versus medication side effects [
23]. While there is no robust data on frequency of urgent admissions to EMUs, cohort studies have shown medication adjustments account for approximately 20–30% of EMU admissions and referrals [
24,
25]. Our model reduces the misclassification rate based on admission status (4% cEEGs misclassified as EMU vs. 11% using a priori stratification on the elective vs. non elective admission status). We did not see a significant change in the misclassification of EMU (27% of EMU admissions misclassified as acute inpatient cEEG vs. 28% using a priori stratification on the elective vs. non elective admission status). While elective vs. emergent and urgent admissions, continues to be the most important predictors in our model, combining them with additional ICD diagnosis, procedure and medication codes can enable identification of acute inpatient cEEGs without a priori exclusion of patients.
Two prior epidemiologic studies have examined the impact of cEEG utilization in critically ill patients using the Nationwide Inpatient Sample [
2,
3]. Both studies found that cEEG utilization is associated with lower in-hospital mortality, and is not associated with increased costs when compared with routine (brief) EEG [
2,
3]. However, to ensure exclusion of EMU admission, the studies excluded all elective admissions. Additionally, to define a cohort of critically ill patients they only included patients that received mechanical ventilation. However, epidemiologic studies have shown that more than half of patients admitted to intensive care units do not receive mechanical ventilation [
26,
27]. Moreover, our data demonstrates that approximately 60% of patients undergoing cEEG were not on mechanical ventilation. Therefore, a priori exclusion of patients not on mechanical ventilation potentially results in an exclusion of a large proportion of patients that are critically ill and could have undergone cEEG, resulting in potential sampling bias in the prior studies. Our model can eliminate the need for upfront inclusion and exclusion or filtering criteria based on admission status, and use of specific medical procedures such as mechanical ventilation, enabling identification of a broader more complete cohort of admissions with inpatient continuous EEG utilization from administrative datasets.
The main limitation of the study is that it is a single center study, therefore may not be generalizable. While the billing and procedure codes, medications and admission data we used in our models may not overlap with all claims datasets, the variables used are routinely available in institutional electronic health and administrative/billing data, as well as in several critical care and population based datasets (e.g. MIMIC, Premier, Nationwide inpatient sample) [
28‐
31]. Other covariates could have been included in the model, such as free text clinical notes, including EEG reports, and discharging providers taxonomy codes, which we propose as future work, along with validation in other administrative datasets.
Acknowledgements
During this research, Dr. Westover was supported by the American Academy of Sleep Medicine through an AASM Foundation Strategic Research Award; and grants from the National Institutes of Health (NIH) (R01NS102190, R01NS102574, R01NS107291, RF1AG064312, RF1NS120947, R01AG073410, R01HL161253), and National Science Foundation (2014431). Dr. Sahar F. Zafar is a clinical neurophysiologist for Corticare, unrelated to this work and was supported by the NIH (K23NS114201). There are no conflicts of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (
http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.