A scoring system derived from electronic health records to identify patients at high risk for noninvasive ventilation failure

Noninvasive ventilation (NIV) plays a key role in the treatment of acute respiratory failure (ARF) and its use is supported by multiple randomized controlled trials [1‐4]. The evidence of benefit is strong for patients with acute hypercapnic respiratory failure [1, 5] and cardiogenic pulmonary edema [6, 7], while consistent benefit in other conditions such as acute hypoxemic respiratory failure was not found [8‐10]. Even so, the use of NIV has dramatically increased in the last two decades in the US for all diagnoses regardless of supporting evidence [11‐14].

Therapy with NIV is considered successful if endotracheal intubation is avoided. Conversely, the term NIV failure is used when a patient initially treated with NIV requires invasive mechanical ventilation (IMV) or dies without being intubated. NIV failure rates range from 5 to 50% and patients who are intubated have an increased risk of death compared to those treated with IMV from the outset [9, 11, 15‐17]. Determining which patients are appropriate for NIV therapy is a complex decision that requires assessment of an individual’s chances of failure and/or survival; improper patient selection is a main reason for poor outcomes [18‐20]. Prior studies have identified several risk factors associated with NIV failure including coexistent pneumonia, tachypnea, hypotension, severe acidemia, higher severity of illness score, or failure to improve in one hour. However, most of these studies were small, were developed in cohorts from randomized trials, or were geared towards specific diagnoses [15, 21‐24].

A simple risk score developed in a real-world cohort to identify patients’ risk for NIV failure may support clinical decision for initiation of NIV and trigger goals of care discussions at the time of NIV initiation. It may also help with decisions regarding monitoring; patients at low risk of failure could be potentially admitted in a step-down unit, whereas those at high risk could benefit from admission to an intensive care unit. Therefore, using data from a large multihospital electronic health record database that contains vitals and laboratory results, we sought to develop a clinical risk score for NIV failure defined as intubation after a trial of NIV based on information routinely available to clinicians at the time of NIV initiation.

There were 94,744 hospitalizations of non-surgical patients with NIV initiated within the first 2 days of admission at 127 hospitals. After exclusion criteria were applied (Fig. 1), our full cohort consisted of 47,749 patients.

Using a large cohort of non-surgical patients treated with NIV at 127 US hospitals, we found that a simple model using data available at hospital presentation successfully predicted intubation after initial treatment with NIV. The final risk score includes number of organ failure, principal diagnosis, acute physiological parameters, and chronic disease comorbidities, and provides a simple method to stratify a patient’s risk of NIV failure into low and high risk categories relative to an intermediate group at average risk. Because of the large size of our cohort and the large network of hospitals contributing data, our model is statistically robust and highly generalizable. This model has significant potential for being incorporated in an online prognostic calculator (see example Additional file 1: figure E2 in the supplement) to help routine decision-making by providers and support appropriate monitoring and/or counseling of patients and families. We have also developed a risk score for NIV failure defined as intubation or death which included the same factors as the intubation only model, although the weight of the predictors changed slightly. Of note, our risk score applies to patients started on NIV soon after admission and not to patients who develop respiratory distress and are treated with NIV later in the course of hospitalization.

The present model differs from prior models used to predict intubation in patients started on NIV in several ways [15, 21‐24]. First, our model was designed to be used in any non-surgical patient started on NIV, regardless of the principal diagnosis, allowing for broader utility. Therefore, our study was not restricted to specific conditions such as COPD or CHF where the evidence for use of NIV is strong. Instead, we developed our predictive model in a large group of patients treated with NIV in routine clinical settings. Several predictive scores exist for specific diagnoses. For example, Confalonieri and colleagues developed a prediction chart of failure risk in patients with COPD [22]. They found that patients with an APACHE II score ≥ 29, a Glasgow coma score < 11, and a respiratory rate ≥ 30 breaths/min have a predicted risk of NIV failure of > 70%. However, inclusion of the APACHE II score makes it less practical due to the multiple variables needed, including laboratory tests. Second, our approach is novel in that is using a large EHR dataset. The variables in our model are easily obtainable and the scoring could be applicable not only for clinical purposes but also for studies with administrative data. Third, we have developed a tool to quantitatively estimate the risk for intubation. If the risk is high, clinicians have to make the difficult decision between NIV and IMV given that those who fail NIV have mortality which is similar or even higher than those started on IMV [9, 16]. Prior studies have shown that at least part of the increase in mortality is related to delayed intubation; this is why, if NIV is started in patients at high risk for failure, these patients need to be closely watched in a highly monitored environment. In this study we found that when NIV was started for unusual diagnoses such as drug overdose or seizure, the risk of NIV failure was high; for this group, the decision to intubate has to be seriously considered. Fourth, the prognostic model can be used as an aid in making decisions about placement of patients in ICU or intermediate care, thereby matching the intensity of monitoring with the needs of the patient [36, 37]. Of note, in a step-down unit, patients are generally monitored with the same technology as in an ICU but frequency of monitoring and the intensity of care provided by the nurses and respiratory therapists is lower [38, 39]. Currently, there is large variation in policies regarding administration of NIV across hospitals, with some institutions restricting NIV utilization to the ICU while others allow it on step-down units [20, 37]. Our scoring system can help tailor these decisions. For example, patients with substance abuse, pneumonia, renal failure (one organ dysfunction), cachexia, a prior year intubation, and tachypnea will have a total score of 28, giving them an 85% probability for NIV failure (Additional file 1: Figure E2); consequently, these patients should be closely watched in the ICU or intubated in the first place.

Our results largely confirm a number of risk factors for NIV failure that have been previously described by other studies [9, 11, 15, 17]. However, a surprising finding of our study is the large number of patients treated with NIV who had neurological, substance abuse, or psychiatric diagnoses; most of which are not typical for acute respiratory patients. Notably, only 35.3% of the 5,973 patients with these diagnoses had a secondary diagnosis of conditions that would suggest an indication for NIV such as CHF, asthma, COPD, AMI, pneumonia or sepsis, raising questions on the purpose of using NIV in this cohort. Furthermore, this group had a higher risk for intubation compared with patients with CHF or COPD: almost one in three patients in this category needed to be intubated after a trial of NIV, demonstrating that they are not good candidates for NIV. We are not able to identify the reason why these patients were started on NIV. One could hypothesize that these patients became lethargic and hypo-ventilated due to their primary diagnosis and consequently became hypoxic or hypercarbic, triggering the use of NIV. While it is true that these are not standard indications for NIV, our data reflect routine care in a large unselected population.

Organ failure was a strong predictor for intubation or NIV failure and patients with two or more organ failures were five times more likely to experience failure than those without organ failure. Although there is strong evidence that organ failure is an important risk factor for intubation or NIV failure, in this real-world cohort 35% of patients treated with NIV and 62% of those who failed NIV had at least one organ failure. This scoring system could help providers to be more vigilant when choosing to deliver NIV to a patient with relative contraindications for NIV.

The results of our study should be interpreted considering its limitations. First, we did not have data on clinical assessments at the 1–2 h time point after initiation of NIV, findings that have been shown to predict NIV success [24]. Nevertheless, our model was intended to provide prognostic information at the time of NIV initiation. Evaluating the response to the NIV is a key aspect of management. However, once the follow-up assessment is made with our risk score, one can adjust the initial prediction (aka ‘prior probability’) based on the new information. Second, our outcome was NIV intubation and did not take in account the competing risk of death (8.9% of patients died without being intubated). For this reason, we have also developed a predictive score for intubation or death. Third, we relied on ICD-9CM and ICD-10CM diagnostic codes which could have resulted in misclassification. Fourth, we lacked information on advance directive status and therefore patients with a do-not-intubate status could have been retained in the cohort. Fifth although the prediction model was validated via a temporal external cohort, future validation in another cohort including additional sites is needed. Sixth, we did not have information about the use of high flow nasal oxygen in this population. Finally, this model does not apply to surgical patients or those with OSA who were excluded from the cohort.

Clinical variables at the time of admission can be used to accurately predict the risk of intubation and of intubation or death, in a broad sample of hospitalized patients using readily available clinical data. The prognostic score may provide quantitative guidance for decision-making about patients with acute respiratory failure who may require conventional mechanical ventilation. Although multiple risk scores for intubation or NIV failure exist few are utilized in routine care. Our score which is applicable to any adult patient for whom a provider is considering NIV could be built in a web-based calculator for easy use at the point-of-care.