Prediction of noninvasive ventilation failure using the ROX index in patients with de novo acute respiratory failure

The use of noninvasive ventilation (NIV) is common in patients with de novo acute respiratory failure [1]. Its use decreases the odds ratio (OR) of intubation relative to conventional oxygen therapy [2]. However, the rate of NIV failure is high in this patient population, ranging from 40% to 65% [3‐6]. Furthermore, a two- to six-fold greater rate of mortality is seen in patients with NIV failure relative to that in patients with NIV success [7]. Among patients with NIV failure, delayed intubation further increases the risk of death [8, 9]. Early identification of high-risk patients and early application of intubation is a promising strategy for reducing mortality [10].

The ratio of SpO₂/FiO₂ to respiratory rate (ROX) index was developed by Roca et al. [11] to predict the failure of high-flow nasal cannula (HFNC). They showed that the area under the receiver operating characteristics curve (AUC) was between 0.66 and 0.80 from initiation to 24 h HFNC when the ROX index was used to predict HFNC failure [12]. The index has also been used to predict HFNC failure in patients with COVID-19 pneumonia [13, 14]. Because measurement of the ROX index is feasible, effective, and reproducible, it is widely used to predict HFNC failure [15, 16]. To the best of our knowledge, the value of the prediction of NIV failure obtained using the ROX index is unclear. Here, we explored the predictive power of the ROX index for NIV failure in patients with de novo acute respiratory failure.

This was a secondary analysis of a multicenter prospective observational study performed to update HACOR scoring [17]. It was conducted in 18 hospitals in China and Turkey from September 2017 to September 2021. The relevant ethics committees approved the study and informed consent was obtained from the patients or their family members. We enrolled patients with de novo acute respiratory failure. However, hypercapnic patients were excluded. De novo acute respiratory failure was defined as occurrence of respiratory failure without chronic respiratory disease, chronic heart disease, asthma, cardiogenic pulmonary edema, cardiac problems other than cardiogenic pulmonary edema, or postoperative hypoxemia [2‐4, 17, 18].

Patients who were admitted to the participating centers were managed by the attending physicians, respiratory therapists, and nurses in charge. NIV was used to avert respiratory failure if the respiratory rate (RR) was > 25 breaths/min, if a clinical presentation of breathlessness at rest emerged (such as active contraction of the accessory inspiratory muscles or paradoxical abdominal motion), or PaO₂ fell to < 60 mmHg at room air pressure or PaO₂/FiO₂ fell to < 300 mmHg with supplemental oxygen [17]. The formula of 21 + 4 × flow (L/min) was used to estimate the FiO₂ if supplemental oxygen was used [19, 20]. A face mask or nasal mask was used to connect the patient to the ventilator. If NIV intolerance occurred, HFNC was used as an alternative strategy to prevent intubation. NIV intolerance was defined as termination of NIV due to discomfort, even in case of intermittent use [21].

We collected diagnoses and underlying diseases at admission. Pneumonia was assessed as new or increasing pulmonary infiltrate in chest radiographs coupled with clinical findings suggesting infection, such as new onset of fever, purulent sputum, cough, chest pain, leukocytosis, decline in oxygenation, and so on [22]. Acute respiratory distress syndrome (ARDS) was diagnosed as follows: (1) presence of acute hypoxemic respiratory failure with PaO₂/FiO₂ less than 300 mmHg; (2) within 1 week of a clinical insult or the presence of new (within 7 days) or worsening respiratory symptoms; (3) bilateral opacities in computed tomographic (CT) scans or chest X-rays not fully explained by effusions, lobar or lung collapse, or nodules; and (4) respiratory failure not fully explained by cardiogenic pulmonary edema or fluid overload [23, 24].

Consciousness was assessed using the Glasgow Coma Scale (GCS). GCS, heart rate, RR, blood pressure, pH, PaCO₂, PaO₂/FiO₂, and SpO₂ were collected before treatment and after 1–2, 12, and 24 h NIV. Disease severity was assessed with the sequential organ failure assessment (SOFA) score. The primary outcome was NIV failure, which was defined as the requirement of intubation [17]. The secondary outcomes were duration of ICU stay and duration of hospital stay.

We used MedCalc (MedCalc Software Ltd, Ostend, Belgium) and SPSS (version 25.0; IBM Corp., Armonk, NY) to analyze the data. Normally distributed continuous variables were analyzed using Student’s t test, and abnormally distributed continuous variables were analyzed using the Mann–Whitney U test when appropriate. Categorical variables were analyzed using the chi-squared test or Fisher’s exact test, where appropriate. The ability to predict NIV failure was tested with the AUC. The Hanley and McNeil method was used to test the difference in AUC between the ROX index and PaO₂/FiO₂ or between the ROX index and PaO₂/FiO₂/RR [25]. Three cutoff values were selected for clinical reference at probabilities of NIV failure equal to 25%, 50%, and 75% [26]. Patients with probabilities of NIV failure less than 25%, 25–50%, 50–75%, and more than 75% after 1–2 h NIV were termed the low, moderate, high, and very high risk for NIV failure groups, respectively. A p value less than 0.05 was considered to indicate statistical significance.

A total of 5413 patients were screened (Fig. 1). In all, 1286 patients were enrolled in the final analysis. Of these, 23 cases (1.8%) had missing data concerning PaO₂ at some time point. The rate of NIV failure was 44% (568/1286). Patients with NIV failure had a higher SOFA score (6.1 ± 2.9 vs. 4.8 ± 2.4, p < 0.01) than those with NIV success (Table 1). They also had higher hospital mortality than successful patients (43% vs. 4%, p < 0.01). The PaO₂/FiO₂ values collected before and after 1–2 h NIV were lower in patients with NIV failure than in those with NIV success (145 ± 76 vs. 167 ± 91 mmHg, p < 0.01; and 156 ± 82 vs. 213 ± 92 mmHg, p < 0.01, respectively). Similar results were found for SpO₂.

In the NIV failure group, the ROX index was much lower than that in the successful NIV group when the index was calculated before NIV (Fig. 2). The rates of NIV failure were 92.3%, 70.5%, 55.3%, 41.1%, 35.1%, and 29.5% in patients with ROX index ≤ 2, 2–4, 4–6, 6–8, 8–10, and > 10, respectively (Fig. 3). Similar results were found when the ROX index was assessed after 1–2, 12, and 24 h NIV.

Before NIV, the AUC was 0.64 (95% confidence interval [CI] 0.61–0.67) when the ROX index was used to predict NIV failure (Fig. 4). It increased to 0.71 (95% CI 0.68–0.74), 0.74 (0.71–0.77), and 0.77 (0.74–0.80) when the ROX index was assessed to predict NIV failure after 1–2, 12, and 24 h NIV, respectively. The sensitivity and specificity to predict NIV failure under different cutoff values of ROX index are presented in Table 2.

After 1–2 h NIV, the probability of NIV failure was 25%, 50%, and 75% for the ROX index cutoff values of 2, 6, and 10, respectively. Patients were classified into low, moderate, high, and very high risk groups for ROX index values of > 10, 6–10, 2–6, and ≤ 2, respectively. The rates of NIV failure were 23%, 34.1%, 64.3%, and 100% in these respective groups when the ROX index was used to stratify patients after 1–2 h NIV. Compared to the ROX index before NIV, improved patients had lower rates of NIV failure than the deteriorated patients after 1–2, 12, and 24 h NIV (Fig. 5).

ROX index values had similar AUCs to PaO₂/FiO₂ when NIV failure was predicted within 24 h NIV (Table 3). Compared to PaO₂/FiO₂/RR, the ROX index also had similar AUCs before NIV and after 12 and 24 h NIV. Only after 1–2 h NIV was the AUC slightly higher in PaO₂/FiO₂/RR than that for the ROX index (0.74 vs. 0.71, p = 0.02).

To the best of our knowledge, this is the largest study to explore the ROX index as a predictor for NIV failure in patients with de novo acute respiratory failure. The index had similar predictive power as PaO₂/FiO₂. It had a similar distinguishing power to PaO₂/FiO₂/RR in most cases. Three cutoff values (2, 6, and 10) were selected to classify patients into low, moderate, high, and very high risk for NIV failure.

The ROX index is mainly used to predict HFNC failure in patients with acute respiratory failure. In a classic study that focused on using the ROX index to predict HFNC failure, AUC was between 0.66 and 0.80 from initiation to 24 h HFNC [12]. In our study, AUC was similar, being between 0.64 and 0.77 for the prediction of NIV failure using the ROX index. This indicates that the predictive power of the ROX index for treatment failure in NIV patients was the same as that in HFNC patients. It is worth noting that the AUC was 0.64 when the index was used to predict NIV failure before NIV. It is difficult to predict NIV failure using only the ROX index at this time point. A combination of other risk factors can improve the predictive power. Using a comprehensive assessment tool such as the HACOR or updated HACOR score is another strategy to improve predictive accuracy [10, 17].

PaO₂/FiO₂ is associated with NIV failure [27]. Patients with lower PaO₂/FiO₂ values are more likely to experience NIV failure. It requires an arterial blood gas test to calculate PaO₂/FiO₂. However, this test is invasive and painful for patients. In addition, it is inconvenient to perform frequently. In contrast, measurement of the ROX index is noninvasive and can be performed at any time. Our study shows that the predictive power of the index for predicting NIV failure was similar to that of PaO₂/FiO₂. Therefore, ROX index can be served as an alternative method to predict NIV failure at any time.

Delayed intubation in NIV patients is associated with increased mortality [8, 9]. However, it is difficult to balance unnecessary and delayed intubation. The ROX index is a feasible assessment tool for predicting NIV failure at the bedside. It can be used to aid in decision-making, as it can stratify patients into low, moderate, high, and very high risk for NIV failure groups. In patients at high risk for NIV failure, NIV should be used cautiously. In those with very high risk for NIV failure, early intubation may be the best choice.

This study had several limitations. First, we only reported the ROX index within 24 h NIV. The predictive power of the index for use of NIV exceeding 24 h was not identified. Second, the use of NIV is a continuous process. Clinicians should assess the ROX index dynamically to avoid delayed intubation. Third, some physicians in this study may have used the ROX index in some patients at some point. This may have influenced their decision regarding intubation. As we did not record this issue, we are unable to report these data. Fourth, we only enrolled patients with de novo acute respiratory failure, due to the lack of clear evidence on the use of NIV in this population as recommended by the ERS/ATS guideline [18]. The index’s predictive power for NIV failure in other populations is unclear.

The ROX index is convenient and reproducible. It has moderate power for predicting NIV failure in patients with de novo acute respiratory failure. Assessment of the ROX index at the bedside is encouraged when NIV is used.