Prediction of obstructive sleep apnea: comparative performance of three screening instruments on the apnea-hypopnea index and the oxygen desaturation index

Data from 235 patients who were monitored by ambulant PSG were retrospectively analyzed. Patient inclusion criteria were patients aging 18 years of age or older, completed clinical data, and completed STOP-Bang questionnaire and NoSAS score. Patient exclusion criteria were previously diagnosed OSA, use of portable sleep studies or respiratory polygraphy, incomplete clinical data, and technically inadequate PSG. In the outpatient clinic, the following clinical parameters were collected for all patients: gender, age, height, weight, body mass index (BMI), neck circumference (NC), self-reported complaints (snoring, daytime sleepiness, and apnea), and self-reported comorbidities (cardiovascular history, hypertension, pulmonary history). The ESS was completed. The clinical parameters were used to calculate the NoSAS score and the STOP-Bang questionnaire.

The STOP-Bang questionnaire consists of four questions used in the STOP questionnaire—snoring, tiredness, observed apneas, and hypertension—plus four demographic queries—BMI > 35 kg/m², age > 50 years old, neck circumference > 40 cm, and male gender. For each question, answering ‘yes’ scores 1, a ‘no’ scores 0. With a total range of 0–8, a total score of ≥ 3 points is considered as a high probability for OSA [6]. The NoSAS score is a 5-item questionnaire which includes neck circumference, obesity, snoring, age, and gender. With a range of 0–17, NoSAS scores 4 points for neck circumference ≥ 40 cm, 3 points for BMI 25–30 kg/m², 5 points for BMI ≥ 30 kg/m², 2 points for snoring, 4 points for age > 55 years old, and 2 points for male gender. The total score of ≥ 8 points is considered as a high probability for OSA [8]. The ESS consists of 8 situations, allowing the patients to assess their degree of dozing off or falling asleep in a particular scene during the day, 0 for no dozing, and 1, 2, and 3 for slight, moderate, and high chance of dozing. A total score of ≥ 10 points is considered as excessive daytime sleepiness [16].

All patients underwent a full-night PSG at home. PSG included electroencephalography, electrooculography, surface electromyography, nasal airflow and air temperature, thoracoabdominal movements, pulse oximetry, body position, and snoring sounds. Breathing was recorded with nasal pressure and temperature sensors. Scoring of the electronic raw data was performed manually, following the recommendations of the American Academy of Sleep Medicine [2]. Apnea was defined as a decrease of at least 90% of airflow from baseline for > 10 s. Hypopnea was defined as a decrease of at least 30% of airflow from baseline for > 10 s, associated with either an arousal or ≥ 3% arterial oxygen saturation decrease. The mean number of apneas and hypopneas per hour of sleep (AHI) was calculated. The ODI was defined as the mean number of arterial oxygen desaturations ≥ 3% per hour. The severity of OSA was categorized both according to the AHI and to the ODI. By using the AHI, patients were classified as mild (5 ≤ AHI < 15 events/h), moderate (15 ≤ AHI < 30 events/h), or severe (AHI ≥ 30 events/h) according to the 2012 American Academy of Sleep Medicine criteria [2]. For classification according to the ODI, patients were divided into similar groups: mild (5 ≤ ODI < 15 events/h), moderate (15 ≤ ODI < 30 events/h), and severe (ODI ≥ 30 events/h) [27]. Other PSG parameters collected included the apnea index (AI), the AHI in supine position, the AHI in non-supine position, minimal arterial oxygen saturation (minimal SpO₂), baseline arterial oxygen saturation (baseline SpO₂), average arterial oxygen saturation (average SpO₂), and percentage of sleep time with arterial oxygen saturation time below 90% (SpO₂ time < 90%).

The statistical analysis was performed by using Statistical Package for Social Studies (IBM SPSS Statistics version 24 for Windows, New York, NY, USA). Continuous data are presented as means with standard deviations. Categorical variables are presented as frequencies with percentages. Comparisons between groups were performed using Chi-square tests for categorical variables, unpaired Student’s t test, and univariate analysis of variance (ANOVA) for continuous variables. Discrimination, the ability of a screening tool to distinguish between patients with and without different outcomes, was estimated from the area under the curve (AUC) obtained by receiver operator characteristic (ROC) curves, which may range from 0.5 (no discrimination) to 1.0 (perfect discrimination) [28]. The AUCs were compared using the algorithm previously described by Hanley et al. [29]. Additionally, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for different AHI and ODI cutoffs using four-grid contingency tables, all estimates are reported with their respective 95% confidence interval (CI). The association between various individual demographic and clinical variables and the presence and degree of OSA was established by using a multivariate logistic regression model (backward stepwise selection, p < 0.05). A two-tailed p value < 0.05 was considered statistically significant.

A total of 201 patients met our inclusion criteria; baseline characteristics are mentioned in Table 1. A total of 148 (73.6%) patients were male, aged 50.0 ± 12.6 years, with a mean BMI of 28.0 ± 4.8 kg/m². Based on the AHI, OSA was present in 159 (79.1%) of the patients; 66 (41.5%) with mild OSA, 45 (28.3%) with moderate OSA, and 48 (30.2%) with severe OSA. Male gender, age, BMI, neck circumference, cardiovascular history, hypertension, snoring, and apneas were all significantly higher in the OSA groups than in the no OSA group. A post hoc Bonferroni test showed a statistically significant difference between no OSA and moderate/severe OSA for male gender (p = 0.008; p = 0.001), age (p = 0.002; p = 0.013), and BMI (p = 0.045; p < 0.001). BMI was also significantly different between mild/moderate OSA and severe OSA (p < 0.001; p = 0.030). Neck circumference (p = 0.043; p = 0.032), cardiovascular history (p = 0.006; p = 0.040), and hypertension (p = 0.004; p = 0.002) all showed a statistically significant difference between no/mild OSA and severe OSA. The ESS did not differ significantly between OSA groups (p = 0.667; p = 0.616). A total of 54.5%, 75.6%, and 85.4% of the patients in the mild, moderate, and severe OSA group, respectively, were classified as high risk of OSA according to the NoSAS score (cutoff ≥ 8; p < 0.001). A total of 97%, 100%, and 100% in the mild, moderate, and severe OSA group, respectively, were classified as high risk of OSA according to the STOP-Bang questionnaire (cutoff ≥ 3; p < 0.001). Polysomnography results (AHI, ODI ≥ 3%, minimal SpO₂, average SpO₂, and SpO₂ time < 90%) were all significantly different between the OSA and no OSA groups (p < 0.001; p < 0.001; p < 0.001; p < 0.001; p = 0.001). Notable is the percentage of patients with positional sleep apnea which was also statistically significant between the groups (p < 0.001). A post hoc Bonferroni test shows that the difference was significant between no OSA and all OSA severity groups (p < 0.001) and mild OSA and severe OSA (p = 0.05).

The predictive performance of the different screening instruments as categorical variable is shown in Table 2. For screening on different cut-off points of AHI and ODI severity, the sensitivity of the NoSAS score varies from 0.70 to 0.92 (AHI > 5 and AHI > 15, respectively). The specificity varies from 0.37 to 0.55 (AHI > 15 and AHI > 5, respectively). The STOP-Bang questionnaire showed the highest sensitivity varying from 0.99 to 1.00. However, the specificity was lower varying from 0.06 to 0.17. The highest specificity was obtained by the ESS, varying from 0.79 to 0.83, with a low sensitivity varying from 0.15 to 0.19. Figure 1 shows the ROC curves and the corresponding AUC of the three screening instruments on different levels of AHI and ODI severity. The screening instruments are presented as continuous variables. The ESS did not show adequate discrimination for screening for AHI and ODI with an AUC ranging from 0.450 to 0.525. The NoSAS score and the STOP-Bang questionnaire were both equally adequate screening tools for the AHI and the ODI with AUC ranging from 0.695 to 0.767 and 0.684 to 0.767, respectively (all comparisons with p value > 0.05). The discriminatory ability of the NoSAS score and the STOP-Bang questionnaire was similar in relation to both the AHI and the ODI (all comparisons with p value > 0.05). When used as categorical variable, the AUC of the NoSAS score ranged from 0.620 to 0.684 (cutoff ≥ 8), the AUC of the STOP-Bang questionnaire ranged from 0.529 to 0.577 (cutoff ≥ 3) (Table 2). Both instruments performed better when used as continuous variable than as categorical variable. However, only for the STOP-Bang questionnaire, this difference proved to be significant (all comparisons except AHI ≥ 5 with p value < 0.05).

Multivariate logistic regression analyses were performed in order to establish the association between various individual demographic and clinical variables and the presence and degree of OSA categorized by the AHI and the ODI. Gender, age, and BMI proved to be the strongest predictors for any OSA (AHI ≥ 5) (p < 0.001; p < 0.001; p = 0.004), moderate to severe OSA (AHI ≥ 15) (p < 0.001; p < 0.001; p < 0.001), ODI ≥ 5 (p = 0.001; p = 0.001; p = 0.001), and ODI ≥ 15 (p < 0.001; p < 0.001; p < 0.001). Gender, BMI, and self-reported history of hypertension proved to be or the strongest predictors for severe OSA (AHI ≥ 30) (p = 0.028; p < 0.001; p = 0.028) and ODI ≥ 30 (p = 0.024; p < 0.001; p = 0.034). The ROC curves of the estimated predictive probability, the NoSAS score, and the STOP-Bang questionnaire with cutoff points AHI ≥ 15 and ODI ≥ 15 are shown in Fig. 2. The AUC of the estimated predicted probability was 0.784 when differentiating for AHI ≥ 15 and 0.805 when differentiating for ODI ≥ 15. The predicted probability performs similar to the NoSAS score and the STOP-Bang questionnaire (all comparisons with p value > 0.05).

This is the first study that evaluated the predictive performance of three different screening instruments with respect to both the AHI and the ODI. This is relevant, due to increasing evidence that the ODI has a higher reproducibility in the clinical setting [19‐21]. Furthermore, significant differences in the severity of OSA have been described between patients with a similar AHI. Presumably, this is due to the fact that the morphology and duration of the apneas are not taken into account in the AHI [22]. In the present study, the NoSAS and STOP-Bang screening instruments both have a high discriminatory ability to predict OSA severity based on the AHI and the ODI. The ESS, however, was not able to detect patients at high risk for OSA and should, therefore, not be used as a screening instrument.

In general, the use of a retrospective analysis to validate the predictive value of different screening instruments is less ideal than a prospective study. In this observational study, however, our center had collected data prior to PSG monitoring, thus maintaining a high credibility for this retrospective study. Most patients were referred to the sleep clinic because they were suspected of having sleep-related problems. Therefore, it is possible that a selection bias was introduced, since the questionnaire was applied only to the suspected individuals. The great prevalence of OSA in this study population could affect the interpretation of the screening instruments. Contrarily, the present study has several important strengths: this is the first study that has evaluated the predictive value of different screening instruments on the ODI. As the ODI is gaining attention as new variable to classify OSA severity, this is an important new insight. Furthermore, all patients were evaluated with a full PSG and scored according to the current guidelines proposed by the American Academy of Sleep Medicine in 2012 [2].