Should the ultrasound probe replace your stethoscope? A SICS-I sub-study comparing lung ultrasound and pulmonary auscultation in the critically ill

This was a planned sub-study of the Simple Intensive Care Studies-I (SICS-I), a single-center, prospective observational study designed to evaluate the diagnostic and prognostic value of combinations of clinical examination and critical care ultrasound (CCUS), in critically ill patients [14]. This sub-study and a prespecified hypothesis were added to the SICS-I study [14]. The local institutional review board (Medisch Ethische Toetsingscommissie of the University Medical Center Groningen (UMCG)) approved the study (M15.168207). This manuscript was reported according to the Standards for Reporting of Diagnostic Accuracy Studies guidelines [15].

All acutely admitted patients who were 18 years and older with an expected ICU stay of at least 24 h were eligible for inclusion. Patients were excluded if their ICU admission was planned; if acquiring research data interfered with clinical care due to, for example., continuous resuscitation efforts (e.g., mechanical circulatory support); or if consent was not obtained. In this sub-study, we selected a convenience sample of patients who had bilateral LUS images in at least two scan sites.

All included patients underwent clinical examination followed by CCUS within the first 24 h of their ICU admission. The researchers were senior medical students and junior residents trained by cardiologist-intensivists for both clinical examination and CCUS before contributing to the study. Training included self-study of theory on how to perform auscultation and lung ultrasound, at least 2 h hands-on training from cardiologists-intensivists, practice on healthy individuals during practical sessions, and supervised clinical examination and CCUS in the first 20 patients.

Data from the clinical examination was prospectively collected based on definitions in the protocol, including the presence of crepitations and rhonchi [14]. Abnormal auscultation was defined as the presence of crepitations and/or rhonchi at any of the sites. Pulmonary edema was defined as the presence of three or more B lines; diffuse pulmonary edema was defined as edema in two or more scan sites of LUS bilaterally [16].

Auscultation was performed of the anterior and axillary lung fields in each hemithorax with the patient in a supine position. Subsequently, CCUS was performed following a predefined protocol using a phased array probe (M3S or M4S) set at a frequency of 3.6 MHz, a depth of 15 cm, and maximal image width (Vivid-S6, GE Healthcare, London, UK) [17]. LUS was performed using the Bedside Lung Ultrasound in Emergency (BLUE) protocol, assessing six scan sites per patient (superior, inferior, and lateral, bilateral) (Fig. 1). In each scan site, the numbers of B lines (0–5) were recorded [18]. Measurements were subsequently conducted by researchers, who were not involved in patient care. Researchers were instructed not to share their findings with the attending physicians, so that these were used for research purposes only.

The overall statistical methods were described in the predefined statistical analysis plan (SAP) of the main study (NCT02912624). Continuous variables were reported as means with standard deviation (SD) or median with interquartile range (IQR) depending on the distributions. Categorical data were presented in proportions. Student’s t test, Mann-Whitney U test, or the chi-square tests were used as appropriate. The agreement between LUS and auscultation for pulmonary edema was described by using the Cohen κ coefficient. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy of lung ultrasound against auscultation to detect pulmonary edema were calculated. Analyses were performed using Stata version 15 (StataCorp, College Station, TX, USA). A subgroup analysis was performed to assess whether these results were robust in patients who were not mechanically ventilated. We performed a sensitivity analysis to assess the agreement and diagnostic accuracy of LUS for pulmonary edema on chest X-ray, in patients where a chest X-ray was available shortly before or after study inclusion (i.e., on the same day).

The SICS-I was designed to address multiple hypotheses on six different outcomes, and therefore, the pulmonary edema outcome was adjusted for multiple hypothesis testing. We refer to our SAP for more details, but in short, a p value of 0.015 indicated statistical significance and p values between 0.015 and 0.05 indicated suggestive significance with an increased family-wise error rate [19]. For secondary or sensitivity analyses, a p value below 0.05 indicated statistical significance due to the hypothesis-generating purpose. Accordingly, the primary analyses are presented with 98.5%CIs and secondary (subgroup) analyses with 95%CIs.

The criteria for pulmonary edema diagnosed by LUS were met in 307 of 926 patients (33%). In 156 of these patients (51%), auscultation was normal. A total of 302 of 926 patients (32%) had pulmonary edema diagnosed by pulmonary auscultation. From these patients, 151 patients (50%) had pulmonary edema on LUS. Of the 302 patients with pulmonary edema on auscultation, 130 patients had crepitations and 209 patients had rhonchi.

Diagnostic performance measures of crepitations, rhonchi, and auscultation for the detection of pulmonary edema are displayed in Table 2. The sensitivity of crepitations was 66% (98.5% CI 55–76), specificity was 71% (98.5% CI 67–75), positive predictive value was 28% (98.5% CI 22–34), and negative predictive value was 93% (98.5% CI 90–95). The overall diagnostic accuracy of crepitations was 72% (98.5% CI 69–74). The sensitivity of rhonchi was 47% (98.5% CI 39–56), specificity was 69% (98.5% CI 65–74), positive predictive value was 31% (98.5% CI 25–38), and the negative predictive value was 82% (98.5% CI 77–85). The overall diagnostic accuracy of rhonchi was 64% (98.5% CI 61–67).

The sensitivity of abnormal auscultation overall was 52% (98.5% CI 45–59), specificity was 74% (98.5% CI 70–79), positive predictive value was 49% (98.5% CI 42–56), and the negative predictive value was 76% (98.5% CI 72–80). The overall diagnostic accuracy of auscultation was 67% (98.5% CI 64–70).

Diagnostic accuracy of auscultation improved if patients were not mechanically ventilated (Table 3). The overall accuracy for auscultation was 69% (95% CI 64–74) in non-mechanically ventilated patients and 67% (98.5%CI 64–70) in all patients (p < 0.001). The overall accuracy for crepitations was 71% (95% CI 67–76) for rhonchi and 66% (95%CI 61–71) in non-ventilated patients. The agreement between auscultation and LUS improved in non-mechanically ventilated patients (κ statistic 0.31).

Radiologists’ reports assessing the chest X-ray were analyzed in a subset of 315 patients as this was part of the standard ICU management until November 21, 2016. The baseline characteristics of these patients were comparable to the overall population (Additional file 1: Table S1). The median time lag between LUS and chest X-ray was 4 h (2–7 h). In 89 of these patients (28%), the radiologist reported the diagnosis of edema; in 6 patients (2%), it was unclear; and in 220 patients (70%), there was no pulmonary edema on chest X-ray according to the radiologist (Additional file 1: Table S2). The agreement and diagnostic accuracy of LUS for pulmonary edema as diagnosed on chest X-ray were limited (κ statistic 0.12; Additional file 1: Table S3).

Improved diagnostic accuracy for detecting pulmonary edema could lead to improved treatment leading to increased benefits and decreased harms for the patient. In critically ill patients, typically multiple pathophysiological processes are co-occurring at the same time, which hampers the extrapolation of the test characteristics for diagnosing abnormalities in these patients, such as pulmonary edema. As some physicians still use auscultation to detect pulmonary edema, we think our study clarifies that auscultation may not be as reliable for detecting pulmonary edema as classically perceived, especially in the ICU. Ultrasonography becomes increasingly available, and our data add nuance to the discussion surrounding how this technology might be properly integrated into clinical practice in the care of the critically ill. These observations encourage further research of LUS; the need for external validation remains to increase the generalizability of this diagnostic modality.

Several limitations of this study must be acknowledged. First, the clinical examination and ultrasonography were conducted as early as possible after ICU admission which limits the applicability of use in patients with prolonged admission. Further studies should explicate how auscultation and LUS compare in other departments and more specifically other pathologies such as a pneumothorax. Second, we were not able to validate all our LUS assessments by experts, also because there are no reference standards for the interpretation of LUS. Chest X-ray and CT are other diagnostic methods that are frequently used for the assessment of pulmonary edema. However, previous studies have suggested that LUS is superior to chest X-ray and comparable to chest CT scan for diagnosing pulmonary edema [3, 8]. Therefore, we decided not to use these modalities as a reference standard and only included a sensitivity analysis of chest X-ray. We limited LUS reporting to the number of B lines per field and did not use further qualitative commentary. Third, the auscultation was not standardized. During clinical examination, researchers performed both auscultation and LUS; however, in contrast to LUS, we did not describe in detail the location of auscultation. In practice, these were similar to the LUS scan sites. Therefore, we think the influence on our results is minimal. Also, the researchers only specified whether they heard significant crepitation or rhonchi on auscultation. Other abnormal breathing sounds were not recorded and we only documented their overall presence or absence; we are unable to compare auscultation with LUS for each specific scan site. In addition, ideally, we ask the patient to cough to distinguish between rhonchi and/or crepitations. Unfortunately, the large majority of the patients in the ICU are not cooperative with this request. Fourth, even though the researchers who performed the measurements were not involved in patient care, they were not blinded for patient information, such as admission diagnoses, other clinical variables and the results of auscultation when performing the CCUS. However, as ultrasonography was always performed after auscultation, we believe it is proper to discuss this potential source of bias but do not believe that it substantially influenced our results due to the objective nature of B line appearance. Fifth, since researchers were senior medical students and junior residents, auscultation by more experienced medical doctors could potentially improve the diagnostic accuracy. Last, 83 (8%) patients were excluded from the analyses due to the absence of LUS or auscultation data. However, the relatively small proportion of this excluded patient group makes it unlikely that excluded patients would have altered the conclusions. Despite the potential biases and limitations, we showed that the agreement between auscultation and lung ultrasound was poor. This is important as current data is scarce on the diagnostic value of new non-invasive bed tools such as CCUS, especially in comparison with clinical examination in critically ill patients.