Pleural ultrasonography versus chest radiography for the diagnosis of pneumothorax: review of the literature and meta-analysis

Chest ultrasonography (US) is gaining more attention in critical care and emergency medicine literature. US has been used recently for evaluation of pneumothorax and other lung pathologies. Several early trials [1‐3] by Litchenstein et al. established the diagnostic signs of pneumothorax on US and showed a strong superiority in favor of US over chest radiography (CXR). Despite those and other cumulating original research evidence favoring ultrasonography, US remained underused. In fact, the most recent British thoracic society guidelines on pleural procedures and thoracic ultrasound stated that “The utility of thoracic ultrasound for diagnosing a pneumothorax is limited in hospital practice due to the ready availability of chest x-rays and conflicting data from published reports” [4]. During the years 2011 and 2012, an increasing number of original research publications compared US with CXR for pneumothorax evaluation, reflecting the expanding rule of US in evaluation of pneumothorax and the growing interest in it to replace CXR. Two meta-analyses [5, 6] attempted for the first time to evaluate the diagnostic accuracy of US in comparison with CXR. In both studies, chest ultrasonography was a near-perfect test with very high sensitivity and specificity. Since the publication of those analyses, few other high-quality original research articles were published comparing the two tests for pneumothorax detection.

By using a regression-analysis model of the published literature, Lijmer et al. [7] showed that inclusion of a diseased population in meta-analyses of diagnostic tests overestimates the diagnostic odds ratio (DOR) by threefold and that using a different reference standard in those studies overestimates the DOR by at least twofold. DOR is defined as the odds for a positive test result in diseased persons relative to the odds of a positive result in nondiseased persons [7]. It is an indicator of test performance that is independent of the prevalence of the disease [7]. Studies with lower methodologic quality tend to include patients with known disease and apply the index test on them, which leads to overestimation of DOR by increasing the odds of having a positive test in diseased subjects. By using a different reference standard (in our analysis: obtaining CT scans preferentially on patients with pneumothorax compared with milder cases without pneumothorax) also overestimated the DOR by increasing the sensitivity and specificity of the index test and introduced possible selection biases to the study. Such studies were included in the previously published meta-analyses comparing the two tests.

We aimed to conduct an accurate meta-analysis of the available literature that included high-quality articles, and avoiding studies that evaluated populations with known pneumothorax and studies that used different verification methods. Additionally, we evaluated the studies for other possible sources of bias. We also intended to include the recent publications that were not included in previous analyses. Furthermore, we specifically planned to address the inherent heterogeneity found that could not be addressed in the previously published meta-analyses. We addressed issues related to operator, type of probe, and patient-population specifics. We believe that our study adds valuable information to the current literature in this field that could guide the application of pleural ultrasonography in the clinicians’ daily practices.

From the 13 chosen studies (Table 1), we extracted the data from each study and conducted a random-effect model meta-analysis. In addition to quality assessment, we assessed for risk of bias and considered covariates that can affect heterogeneity. A total of 3,028 hemithoraces from 1,514 patients were included in the analysis. Our study revealed a clear superiority of US over CXR. The calculated pooled sensitivity for US and CXR were: 78.6% (95% CI, 68.1 to 98.1) and 39.8% (95% CI, 29.4 to 50.3), respectively. The pooled specificity for US and CXR were 98.4% (95% CI, 97.3 to 99.5) and 99.3% (95% CI, 98.4 to 100), respectively (Figures 2 and 3). The pooled DOR for US was 279.31 (95% CI, 106.29 to 733.94), whereas for CXR, the pooled DOR was 87.19 (95% CI, 33.44 to 229.34) (Figure 4). The summary receiver operating characteristic (sROC) curves for US and CXR are depicted in Figure 5. The areas under the curve (AUC) for US and CXR are 0.98 (SE, 0.0065) and 0.959 (SE, 0.014), respectively. Our results, as in previous meta-analyses [5, 6], revealed a high degree of heterogeneity. Our analysis indicated that the observed variations were more likely between studies than within studies, as indicated by a high ratio of the variance of observed effects to total variance (denoted by the I² statistic). We assessed possible sources of heterogeneity by performing meta-regression (and subgroup analyses when applicable) by using data on observed covariates for CXR (Additional file 1: eTable S2) and US (Additional file 1: eTable S3), and results are expressed as a relative diagnostic odds ratio (RDOR). By using subgroup analysis, type of sampling for patient selection and trauma/nontrauma setting were the significant covariates. The amount of variation in US was higher than that in CXR in the included studies (Tau-squared estimate, 2.2 versus 1.65; Figure 4).

For CXR, both consecutive sampling (RDOR = 6.77; 95% CI, 1.53 to 29.93; P = 0.017) and trauma settings (RDOR = 27.42; 95% CI, 5.36 to 140.31; P = 0.0013) were significantly associated with different results, with a higher DOR for consecutive sampling and trauma. Heterogeneity in CXR studies was fully explained by these factors with negligible residual T² (Additional file 1: eTable S2). Subgroup analysis showed improvement in pooled sensitivity in consecutive-sampling studies to 49.7% (95% CI, 33.0 to 66.3). In contrast, in convenience sampling studies, the pooled sensitivity was significantly lower at 32.6% (95% CI, 20.7 to 44.5), whereas specificity remained comparable in all subgroups.

For US, both trauma settings (RDOR = 32.87; 95% CI, 2.42 to 447.03; P = 0.018) and consecutive sampling (RDOR = 21.99; 95% CI, 1.98 to 244.93; P = 0.021) were significant contributors to heterogeneity by using meta-regression analysis (Additional file 1: eTable S3). Subgroup analysis also showed that in consecutive sampling studies, the pooled sensitivity improved to 85.3% (95% CI, 68 to 100), whereas in nonconsecutive (convenience) sampling studies, the pooled sensitivity decreased to 73.6% (95% CI, 60.4 to 86.7). Studies that used the high-frequency linear array probe had a pooled sensitivity of 82.2% (95% CI, 68.8 to 95.5), whereas those using a convex array probe had a pooled sensitivity of 76% (95% CI, 59.8 to 92.3). Operator (radiologist versus others) was not a significant variable in our analysis. However, emergency physician-performed US had better sensitivity than nonemergency physicians-performed US (82.3% versus 72.8%). (Additional file 1contains statistical tables for both CXR and US subanalyses).

In the discussion sections of the reviewed articles [11‐23], few conditions were reported to cause false-positive results on US, including pleural adhesions, bullous emphysema, and main-stem intubation on the contralateral side. Subcutaneous emphysema and pleural calcifications were also reported to impede the US waves.

Ultrasonography time, when assessed [12, 14, 18, 20, 21], ranged between <2 minutes and 15 minutes. In all studies that evaluated test-performance time, except one [21], US time was significantly shorter than CXR time (mean, <5 minutes versus 10 to 15 minutes).

In the past 3 years, at least nine published original prospective research articles addressed the diagnostic accuracy of pleural ultrasonography for the diagnosis of pneumothorax, reflecting the growing interest in this valuable test as an alternative to CXR. Six of those articles [11‐16] met our inclusion and quality criteria and, for the first time, were included in a meta-analysis. The pooled sensitivity of US in our study is the lowest of the previous two analyses [5, 6] (78.6% versus 88% [5] and 90.9% [6]). The DOR reported by a previous meta-analysis [5] is 3.3 times higher than our result (993 with 95% CI, 333 to 2,957) indicating a threefold overestimation of the accuracy of the test and attesting to Lijmer’s previous estimates [7]. Both reported sensitivity and DOR are significantly different from those obtained in previous analyses, as shown from the reported confidence intervals.

We believe that our study provides better estimates of the test parameters because of the inclusion of a large number of good-quality standardized studies and patients (total of 1,514 patients) in the analysis. Our meta-analysis allowed, for the first time, the identification of significant sources of variation in the effect size among the included studies. It is the first to compare CT scan, US, and CXR in the same population on this large scale. On all counts US remains superior to CXR for detection of pneumothorax, even after controlling for possible sources of heterogeneity (the lowest US subgroup sensitivity was 73.6%). With positive test results, patients tested with US have greater odds of having an accurate diagnosis of pneumothorax than do those tested with CXR (DOR, 279.31 versus 87.19). The majority of studies included in our analysis were in trauma settings; this was expected, as an indication for CT scan of the chest is readily available in this setting. Our results indicated that a linear probe provided better sensitivity (82% versus 76%); this is likely because of the better views of the lung sliding sign obtained with this high-resolution probe.

In our study, emergency physicians performed better US than did nonemergency physicians (sensitivity, 82.3% versus 72.8%). This could be related to their early experience in thoracic US use as part of the eFAST (Extended Focused Assessment with Sonography for Trauma) that emphasized the importance of training and experience in this operator-dependent test.

Our study is not without limitations. Despite meticulous efforts to explain possible causes of heterogeneity, we were unable to account for some minor sources, especially on the US analysis, which had some minimal residual heterogeneity after meta-regression (T² = 0.2; Additional file 1: eTable S3). We did not evaluate our meta-analysis for publication bias. We kept our search very broad initially to overcome this issue but included all studies that performed US and CXR as well as CT scan as the gold standard to maintain accuracy and avoid overestimates of diagnostic accuracy.

Studies published in languages other than English, with the exception of one [10], were not identified in our search and were not included. This probably has a negligible effect. CXR was performed in supine position in the vast majority of our study patients, and this may have underestimated test performance. In nontrauma studies [16, 21], in which a large number of patients had semierect CXR, the sensitivity only increased to 42.3% (95% CI, 14.3 to 70.3). A previous meta-analysis [5] included studies of CXR alone to neutralize the effect of patient position, and their reported sensitivity was only 52%.

As stated earlier, most of our included studies were in trauma/emergency department settings. This was mainly the result of including studies that compared both tests with the gold standard (CT scan of the chest). It should be noted that the severity of trauma was not assessed in the majority of those studies. The consecutive-sampling studies (which showed higher sensitivity) may have allowed a wide spectrum of patients to be included in those studies. Furthermore, most pneumothoraces missed by CXR were occult and detected only by US and CT scans. In one study [22] that reported injury severity score (ISS), occult pneumothorax was 8 times more likely to be present in patients with ISS >16. Studies comparing US and CXR with CT scans in evaluation of postbronchoscopy pneumothorax, for example, would expose patients to unnecessary radiation from CT scan and would be unethical. A small pneumothorax might not require invasive treatment, but it is important to recognize, in certain clinical situations, such as positive-pressure ventilation and air transportation.

It is important to note that the test characteristics are only part of the assessment of a diagnostic test performance, and the value of any test ultimately lies in its effects on patient outcome. Other important factors such as potential of harm as a consequence of the test (in our case, possible exposure to unnecessary procedures to treat a small pneumothorax or exposure to ionizing radiation), physician’s perception and confidence in test results, as well as the ability to make treatment decisions based on test results were not addressed in our study [25]. Future research can be designed to address the downstream effects of two separate testing strategies specifically for pneumothorax: one that uses CXR and another for US. Possible outcome measures are number of invasive procedures and subsequent tests resulting from the index test, and total condition-related cost of care. We expect US to be safer, more convenient, more cost effective, and to outperform CXR in most aspects.

Despite the lower sensitivity and lower DOR found in our analysis, US remains much more sensitive than CXR for identification of pneumothorax. Our analysis supports the available evidence in favor of ultrasonography over chest radiography and provides an objective assessment of the diagnostic performance of both tests in the well-designed published studies that we included in our meta-analysis. Our analysis identifies several important factors that increase the accuracy of US in detection of pneumothorax, including operator experience, patient population, and the type of probe used.

Saadah Alrajab, MD, MPH; department of pulmonary, critical care and sleep medicine. Louisiana State University Health Sciences center and Cogent-The Intensivist Group, St.Bernardine Medical Center (California). Asser M Ypussef, MD; department of surgery and trauma. Louisiana State University Health Sciences center. Nuri I Akkus, MD; department of cardiology. Louisiana State University Health Sciences center. Gloria Caldito, PhD; department of biometry. Louisiana State University Health Sciences center.