Development and validation of the PET-CT score for diagnosis of malignant pleural effusion

From May 1, 2014 through October 31, 2015, all consecutive adult patients admitted to the Department of Respiratory and Critical Care Medicine of Beijing Chaoyang Hospital, China who underwent investigation of exudative pleural effusions and ¹⁸F-FDG PET-CT were enrolled in the derivation cohort. From November 1, 2015 to October 31, 2016, all consecutive adult patients with pleural effusions who underwent ¹⁸F-FDG PET-CT at the Department of Nuclear Medicine of Beijing Chaoyang Hospital were enrolled in a validation cohort. For both derivation and validation cohorts, obtaining a definite cause of the pleural effusion was required for final inclusion into the study.

Following STARD guidelines, this study was conducted in accordance with the Declaration of Helsinki and approved by the institutional ethics committee of Beijing Chaoyang Hospital, Beijing, China (ID 2014-ke-98). Patients provided written informed consent.

The diagnosis of MPE was established if malignant cells were detected upon cytological examination of the pleural fluid or biopsy specimens that were obtained during the same admission with the PET-CT examination. Tuberculous pleural effusion was diagnosed when Ziehl–Neelsen stains or Lowenstein–Jensen cultures of pleural fluid, sputum, or pleural biopsy specimens were positive or when granulomas were found in the parietal pleural biopsies. A parapneumonic effusion was defined as any effusion associated with bacterial pneumonia, lung abscess, bronchiectasis, or empyema, when reported with the presence of pus within the pleural space. Other causes of pleural effusions followed well-established clinical criteria. The patients were followed up for at least 12 months to ensure the absence of malignant pleural processes if they were diagnosed to have benign effusion [13].

The integrated ¹⁸F-FDG PET-CT study was performed on a GE Discovery STE device using a standard protocol before invasive procedures were performed. All patients fasted for at least 6 h and had a blood glucose level of <200 mg/dL before ¹⁸F-FDG administration. Whole body PET-CT scans were acquired 55–73 min (mean 63.2 ± 7.3 min) after intravenous injection of 3.7 MBq/kg of ¹⁸F-FDG. A body scan from the skull base to the upper thighs was firstly performed and then was followed by a head scan. CT parameters for body scan were: 140 kV, 120 mA, and slice thickness of 3.75 mm. CT parameters for head scan were: 120 kV, 200 mA, and slice thickness of 3.75 mm. PET parameters were: 2.5 min/bed for body scan and 5 min/bed for head scan in 3-dimension mode. Attenuation-corrected PET images (voxel size: 3.9 × 3.9 × 3.3-mm for both body and head scan) were reconstructed using a 3-dimensional ordered-subset expectation maximization algorithm (14 subsets and 2 iterations for body scan, and 28 subsets and 2 iterations for head scan). Integrated PET and CT images were obtained automatically on AW VolumeShare2 (GE Healthcare).

One radiologist (TJ) and one nuclear physician (MFY) evaluated the PET-CT images together and a final consensus was obtained on all imaging findings. Both observers were blinded to the final diagnosis of pleural effusion. Pleural ¹⁸F-FDG uptake was calculated by manually drawing regions of interest (ROIs) on PET and CT registered images slice by slice, and the maximum standardized uptake value (SUVmax) was selected to represent ¹⁸F-FDG uptake. ¹⁸F-FDG uptake of pleural fluid was evaluated on the registered slice with the deepest fluid. A circular ROI of 5 mm diameter was placed 5 mm to the parietal pleura, and the SUVmax of pleural fluid was recorded. To obtain a background value of ¹⁸F-FDG uptake, mediastinal uptake was measured by placing an ROI on the superior vena cava and the SUVmean was recorded. All SUVs were normalized to body weight. Then, a target-to-background ratio (TBR) was determined by calculating the ratio of the SUVmax of the pleura or pleural fluid, and the SUVmean of the mediastinum. Lung nodules and/or masses (diameter ≥ 8 mm) were identified, and the SUVmax was measured. Lymph nodes with high uptake (higher than that in the surrounding normal soft tissues) but without calcification and without attenuation higher than 70 HU were regarded as positive. Other organs were classified as positive when there was focal ¹⁸F-FDG uptake, compared with the surrounding normal organ (tissue) or ¹⁸F-FDG uptake that could not be explained by physiologic activity.

Continuous data of PET-CT features (pleural ¹⁸F-FDG uptake, pleural effusion ¹⁸F-FDG uptake, and pleural effusion depth) were first transferred to categorical data by means of drawing the receiver-operating-characteristic (ROC) curves and calculating the cut-off values. Pleural effusion depth was measured in centimeters for maximum anteroposterior depth of the effusion on chest CT scans. The cut-off value of the SUVmax of lung nodule was set as 2.5.

PET-CT parameters evaluated for the discriminating analysis included: (1) pleural thickening (≥3 mm); (2) pleural nodule(s) (≥1 cm); (3) increased pleural ¹⁸F-FDG uptake (TBR > 1.8); (4) pleural thickening (≥3 mm) with increased ¹⁸F-FDG uptake (TBR > 1.8); (5) pleural nodule(s) with increased uptake (TBR > 1.8); (6) pleural calcifications; (7) unilateral effusion; (8) massive pleural effusion (depth > 16.5 cm); (9) increased pleural effusion ¹⁸F-FDG uptake (TBR > 1.1); (10) pleural loculations (i.e., an effusion that is compartmentalized, has septations or a convex shape facing the lung parenchyma, or accumulated in a nondependent portion); (11) lung nodules and/or masses; (12) lung nodules and/or masses with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5); (13) lung single nodule or mass with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5); (14) multiple nodules or masses (uni- or bilateral lungs) with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5); (15) unilateral lung nodules and/or masses with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5); (16) bilateral lung nodules and/or masses with increased 18F-FDG uptake (SUVmax ≥ 2.5); (17) obstructive atelectasis or pneumonia (i.e., FDG uptake was visibly increased at the obstruction site of the bronchus, regardless of the FDG uptake within the distal collapsed lung); (18) mediastinal positive lymph node(s); (19) unilateral positive hilar lymph node(s); (20) bilateral positive hilar lymph node(s); (21) positive hilar lymph node(s); (22) extra-thoracic positive lymph node(s); (23) pericardial effusion; (24) pericardial effusion with increased ¹⁸F-FDG uptake (TBR > 1.1); (25) cardiomegaly (cardiothoracic ratio > 0.5 in axial images); (26) dilation of the inferior vena cava (diameter > 1.7 cm, measured just above the entrance of the suprahepatic veins); (27) ascites; (28) ascites with increased ¹⁸F-FDG uptake (TBR > 1.1); and (29) extrapulmonary malignancies (primary/metastatic).

Categorical and continuous data were expressed as numbers (percentages) and means ± SD, respectively. Between-group comparisons were performed with X², Fisher’s exact, and Student’s t tests, as appropriate. We calculated differences between groups by using ANOVA and univariate analysis, entering only variables with a p value < 0.01 into the multiple regression models. A logistic regression analysis with backward conditional method served to select those imaging variables entering the scoring system. Weight values to each variable were assigned proportionally to the magnitude of the logistic equation’s coefficients.

ROC curves were drawn and the areas under the curve (AUCs) were calculated to determine the diagnostic value of the PET-CT measurement or score, including sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio [14, 15]; and AUCs were compared using z-statistic with the Hanley and McNeil procedure [16]. The optimum cut-off values were defined based on their maximum Youden index (sensitivity + specificity − 1). The parameters of diagnostic accuracy are reported together with their 95% confidence intervals (CIs). To verify the diagnostic accuracy of the PET-CT score in the validation cohort, cases with the PET-CT score above the cut-off value obtained in the derivation cohort were considered as positive results. Interobserver agreement about the PET-CT parameters that make up the PET-CT score was evaluated using κ statistics. The statistical significance level was set at 0.05 (two-tailed). All analyses were conducted with MedCalc and SPSS version 23.0 statistical software.

As shown in Fig. 1, 41 patients in the derivation cohort and 39 in the validation cohort were excluded for the following reasons: (1) suspected benign effusion but follow-up <12 months; (2) with malignant primary disease but no definite diagnosis of the plerual effusion or pleura during the same admission with the PET-CT examination; and (3) no confirmed primary disease and no definite diagnosis of the effusion with a follow-up of 12 months. The characteristics and diagnostic work-up of the excluded patients were presented in Supplemental Table 1. Eventually, a total of 199 patients were included in the derivation cohort and 74 patients in the validation cohort, and their baseline characteristics are shown in Table 1. There were 113 males and 86 females in the derivation cohort, and the average age of the patients was 60.3 ± 16.4 years (range, 21–88 years). In the validation cohort, 43 patients were males and 31 females, and the average age was 61.7 ± 15.3 years (range, 18–90 years).

Patients in the derivation cohort were similar to those in the validation cohort in terms of the distribution of age, sex, and etiology of MPE. Differences in the etiological distribution of benign pleural effusions were found between the two cohorts (X² = 23.56, p < 0.001, Table 1). A total of 42.2% of the patients in the derivation cohort and 52.7% of the patients in the validation cohort were confirmed to have MPE; 57.8% of the patients in the derivation cohort and 47.3% of the patients in the validation cohort suffered from benign pleural effusion (Table 1). The most frequent cause of MPE is lung cancer, followed by mesothelioma in both cohorts. The percentage of tuberculous pleural effusion in the derivation cohort was significantly higher than that in the validation cohort (49.6% vs. 20.0%, p = 0.002).

All PET and CT images were qualified. Initially, we evaluated 29 PET-CT parameters in the bivariate analysis using univariate analysis, and 19 of them displayed statistical significance with p < 0.01 in the discriminating analysis (Table 2). These 19 PET-CT variables were further processed using a multivariable logistic regression model. The model selected five variables that were predictive of malignancy, which were included to establish weight scores as follows: (1) unilateral lung nodules and/or masses with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5) (3 points); (2) extrapulmonary malignancies (primary/metastatic) (3 points); (3) pleural thickening (≥3 mm) with increased ¹⁸F-FDG uptake (TBR > 1.8) (2 points); (4) multiple nodules or masses (uni- or bilateral lungs) with increased ¹⁸F-FDG uptake (SUVmax ≥ 2.5) (1 point); and (5) increased pleural effusion ¹⁸F-FDG uptake (TBR > 1.1) (1 point). For the above variables comprising the PET-CT score, interobserver agreement of measurement was perfect or substantial, with κ coefficients of 0.965, 0.981, 0.888, 0.914, and 0.760, respectively. Thus, the PET-CT scores ranged from 0 to 10 (Table 3).

First, we explored the diagnostic accuracy of each parameter that comprises the PET-CT score in discriminating MPE from benign effusion (Table 4). Although none of these parameters were individually satisfactory for diagnostic purposes in a clinical setting there were meaningful results when examined as a group. At the best cut-off of 4 points, the PET-CT score yielded 83.3% sensitivity (95% CI: 73.6–90.6%), 92.2% specificity (85.7–96.4%), 10.7 positive likelihood ratio (5.6–20.1), 0.2 negative likelihood ratio (0.1–0.3), and AUC 0.949 (0.908–0.975) (Table 5 and Fig. 2a and b), indicating that the PET-CT score provides acceptable differential diagnostic accuracy for patients with MPE and performs significantly better than any single PET-CT parameter.

Next, we separately analyzed the diagnostic accuracy of the PET-CT score in differentiating MPE from tuberculous effusion and from non-tuberculosis effusion. By applying the same cut-off value, we noted that the sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and the AUC of the PET-CT score in differentiating MPE (n = 84) from tuberculous effusion (n = 57) were 83.3% (73.6–90.6%), 89.5% (78.5–96.0%), 7.9 (3.7–17.0), 0.2 (0.1–0.3), and 0.927 (0.871–0.964), respectively (Fig. 2c and d), and that those for differentiating MPE from non-tuberculosis effusion (n = 58) were 83.3% (73.6–90.6%), 94.8% (85.6–98.9%), 16.1 (5.3–48.7), 0.2 (0.1–0.3), and 0.970 (0.927–0.991), respectively (Fig. 2e and f). We also noted that the AUC of the PET-CT score in the differential diagnosis of MPE from tuberculous effusion was similar to that of non-tuberculosis effusion, with no statistical difference (z = 1.870, p = 0.969), and both AUCs did not differ from that of the PET-CT score in discriminating MPE from overall benign effusion (z = 0.880 and 1.105; p = 0.811 and 0.864, respectively).

We performed another blinded validation study in an independent population (39 MPE and 35 benign effusions) to validate the diagnostic accuracy of a PET-CT score ≥ 4 in discriminating MPE from benign effusion. Figure 3 shows that the acceptable discrimination between MPE and benign effusion was confirmed in the validation cohort: sensitivity 89.7% (95% CI: 75.8–97.1%), specificity 88.6% (73.3–96.8%), positive likelihood ratio 7.9 (3.1–19.9), negative likelihood ratio 0.1 (0.1–0.3), and AUC 0.942 (0.863–0.983), respectively.