Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Prognostic Value of 18F-FDG PET/CT in Surgical Non-Small Cell Lung Cancer: A Meta-Analysis

  • Jing Liu,

    Affiliation Department of Radiation Oncology and Shandong Key Laboratory of Radiation Oncology, Shandong Cancer Hospital and Institute, Jinan, Shandong, China

    ⨯
  • Min Dong,

    Affiliation Department of Oncology, the People’s Hospital of Pingyi County, Pingyi, Shandong, China

    ⨯
  • Xiaorong Sun,

    Affiliation Department of Radiology, Shandong Cancer Hospital and Institute, Jinan, Shandong, China

    ⨯
  • Wenwu Li,

    Affiliation Department of Radiology, Shandong Cancer Hospital and Institute, Jinan, Shandong, China

    ⨯
  • Ligang Xing ,

    xinglg@medmail.com.cn

    Affiliation Department of Radiation Oncology and Shandong Key Laboratory of Radiation Oncology, Shandong Cancer Hospital and Institute, Jinan, Shandong, China

    ⨯
  • Jinming Yu

    Affiliation Department of Radiation Oncology and Shandong Key Laboratory of Radiation Oncology, Shandong Cancer Hospital and Institute, Jinan, Shandong, China

    ⨯

Abstract

Background

The identification of surgical non-small cell lung cancer (NSCLC) patients with poor prognosis is a priority in clinical oncology because of their high 5-year mortality. This meta-analysis explored the prognostic value of maximal standardized uptake value (SUVmax), metabolic tumor volume (MTV) and total lesion glycolysis (TLG) on disease-free survival (DFS) and overall survival (OS) in surgical NSCLC patients.

Materials and Methods

MEDLINE, EMBASE and Cochrane Libraries were systematically searched until August 1, 2015. Prospective or retrospective studies that evaluated the prognostic roles of preoperative 18F-FDG PET/CT with complete DFS and OS data in surgical NSCLC patients were included. The impact of SUVmax, MTV or TLG on survival was measured using hazard ratios (HR). Sub-group analyses were performed based on disease stage, pathological classification, surgery only and cut-off values.

Results

Thirty-six studies comprised of 5807 patients were included. The combined HRs for DFS were 2.74 (95%CI 2.33–3.24, unadjusted) and 2.43 (95%CI: 1.76–3.36, adjusted) for SUVmax, 2.27 (95%CI 1.77–2.90, unadjusted) and 2.49 (95%CI 1.23–5.04, adjusted) for MTV, and 2.46 (95%CI 1.91–3.17, unadjusted) and 2.97 (95%CI 1.68–5.28, adjusted) for TLG. The pooled HRs for OS were 2.54 (95%CI 1.86–3.49, unadjusted) and 1.52 (95%CI 1.16–2.00, adjusted) for SUVmax, 2.07 (95%CI 1.16–3.69, unadjusted) and 1.91 (95%CI 1.13–3.22, adjusted) for MTV, and 2.47 (95%CI 1.38–4.43, unadjusted) and 1.94 (95%CI 1.12–3.33, adjusted) for TLG. Begg’s test detected publication bias, the trim and fill procedure was performed, and similar HRs were obtained. The prognostic role of SUVmax, MTV and TLG remained similar in the sub-group analyses.

Conclusions

High values of SUVmax, MTV and TLG predicted a higher risk of recurrence or death in patients with surgical NSCLC. We suggest the use of FDG PET/CT to select patients who are at high risk of disease recurrence or death and may benefit from aggressive treatments.

Citation: Liu J, Dong M, Sun X, Li W, Xing L, Yu J (2016) Prognostic Value of 18F-FDG PET/CT in Surgical Non-Small Cell Lung Cancer: A Meta-Analysis. PLoS ONE 11(1): e0146195. https://doi.org/10.1371/journal.pone.0146195

Editor: Santosh Patnaik, Roswell Park Cancer Institute, UNITED STATES

Received: September 17, 2015; Accepted: December 13, 2015; Published: January 4, 2016

Copyright: © 2016 Liu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This work was supported in part by the grant from National Health and Family Planning Commission Institutes of the People’s Republic of China (201402011), and the grants from National Natural Science Foundation of China (81272502 and L1322016) to LX. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The application of advanced diagnostic and screening techniques led to the increased detection of early staged non-small cell lung cancers (NSCLC) and improved cures using standard surgery [1]. The 5-year survival after resection of localized NSCLC approaches a modest 50% despite improved surgical techniques and advanced adjuvant therapy [2, 3]. No prognostic factor, except stage and performance status, was definitively established in NSCLC. Accurate markers would be invaluable to stratify patients for adjuvant therapy and predict outcomes.

18F-fluorodeoxyglucose (FDG) positron emission tomography/computed tomography (PET/CT) is the standard modality for staging, treatment response monitoring and prognosis prediction for a variety of tumors, including NSCLC [4, 5]. Standardized uptake value (SUV) is a semi-quantitative determination of the normalized concentration of radioactivity, and maximum SUV (SUVmax) is the most widely applied parameter in clinical practice [6]. Volumetric parameters, including metabolic tumor volume (MTV) and total lesion glycolysis (TLG), were also used recently to reflect disease burden and tumor aggressiveness in NSCLC [4, 7]. Several recent systematic reviews and meta-analyses [8–10] found that SUV was negatively correlated with prognosis in heterogeneous groups of NSCLC patients. Im et al. [11] reported significant prognostic values of MTV and TLG on survival in NSCLC patients. However, the quality of existing studies has not been systematically assessed, and their clinical features have not been fully assessed to further evaluate the potential association between SUV or volumetric parameters and prognosis in surgical NSCLC.

Therefore, we performed a meta-analysis to identify, appraise, and synthesize results from published studies that examined the prognostic value of SUVmax, MTV and TLG on disease-free survival (DFS) and overall survival (OS) in surgical NSCLC patients.

Materials and Methods

Search Strategy and Eligible Criteria

MEDLINE, EMBASE and Cochrane Library were searched and updated through August 1, 2015. The following terms were used: “non-small cell lung cancer” OR “NSCLC” OR “carcinoma, non-small cell lung” AND “18F-FDG” OR “fluorodeoxyglucose” OR “PET” OR “positron emission tomography” AND “survival” OR “local control” OR “prognostic” OR “outcome” OR “predict” AND “surgery” OR “resect” OR “operation”. Reviews, case studies, conference abstracts and editorials were excluded.

Two authors independently searched articles and performed an initial screening of identified titles and abstracts. Articles were further reviewed if they reported the prognosis of surgically resected NSCLC patients with pre-operative 18F-FDG PET/CT imaging from original data. Full-text articles were used for the second screening. The following inclusion criteria for the meta-analysis were used: (1) prospective or retrospective studies investigating the correlation of FDG uptake with DFS, recurrence-free survival (RFS), and/or OS; (2) pathological stage I-IIIA NSCLC patients who received diagnostic 18F-FDG PET/CT scanning before treatments; (3) treated with surgery alone or adjuvant therapy; (4) survival data assessed in detail; and (5) surgical procedures included either full anatomical resections or limited lung resection regardless of whether they were performed via open thoracotomy or video-assisted thoracic surgery. A consensus resolved any discrepancies.

Studies that included patients with small cell lung cancer (SCLC) were eligible if more than 95% of patients had NSCLC. Patients with an advanced stage (IIIB-IV) also accounted for less than 5% of the included studies. Data were partially extracted when only certain sub-group analyses met our inclusion criteria. Studies that included patients who received neoadjuvant therapy were excluded. Only the most recent or complete report was included when the survival results of the same patient population were reported more than once.

Data Extraction

Data extraction was conducted in agreement with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance (S1 PRISMA Checklist) [12]. Two investigators independently extracted information, including the first author, publication year, country, study design, sample size, stage, treatment, and survival endpoints. The primary endpoint was DFS, which was measured from the defined starting point in each study to the date of recurrence or first progression. OS was taken as the secondary endpoint and defined as the time from the starting point applied in each study to death.

Study Quality Control

Three investigators reviewed and scored each article independently using a quality scale (S1 File). Quality assessment included four modified parts based on similar studies: scientific design, the generalizability of the results, data analyses, and PET reports [13–15]. Five items were observed in each part. A point value of 0, 1, or 2 was scored to each item. A consensus was obtained of all investigators present, which ensured the objectivity of the scores and correct interpretation. Final scores are expressed as percentages, and higher values reflect a greater consistence with quality assessment standards. Any article with a final score < 60% was excluded.

Statistical Analysis

Review Manager statistical software (RevMan, version 5.3) was used. The impact of SUVmax, MTV and TLG on DFS and OS was measured using hazard ratios (HRs). Survival data were extracted using the methodology suggested by Tierney et al. [16]. Cut-off values of SUVmax, MTV and TLG and the delineation thresholds applied to MTV and TLG were determined based on the definition applied in each individual study. Unadjusted and adjusted values were extracted for risk measurements. We extracted the HR estimate and 95% confidence intervals (CIs) directly from each study when provided by the authors. P values of the log-rank test, number analyzed in each group, and the number of events were extracted to estimate the univariate HR indirectly. Correlations between the quality scores and the number of patients were measured using the Spearman’s rank correlation coefficient.

Heterogeneity was evaluated using Cochrane’s Q test and I2 [17]. P<0.05 in Q test was considered significant heterogeneity. An I2 value of 0% indicates no heterogeneity, a value less than 25% indicates low heterogeneity, a value of 25.1–50% indicates moderate heterogeneity, and a value greater than 50% indicates substantial heterogeneity [18]. A fixed effect model was used to calculate the pooled HRs when no, low or moderate heterogeneity was observed. A random effects model was applied when substantial or significant heterogeneity was observed. An HR greater than 1 implied worse survival outcome for patients with high SUVmax, MTV or TLG, but an HR less than 1 implied a survival benefit for patients with high SUVmax, MTV or TLG. Sub-group analyses were performed based on histological subtypes, pathological stage, surgery only and cut-off values.

The possibility of publication bias was estimated using visual inspection of a funnel plot. Begg’s test was performed for meta-analysis that included more than 10 studies [19, 20]. We also performed non-parametric “trim and fill” procedures to further estimate the potential influence of publication bias [21, 22]. This procedure calculates a new pooled HR that incorporates hypothetical missing studies.

Results

Study Characteristics and Qualitative Assessment

Thirty-six eligible studies were included in the meta-analysis [23–58] (Fig 1, Tables 1 and 2). Only two studies [31, 37] were prospectively designed. The studies were published between 2000 and 2015, and the sample size varied from 49 to 530 subjects (median 102). Only 5 SCLC [40, 51] patients were mixed into the analysis of 5807 patients. Four studies lacked raw data of stage [23, 29, 40, 49], but the distribution of stages I, II, III and IV were 80.4%, 14.2%, 4.5% (2.7% IIIA, 0.9% IIIB, and 0.9% stage III) and 0.9%, respectively. Table 1 lists PET/CT scans and models. The dose of FDG injected varied from 150 to 666 MBq based on different individual scanning protocols. The time duration before scanning was 40–60 minutes in 28 studies, 81 minutes in 1 study, 90 minutes in 1 study and not reported in 6 studies. SUVmax was measured in 34 studies [23–25, 28–58], which normalized values by body weight. MTV was measured in 7 studies [24, 26–29, 52, 53], and TLG was measured in 7 studies [24, 26, 27, 29, 52, 53, 56]. A fixed SUV of 2.5 [27, 28, 52, 56], the gradient segmentation method [29], a 50% of SUVmax [24], a 42% of SUVmax [53], and mediastinal background SUVmean plus 2 standard deviations [26] were adapted to segment volumes of interest. A minimum P value, receiver operating characteristics (ROCs), and median value were applied in most studies to determine cut-off values. Median cut-off points were 5.90 (2.4 to 20) for SUVmax. The cut-off values of MTV were between 2.95 and 37.34 cm3 (median 11.197 for OS and 10.29 for DFS), and TLG values ranged from 9.61 to 407.48 (median 26.739 for OS and 29.221 for DFS).

thumbnail
Download:
Fig 1. Flowchart of Study Selection.

https://doi.org/10.1371/journal.pone.0146195.g001

thumbnail
Download:
Table 1. Studies Included in Meta-Analysis.

https://doi.org/10.1371/journal.pone.0146195.t001

thumbnail
Download:
Table 2. Clinical Characteristics of Included Studies.

https://doi.org/10.1371/journal.pone.0146195.t002

Adjusted HRs were determined for 25 studies. Most risk measures were adjusted for tumor size or T stage, stage, age, gender and histology, and other studies were adjusted for lymph node status, differentiation and CEA level.

Twenty-seven studies published complete resection rates as 100%, while the remaining studies did not report rates. Average (mean or median) follow-up duration was given in 29 studies and ranged from 16.6 to 64 months (median 32.0 months). The follow-up design was reported in detail in 11 studies, but it was not indicated in 20 studies.

Attempts to contact the authors to obtain missing information of methodological quality were made when necessary, and the mean quality score was 77.5% (70.0% to 87.5%). Spearman’s correlation coefficient was 0.326 between the quality score and the number of patients (P = 0.36).

Primary Outcome: DFS

Unadjusted analysis of SUVmax and DFS (2845 patients) revealed a combined HR of 2.74 (95%CI: 2.33–3.24, P = 0.07, I2 = 32%) (Fig 2A). The funnel plots for publication bias exhibited significant asymmetry with statistical significance (Begg’s test, z = 3.59, P<0.001). The pooled HR was 2.37 (95% CI: 2.03–2.75) after the trim and fill analysis (Fig 3A). Sensitively analysis was further conducted with significant heterogeneity (I2 = 72%, P<0.00001, HR 2.43, 95%CI: 1.76–3.36) for multivariate analysis of SUVmax and DFS (2279 patients) (Fig 2B) to estimate the effect of each study on the pooled HR by omitting one study at a time. Three studies [34, 44, 55] were omitted, and an HR of 3.24 (2.43–4.33) using a fixed-model was obtained with a decreased I2 of 38% and a P value of 0.07 in the Q test. The Begg’s test was statistically significant (z = 2.23, P = 0.026). The pooled HR was 1.77 (95% CI: 1.29–2.43) after the trim and fill analysis (Fig 3B).

thumbnail
Download:
Fig 2. Forest plots of HR for recurrence with SUVmax (A, unadjusted; B, adjusted), MTV (C, unadjusted; D, adjusted) and TLG (E, unadjusted; F, adjusted).

The Chi2 test is a measurement of heterogeneity. P<0.05 indicates significant heterogeneity. Squares = individual study point estimates. Horizontal lines = 95% CIs. Rhombus = summarized estimate and its 95% CI. Fixed: fixed effect model. Random: random effect model.

https://doi.org/10.1371/journal.pone.0146195.g002

thumbnail
Download:
Fig 3. Funnel plots without (left column) and with (right column) trim and fill.

The pseudo 95% confidence interval (CI) is computed as part of the analysis that produced the funnel plot and corresponds to the expected 95% CI for a given standard error (SE). HR indicates hazard ratio.

https://doi.org/10.1371/journal.pone.0146195.g003

Five studies (881 patients) indicated that a larger MTV predicted worse local control using univariate analysis with a pooled HR of 2.27 (95% CI: 1.77–2.90, P = 0.16, I2 = 39%) (Fig 2C). Three studies obtained the same result [26, 27, 52] for multivariate analysis, and 1 study reported undetermined results [28]. The combined HR of all 4 studies (782 patients) was 2.49 (95% CI: 1.23–5.04, P = 0.04, I2 = 63%) (Fig 2D). High TLG was associated with poor local control in 4 studies (821 patients) using univariate analysis with a combined HR of 2.46 (95% CI: 1.91–3.17, P = 0.39, I2 = 1%) (Fig 2E). Multivariate analysis with 3 studies (730 patients) revealed a combined HR of 2.97 (95%CI: 1.68–5.28, P = 0.09, I2 = 59%) (Fig 2F).

Secondary Outcome: OS

Univariate analysis of 19 studies (3178 patients) explored the prognostic role of SUVmax for OS and demonstrated a combined HR of 2.54 (95%CI: 1.86–3.49, P<0.00001, I2 = 86%) (Fig 4A). I2 was not statistically significant (17 studies, 2698 patients, P = 0.19, I2 = 23%) after the exclusion of two studies [34, 39] with an HR of 2.26 (95%CI: 1.94–2.64). Begg’s test revealed no significant publication bias (z = 1.47, P = 0.141). Heterogeneity also existed (I2 = 68%, P = 0.002, HR = 1.52, 95%CI: 1.16–2.00) in adjusted analyses of SUVmax and OS rate (9 studies, 1467 patients) (Fig 4B). Exclusion of the report from Bille et al.[39] reduced this heterogeneity and led to a P value of 0.69 (8 studies, 1063 patients, I2 = 0%). A fixed-effect model revealed that the combined HR reached 1.64 (95%CI: 1.34–1.99).

thumbnail
Download:
Fig 4. Forest plots of HR for deaths with SUVmax (A, unadjusted; B, adjusted), MTV (C, unadjusted; D, adjusted) and TLG (E, unadjusted; F, adjusted).

The Chi2 test is a measurement of heterogeneity. P<0.05 indicates significant heterogeneity. Squares = individual study point estimates. Horizontal lines = 95% CIs. Rhombus = summarized estimate and its 95%CI. Fixed: fixed effect model. Random: random effect model.

https://doi.org/10.1371/journal.pone.0146195.g004

Larger MTV predicted poor OS in univariate and multivariate analyses. Significant disparities were determined in unadjusted (4 studies, 860 patients, HR 2.07, 95%CI: 1.16–3.69, P = 0.005, I2 = 77%) and adjusted analyses (3 studies, 679 patients, HR = 1.91, 95%CI: 1.13–3.22, P = 0.11, I2 = 55%) (Fig 4C and 4D). High TLG was associated with poor OS in univariate analysis (5 studies, 1108 patients) with a combined HR of 2.47 (95%CI: 1.38–4.43, P = 0.0006, I2 = 79%). Multivariate analysis of 3 studies with 836 patients also demonstrated a significant prognostic role of TLG for OS (HR 1.94, 95%CI: 1.12–3.33, P = 0.04, I2 = 69%) (Fig 4E and 4F).

Stratified Analyses of SUVmax

Table 3 summarizes total and stratified results. The combined HRs of SUVmax for stage I in the sub-group analysis for DFS according to disease stage were 3.62 (95%CI: 2.72–4.81, P = 0.07, I2 = 41%) from univariate analysis and 3.35 (95%CI: 2.18–5.16, P = 0.46, I2 = 0%) from multivariate analysis. Publication bias existed (Z = 2.81, P = 0.005) in univariate analysis, and the pooled HR was 3.00 (95%CI: 2.30–3.91) after the trim and fill process (Fig 3C). The combined HRs of SUVmax from univariate analysis of stage I and stage II for OS were 3.43 (95%CI: 1.75–6.75, P = 0.01, I2 = 66%) and 2.64 (95%CI: 1.11–6.31, P = 0.23, I2 = 31%), respectively.

thumbnail
Download:
Table 3. Total and subgroup analyses of SUVmax, MTV and TLG in surgical NSCLC.

https://doi.org/10.1371/journal.pone.0146195.t003

Sub-group analysis based on the histology type revealed that the combined HRs of SUVmax on DFS from univariate analysis for adenocarcinoma and non-adenocarcinoma were 4.81 (95%CI: 2.87–8.08, P = 0.13, I2 = 47%) and 1.98 (95%CI: 1.04–3.79, P = 0.64, I2 = 0%), respectively. Four studies provided multivariate analysis of SUVmax on DFS for adenocarcinoma patients, with a pooled HR of 2.92 (95%CI: 1.19–7.17, P = 0.005, I2 = 77%).

The combined HRs of SUVmax on DFS were 2.75 (95%CI: 2.30–3.29, P = 0.03, I2 = 41%) and 2.30 (95%CI: 1.65–3.20, P<0.01, I2 = 72%) in unadjusted and adjusted analysis, respectively, when analyses were narrowed to surgical only patients without adjuvant therapy. The pooled HRs of SUVmax in univariate and multivariate analysis were 2.27 (95%CI: 1.93–2.66, P = 0.02, I2 = 48%) and 1.59 (95%CI: 1.30–1.95, P = 0.79, I2 = 0%), respectively, for OS.

Cut-off values of SUVmax in each individual study were determined to be high (>5.9) or low (< = 5.9) based on the median value. Subgroup analyses demonstrated that the combined HRs of SUVmax for high cut-off value were 2.42 (95%CI: 1.89–3.11, P = 0.10, I2 = 38%) from univariate analysis and 1.68 (95%CI: 1.07–2.63, P = 0.03, I2 = 58%) from multivariate analysis. The low cut-off value studies demonstrated that the combined HRs of SUVmax (univariate analysis: HR 3.02, 95%CI: 2.42–3.77, P = 0.03, I2 = 46%; multivariate analysis: HR 4.63, 95%CI: 2.53–8.48, P = 0.003, I2 = 62%) were larger than the high cut-off value subgroup. Analysis of OS data also revealed a similar trend with pooled HRs of 3.47 (95%CI: 2.10–5.71, P = 0.003, I2 = 68%) and 1.61 (95%CI: 1.22–2.12, P = 0.38, I2 = 0%) for high cut-off group and 2.12 (95%CI: 1.44–3.13, P<0.01, I2 = 85%) and 1.42 (95%CI: 0.97–2.08, P = 0.02, I2 = 67%) for low cut-off group in unadjusted and adjusted analyses, respectively.

Discussion

There is a high risk of local relapse and distant metastasis after curative resection for early-stage and localized NSCLC. Therefore, adjuvant therapy was explored to eliminate occult metastases and/or loco-regional residual tumor cells with a consequent reduction on recurrence and prolonged survival. It is essential to identify prognostic factors that may predict patients who are at a high risk of recurrence who will attain the most benefit from the adjuvant therapy to optimize the treatment. The evidence-based use of adjuvant therapy is highly dependent on clinical-pathological tumor staging information in the clinical setting. The role of 18F-FDG PET/CT imaging for the prediction of local control and OS in surgical NSCLC must be investigated because it may provide important biological information beyond TNM staging. The present systemic review and meta-analysis found that higher values of SUVmax, MTV and TLG predicted a higher risk of disease recurrence or death in patients with surgical NSCLC. The positive association remained statistically significant across stratified analyses according to stage, pathology and cut-off values. FDG PET/CT may be used to select patients who are at high risk of tumor recurrence or death and may benefit from subsequent more aggressive treatments.

SUVmax is the most commonly used parameter in 18F-FDG PET/CT diagnosis and response monitoring because of high reproducibility and availability. The potential prognostic value of SUVmax for primary lung cancer was widely reported in various staged and treated populations [8–10, 14] (Table 4). Therefore, our meta-analysis focused on surgical NSCLC only and provided the most comprehensive information for the total population and sub-groups based on disease stage, pathological classification and cut-off values. However, SUVmax only provides information about a single volumetric pixel within the tumor, and it does not measure the volume or heterogeneity of metabolically active disease. Volumetric parameters, such as MTV and TLG, were investigated recently. The prognostic role of MTV and TLG was meta-analyzed in NSCLC patients with different stages [11]. Similar results were derived in our study, which focused on surgical NSCLC patients. Volume-based parameters exhibit advantages in the measurement of metabolic tumor burden, but controversy on the most appropriate segmentation method to measure MTV and TLG remains. Potential preferable performance of volumetric parameters to SUVmax as prognostic factors were reported by the studies [24, 28, 29, 52, 53] that reported complete data of FDG PET/CT-derived parameters. The present meta-analysis demonstrated that SUVmax performed equally with volumetric parameters based on existing data because of the limited data of volumetric parameters compared with FDG uptake. Other FDG PET/CT imaging characteristics beyond traditional parameters were also studied, such as intratumor FDG uptake heterogeneity. This parameter, as an area under the curve (AUC) of the cumulative histogram, and texture analysis predict tumor control [59] and are independent prognostic factors for survival [60–62] in NSCLC. However, these reports were not included in present meta-analysis because the study population was relatively small.

thumbnail
Download:
Table 4. Previous meta-analyses of 18F-FDG PET/CT on survival of NSCLC patients.

https://doi.org/10.1371/journal.pone.0146195.t004

Patient heterogeneity, statistical data mining, retrospective cohorts, PET acquisition and calculations of SUVmax are significant contributors to heterogeneity, which limited the application of glucose uptake as a companion diagnostic/prognostic marker. NSCLC is a heterogeneous disease. Patients with different histological types, stages, surgical procedures and adjuvant treatments were included in the meta-analysis. For example, Higashi et al. [41] and Stiles et al. [44] applied similar thresholds for FDG uptake. Significant differences were found in Higashi’s study in DFS (HR 8.17, 95%CI: 2.83–23.53), but statistically significant differences in DFS were not found in Stiles’s study (HR 1.54, 95%CI: 0.92–2.56). There were more patients with stage I NSCLC (80.7% versus 76.8%) and more patients with bronchioloalveolar cell carcinoma (22.8% of BAC versus <8.3%) in Higashi’s study, which may explain the lower risk of recurrence in patients with low tumor FDG uptake. The heterogeneity in PET imaging thresholds was also obvious between the studies, which be explained by many factors, including the type of PET machine, the algorithms for iteration and reconstruction, the time elapsed between FDG injection and emission scan, and the method for threshold determination. Differences in defining the regions of interest [63] and timing of the data acquisition [64] may also result in different absolute SUV estimates.

Heterogeneity between the included reports was the main limitation of this meta-analysis. Non-English articles were excluded. The fact that small sample studies with negative results are less frequently published or published with simple descriptions led to the phenomenon of increased standard error for higher HRs. The trim and fill sensitivity analysis in the present study, which incorporates the hypothetical missing studies, did not change the general result, which suggests that the association was convincible. Individual HRs from small sample studies weighed less in the total HR, and it was also helpful to ensure the reliability of results. MTV and TLG were measured in 7 studies only. Multivariate analyses were based on 5 studies for MTV and 4 for TLG. Too little data were available to meta-analyze the values of volumetric PET/CT parameters for the prediction of patient’s prognosis. Only 2 included studies were prospectively designed, but PET as a biomarker to prognosticate or predict the response to therapy was assessed over 10 years. The prospectively designed studies [65, 66] that were ineligible for the present meta-analysis also reported primarily positive results on various FDG PET/CT-derived parameters of lung cancer patients. Our meta-analysis offers a considerably valid conclusion for clinical practice under the circumstance of insufficient evidence from prospectively designed data.

In summary, this meta-analysis demonstrated that high values of SUVmax and MTV derived from the pretreatment of 18F-FDG PET/CT predicted a higher risk of recurrence or death in surgical NSCLC patients. Our findings suggest that FDG PET/CT may be used for risk stratification in disease control and survival. Patients with tumors who exhibit intense FDG uptake may be considered at a high risk of treatment failure and may benefit from more aggressive treatment. Further individual patient data should be meta-analyzed to determine the optimal threshold for PET imaging parameters.

Supporting Information

S1 PRISMA Checklist. PRISMA checklist.

https://doi.org/10.1371/journal.pone.0146195.s001

(DOC)

S1 File. The quality scale used in this study.

https://doi.org/10.1371/journal.pone.0146195.s002

(DOCX)

Author Contributions

Conceived and designed the experiments: LX JY JL. Performed the experiments: JL MD XS WL LX JY. Analyzed the data: JL MD XS WL LX. Contributed reagents/materials/analysis tools: JL MD XS LX JY. Wrote the paper: JL MD LX.

References

  1. 1. Zheng X, Schipper M, Kidwell K, Lin J, Reddy R, Ren Y, et al. Survival outcome after stereotactic body radiation therapy and surgery for stage I non-small cell lung cancer: a meta-analysis. International journal of radiation oncology, biology, physics. 2014;90(3):603–11. pmid:25052562.
  2. 2. Goldstraw P, Crowley J, Chansky K, Giroux DJ, Groome PA, Rami-Porta R, et al. The IASLC Lung Cancer Staging Project: proposals for the revision of the TNM stage groupings in the forthcoming (seventh) edition of the TNM Classification of malignant tumours. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2007;2(8):706–14. pmid:17762336.
  3. 3. Naruke T, Tsuchiya R, Kondo H, Asamura H. Prognosis and survival after resection for bronchogenic carcinoma based on the 1997 TNM-staging classification: the Japanese experience. The Annals of thoracic surgery. 2001;71(6):1759–64. pmid:11426744.
  4. 4. Moon SH, Hyun SH, Choi JY. Prognostic significance of volume-based PET parameters in cancer patients. Korean journal of radiology: official journal of the Korean Radiological Society. 2013;14(1):1–12. pmid:23323025; PubMed Central PMCID: PMC3542291.
  5. 5. Zhu A, Lee D, Shim H. Metabolic positron emission tomography imaging in cancer detection and therapy response. Seminars in oncology. 2011;38(1):55–69. pmid:21362516; PubMed Central PMCID: PMC3075495.
  6. 6. Paidpally V, Chirindel A, Lam S, Agrawal N, Quon H, Subramaniam RM. FDG-PET/CT imaging biomarkers in head and neck squamous cell carcinoma. Imaging in medicine. 2012;4(6):633–47. pmid:23482696; PubMed Central PMCID: PMC3587845.
  7. 7. Davison J, Mercier G, Russo G, Subramaniam RM. PET-based primary tumor volumetric parameters and survival of patients with non-small cell lung carcinoma. AJR American journal of roentgenology. 2013;200(3):635–40. pmid:23436855.
  8. 8. Na F, Wang J, Li C, Deng L, Xue J, Lu Y. Primary tumor standardized uptake value measured on F18-Fluorodeoxyglucose positron emission tomography is of prediction value for survival and local control in non-small-cell lung cancer receiving radiotherapy: meta-analysis. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2014;9(6):834–42. pmid:24787963; PubMed Central PMCID: PMC4219540.
  9. 9. Nair VS, Krupitskaya Y, Gould MK. Positron emission tomography 18F-fluorodeoxyglucose uptake and prognosis in patients with surgically treated, stage I non-small cell lung cancer: a systematic review. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2009;4(12):1473–9. pmid:19887967.
  10. 10. Paesmans M, Berghmans T, Dusart M, Garcia C, Hossein-Foucher C, Lafitte JJ, et al. Primary tumor standardized uptake value measured on fluorodeoxyglucose positron emission tomography is of prognostic value for survival in non-small cell lung cancer: update of a systematic review and meta-analysis by the European Lung Cancer Working Party for the International Association for the Study of Lung Cancer Staging Project. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2010;5(5):612–9. pmid:20234323.
  11. 11. Im HJ, Pak K, Cheon GJ, Kang KW, Kim SJ, Kim IJ, et al. Prognostic value of volumetric parameters of (18)F-FDG PET in non-small-cell lung cancer: a meta-analysis. European journal of nuclear medicine and molecular imaging. 2015;42(2):241–51. pmid:25193652.
  12. 12. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 2009;6(7):e1000097. pmid:19621072; PubMed Central PMCID: PMC2707599.
  13. 13. Pak K, Cheon GJ, Nam HY, Kim SJ, Kang KW, Chung JK, et al. Prognostic Value of Metabolic Tumor Volume and Total Lesion Glycolysis in Head and Neck Cancer: A Systematic Review and Meta-Analysis. Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2014;55(6):884–90. pmid:24752671.
  14. 14. Berghmans T, Dusart M, Paesmans M, Hossein-Foucher C, Buvat I, Castaigne C, et al. Primary tumor standardized uptake value (SUVmax) measured on fluorodeoxyglucose positron emission tomography (FDG-PET) is of prognostic value for survival in non-small cell lung cancer (NSCLC): a systematic review and meta-analysis (MA) by the European Lung Cancer Working Party for the IASLC Lung Cancer Staging Project. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2008;3(1):6–12. pmid:18166834.
  15. 15. Pan L, Gu P, Huang G, Xue H, Wu S. Prognostic significance of SUV on PET/CT in patients with esophageal cancer: a systematic review and meta-analysis. European journal of gastroenterology & hepatology. 2009;21(9):1008–15.
  16. 16. Tierney JF, Stewart LA, Ghersi D, Burdett S, Sydes MR. Practical methods for incorporating summary time-to-event data into meta-analysis. Trials. 2007;8:16. pmid:17555582; PubMed Central PMCID: PMC1920534.
  17. 17. Dinnes J, Deeks J, Kirby J, Roderick P. A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. Health Technol Assess. 2005;9(12):1–113, iii. pmid:15774235.
  18. 18. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60. pmid:12958120; PubMed Central PMCID: PMC192859.
  19. 19. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34. pmid:9310563; PubMed Central PMCID: PMC2127453.
  20. 20. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088–101. pmid:7786990.
  21. 21. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in medicine. 2002;21(11):1539–58. pmid:12111919.
  22. 22. Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–63. pmid:10877304.
  23. 23. Cistaro A, Quartuccio N, Mojtahedi A, Fania P, Filosso PL, Campenni A, et al. Prediction of 2 years-survival in patients with stage I and II non-small cell lung cancer utilizing (18)F-FDG PET/CT SUV quantification. Radiology and oncology. 2013;47(3):219–23. pmid:24133385; PubMed Central PMCID: PMC3794876.
  24. 24. Melloni G, Gajate AM, Sestini S, Gallivanone F, Bandiera A, Landoni C, et al. New positron emission tomography derived parameters as predictive factors for recurrence in resected stage I non-small cell lung cancer. European journal of surgical oncology: the journal of the European Society of Surgical Oncology and the British Association of Surgical Oncology. 2013;39(11):1254–61. pmid:23948705.
  25. 25. Ohtaka K, Hida Y, Kaga K, Okamoto S, Shiga T, Tamaki N, et al. Outcome analysis of (18)F-fluorodeoxyglucose positron-emission tomography in patients with lung cancer after partial volume correction. Anticancer research. 2013;33(11):5193–8. pmid:24222169.
  26. 26. Hyun SH, Choi JY, Kim K, Kim J, Shim YM, Um SW, et al. Volume-based parameters of (18)F-fluorodeoxyglucose positron emission tomography/computed tomography improve outcome prediction in early-stage non-small cell lung cancer after surgical resection. Annals of surgery. 2013;257(2):364–70. Epub 2012/09/13. pmid:22968069.
  27. 27. Kim K, Kim SJ, Kim IJ, Kim YS, Pak K, Kim H. Prognostic value of volumetric parameters measured by F-18 FDG PET/CT in surgically resected non-small-cell lung cancer. Nuclear medicine communications. 2012;33(6):613–20. pmid:22407127.
  28. 28. Lin Y, Lin WY, Kao CH, Yen KY, Chen SW, Yeh JJ. Prognostic value of preoperative metabolic tumor volumes on PET-CT in predicting disease-free survival of patients with stage I non-small cell lung cancer. Anticancer research. 2012;32(11):5087–91. pmid:23155285.
  29. 29. Zhang H, Wroblewski K, Liao S, Kampalath R, Penney BC, Zhang Y, et al. Prognostic value of metabolic tumor burden from (18)F-FDG PET in surgical patients with non-small-cell lung cancer. Academic radiology. 2013;20(1):32–40. pmid:22999369.
  30. 30. Agarwal M, Brahmanday G, Bajaj SK, Ravikrishnan KP, Wong CY. Revisiting the prognostic value of preoperative (18)F-fluoro-2-deoxyglucose ((18)F-FDG) positron emission tomography (PET) in early-stage (I & II) non-small cell lung cancers (NSCLC). European journal of nuclear medicine and molecular imaging. 2010;37(4):691–8. pmid:19915840; PubMed Central PMCID: PMC2844956.
  31. 31. Dooms C, van Baardwijk A, Verbeken E, van Suylen RJ, Stroobants S, De Ruysscher D, et al. Association between 18F-fluoro-2-deoxy-D-glucose uptake values and tumor vitality: prognostic value of positron emission tomography in early-stage non-small cell lung cancer. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2009;4(7):822–8. pmid:19487964.
  32. 32. Goodgame B, Pillot GA, Yang Z, Shriki J, Meyers BF, Zoole J, et al. Prognostic value of preoperative positron emission tomography in resected stage I non-small cell lung cancer. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2008;3(2):130–4. pmid:18303432.
  33. 33. Hanin FX, Lonneux M, Cornet J, Noirhomme P, Coulon C, Distexhe J, et al. Prognostic value of FDG uptake in early stage non-small cell lung cancer. European journal of cardio-thoracic surgery: official journal of the European Association for Cardio-thoracic Surgery. 2008;33(5):819–23. pmid:18374589.
  34. 34. Kim YS, Kim SJ, Kim YK, Kim IJ, Kim YD, Lee MK. Prediction of survival and cancer recurrence using F-18 FDG PET/CT in patients with surgically resected early stage (Stage I and II) non-small cell lung cancer. Neoplasma. 2011;58(3):245–50. pmid:21391742.
  35. 35. Nair VS, Barnett PG, Ananth L, Gould MK, Veterans Affairs Solitary Nodule Accuracy Project Cooperative Studies G. PET scan 18F-fluorodeoxyglucose uptake and prognosis in patients with resected clinical stage IA non-small cell lung cancer. Chest. 2010;137(5):1150–6. pmid:20038738.
  36. 36. Um SW, Kim H, Koh WJ, Suh GY, Chung MP, Kwon OJ, et al. Prognostic value of 18F-FDG uptake on positron emission tomography in patients with pathologic stage I non-small cell lung cancer. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2009;4(11):1331–6. pmid:19701106.
  37. 37. Vesselle H, Freeman JD, Wiens L, Stern J, Nguyen HQ, Hawes SE, et al. Fluorodeoxyglucose uptake of primary non-small cell lung cancer at positron emission tomography: new contrary data on prognostic role. Clinical cancer research: an official journal of the American Association for Cancer Research. 2007;13(11):3255–63. pmid:17545531.
  38. 38. Zhang ZJ, Chen JH, Meng L, Du JJ, Zhang L, Liu Y, et al. 18F-FDG uptake as a biologic factor predicting outcome in patients with resected non-small-cell lung cancer. Chinese medical journal. 2007;120(2):125–31. pmid:17335654.
  39. 39. Bille A, Okiror L, Skanjeti A, Errico L, Arena V, Penna D, et al. The prognostic significance of maximum standardized uptake value of primary tumor in surgically treated non-small-cell lung cancer patients: analysis of 413 cases. Clinical lung cancer. 2013;14(2):149–56. pmid:22682667.
  40. 40. Dhital K, Saunders CA, Seed PT, O'Doherty MJ, Dussek J. [(18)F]Fluorodeoxyglucose positron emission tomography and its prognostic value in lung cancer. European journal of cardio-thoracic surgery: official journal of the European Association for Cardio-thoracic Surgery. 2000;18(4):425–8. pmid:11024379.
  41. 41. Higashi K, Ueda Y, Arisaka Y, Sakuma T, Nambu Y, Oguchi M, et al. 18F-FDG uptake as a biologic prognostic factor for recurrence in patients with surgically resected non-small cell lung cancer. Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2002;43(1):39–45. pmid:11801701.
  42. 42. Kim HR, Kim DJ, Lee WW, Jheon S, Sung SW. The significance of maximum standardized uptake values in patients with stage I pulmonary adenocarcinoma. European journal of cardio-thoracic surgery: official journal of the European Association for Cardio-thoracic Surgery. 2009;35(4):712–6; discussion 6–7. pmid:19211260.
  43. 43. Ohtsuka T, Nomori H, Watanabe K, Kaji M, Naruke T, Suemasu K, et al. Prognostic significance of [(18)F]fluorodeoxyglucose uptake on positron emission tomography in patients with pathologic stage I lung adenocarcinoma. Cancer. 2006;107(10):2468–73. pmid:17036361.
  44. 44. Stiles BM, Nasar A, Mirza F, Paul S, Lee PC, Port JL, et al. Ratio of positron emission tomography uptake to tumor size in surgically resected non-small cell lung cancer. The Annals of thoracic surgery. 2013;95(2):397–403; discussion 4. pmid:23000262.
  45. 45. Tomita M, Shimizu T, Ayabe T, Onitsuka T. Maximum SUV on positron emission tomography and serum CEA level as prognostic factors after curative resection for non-small cell lung cancer. Asia-Pacific journal of clinical oncology. 2012;8(3):244–7. pmid:22897792.
  46. 46. Tsutani Y, Miyata Y, Misumi K, Ikeda T, Mimura T, Hihara J, et al. Difference in prognostic significance of maximum standardized uptake value on [18F]-fluoro-2-deoxyglucose positron emission tomography between adenocarcinoma and squamous cell carcinoma of the lung. Japanese journal of clinical oncology. 2011;41(7):890–6. pmid:21613306.
  47. 47. van Baardwijk A, Dooms C, van Suylen RJ, Verbeken E, Hochstenbag M, Dehing-Oberije C, et al. The maximum uptake of (18)F-deoxyglucose on positron emission tomography scan correlates with survival, hypoxia inducible factor-1alpha and GLUT-1 in non-small cell lung cancer. Eur J Cancer. 2007;43(9):1392–8. Epub 2007/05/22. pmid:17512190.
  48. 48. Downey RJ, Akhurst T, Gonen M, Park B, Rusch V. Fluorine-18 fluorodeoxyglucose positron emission tomographic maximal standardized uptake value predicts survival independent of clinical but not pathologic TNM staging of resected non-small cell lung cancer. The Journal of thoracic and cardiovascular surgery. 2007;133(6):1419–27. pmid:17532932.
  49. 49. Koo HK, Jin SM, Lee CH, Lim HJ, Yim JJ, Kim YT, et al. Factors associated with recurrence in patients with curatively resected stage I-II lung cancer. Lung Cancer. 2011;73(2):222–9. pmid:21168237.
  50. 50. Pelosi E, Bille A, Skanjeti A, Errico L, Arena V, Ardissone F, et al. Prognostic role of the PET parameter maximum standardized uptake value in non small cell lung cancer: analysis in tumour of diameter >/ = and <25 mm. Q J Nucl Med Mol Imaging. 2011;55(1):72–80. pmid:20539268.
  51. 51. Shiono S, Abiko M, Sato T. Positron emission tomography/computed tomography and lymphovascular invasion predict recurrence in stage I lung cancers. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer. 2011;6(1):43–7. pmid:21079522.
  52. 52. Kim DH, Son SH, Kim CY, Hong CM, Oh JR, Song BI, et al. Prediction for recurrence using F-18 FDG PET/CT in pathologic N0 lung adenocarcinoma after curative surgery. Annals of surgical oncology. 2014;21(2):589–96. pmid:24046125.
  53. 53. Domachevsky L, Groshar D, Galili R, Saute M, Bernstine H. Survival Prognostic Value of Morphological and Metabolic variables in Patients with Stage I and II Non-Small Cell Lung Cancer. European radiology. 2015. pmid:25929940.
  54. 54. Ko KH, Hsu HH, Huang TW, Gao HW, Cheng CY, Hsu YC, et al. Predictive value of 18F-FDG PET and CT morphologic features for recurrence in pathological stage IA non-small cell lung cancer. Medicine. 2015;94(3):e434. pmid:25621697.
  55. 55. Motono N, Ueno M, Tanaka M, Machida Y, Usuda K, Sakuma T, et al. Differences in the prognostic significance of the SUVmax between patients with resected pulmonary Adenocarcinoma and squamous cell carcinoma. Asian Pacific journal of cancer prevention: APJCP. 2014;15(23):10171–4. pmid:25556443.
  56. 56. Park SY, Cho A, Yu WS, Lee CY, Lee JG, Kim DJ, et al. Prognostic value of total lesion glycolysis by 18F-FDG PET/CT in surgically resected stage IA non-small cell lung cancer. Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2015;56(1):45–9. pmid:25525185.
  57. 57. Shimizu K, Maeda A, Yukawa T, Nojima Y, Saisho S, Okita R, et al. Difference in prognostic values of maximal standardized uptake value on fluorodeoxyglucose-positron emission tomography and cyclooxygenase-2 expression between lung adenocarcinoma and squamous cell carcinoma. World journal of surgical oncology. 2014;12:343. pmid:25392182; PubMed Central PMCID: PMC4254182.
  58. 58. Yoo Ie R, Chung SK, Park HL, Choi WH, Kim YK, Lee KY, et al. Prognostic value of SUVmax and metabolic tumor volume on 18F-FDG PET/CT in early stage non-small cell lung cancer patients without LN metastasis. Bio-medical materials and engineering. 2014;24(6):3091–103. pmid:25227018.
  59. 59. Vaidya M, Creach KM, Frye J, Dehdashti F, Bradley JD, El Naqa I. Combined PET/CT image characteristics for radiotherapy tumor response in lung cancer. Radiotherapy and oncology: journal of the European Society for Therapeutic Radiology and Oncology. 2012;102(2):239–45. pmid:22098794.
  60. 60. Tixier F, Hatt M, Valla C, Fleury V, Lamour C, Ezzouhri S, et al. Visual Versus Quantitative Assessment of Intratumor 18F-FDG PET Uptake Heterogeneity: Prognostic Value in Non-Small Cell Lung Cancer. Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2014;55(8):1235–41. pmid:24904113.
  61. 61. Kang SR, Song HC, Byun BH, Oh JR, Kim HS, Hong SP, et al. Intratumoral Metabolic Heterogeneity for Prediction of Disease Progression After Concurrent Chemoradiotherapy in Patients with Inoperable Stage III Non-Small-Cell Lung Cancer. Nuclear medicine and molecular imaging. 2014;48(1):16–25. pmid:24900134; PubMed Central PMCID: PMC4035157.
  62. 62. Cook GJ, Yip C, Siddique M, Goh V, Chicklore S, Roy A, et al. Are pretreatment 18F-FDG PET tumor textural features in non-small cell lung cancer associated with response and survival after chemoradiotherapy? Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2013;54(1):19–26. pmid:23204495.
  63. 63. Boellaard R, Krak NC, Hoekstra OS, Lammertsma AA. Effects of noise, image resolution, and ROI definition on the accuracy of standard uptake values: a simulation study. Journal of nuclear medicine: official publication, Society of Nuclear Medicine. 2004;45(9):1519–27. pmid:15347719.
  64. 64. Stahl A, Ott K, Schwaiger M, Weber WA. Comparison of different SUV-based methods for monitoring cytotoxic therapy with FDG PET. European journal of nuclear medicine and molecular imaging. 2004;31(11):1471–8. pmid:15257418.
  65. 65. Lee P, Weerasuriya DK, Lavori PW, Quon A, Hara W, Maxim PG, et al. Metabolic tumor burden predicts for disease progression and death in lung cancer. International journal of radiation oncology, biology, physics. 2007;69(2):328–33. pmid:17869659.
  66. 66. Machtay M, Duan F, Siegel BA, Snyder BS, Gorelick JJ, Reddin JS, et al. Prediction of survival by [18F]fluorodeoxyglucose positron emission tomography in patients with locally advanced non-small-cell lung cancer undergoing definitive chemoradiation therapy: results of the ACRIN 6668/RTOG 0235 trial. Journal of clinical oncology: official journal of the American Society of Clinical Oncology. 2013;31(30):3823–30. pmid:24043740; PubMed Central PMCID: PMC3795891.