Skip to main content
Erschienen in: European Radiology 10/2019

01.04.2019 | Chest

Post-imaging pulmonary nodule mathematical prediction models: are they clinically relevant?

verfasst von: Johanna Uthoff, Nicholas Koehn, Jared Larson, Samantha K. N. Dilger, Emily Hammond, Ann Schwartz, Brian Mullan, Rolando Sanchez, Richard M. Hoffman, Jessica C. Sieren, for the COPDGene Investigators

Erschienen in: European Radiology | Ausgabe 10/2019

Einloggen, um Zugang zu erhalten

Abstract

Objectives

Post-imaging mathematical prediction models (MPMs) provide guidance for the management of solid pulmonary nodules by providing a lung cancer risk score from demographic and radiologists-indicated imaging characteristics. We hypothesized calibrating the MPM risk score threshold to a local study cohort would result in improved performance over the original recommended MPM thresholds. We compared the pre- and post-calibration performance of four MPM models and determined if improvement in MPM prediction occurs as nodules are imaged longitudinally.

Materials and methods

A common cohort of 317 individuals with computed tomography-detected, solid nodules (80 malignant, 237 benign) were used to evaluate the MPM performance. We created a web-based application for this study that allows others to easily calibrate thresholds and analyze the performance of MPMs on their local cohort. Thirty patients with repeated imaging were tested for improved performance longitudinally.

Results

Using calibrated thresholds, Mayo Clinic and Brock University (BU) MPMs performed the best (AUC = 0.63, 0.61) compared to the Veteran’s Affairs (0.51) and Peking University (0.55). Only BU had consensus with the original MPM threshold; the other calibrated thresholds improved MPM accuracy. No significant improvements in accuracy were found longitudinally between time points.

Conclusions

Calibration to a common cohort can select the best-performing MPM for your institution. Without calibration, BU has the most stable performance in solid nodules ≥ 8 mm but has only moderate potential to refine subjects into appropriate workup. Application of MPM is recommended only at initial evaluation as no increase in accuracy was achieved over time.

Key Points

• Post-imaging lung cancer risk mathematical predication models (MPMs) perform poorly on local populations without calibration.
• An application is provided to facilitate calibration to new study cohorts: the Mayo Clinic model, the U.S. Department of Veteran’s Affairs model, the Brock University model, and the Peking University model.
• No significant improvement in risk prediction occurred in nodules with repeated imaging sessions, indicating the potential value of risk prediction application is limited to the initial evaluation.
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
2.
Zurück zum Zitat MacMahon H, Naidich DP, Goo JM et al (2017) Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology 284:228–243CrossRefPubMed MacMahon H, Naidich DP, Goo JM et al (2017) Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology 284:228–243CrossRefPubMed
4.
Zurück zum Zitat Gould MK, Donington J, Lynch WR et al (2013) Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:e93S–e120SCrossRefPubMedPubMedCentral Gould MK, Donington J, Lynch WR et al (2013) Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:e93S–e120SCrossRefPubMedPubMedCentral
5.
Zurück zum Zitat Mehta HJ, Mohammed TL, Jantz MA (2017) The American College of Radiology Lung Imaging Reporting and Data System: potential drawbacks and need for revision. Chest 151:539–543CrossRefPubMed Mehta HJ, Mohammed TL, Jantz MA (2017) The American College of Radiology Lung Imaging Reporting and Data System: potential drawbacks and need for revision. Chest 151:539–543CrossRefPubMed
6.
Zurück zum Zitat Pinsky PF, Gierada DS, Black W et al (2015) Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment. Ann Intern Med 162:485–491CrossRefPubMedPubMedCentral Pinsky PF, Gierada DS, Black W et al (2015) Performance of Lung-RADS in the National Lung Screening Trial: a retrospective assessment. Ann Intern Med 162:485–491CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat van Riel SJ, Ciompi F, Jacobs C et al (2017) Malignancy risk estimation of screen-detected nodules at baseline CT: comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 27:4019–4029CrossRefPubMedPubMedCentral van Riel SJ, Ciompi F, Jacobs C et al (2017) Malignancy risk estimation of screen-detected nodules at baseline CT: comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 27:4019–4029CrossRefPubMedPubMedCentral
8.
Zurück zum Zitat Gray EP, Teare MD, Stevens J, Archer R (2016) Risk prediction models for lung cancer: a systematic review. Clin Lung Cancer 17:95–106CrossRefPubMed Gray EP, Teare MD, Stevens J, Archer R (2016) Risk prediction models for lung cancer: a systematic review. Clin Lung Cancer 17:95–106CrossRefPubMed
9.
Zurück zum Zitat Swensen SJ, Silverstein MD, Ilstrup DM, Schleck CD, Edell ES (1997) The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules. Arch Intern Med 157:849–855CrossRefPubMed Swensen SJ, Silverstein MD, Ilstrup DM, Schleck CD, Edell ES (1997) The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules. Arch Intern Med 157:849–855CrossRefPubMed
10.
Zurück zum Zitat Gould MK, Ananth L, Barnett PG, Veterans Affairs SNAP Cooperative Study Group (2007) A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules. Chest 131:383–388 Gould MK, Ananth L, Barnett PG, Veterans Affairs SNAP Cooperative Study Group (2007) A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules. Chest 131:383–388
11.
Zurück zum Zitat McWilliams A, Tammemagi MC, Mayo JR et al (2013) Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 369:910–919CrossRefPubMedPubMedCentral McWilliams A, Tammemagi MC, Mayo JR et al (2013) Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 369:910–919CrossRefPubMedPubMedCentral
12.
Zurück zum Zitat Li Y, Wang J (2012) A mathematical model for predicting malignancy of solitary pulmonary nodules. World J Surg 36:830–835CrossRefPubMed Li Y, Wang J (2012) A mathematical model for predicting malignancy of solitary pulmonary nodules. World J Surg 36:830–835CrossRefPubMed
13.
Zurück zum Zitat Herder GJ, van Tinteren H, Golding RP et al (2005) Clinical prediction model to characterize pulmonary nodules: validation and added value of 18F-fluorodeoxyglucose positron emission tomography. Chest 128:2490–2496CrossRefPubMed Herder GJ, van Tinteren H, Golding RP et al (2005) Clinical prediction model to characterize pulmonary nodules: validation and added value of 18F-fluorodeoxyglucose positron emission tomography. Chest 128:2490–2496CrossRefPubMed
14.
Zurück zum Zitat Baldwin DR, Callister ME (2015) The British Thoracic Society guidelines on the investigation and management of pulmonary nodules. Thorax 70:794–798CrossRefPubMed Baldwin DR, Callister ME (2015) The British Thoracic Society guidelines on the investigation and management of pulmonary nodules. Thorax 70:794–798CrossRefPubMed
15.
Zurück zum Zitat Hammer MM, Nachiappan AC, Barbosa EJM Jr (2018) Limited utility of pulmonary nodule risk calculators for managing large nodules. Curr Probl Diagn Radiol 47:23–27 Hammer MM, Nachiappan AC, Barbosa EJM Jr (2018) Limited utility of pulmonary nodule risk calculators for managing large nodules. Curr Probl Diagn Radiol 47:23–27
16.
Zurück zum Zitat Al-Ameri A, Malhotra P, Thygesen H et al (2015) Risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 89:27–30CrossRefPubMed Al-Ameri A, Malhotra P, Thygesen H et al (2015) Risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 89:27–30CrossRefPubMed
17.
Zurück zum Zitat Mehta HJ, Ravenel JG, Shaftman SR et al (2014) The utility of nodule volume in the context of malignancy prediction for small pulmonary nodules. Chest 145:464–472CrossRefPubMed Mehta HJ, Ravenel JG, Shaftman SR et al (2014) The utility of nodule volume in the context of malignancy prediction for small pulmonary nodules. Chest 145:464–472CrossRefPubMed
18.
Zurück zum Zitat Perandini S, Soardi GA, Motton M, Montemezzi S (2015) Critique of Al-Ameri et al. (2015) - risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 90:118–119CrossRefPubMed Perandini S, Soardi GA, Motton M, Montemezzi S (2015) Critique of Al-Ameri et al. (2015) - risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 90:118–119CrossRefPubMed
19.
Zurück zum Zitat Regan EA, Hokanson JE, Murphy JR et al (2010) Genetic epidemiology of COPD (COPDGene) study design. COPD 7:32–43CrossRefPubMed Regan EA, Hokanson JE, Murphy JR et al (2010) Genetic epidemiology of COPD (COPDGene) study design. COPD 7:32–43CrossRefPubMed
20.
Zurück zum Zitat Schwartz AG, Lusk CM, Wenzlaff AS et al (2016) Risk of lung cancer associated with COPD phenotype based on quantitative image analysis. Cancer Epidemiol Biomarkers Prevv 25:1341–1347 Schwartz AG, Lusk CM, Wenzlaff AS et al (2016) Risk of lung cancer associated with COPD phenotype based on quantitative image analysis. Cancer Epidemiol Biomarkers Prevv 25:1341–1347
21.
Zurück zum Zitat Swensen SJ, Silverstein MD, Edell ES et al (1999) Solitary pulmonary nodules: clinical prediction model versus physicians. Mayo Clin Proc 74:319–329CrossRefPubMed Swensen SJ, Silverstein MD, Edell ES et al (1999) Solitary pulmonary nodules: clinical prediction model versus physicians. Mayo Clin Proc 74:319–329CrossRefPubMed
22.
Zurück zum Zitat Cummings SR, Lillington GA, Richard RJ (1986) Managing solitary pulmonary nodules. The choice of strategy is a “close call”. Am Rev Respir Dis 134:453–460PubMed Cummings SR, Lillington GA, Richard RJ (1986) Managing solitary pulmonary nodules. The choice of strategy is a “close call”. Am Rev Respir Dis 134:453–460PubMed
23.
Zurück zum Zitat Chung K, Mets OM, Gerke PK et al (2018) Brock malignancy risk calculator for pulmonary nodules: validation outside a lung cancer screening population. Thorax 73:857–863CrossRefPubMed Chung K, Mets OM, Gerke PK et al (2018) Brock malignancy risk calculator for pulmonary nodules: validation outside a lung cancer screening population. Thorax 73:857–863CrossRefPubMed
25.
Zurück zum Zitat Jesse D, Goadrich M (2006) The relationship between precision-recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, 25–29 June 2006, pp 233–240 Jesse D, Goadrich M (2006) The relationship between precision-recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, 25–29 June 2006, pp 233–240
26.
Zurück zum Zitat McNemar Q (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12:153–157CrossRefPubMed McNemar Q (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12:153–157CrossRefPubMed
27.
Zurück zum Zitat Callister ME, Baldwin DR, Akram AR et al (2015) British Thoracic Society guidelines for the investigation and management of pulmonary nodules. Thorax 70(Suppl 2):ii1–ii54CrossRefPubMed Callister ME, Baldwin DR, Akram AR et al (2015) British Thoracic Society guidelines for the investigation and management of pulmonary nodules. Thorax 70(Suppl 2):ii1–ii54CrossRefPubMed
28.
Zurück zum Zitat McNitt-Gray MF, Kim GH, Zhao B et al (2015) Determining the variability of lesion size measurements from CT patient data sets acquired under “no change” conditions. Transl Oncol 8:55–64CrossRefPubMedPubMedCentral McNitt-Gray MF, Kim GH, Zhao B et al (2015) Determining the variability of lesion size measurements from CT patient data sets acquired under “no change” conditions. Transl Oncol 8:55–64CrossRefPubMedPubMedCentral
29.
Zurück zum Zitat Lin H, Huang C, Wang W, Luo J, Yang X, Liu Y (2017) Measuring interobserver disagreement in rating diagnostic characteristics of pulmonary nodule using the lung imaging database consortium and image database resource initiative. Acad Radiol 24:401–410CrossRefPubMed Lin H, Huang C, Wang W, Luo J, Yang X, Liu Y (2017) Measuring interobserver disagreement in rating diagnostic characteristics of pulmonary nodule using the lung imaging database consortium and image database resource initiative. Acad Radiol 24:401–410CrossRefPubMed
30.
Zurück zum Zitat Maiga AW, Deppen SA, Massion PP et al (2018) Communication about the probability of cancer in indeterminate pulmonary nodules. JAMA Surg 153:353–357CrossRefPubMed Maiga AW, Deppen SA, Massion PP et al (2018) Communication about the probability of cancer in indeterminate pulmonary nodules. JAMA Surg 153:353–357CrossRefPubMed
31.
Zurück zum Zitat Dilger SK, Uthoff J, Judisch A et al (2015) Improved pulmonary nodule classification utilizing quantitative lung parenchyma features. J Med Imaging (Bellingham) 2:041004CrossRef Dilger SK, Uthoff J, Judisch A et al (2015) Improved pulmonary nodule classification utilizing quantitative lung parenchyma features. J Med Imaging (Bellingham) 2:041004CrossRef
32.
Zurück zum Zitat Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N (2016) A combination of shape and texture features for classification of pulmonary nodules in lung CT images. J Digit Imaging 29:466–475CrossRefPubMedPubMedCentral Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N (2016) A combination of shape and texture features for classification of pulmonary nodules in lung CT images. J Digit Imaging 29:466–475CrossRefPubMedPubMedCentral
33.
Zurück zum Zitat Ferreira JR Jr, Oliveira MC, de Azevedo-Marques PM (2017) Characterization of pulmonary nodules based on features of margin sharpness and texture. J Digit Imaging 31:451–463 Ferreira JR Jr, Oliveira MC, de Azevedo-Marques PM (2017) Characterization of pulmonary nodules based on features of margin sharpness and texture. J Digit Imaging 31:451–463
34.
Zurück zum Zitat Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J Comput Assist Radiol Surg 12:1799–1808CrossRefPubMed Nibali A, He Z, Wollersheim D (2017) Pulmonary nodule classification with deep residual networks. Int J Comput Assist Radiol Surg 12:1799–1808CrossRefPubMed
35.
Zurück zum Zitat Sun W, Zheng B, Qian W (2017) Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput Biol Med 89:530–539CrossRefPubMed Sun W, Zheng B, Qian W (2017) Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput Biol Med 89:530–539CrossRefPubMed
36.
Zurück zum Zitat Way TW, Sahiner B, Chan HP et al (2009) Computer-aided diagnosis of pulmonary nodules on CT scans: improvement of classification performance with nodule surface features. Med Phys 36:3086–3098CrossRefPubMedPubMedCentral Way TW, Sahiner B, Chan HP et al (2009) Computer-aided diagnosis of pulmonary nodules on CT scans: improvement of classification performance with nodule surface features. Med Phys 36:3086–3098CrossRefPubMedPubMedCentral
37.
Zurück zum Zitat Zhu Y, Tan Y, Hua Y, Wang M, Zhang G, Zhang J (2010) Feature selection and performance evaluation of support vector machine (SVM)-based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. J Digit Imaging 23:51–65CrossRefPubMed Zhu Y, Tan Y, Hua Y, Wang M, Zhang G, Zhang J (2010) Feature selection and performance evaluation of support vector machine (SVM)-based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. J Digit Imaging 23:51–65CrossRefPubMed
Metadaten
Titel
Post-imaging pulmonary nodule mathematical prediction models: are they clinically relevant?
verfasst von
Johanna Uthoff
Nicholas Koehn
Jared Larson
Samantha K. N. Dilger
Emily Hammond
Ann Schwartz
Brian Mullan
Rolando Sanchez
Richard M. Hoffman
Jessica C. Sieren
for the COPDGene Investigators
Publikationsdatum
01.04.2019
Verlag
Springer Berlin Heidelberg
Erschienen in
European Radiology / Ausgabe 10/2019
Print ISSN: 0938-7994
Elektronische ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-019-06168-x

Weitere Artikel der Ausgabe 10/2019

European Radiology 10/2019 Zur Ausgabe

Screening-Mammografie offenbart erhöhtes Herz-Kreislauf-Risiko

26.04.2024 Mammografie Nachrichten

Routinemäßige Mammografien helfen, Brustkrebs frühzeitig zu erkennen. Anhand der Röntgenuntersuchung lassen sich aber auch kardiovaskuläre Risikopatientinnen identifizieren. Als zuverlässiger Anhaltspunkt gilt die Verkalkung der Brustarterien.

S3-Leitlinie zu Pankreaskrebs aktualisiert

23.04.2024 Pankreaskarzinom Nachrichten

Die Empfehlungen zur Therapie des Pankreaskarzinoms wurden um zwei Off-Label-Anwendungen erweitert. Und auch im Bereich der Früherkennung gibt es Aktualisierungen.

Fünf Dinge, die im Kindernotfall besser zu unterlassen sind

18.04.2024 Pädiatrische Notfallmedizin Nachrichten

Im Choosing-Wisely-Programm, das für die deutsche Initiative „Klug entscheiden“ Pate gestanden hat, sind erstmals Empfehlungen zum Umgang mit Notfällen von Kindern erschienen. Fünf Dinge gilt es demnach zu vermeiden.

„Nur wer sich gut aufgehoben fühlt, kann auch für Patientensicherheit sorgen“

13.04.2024 Klinik aktuell Kongressbericht

Die Teilnehmer eines Forums beim DGIM-Kongress waren sich einig: Fehler in der Medizin sind häufig in ungeeigneten Prozessen und mangelnder Kommunikation begründet. Gespräche mit Patienten und im Team können helfen.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.