Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 1/2023

26.05.2022 | Original Article

Performance improvement in multi-label thoracic abnormality classification of chest X-rays with noisy labels

verfasst von: Mingyan Yang, Hisashi Tanaka, Takayuki Ishida

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 1/2023

Einloggen, um Zugang zu erhalten

Abstract

Purpose

This study aimed at developing a deep learning-based method for multi-label thoracic abnormality classification on frontal view chest X-ray (CXR). To improve the performance of classification, issues of class imbalance, noisy labels and ensemble of networks are addressed in the paper.

Methods

The experiments were performed on a public dataset called Chest X-ray 14 (CXR14), which includes 112,120 frontal view CXRs from 30,805 patients. We came up with an ensemble learning framework to improve the classification and a noisy label detection method to detect the CXRs with noisy labels. The detected CXRs were reviewed by two board-certificated radiologists in a consensus fashion to evaluate detected noisy labels. The classification was assessed on CXR14 with area under the receiver operating characteristic curve (AUC).

Results

Report from the radiologists indicated that detected noisy labels had high possibility to be true positives. A notable improvement from baseline in performance of classification was observed with the ensemble learning framework. After removing the CXRs with detected noisy labels, 8 out of 14 abnormalities improved significantly on CXR14. The suggested framework achieved AUC score of 0.827 on CXR14.

Conclusion

The methods of this study boost the classification on CXR with awareness of the label noise. Expanded experimental results show that all of them were able to improve multi-label thoracic abnormality classification performance, respectively. A new state-of-the-art is achieved in this study.
Literatur
2.
Zurück zum Zitat Royal College of Radiologists (2020) 2019 Clinical radiology UK workforce census. R Coll Radiol Royal College of Radiologists (2020) 2019 Clinical radiology UK workforce census. R Coll Radiol
4.
Zurück zum Zitat Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. J Big Data 6:1–54CrossRef Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. J Big Data 6:1–54CrossRef
5.
Zurück zum Zitat Oakden-Rayner L (2020) Exploring large-scale public medical image datasets. Acad Radiol 27:106–112CrossRef Oakden-Rayner L (2020) Exploring large-scale public medical image datasets. Acad Radiol 27:106–112CrossRef
6.
Zurück zum Zitat Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2019) ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases. In: Lu L, Wang X, Carneiro G, Yang L (eds) Advances in Computer Vision and Pattern Recognition. Springer International Publishing, Cham, pp 369–392 Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2019) ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases. In: Lu L, Wang X, Carneiro G, Yang L (eds) Advances in Computer Vision and Pattern Recognition. Springer International Publishing, Cham, pp 369–392
7.
Zurück zum Zitat Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J, Mong DA, Halabi SS, Sandberg JK, Jones R, Larson DB, Langlotz CP, Patel BN, Lungren MP, Ng AY (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence, pp 590–597 Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J, Mong DA, Halabi SS, Sandberg JK, Jones R, Larson DB, Langlotz CP, Patel BN, Lungren MP, Ng AY (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence, pp 590–597
8.
Zurück zum Zitat Johnson AEW, Pollard TJ, Greenbaum NR, Lungren MP, Deng C, Peng Y, Lu Z, Mark RG, Berkowitz SJ, Horng S (2019) MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv Prepr arXiv190107042 Johnson AEW, Pollard TJ, Greenbaum NR, Lungren MP, Deng C, Peng Y, Lu Z, Mark RG, Berkowitz SJ, Horng S (2019) MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv Prepr arXiv190107042
9.
Zurück zum Zitat Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K, Lungren MP, Ng AY (2017) CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv Prepr arXiv171105225 Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K, Lungren MP, Ng AY (2017) CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv Prepr arXiv171105225
10.
Zurück zum Zitat Kundu R, Das R, Geem ZW, Han G-T, Sarkar R (2021) Pneumonia detection in chest X-ray images using an ensemble of deep learning models. PLoS ONE 16:e0256630CrossRef Kundu R, Das R, Geem ZW, Han G-T, Sarkar R (2021) Pneumonia detection in chest X-ray images using an ensemble of deep learning models. PLoS ONE 16:e0256630CrossRef
11.
Zurück zum Zitat Yao L, Prosky J, Poblenz E, Covington B, Lyman K (2018) Weakly supervised medical diagnosis and localization from multiple resolutions. arXiv Prepr arXiv180307703 Yao L, Prosky J, Poblenz E, Covington B, Lyman K (2018) Weakly supervised medical diagnosis and localization from multiple resolutions. arXiv Prepr arXiv180307703
12.
Zurück zum Zitat Gündel S, Grbic S, Georgescu B, Liu S, Maier A, Comaniciu D (2019) Learning to recognize abnormalities in chest X-rays with location-aware dense networks. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, pp 757–765 Gündel S, Grbic S, Georgescu B, Liu S, Maier A, Comaniciu D (2019) Learning to recognize abnormalities in chest X-rays with location-aware dense networks. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, pp 757–765
14.
Zurück zum Zitat Ma Y, Zhou Q, Chen X, Lu H, Zhao Y (2019) Multi-attention network for thoracic disease classification and localization. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1378–1382. IEEE Ma Y, Zhou Q, Chen X, Lu H, Zhao Y (2019) Multi-attention network for thoracic disease classification and localization. In: ICASSP 2019–2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1378–1382. IEEE
16.
Zurück zum Zitat Dubey AK, Young MT, Stanley C, Lunga D, Hinkle J (2021) Computer-aided abnormality detection in chest radiographs in a clinical setting via domain-adaptation Dubey AK, Young MT, Stanley C, Lunga D, Hinkle J (2021) Computer-aided abnormality detection in chest radiographs in a clinical setting via domain-adaptation
17.
Zurück zum Zitat Huang J, Qu L, Jia R, Zhao B (2019) O2U-Net: a simple noisy label detection approach for deep neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 3325–3333 Huang J, Qu L, Jia R, Zhao B (2019) O2U-Net: a simple noisy label detection approach for deep neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 3325–3333
18.
Zurück zum Zitat Kim CD, Jeong J, Moon S, Kim G (2021) Continual learning on noisy data streams via self-purified replay. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 537–547 Kim CD, Jeong J, Moon S, Kim G (2021) Continual learning on noisy data streams via self-purified replay. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 537–547
19.
Zurück zum Zitat Gündel S, Setio AAA, Ghesu FC, Grbic S, Georgescu B, Maier A, Comaniciu D (2021) Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment. Med Image Anal 72:102087CrossRef Gündel S, Setio AAA, Ghesu FC, Grbic S, Georgescu B, Maier A, Comaniciu D (2021) Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment. Med Image Anal 72:102087CrossRef
21.
Zurück zum Zitat Tan M, Le Q V. (2019) EfficientNet: rethinking model scaling for convolutional neural networks. In: 36th international conference on machine learning ICML, pp 6105-6114 Tan M, Le Q V. (2019) EfficientNet: rethinking model scaling for convolutional neural networks. In: 36th international conference on machine learning ICML, pp 6105-6114
22.
Zurück zum Zitat Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR, pp 2261–2269 Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR, pp 2261–2269
23.
Zurück zum Zitat Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500 Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500
24.
Zurück zum Zitat Cui Y, Jia M, Lin T-Y, Song Y, Belongie S (2019) Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9268–9277 Cui Y, Jia M, Lin T-Y, Song Y, Belongie S (2019) Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9268–9277
25.
Zurück zum Zitat Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT 2010—19th international conference on computational statistics, keynote, invited and contributed papers. Springer, pp 177–186 Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT 2010—19th international conference on computational statistics, keynote, invited and contributed papers. Springer, pp 177–186
26.
Zurück zum Zitat DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRef DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRef
27.
Zurück zum Zitat Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626 Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626
29.
Zurück zum Zitat Ma C, Wang H, Hoi SCH (2019) Multi-label thoracic disease image classification with cross-attention networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 730–738 Ma C, Wang H, Hoi SCH (2019) Multi-label thoracic disease image classification with cross-attention networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 730–738
Metadaten
Titel
Performance improvement in multi-label thoracic abnormality classification of chest X-rays with noisy labels
verfasst von
Mingyan Yang
Hisashi Tanaka
Takayuki Ishida
Publikationsdatum
26.05.2022
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 1/2023
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-022-02684-2

Weitere Artikel der Ausgabe 1/2023

International Journal of Computer Assisted Radiology and Surgery 1/2023 Zur Ausgabe

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Ein Drittel der jungen Ärztinnen und Ärzte erwägt abzuwandern

07.05.2024 Klinik aktuell Nachrichten

Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.