Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 3/2020

14.01.2020 | Original Article

Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN

verfasst von: Bin Liu, Jianxu Luo, Huan Huang

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 3/2020

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Knee osteoarthritis (OA) is a common disease that impairs knee function and causes pain. Radiologists usually review knee X-ray images and grade the severity of the impairments according to the Kellgren–Lawrence grading scheme. However, this approach becomes inefficient in hospitals with high throughput as it is time-consuming, tedious and also subjective. This paper introduces a model for automatic diagnosis of knee OA based on an end-to-end deep learning method.

Method

In order to process the input images with location and classification simultaneously, we use Faster R-CNN as baseline, which consists of region proposal network (RPN) and Fast R-CNN. The RPN is trained to generate region proposals, which contain knee joint and then be used by Fast R-CNN for classification. Due to the localized classification via CNNs, the useless information in X-ray images can be filtered and we can extract clinically relevant features. For the further improvement in the model’s performance, we use a novel loss function whose weighting scheme allows us to address the class imbalance. Besides, larger anchors are used to overcome the problem that anchors don’t match the object when increasing the input size of X-ray images.

Result

The performance of the proposed model is thoroughly assessed using various measures. The results show that our adjusted model outperforms the Faster R-CNN, achieving a mean average precision nearly 0.82 with a sensitivity above 78% and a specificity above 94%. It takes 0.33 s to test each image, which achieves a better trade-off between accuracy and speed.

Conclusion

The proposed end-to-end fully automatic model which is computationally efficient has the potential to achieve the real automatic diagnosis of knee OA and be used as computer-aided diagnosis tools in clinical applications.
Literatur
1.
Zurück zum Zitat Zhang Y, Jordan JM (2010) Epidemiology of osteoarthritis. Clin Geriatr Med 26(3):355–369 CrossRef Zhang Y, Jordan JM (2010) Epidemiology of osteoarthritis. Clin Geriatr Med 26(3):355–369 CrossRef
2.
Zurück zum Zitat Cross M, Smith E, Hoy D, Nolte S, Ackerman I, Fransen M, Bridgett L, Williams S, Guillemin F, Hill CL (2014) The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Ann Rheum Dis 73(7):1323–1330CrossRef Cross M, Smith E, Hoy D, Nolte S, Ackerman I, Fransen M, Bridgett L, Williams S, Guillemin F, Hill CL (2014) The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Ann Rheum Dis 73(7):1323–1330CrossRef
3.
Zurück zum Zitat Karsdal MA, Michaelis M, Ladel C, Siebuhr AS, Bihlet A, Andersen JR, Guehring H, Christiansen C, Bayjensen AC, Kraus VB (2016) Disease-modifying treatments for osteoarthritis (DMOADs) of the knee and hip: lessons learned from failures and opportunities for the future. Osteoarthr Cartil 24(12):2013–2021CrossRef Karsdal MA, Michaelis M, Ladel C, Siebuhr AS, Bihlet A, Andersen JR, Guehring H, Christiansen C, Bayjensen AC, Kraus VB (2016) Disease-modifying treatments for osteoarthritis (DMOADs) of the knee and hip: lessons learned from failures and opportunities for the future. Osteoarthr Cartil 24(12):2013–2021CrossRef
4.
Zurück zum Zitat Braun HJ, Gold GE (2012) Diagnosis of osteoarthritis: imaging. Bone 51(2):278–288CrossRef Braun HJ, Gold GE (2012) Diagnosis of osteoarthritis: imaging. Bone 51(2):278–288CrossRef
5.
Zurück zum Zitat Yoo TK, Kim DW, Choi SB, Park JS (2016) Simple scoring system and artificial neural network for knee osteoarthritis risk prediction: a cross-sectional study. PloS ONE 11(2):e0148724CrossRef Yoo TK, Kim DW, Choi SB, Park JS (2016) Simple scoring system and artificial neural network for knee osteoarthritis risk prediction: a cross-sectional study. PloS ONE 11(2):e0148724CrossRef
6.
Zurück zum Zitat Orlov N, Shamir L, Macura T, Johnston J, Eckley DM, Goldberg IG (2008) WND-CHARM: multi-purpose image classification using compound image transforms. Pattern Recognit Lett 29(11):1684–1693CrossRef Orlov N, Shamir L, Macura T, Johnston J, Eckley DM, Goldberg IG (2008) WND-CHARM: multi-purpose image classification using compound image transforms. Pattern Recognit Lett 29(11):1684–1693CrossRef
7.
Zurück zum Zitat Oka H, Muraki S, Akune T, Mabuchi A, Suzuki T, Yoshida H, Yamamoto S, Nakamura K, Yoshimura N, Kawaguchi H (2008) Fully automatic quantification of knee osteoarthritis severity on plain radiographs. Osteoarthr Cartil 16(11):1300–1306CrossRef Oka H, Muraki S, Akune T, Mabuchi A, Suzuki T, Yoshida H, Yamamoto S, Nakamura K, Yoshimura N, Kawaguchi H (2008) Fully automatic quantification of knee osteoarthritis severity on plain radiographs. Osteoarthr Cartil 16(11):1300–1306CrossRef
8.
Zurück zum Zitat Juefei-Xu F, Naresh Boddeti V , Savvides M (2017) Local binary convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 19–28 Juefei-Xu F, Naresh Boddeti V , Savvides M (2017) Local binary convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 19–28
9.
Zurück zum Zitat Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Laak JAVD, Ginneken BV, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Laak JAVD, Ginneken BV, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef
10.
Zurück zum Zitat Antony J, McGuinness K, O’Connor NE, Moran K (2016) Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 1195–1200 Antony J, McGuinness K, O’Connor NE, Moran K (2016) Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, pp 1195–1200
11.
Zurück zum Zitat Antony J, McGuinness K, Moran K, O’Connor NE (2017) Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. In: International conference on machine learning and data mining in pattern recognition. Springer, pp 376–390 Antony J, McGuinness K, Moran K, O’Connor NE (2017) Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. In: International conference on machine learning and data mining in pattern recognition. Springer, pp 376–390
12.
Zurück zum Zitat Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440 Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
13.
Zurück zum Zitat Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8(1):1727CrossRef Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S (2018) Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 8(1):1727CrossRef
14.
Zurück zum Zitat Abedin J, Antony J, McGuinness K, Moran K, O’Connor NE, Rebholz-Schuhmann D, Newell J (2019) Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images. Sci Rep 9(1):5761CrossRef Abedin J, Antony J, McGuinness K, Moran K, O’Connor NE, Rebholz-Schuhmann D, Newell J (2019) Predicting knee osteoarthritis severity: comparative modeling based on patient’s data and plain X-ray images. Sci Rep 9(1):5761CrossRef
15.
Zurück zum Zitat Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988 Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
16.
Zurück zum Zitat Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99 Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99
17.
Zurück zum Zitat Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587
18.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRef He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRef
19.
Zurück zum Zitat Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448 Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448
20.
Zurück zum Zitat Neubeck A, Van Gool L (2006) Efficient non-maximum suppression. In: 18th international conference on pattern recognition (ICPR’06), vol 3. IEEE, pp 850–855 Neubeck A, Van Gool L (2006) Efficient non-maximum suppression. In: 18th international conference on pattern recognition (ICPR’06), vol 3. IEEE, pp 850–855
21.
Zurück zum Zitat Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788 Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788
22.
Zurück zum Zitat Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271 Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271
24.
Zurück zum Zitat Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37 Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37
25.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Metadaten
Titel
Toward automatic quantification of knee osteoarthritis severity using improved Faster R-CNN
verfasst von
Bin Liu
Jianxu Luo
Huan Huang
Publikationsdatum
14.01.2020
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 3/2020
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-02096-9

Weitere Artikel der Ausgabe 3/2020

International Journal of Computer Assisted Radiology and Surgery 3/2020 Zur Ausgabe

Akuter Schwindel: Wann lohnt sich eine MRT?

28.04.2024 Schwindel Nachrichten

Akuter Schwindel stellt oft eine diagnostische Herausforderung dar. Wie nützlich dabei eine MRT ist, hat eine Studie aus Finnland untersucht. Immerhin einer von sechs Patienten wurde mit akutem ischämischem Schlaganfall diagnostiziert.

Screening-Mammografie offenbart erhöhtes Herz-Kreislauf-Risiko

26.04.2024 Mammografie Nachrichten

Routinemäßige Mammografien helfen, Brustkrebs frühzeitig zu erkennen. Anhand der Röntgenuntersuchung lassen sich aber auch kardiovaskuläre Risikopatientinnen identifizieren. Als zuverlässiger Anhaltspunkt gilt die Verkalkung der Brustarterien.

S3-Leitlinie zu Pankreaskrebs aktualisiert

23.04.2024 Pankreaskarzinom Nachrichten

Die Empfehlungen zur Therapie des Pankreaskarzinoms wurden um zwei Off-Label-Anwendungen erweitert. Und auch im Bereich der Früherkennung gibt es Aktualisierungen.

Fünf Dinge, die im Kindernotfall besser zu unterlassen sind

18.04.2024 Pädiatrische Notfallmedizin Nachrichten

Im Choosing-Wisely-Programm, das für die deutsche Initiative „Klug entscheiden“ Pate gestanden hat, sind erstmals Empfehlungen zum Umgang mit Notfällen von Kindern erschienen. Fünf Dinge gilt es demnach zu vermeiden.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.