nach oben

Erschienen in:

14.07.2022 | Supplement article

External validation of the deep learning system “SpineNet” for grading radiological features of degeneration on MRIs of the lumbar spine

verfasst von: Alexandra Grob, Markus Loibl, Amir Jamaludin, Sebastian Winklhofer, Jeremy C. T. Fairbank, Tamás Fekete, François Porchet, Anne F. Mannion

Erschienen in: European Spine Journal | Ausgabe 8/2022

Einloggen, um Zugang zu erhalten

Abstract

Background

Magnetic resonance imaging (MRI) is used to detect degenerative changes of the lumbar spine. SpineNet (SN), a computer vision-based system, performs an automated analysis of degenerative features in MRI scans aiming to provide high accuracy, consistency and objectivity. This study evaluated SN’s ratings compared with those of an expert radiologist.

Method

MRIs of 882 patients (mean age, 72 ± 8.8 years) with degenerative spinal disorders from two previous trials carried out in our spine center between 2011 and 2019, were analyzed by an expert radiologist. Lumbar segments (L1/2–L5/S1) were graded for Pfirrmann Grades (PG), Spondylolisthesis (SL) and Central Canal Stenosis (CCS). SN’s analysis for the equivalent parameters was generated. Agreement between methods was analyzed using kappa (κ), Spearman correlation (ρ) and Lin’s concordance correlation (ρ_c) coefficients and class average accuracy (CAA).

Results

4410 lumbar segments were analyzed. κ statistics showed moderate to substantial agreement in PG between the radiologist and SN depending on spinal level (range κ 0.63–0.77, all levels together 0.72; range CAA 45–68%, all levels 55%), slight to substantial agreement for SL (range κ 0.07–0.60, all levels 0.63; range CAA 47–57%, all levels 56%) and CCS (range κ 0.17–0.57, all levels 0.60; range CAA 35–41%, all levels 43%). SN tended to record more pathological features in PG than did the radiologist whereas the contrary was the case for CCS. SL showed an even distribution between methods.

Conclusion

SN is a robust and reliable tool with the ability to grade degenerative features such as PG, SL or CCS in lumbar MRIs with moderate to substantial agreement compared to the current gold-standard, the radiologist. It is a valuable alternative for analyzing MRIs from large cohorts for diagnostic and research purposes.

Palmer KT, Walsh K, Bendall H, Cooper C, Coggon D (2000) Back pain in Britain: comparison of two prevalence surveys at an interval of 10 years. BMJ 320:1577–1578. https://doi.org/10.1136/bmj.320.7249.1577CrossRefPubMedPubMedCentral

de Schepper EI, Damen J, van Meurs JB, Ginai AZ, Popham M, Hofman A, Koes BW, Bierma-Zeinstra SM (2010) The association between lumbar disc degeneration and low back pain: the influence of age, gender, and individual radiographic features. Spine (Phila Pa 1976) 35:531–536. https://doi.org/10.1097/BRS.0b013e3181aa5b33CrossRef

Brinjikji W, Luetmer PH, Comstock B, Bresnahan BW, Chen LE, Deyo RA, Halabi S, Turner JA, Avins AL, James K, Wald JT, Kallmes DF, Jarvik JG (2015) Systematic literature review of imaging features of spinal degeneration in asymptomatic populations. AJNR Am J Neuroradiol 36:811–816. https://doi.org/10.3174/ajnr.A4173CrossRefPubMedPubMedCentral

Stadnik TW, Lee RR, Coen HL, Neirynck EC, Buisseret TS, Osteaux MJ (1998) Annular tears and disk herniation: prevalence and contrast enhancement on MR images in the absence of low back pain or sciatica. Radiology 206:49–55CrossRef

Weishaupt D, Zanetti M, Hodler J, Boos N (1998) MR imaging of the lumbar spine: prevalence of intervertebral disk extrusion and sequestration, nerve root compression, end plate abnormalities, and osteoarthritis of the facet joints in asymptomatic volunteers. Radiology 209:661–666. https://doi.org/10.1148/radiology.209.3.9844656CrossRefPubMed

Jensen MC, Brant-Zawadzki MN, Obuchowski N, Modic MT, Malkasian D, Ross JS (1994) Magnetic resonance imaging of the lumbar spine in people without back pain. N Engl J Med 331:69–73. https://doi.org/10.1056/NEJM199407143310201CrossRefPubMed

Galbusera F, Casaroli G, Bassani T (2019) Artificial intelligence and machine learning in spine research. JOR Spine 2:e1044. https://doi.org/10.1002/jsp2.1044CrossRefPubMedPubMedCentral

Zhou Y, Liu Y, Chen Q, Gu G, Sui X (2019) Automatic lumbar MRI detection and identification based on deep learning. J Digit Imaging 32:513–520. https://doi.org/10.1007/s10278-018-0130-7CrossRefPubMed

Wimmer M, Major D, Novikov AA, Bühler K (2018) Fully automatic cross-modality localization and labeling of vertebral bodies and intervertebral discs in 3D spinal images. Int J Comput Assist Radiol Surg 13:1591–1603. https://doi.org/10.1007/s11548-018-1818-3CrossRefPubMed

10.

Cai Y, Landis M, Laidley DT, Kornecki A, Lum A, Li S (2016) Multi-modal vertebrae recognition using transformed deep convolution network. Comput Med Imaging Graph 51:11–19. https://doi.org/10.1016/j.compmedimag.2016.02.002CrossRefPubMed

11.

Oktay AB, Akgul YS (2013) Simultaneous localization of lumbar vertebrae and intervertebral discs with SVM-based MRF. IEEE Trans Biomed Eng 60:2375–2383. https://doi.org/10.1109/TBME.2013.2256460CrossRefPubMed

12.

Castro-Mateos I, Hua R, Pozo JM, Lazary A, Frangi AF (2016) Intervertebral disc classification by its degree of degeneration from T2-weighted magnetic resonance images. Eur Spine J 25:2721–2727. https://doi.org/10.1007/s00586-016-4654-6CrossRefPubMed

13.

Jamaludin A, Lootus M, Kadir T, Zisserman A, Urban J, Battié MC, Fairbank J, McCall I, Consotium G (2017) ISSLS prize in bioengineering science 2017: automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J 26:1374–1383. https://doi.org/10.1007/s00586-017-4956-3CrossRefPubMed

14.

Jamaludin A, Kadir T, Zisserman A (2017) SpineNet: automated classification and evidence visualization in spinal MRIs. Med Image Anal 41:63–73. https://doi.org/10.1016/j.media.2017.07.002CrossRefPubMed

15.

Ishimoto Y, Jamaludin A, Cooper C, Walker-Bone K, Yamada H, Hashizume H, Oka H, Tanaka S, Yoshimura N, Yoshida M, Urban J, Kadir T, Fairbank J (2020) Could automated machine-learned MRI grading aid epidemiological studies of lumbar spinal stenosis? Validation within the Wakayama spine study. BMC Musculoskelet Disord 21:158. https://doi.org/10.1186/s12891-020-3164-1CrossRefPubMedPubMedCentral

16.

Becker HJ, Nauer S, Porchet F, Kleinstück FS, Haschtmann D, Fekete TF, Steurer J, Mannion AF (2017) A novel use of the Spine Tango registry to evaluate selection bias in patient recruitment into clinical studies: an analysis of patients participating in the Lumbar Spinal Stenosis Outcome Study (LSOS). Eur Spine J 26:441–449. https://doi.org/10.1007/s00586-016-4850-4CrossRefPubMed

17.

Mannion AF, Mariaux F, Fekete T, Kleinstück FS, Jeszenszky D, Köhler M, Haschtmann D, Lurie J, Pearson A, Otten P, Norberg M, Loibl M, Pittet V, Porchet F (2020) A prospective, controlled, multicentre study to evaluate the association between “appropriate use of surgery” and outcome in degenerative spondylolisthesis. Swiss Med Wkly 149 (Suppl. 244):16S

18.

Jamaludin A, Windsor R (2021) Oxford SpineNet Version2. http://zeus.robots.ox.ac.uk/spinenet2/. Accessed 23 june 2021

19.

Pfirrmann CW, Metzdorf A, Zanetti M, Hodler J, Boos N (2001) Magnetic resonance classification of lumbar intervertebral disc degeneration. Spine (Phila Pa 1976) 26:1873–1878. https://doi.org/10.1097/00007632-200109010-00011CrossRef

20.

Kalichman L, Hunter DJ (2008) Diagnosis and conservative management of degenerative lumbar spondylolisthesis. Eur Spine J 17:327–335. https://doi.org/10.1007/s00586-007-0543-3CrossRefPubMed

21.

Wiltse LL, Newman PH, Macnab I (1976) Classification of spondylolisis and spondylolisthesis. Clin Orthop Relat Res. https://doi.org/10.1097/00003086-197606000-00003CrossRefPubMed

22.

Meyerding HW (1956) Spondylolisthesis; surgical fusion of lumbosacral portion of spinal column and interarticular facets; use of autogenous bone grafts for relief of disabling backache. J Int Coll Surg 26:566–591PubMed

23.

Lurie JD, Tosteson AN, Tosteson TD, Carragee E, Carrino JA, Carrino J, Kaiser J, Sequeiros RT, Lecomte AR, Grove MR, Blood EA, Pearson LH, Weinstein JN, Herzog R (2008) Reliability of readings of magnetic resonance imaging features of lumbar spinal stenosis. Spine (Phila Pa 1976) 33:1605–1610. https://doi.org/10.1097/BRS.0b013e3181791af3CrossRef

24.

Windsor R, Jamaludin A (2020) SpineNet Version 2 Online Demo. University of Oxford, Great Britain. http://zeus.robots.ox.ac.uk/spinenet2/. Accessed May 2022

25.

Cohen J (1968) Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 70:213–220. https://doi.org/10.1037/h0026256CrossRefPubMed

26.

Ranganathan P, Pramesh CS, Aggarwal R (2017) Common pitfalls in statistical analysis: measures of agreement. Perspect Clin Res 8:187–191. https://doi.org/10.4103/picr.PICR_123_17CrossRefPubMedPubMedCentral

27.

Fleiss JL, Cohen J (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 33:613–619. https://doi.org/10.1177/001316447303300309CrossRef

28.

Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174CrossRef

29.

Schober P, Boer C, Schwarte LA (2018) Correlation coefficients: appropriate use and interpretation. Anesth Analg 126:1763–1768. https://doi.org/10.1213/ANE.0000000000002864CrossRefPubMed

30.

Lin LI (1989) A concordance correlation coefficient to evaluate reproducibility. Biometrics 45:255–268CrossRef

31.

Lin L, Hedayat AS, Wu W (2007) A unified approach for assessing agreement for continuous and categorical data. J Biopharm Stat 17:629–652. https://doi.org/10.1080/10543400701376498CrossRefPubMed

32.

Robieson WZ (1999) On weighted kappa and concordance correlation coefficient. University of Illinois, Chicago

33.

Brodersen KH, Ong CS, Stephen KE, Buhmann JM (2010) The Balanced Accuracy and Its Posterior Distribution. In: Proceedings 2010 20th international conference on pattern recognition, pp 3121–3124

34.

Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manag 45:427–437. https://doi.org/10.1016/j.ipm.2009.03.002CrossRef

35.

Altman DG, Bland JM (1994) Diagnostic tests. 1: sensitivity and specificity. BMJ 308(6943):1552. https://doi.org/10.1136/bmj.308.6943.1552CrossRefPubMedPubMedCentral

36.

Miller E (1969) The signed-rank (Wilcoxon)test. Lancet 1:371. https://doi.org/10.1016/s0140-6736(69)91330-0CrossRefPubMed

37.

Windsor R, Jamaludin A, Kadir T, Zisserman A (2020) A convolutional approach to vertebrae detection and labelling in whole spine MRI. Springer International Publishing, Cham, pp 712–722

38.

Winklhofer S, Held U, Burgstaller JM, Finkenstaedt T, Bolog N, Ulrich N, Steurer J, Andreisek G, Del Grande F (2017) Degenerative lumbar spinal canal stenosis: intra- and inter-reader agreement for magnetic resonance imaging parameters. Eur Spine J 26:353–361. https://doi.org/10.1007/s00586-016-4667-1CrossRefPubMed

39.

Warrens MJ (2013) Cohen’s weighted kappa with additive weights. Adv Data Anal Classif 7:41–55. https://doi.org/10.1007/s11634-013-0123-9CrossRef

40.

Won D, Lee HJ, Lee SJ, Park SH (2020) Spinal stenosis grading in magnetic resonance imaging using deep convolutional neural networks. Spine (Phila Pa 1976) 45:804–812. https://doi.org/10.1097/BRS.0000000000003377CrossRef

41.

Lu J-T, Pedemonte S, Bizzo B, Doyle S, Andriole K, Michalski M, Gonzalez R, Pomerantz S (2018) DeepSPINE: automated lumbar vertebral segmentation, disc-level designation, and spinal stenosis grading using deep learning.

42.

Speciale AC, Pietrobon R, Urban CW, Richardson WJ, Helms CA, Major N, Enterline D, Hey L, Haglund M, Turner DA (2002) Observer variability in assessing lumbar spinal stenosis severity on magnetic resonance imaging and its relation to cross-sectional spinal canal area. Spine (Phila Pa 1976) 27:1082–1086. https://doi.org/10.1097/00007632-200205150-00014CrossRef

43.

Birkenbach X (1984) Halo, Central Tendency, and Leniency in performance appraisel: a comparison between a graphic rating scale and a behaviourally based measure. S Afr J Ind Psychol. https://doi.org/10.4102/sajip.v0i0.351CrossRef

44.

Yu LP, Qian WW, Yin GY, Ren YX, Hu ZY (2012) MRI assessment of lumbar intervertebral disc degeneration with lumbar degenerative disease using the Pfirrmann grading systems. PLoS ONE 7:e48074. https://doi.org/10.1371/journal.pone.0048074CrossRefPubMedPubMedCentral

45.

Koslosky E, Gendelberg D (2020) Classification in brief: the Meyerding classification system of spondylolisthesis. Clin Orthop Relat Res 478:1125–1130. https://doi.org/10.1097/CORR.0000000000001153CrossRefPubMedPubMedCentral

46.

Mac-Thiong JM, Duong L, Parent S, Hresko MT, Dimar JR, Weidenbaum M, Labelle H (2012) Reliability of the Spinal Deformity Study Group classification of lumbosacral spondylolisthesis. Spine (Phila Pa 1976) 37:E95-102. https://doi.org/10.1097/BRS.0b013e3182233969CrossRef

47.

Ogikubo O, Forsberg L, Hansson T (2007) The relationship between the cross-sectional area of the cauda equina and the preoperative symptoms in central lumbar spinal stenosis. Spine (Phila Pa 1976) 32:1423–1428. https://doi.org/10.1097/BRS.0b013e318060a5f5CrossRef

48.

Schizas C, Theumann N, Burn A, Tansey R, Wardlaw D, Smith FW, Kulik G (2010) Qualitative grading of severity of lumbar spinal stenosis based on the morphology of the dural sac on magnetic resonance images. Spine (Phila Pa 1976) 35:1919–1924. https://doi.org/10.1097/BRS.0b013e3181d359bdCrossRef

49.

Jamaludin A, Kadir T, Zisserman A (2019) SpineNet Version 1 online Demo. http://zeus.robots.ox.ac.uk/spinenet/demo.html. Accessed March 2019

50.

Roller BL, Boutin RD, O’Gara TJ, Knio ZO, Jamaludin A, Tan J, Lenchik L (2021) Accurate prediction of lumbar microdecompression level with an automated MRI grading system. Skeletal Radiol 50:69–78. https://doi.org/10.1007/s00256-020-03505-wCrossRefPubMed

51.

Szpalski M, Gunzburg R (2003) Lumbar spinal stenosis in the elderly: an overview. Eur Spine J 12(Suppl 2):S170-175. https://doi.org/10.1007/s00586-003-0612-1CrossRefPubMedPubMedCentral

52.

Sim J, Wright CC (2005) The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 85:257–268CrossRef

53.

de Vet HC, Mokkink LB, Terwee CB, Hoekstra OS, Knol DL (2013) Clinicians are right not to like Cohen’s κ. BMJ 346:f2125. https://doi.org/10.1136/bmj.f2125CrossRefPubMed

Titel: External validation of the deep learning system “SpineNet” for grading radiological features of degeneration on MRIs of the lumbar spine
verfasst von: Alexandra Grob
Markus Loibl
Amir Jamaludin
Sebastian Winklhofer
Jeremy C. T. Fairbank
Tamás Fekete
François Porchet
Anne F. Mannion
Publikationsdatum: 14.07.2022
Verlag: Springer Berlin Heidelberg
Erschienen in: European Spine Journal / Ausgabe 8/2022
Print ISSN: 0940-6719
Elektronische ISSN: 1432-0932
DOI: https://doi.org/10.1007/s00586-022-07311-x

Update Orthopädie und Unfallchirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Die Highlights vom Kongress des American College of Cardiology 2024

Springer Medizin

External validation of the deep learning system “SpineNet” for grading radiological features of degeneration on MRIs of the lumbar spine