nach oben

Erschienen in:

22.10.2019 | Original Article

External validation of a prediction model for pain and functional outcome after elective lumbar spinal fusion

verfasst von: Ayesha Quddusi, Hubert A. J. Eversdijk, Anita M. Klukowska, Marlies P. de Wispelaere, Julius M. Kernbach, Marc L. Schröder, Victor E. Staartjes

Erschienen in: European Spine Journal | Ausgabe 2/2020

Einloggen, um Zugang zu erhalten

Abstract

Objective

Patient-reported outcome measures following elective lumbar fusion surgery demonstrate major heterogeneity. Individualized prediction tools can provide valuable insights for shared decision-making. We externally validated the spine surgical care and outcomes assessment programme/comparative effectiveness translational network (SCOAP-CERTAIN) model for prediction of 12-month minimum clinically important difference in Oswestry Disability Index (ODI) and in numeric rating scales for back (NRS-BP) and leg pain (NRS-LP) after elective lumbar fusion.

Methods

Data from a prospective registry were obtained. We calculated the area under the curve (AUC), calibration slope and intercept, and Hosmer–Lemeshow values to estimate discrimination and calibration of the models.

Results

We included 100 patients, with average age of 50.4 ± 11.4 years. For 12-month ODI, AUC was 0.71 while the calibration intercept and slope were 1.08 and 0.95, respectively. For NRS-BP, AUC was 0.72, with a calibration intercept of 1.02, and slope of 0.74. For NRS-LP, AUC was 0.83, with a calibration intercept of 1.08, and slope of 0.95. Sensitivity ranged from 0.64 to 1.00, while specificity ranged from 0.38 to 0.65. A lack of fit was found for all three models based on Hosmer–Lemeshow testing.

Conclusions

The SCOAP-CERTAIN tool can accurately predict which patients will achieve favourable outcomes. However, the predicted probabilities—which are the most valuable in clinical practice—reported by the tool do not correspond well to the true probability of a favourable outcome. We suggest that any prediction tool should first be externally validated before it is applied in routine clinical practice.

Graphic abstract

These slides can be retrieved under Electronic Supplementary Material.

Nur mit Berechtigung zugänglich

Steinmetz MP, Mroz T (2018) Value of adding predictive clinical decision tools to spine surgery. JAMA Surg. https://doi.org/10.1001/jamasurg.2018.0078 CrossRefPubMed

Khor S, Lavallee D, Cizik AM et al (2018) Development and validation of a prediction model for pain and functional outcomes after lumbar spine surgery. JAMA Surg. https://doi.org/10.1001/jamasurg.2018.0072 CrossRefPubMedPubMedCentral

Siccoli A, de Wispelaere MP, Schröder ML, Staartjes VE (2019) Machine learning–based preoperative predictive analytics for lumbar spinal stenosis. Neurosurg Focus 46:E5. https://doi.org/10.3171/2019.2.FOCUS18723 CrossRefPubMed

Staartjes VE, de Wispelaere MP, Vandertop WP, Schröder ML (2018) Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar diskectomy: feasibility of center-specific modeling. Spine J Off J North Am Spine Soc. https://doi.org/10.1016/j.spinee.2018.11.009 CrossRef

Janssen DMC, van Kuijk SMJ, d’Aumerie B, Willems P (2019) A prediction model of surgical site infection after instrumented thoracolumbar spine surgery in adults. Eur Spine J. https://doi.org/10.1007/s00586-018-05877-z CrossRefPubMed

Janssen DMC, van Kuijk SMJ, d’Aumerie BB, Willems PC (2018) External validation of a prediction model for surgical site infection after thoracolumbar spine surgery in a Western European cohort. J Orthop Surg. https://doi.org/10.1186/s13018-018-0821-2 CrossRef

Senders JT, Staples PC, Karhade AV et al (2018) Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg 109:476–486.e1. https://doi.org/10.1016/j.wneu.2017.09.149 CrossRefPubMed

Brusko GD, Kolcun JPG, Wang MY (2018) Machine-learning models: the future of predictive analytics in neurosurgery. Neurosurgery 83:E3–E4. https://doi.org/10.1093/neuros/nyy166 CrossRefPubMed

Cabitza F, Rasoini R, Gensini GF (2017) Unintended consequences of machine learning in medicine. JAMA 318:517–518. https://doi.org/10.1001/jama.2017.7797 CrossRefPubMed

10.

Collins GS, Ogundimu EO, Le Manach Y (2015) Assessing calibration in an external validation study. Spine J 15:2446–2447. https://doi.org/10.1016/j.spinee.2015.06.043 CrossRefPubMed

11.

Debray TPA, Vergouwe Y, Koffijberg H et al (2015) A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 68:279–289. https://doi.org/10.1016/j.jclinepi.2014.06.018 CrossRefPubMed

12.

Tetreault LA, Côté P, Kopjar B et al (2015) A clinical prediction model to assess surgical outcome in patients with cervical spondylotic myelopathy: internal and external validations using the prospective multicenter AOSpine North American and international datasets of 743 patients. Spine J 15:388–397. https://doi.org/10.1016/j.spinee.2014.12.145 CrossRefPubMed

13.

Riley RD, Ensor J, Snell KIE et al (2016) External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ 353:i3140. https://doi.org/10.1136/bmj.i3140 CrossRefPubMedPubMedCentral

14.

Collins GS, de Groot JA, Dutton S et al (2014) External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol 14:40. https://doi.org/10.1186/1471-2288-14-40 CrossRefPubMedPubMedCentral

15.

Collins GS, Ogundimu EO, Altman DG (2016) Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med 35:214–226. https://doi.org/10.1002/sim.6787 CrossRefPubMed

16.

Staartjes VE, de Wispelaere MP, Schröder ML (2019) Improving recovery after elective degenerative spine surgery: 5-year experience with an enhanced recovery after surgery (ERAS) protocol. Neurosurg Focus 46:E7. https://doi.org/10.3171/2019.1.FOCUS18646 CrossRefPubMed

17.

Schröder ML, Staartjes VE (2017) Revisions for screw malposition and clinical outcomes after robot-guided lumbar fusion for spondylolisthesis. Neurosurg Focus 42:E12. https://doi.org/10.3171/2017.3.FOCUS16534 CrossRefPubMed

18.

Staartjes VE, Schröder ML (2018) Effectiveness of a decision-making protocol for the surgical treatment of lumbar stenosis with grade 1 degenerative spondylolisthesis. World Neurosurg 110:e355–e361. https://doi.org/10.1016/j.wneu.2017.11.001 CrossRefPubMed

19.

Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594CrossRefPubMed

20.

Staartjes VE, Siccoli A, de Wispelaere MP, Schröder ML (2018) Patient-reported outcomes unbiased by length of follow-up after lumbar degenerative spine surgery: do we need 2 years of follow-up? Spine J Off J North Am Spine Soc. https://doi.org/10.1016/j.spinee.2018.10.004 CrossRef

21.

Van Hooff ML, Spruit M, Fairbank JCT et al (2015) The Oswestry Disability Index (version 2.1a): validation of a Dutch language version. Spine 40:E83–E90. https://doi.org/10.1097/BRS.0000000000000683 CrossRefPubMed

22.

Steyerberg EW, Vickers AJ, Cook NR et al (2010) Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiol Camb Mass 21:128–138. https://doi.org/10.1097/EDE.0b013e3181c30fb2 CrossRef

23.

Hosmer DW, Lemeshow S, Sturdivant RX (2013) Assessing the fit of the model. In: Hosmer DW Jr, Lemeshow S, Sturdivant RX (eds) Applied logistic regression. Wiley, Hoboken, pp 153–225CrossRef

24.

Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78:1–3. https://doi.org/10.1175/1520-0493(1950)078%3c0001:VOFEIT%3e2.0.CO;2

25.

Van Hoorde K, Van Huffel S, Timmerman D et al (2015) A spline-based tool to assess and visualize the calibration of multiclass risk predictions. J Biomed Inform 54:283–293. https://doi.org/10.1016/j.jbi.2014.12.016 CrossRefPubMed

26.

Core Team R (2018) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna

27.

Falavigna A, Dozza DC, Teles AR et al (2017) Current status of worldwide use of patient-reported outcome measures (PROMs) in spine care. World Neurosurg 108:328–335. https://doi.org/10.1016/j.wneu.2017.09.002 CrossRefPubMed

28.

Glassman SD, Schwab F, Bridwell KH et al (2009) Do 1-year outcomes predict 2-year outcomes for adult deformity surgery? Spine J Off J North Am Spine Soc 9:317–322. https://doi.org/10.1016/j.spinee.2008.06.450 CrossRef

29.

van Niftrik CHB, van der Wouden F, Staartjes VE et al (2019) Machine learning algorithm identifies patients at high risk for early complications after intracranial tumor surgery: registry-based cohort study. Neurosurgery. https://doi.org/10.1093/neuros/nyz145 CrossRefPubMed

30.

Durand WM, DePasse JM, Daniels AH (2018) Predictive modeling for blood transfusion after adult spinal deformity surgery: a tree-based machine learning approach. Spine 43:1058. https://doi.org/10.1097/BRS.0000000000002515 CrossRefPubMed

31.

Ehlers AP, Roy SB, Khor S et al (2017) Improved risk prediction following surgery using machine learning algorithms. eGEMs. https://doi.org/10.13063/2327-9214.1278 CrossRefPubMedPubMedCentral

32.

Kim JS, Merrill RK, Arvind V et al (2018) Examining the ability of artificial neural networks machine learning models to accurately predict complications following posterior lumbar spine fusion. Spine. https://doi.org/10.1097/BRS.0000000000002442 CrossRefPubMedPubMedCentral

33.

van Rein EAJ, van der Sluijs R, Voskens FJ et al (2019) Development and validation of a prediction model for prehospital triage of trauma patients. JAMA Surg. https://doi.org/10.1001/jamasurg.2018.4752 CrossRefPubMedPubMedCentral

34.

Janssen KJM, Moons KGM, Kalkman CJ et al (2008) Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 61:76–86. https://doi.org/10.1016/j.jclinepi.2007.04.018 CrossRefPubMed

35.

Niculescu-Mizil A, Caruana R (2005) Predicting Good Probabilities with Supervised Learning. In: Proceedings of the 22nd international conference on machine learning. ACM, New York, pp 625–632

36.

Staartjes VE, Schröder ML (2018) Letter to the editor. Class imbalance in machine learning for neurosurgical outcome prediction: are our models valid? J Neurosurg Spine. https://doi.org/10.3171/2018.5.SPINE18543 CrossRefPubMed

37.

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357. https://doi.org/10.1613/jair.953 CrossRef

38.

Goldstein CL, Phillips FM, Rampersaud YR (2016) Comparative effectiveness and economic evaluations of open versus minimally invasive posterior or transforaminal lumbar interbody fusion: a systematic review. Spine 41(Suppl 8):S74–S89. https://doi.org/10.1097/BRS.0000000000001462 CrossRefPubMed

39.

Goldstein CL, Macwan K, Sundararajan K, Rampersaud YR (2016) Perioperative outcomes and adverse events of minimally invasive versus open posterior lumbar fusion: meta-analysis and systematic review. J Neurosurg Spine 24:416–427. https://doi.org/10.3171/2015.2.SPINE14973 CrossRefPubMed

40.

Schröder ML, de Wispelaere MP, Staartjes VE (2018) Are patient-reported outcome measures biased by method of follow-up? Evaluating paper-based and digital follow-up after lumbar fusion surgery. Spine J Off J North Am Spine Soc 2:2. https://doi.org/10.1016/j.spinee.2018.05.002 CrossRef

Titel: External validation of a prediction model for pain and functional outcome after elective lumbar spinal fusion
verfasst von: Ayesha Quddusi
Hubert A. J. Eversdijk
Anita M. Klukowska
Marlies P. de Wispelaere
Julius M. Kernbach
Marc L. Schröder
Victor E. Staartjes
Publikationsdatum: 22.10.2019
Verlag: Springer Berlin Heidelberg
Erschienen in: European Spine Journal / Ausgabe 2/2020
Print ISSN: 0940-6719
Elektronische ISSN: 1432-0932
DOI: https://doi.org/10.1007/s00586-019-06189-6

Arthropedia

Grundlagenwissen der Arthroskopie und Gelenkchirurgie. Erweitert durch Fallbeispiele, Videos und Abbildungen.
» Jetzt entdecken

Neu im Fachgebiet Orthopädie und Unfallchirurgie

Häusliche Gewalt in der orthopädischen Notaufnahme oft nicht erkannt

28.05.2024 Traumatologische Notfälle Nachrichten

In der Notaufnahme wird die Chance, Opfer von häuslicher Gewalt zu identifizieren, von Orthopäden und Orthopädinnen offenbar zu wenig genutzt. Darauf deuten die Ergebnisse einer Fragebogenstudie an der Sahlgrenska-Universität in Schweden hin.

Fehlerkultur in der Medizin – Offenheit zählt!

28.05.2024 Internistische Arzneimitteltherapie Podcast

Darüber reden und aus Fehlern lernen, sollte das Motto in der Medizin lauten. Und zwar nicht nur im Sinne der Patientensicherheit. Eine negative Fehlerkultur kann auch die Behandelnden ernsthaft krank machen, warnt Prof. Dr. Reinhard Strametz. Ein Plädoyer und ein Leitfaden für den offenen Umgang mit kritischen Ereignissen in Medizin und Pflege.

Mehr Frauen im OP – weniger postoperative Komplikationen

21.05.2024 Allgemeine Chirurgie Nachrichten

Ein Frauenanteil von mindestens einem Drittel im ärztlichen Op.-Team war in einer großen retrospektiven Studie aus Kanada mit einer signifikanten Reduktion der postoperativen Morbidität assoziiert.

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Update Orthopädie und Unfallchirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.

Newsletter bestellen

Live-Webinar "Urologie und Sexualmedizin in der Praxis"

Springer Medizin

External validation of a prediction model for pain and functional outcome after elective lumbar spinal fusion