Skip to main content
Erschienen in: Clinical Orthopaedics and Related Research® 11/2013

01.11.2013 | Clinical Research

Have Levels of Evidence Improved the Quality of Orthopaedic Research?

verfasst von: Brian P. Cunningham, MD, Samuel Harmsen, MD, Chris Kweon, MD, Jason Patterson, MD, Robert Waldrop, MD, Alex McLaren, MD, Ryan McLemore, PhD

Erschienen in: Clinical Orthopaedics and Related Research® | Ausgabe 11/2013

Einloggen, um Zugang zu erhalten

Abstract

Background

Since 2003 many orthopaedic journals have adopted grading systems for levels of evidence (LOE). It is unclear if the quality of orthopaedic literature has changed since LOE was introduced.

Questions/purposes

We asked three questions: (1) Have the overall number and proportion of Level I and II studies increased in the orthopaedic literature since the introduction of LOE? (2) Is a similar pattern seen in individual orthopaedic subspecialty journals? (3) What is the interobserver reliability of grading LOE?

Methods

We assigned LOE to therapeutic studies published in 2000, 2005, and 2010 in eight major orthopaedic subspecialty journals. Number and proportion of Level I and II publications were determined. Data were evaluated using log-linear models. Twenty-six reviewers (13 residents and 13 attendings) graded LOE of 20 blinded therapeutic articles from the Journal of Bone and Joint Surgery for 2009. Interobserver agreement relative to the Journal of Bone and Joint Surgery was assessed using a weighted kappa.

Results

The total number of Level I and II publications in subspecialty journals increased from 150 in 2000 to 239 in 2010. The proportion of high-quality publications increased with time (p < 0.001). All subspecialty journals other than the Journal of Pediatric Orthopaedics and the Journal of Orthopaedic Trauma showed a similar behavior. Average weighted kappa was 0.791 for residents and 0.842 for faculty (p = 0.209).

Conclusions

The number and proportion of Level I and II publications have increased. LOE can be graded reliably with high interobserver agreement. The number and proportion of high-level studies should continue to increase.
Literatur
1.
Zurück zum Zitat Altman DG. Practical Statistics for Medical Research. 2nd ed. London, England: Chapman & Hall/CRC; 2006. Altman DG. Practical Statistics for Medical Research. 2nd ed. London, England: Chapman & Hall/CRC; 2006.
2.
Zurück zum Zitat Bhandari M, Swiontkowski MF, Einhorn TA, Tornetta P 3rd, Schemitsch EH, Leece P, Sprague S, Wright JG. Interobserver agreement in the application of levels of evidence to scientific papers in the American volume of the Journal of Bone and Joint Surgery. J Bone Joint Surg Am. 2004;86:1717–1720.PubMed Bhandari M, Swiontkowski MF, Einhorn TA, Tornetta P 3rd, Schemitsch EH, Leece P, Sprague S, Wright JG. Interobserver agreement in the application of levels of evidence to scientific papers in the American volume of the Journal of Bone and Joint Surgery. J Bone Joint Surg Am. 2004;86:1717–1720.PubMed
3.
Zurück zum Zitat Brophy RH, Gardner MJ, Saleem O, Marx RG. An assessment of the methodological quality of research published in The American Journal of Sports Medicine. Am J Sports Med. 2005;33:1812–1815.PubMedCrossRef Brophy RH, Gardner MJ, Saleem O, Marx RG. An assessment of the methodological quality of research published in The American Journal of Sports Medicine. Am J Sports Med. 2005;33:1812–1815.PubMedCrossRef
4.
Zurück zum Zitat Cashin MS, Kelley SP, Douziech JR, Varghese RA, Hamilton QP, Mulpuri K. The levels of evidence in pediatric orthopaedic journals: where are we now? J Pediatr Orthop. 2011;31:721–725.PubMedCrossRef Cashin MS, Kelley SP, Douziech JR, Varghese RA, Hamilton QP, Mulpuri K. The levels of evidence in pediatric orthopaedic journals: where are we now? J Pediatr Orthop. 2011;31:721–725.PubMedCrossRef
5.
Zurück zum Zitat Chang EY, Pannucci CJ, Wilkins EG. Quality of clinical studies in aesthetic surgery journals: a 10-year review. Aesthet Surg J. 2009;29:144–147; discussion 147–149. Chang EY, Pannucci CJ, Wilkins EG. Quality of clinical studies in aesthetic surgery journals: a 10-year review. Aesthet Surg J. 2009;29:144–147; discussion 147–149.
6.
Zurück zum Zitat Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to “Effectiveness and Efficiency: Random Reflections on the Health Services” 1972. Control Clin Trials. 1989;10:428–433. Cochrane AL. Archie Cochrane in his own words. Selections arranged from his 1972 introduction to “Effectiveness and Efficiency: Random Reflections on the Health Services” 1972. Control Clin Trials. 1989;10:428–433.
7.
Zurück zum Zitat Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468:1428–1435.PubMedCrossRef Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468:1428–1435.PubMedCrossRef
8.
Zurück zum Zitat Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. Lancet. 1995;346:407–410.PubMedCrossRef Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. Lancet. 1995;346:407–410.PubMedCrossRef
9.
Zurück zum Zitat Esses SI, McGuire R, Jenkins J, Finkelstein J, Woodard E, Watters WC 3rd, Goldberg MJ, Keith M, Turkelson CM, Wies JL, Sluka P, Boyer KM, Hitchcock K. The treatment of symptomatic osteoporotic spinal compression fractures. J Am Acad Orthop Surg. 2011;19:176–182.PubMed Esses SI, McGuire R, Jenkins J, Finkelstein J, Woodard E, Watters WC 3rd, Goldberg MJ, Keith M, Turkelson CM, Wies JL, Sluka P, Boyer KM, Hitchcock K. The treatment of symptomatic osteoporotic spinal compression fractures. J Am Acad Orthop Surg. 2011;19:176–182.PubMed
10.
Zurück zum Zitat Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268:2420–2425.CrossRef Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268:2420–2425.CrossRef
11.
Zurück zum Zitat Hanzlik S, Mahabir RC, Baynosa RC, Khiabani KT. Levels of evidence in research published in The Journal of Bone and Joint Surgery (American Volume) over the last thirty years. J Bone Joint Surg Am. 2009;91:425–428.PubMedCrossRef Hanzlik S, Mahabir RC, Baynosa RC, Khiabani KT. Levels of evidence in research published in The Journal of Bone and Joint Surgery (American Volume) over the last thirty years. J Bone Joint Surg Am. 2009;91:425–428.PubMedCrossRef
12.
Zurück zum Zitat Keith MW, Masear V, Amadio PC, Andary M, Barth RW, Graham B, Chung K, Maupin K, Watters WC 3rd, Haralson RH 3rd, Turkelson CM, Wies JL, McGowan R. Treatment of carpal tunnel syndrome. J Am Acad Orthop Surg. 2009;17:397–405.PubMed Keith MW, Masear V, Amadio PC, Andary M, Barth RW, Graham B, Chung K, Maupin K, Watters WC 3rd, Haralson RH 3rd, Turkelson CM, Wies JL, McGowan R. Treatment of carpal tunnel syndrome. J Am Acad Orthop Surg. 2009;17:397–405.PubMed
13.
Zurück zum Zitat Kuhn JE. Exercise in the treatment of rotator cuff impingement: a systematic review and a synthesized evidence-based rehabilitation protocol. J Shoulder Elbow Surg. 2009;18:138–160.PubMedCrossRef Kuhn JE. Exercise in the treatment of rotator cuff impingement: a systematic review and a synthesized evidence-based rehabilitation protocol. J Shoulder Elbow Surg. 2009;18:138–160.PubMedCrossRef
14.
Zurück zum Zitat Lichtman DM, Bindra RR, Boyer MI, Putnam MD, Ring D, Slutsky DJ, Taras JS, Watters WC 3rd, Goldberg MJ, Keith M, Turkelson CM, Wies JL, Haralson RH 3rd, Boyer KM, Hitchcock K, Raymond L. Treatment of distal radius fractures. J Am Acad Orthop Surg. 2010;18:180–189.PubMed Lichtman DM, Bindra RR, Boyer MI, Putnam MD, Ring D, Slutsky DJ, Taras JS, Watters WC 3rd, Goldberg MJ, Keith M, Turkelson CM, Wies JL, Haralson RH 3rd, Boyer KM, Hitchcock K, Raymond L. Treatment of distal radius fractures. J Am Acad Orthop Surg. 2010;18:180–189.PubMed
15.
Zurück zum Zitat Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121:207e–211e.PubMedCrossRef Loiselle F, Mahabir RC, Harrop AR. Levels of evidence in plastic surgery research over 20 years. Plast Reconstr Surg. 2008;121:207e–211e.PubMedCrossRef
17.
Zurück zum Zitat Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005;87:2632–2638.PubMedCrossRef Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005;87:2632–2638.PubMedCrossRef
18.
Zurück zum Zitat Richmond J, Hunter D, Irrgang J, Jones MH, Levy B, Marx R, Snyder-Mackler L, Watters WC 3rd, Haralson RH 3rd, Turkelson CM, Wies JL, Boyer KM, Anderson S, St Andre J, Sluka P, McGowan R; American Academy of Orthopaedic Surgeons. Treatment of osteoarthritis of the knee (nonarthroplasty). J Am Acad Orthop Surg. 2009;17:591–600. Richmond J, Hunter D, Irrgang J, Jones MH, Levy B, Marx R, Snyder-Mackler L, Watters WC 3rd, Haralson RH 3rd, Turkelson CM, Wies JL, Boyer KM, Anderson S, St Andre J, Sluka P, McGowan R; American Academy of Orthopaedic Surgeons. Treatment of osteoarthritis of the knee (nonarthroplasty). J Am Acad Orthop Surg. 2009;17:591–600.
19.
Zurück zum Zitat Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986;89(2 suppl):2S–3S.PubMed Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986;89(2 suppl):2S–3S.PubMed
20.
Zurück zum Zitat Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–72.PubMedCrossRef Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–72.PubMedCrossRef
21.
Zurück zum Zitat Schmidt AH, Zhao G, Turkelson C. Levels of evidence at the AAOS meeting: can authors rate their own submissions, and do other raters agree? J Bone Joint Surg Am. 2009;91:867–873.PubMedCrossRef Schmidt AH, Zhao G, Turkelson C. Levels of evidence at the AAOS meeting: can authors rate their own submissions, and do other raters agree? J Bone Joint Surg Am. 2009;91:867–873.PubMedCrossRef
22.
Zurück zum Zitat Schwartz WB, Gorry GA, Kassirer JP, Essig A. Decision analysis and clinical judgment. Am J Med. 1973;55:459–472.PubMedCrossRef Schwartz WB, Gorry GA, Kassirer JP, Essig A. Decision analysis and clinical judgment. Am J Med. 1973;55:459–472.PubMedCrossRef
23.
Zurück zum Zitat Thornton J, Alderson P, Tan T, Turner C, Latchem S, Shaw E, Ruiz F, Reken S, Mugglestone MA, Hill J, Neilson J, Westby M, Francis K, Whittington C, Siddiqui F, Sharma T, Kelly V, Ayiku L, Chamberlain K. Introducing GRADE across the NICE clinical guideline program. J Clin Epidemiol. 2013;66:124–131.PubMedCrossRef Thornton J, Alderson P, Tan T, Turner C, Latchem S, Shaw E, Ruiz F, Reken S, Mugglestone MA, Hill J, Neilson J, Westby M, Francis K, Whittington C, Siddiqui F, Sharma T, Kelly V, Ayiku L, Chamberlain K. Introducing GRADE across the NICE clinical guideline program. J Clin Epidemiol. 2013;66:124–131.PubMedCrossRef
24.
Zurück zum Zitat Turpen RM, Fesperman SF, Sultan S, Borawski KM, Norris RD, Klink J, Sur RL, Breau RH, Krupski TL, Dahm P. Levels of evidence ratings in the urological literature: an assessment of interobserver agreement. BJU Int. 2010;105:602–606.PubMedCrossRef Turpen RM, Fesperman SF, Sultan S, Borawski KM, Norris RD, Klink J, Sur RL, Breau RH, Krupski TL, Dahm P. Levels of evidence ratings in the urological literature: an assessment of interobserver agreement. BJU Int. 2010;105:602–606.PubMedCrossRef
26.
Zurück zum Zitat US Preventive Services Task Force. Guide to Clinical Preventive Services: Report of the US Preventive Services Task Force. Collingdale, PA: DIANE Publishing; 1989. US Preventive Services Task Force. Guide to Clinical Preventive Services: Report of the US Preventive Services Task Force. Collingdale, PA: DIANE Publishing; 1989.
27.
Zurück zum Zitat Voleti PB, Donegan DJ, Baldwin KD, Lee GC. Level of evidence of presentations at American Academy of Orthopaedic Surgeons annual meetings. J Bone Joint Surg Am. 2012;94:e50.PubMedCrossRef Voleti PB, Donegan DJ, Baldwin KD, Lee GC. Level of evidence of presentations at American Academy of Orthopaedic Surgeons annual meetings. J Bone Joint Surg Am. 2012;94:e50.PubMedCrossRef
28.
Zurück zum Zitat Voleti PB, Donegan DJ, Kim TWB, Lee GC. Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg Am. 2013;95:e2.PubMed Voleti PB, Donegan DJ, Kim TWB, Lee GC. Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg Am. 2013;95:e2.PubMed
29.
Zurück zum Zitat Wasserman JM, Wynn R, Bash TS, Rosenfeld RM. Levels of evidence in otolaryngology journals. Otolaryngol Head Neck Surg. 2006;134:717–723.PubMedCrossRef Wasserman JM, Wynn R, Bash TS, Rosenfeld RM. Levels of evidence in otolaryngology journals. Otolaryngol Head Neck Surg. 2006;134:717–723.PubMedCrossRef
30.
Zurück zum Zitat Wingfield K, Matheson GO, Meeuwisse WH. Preparticipation evaluation: an evidence-based review. Clin J Sport Med. 2004;14:109–122.PubMedCrossRef Wingfield K, Matheson GO, Meeuwisse WH. Preparticipation evaluation: an evidence-based review. Clin J Sport Med. 2004;14:109–122.PubMedCrossRef
31.
Zurück zum Zitat Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85:1–3.PubMedCrossRef Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85:1–3.PubMedCrossRef
32.
Zurück zum Zitat Wupperman R, Davis R, Obremskey WT. Level of evidence in Spine compared to other orthopedic journals. Spine (Phila Pa 1976). 2007;32:388–393. Wupperman R, Davis R, Obremskey WT. Level of evidence in Spine compared to other orthopedic journals. Spine (Phila Pa 1976). 2007;32:388–393.
33.
Zurück zum Zitat Xu CC, Côté DW, Chowdhury RH, Morrissey AT, Ansari K. Trends in level of evidence in facial plastic surgery research. Plast Reconstr Surg. 2011;127:1499–1504.PubMedCrossRef Xu CC, Côté DW, Chowdhury RH, Morrissey AT, Ansari K. Trends in level of evidence in facial plastic surgery research. Plast Reconstr Surg. 2011;127:1499–1504.PubMedCrossRef
Metadaten
Titel
Have Levels of Evidence Improved the Quality of Orthopaedic Research?
verfasst von
Brian P. Cunningham, MD
Samuel Harmsen, MD
Chris Kweon, MD
Jason Patterson, MD
Robert Waldrop, MD
Alex McLaren, MD
Ryan McLemore, PhD
Publikationsdatum
01.11.2013
Verlag
Springer US
Erschienen in
Clinical Orthopaedics and Related Research® / Ausgabe 11/2013
Print ISSN: 0009-921X
Elektronische ISSN: 1528-1132
DOI
https://doi.org/10.1007/s11999-013-3159-4

Weitere Artikel der Ausgabe 11/2013

Clinical Orthopaedics and Related Research® 11/2013 Zur Ausgabe

Letter to the Editor

Letter to the Editor

Symposium: ABJS Carl T. Brighton Workshop on Outcome Measures

Measuring Expectations in Orthopaedic Surgery: A Systematic Review

Symposium: ABJS Carl T. Brighton Workshop on Outcome Measures

Statistical Considerations in the Psychometric Validation of Outcome Measures

Reply to the Letter to the Editor

Reply to the Letter to the Editor

Arthropedia

Grundlagenwissen der Arthroskopie und Gelenkchirurgie. Erweitert durch Fallbeispiele, Videos und Abbildungen. 
» Jetzt entdecken

Notfall-TEP der Hüfte ist auch bei 90-Jährigen machbar

26.04.2024 Hüft-TEP Nachrichten

Ob bei einer Notfalloperation nach Schenkelhalsfraktur eine Hemiarthroplastik oder eine totale Endoprothese (TEP) eingebaut wird, sollte nicht allein vom Alter der Patientinnen und Patienten abhängen. Auch über 90-Jährige können von der TEP profitieren.

Arthroskopie kann Knieprothese nicht hinauszögern

25.04.2024 Gonarthrose Nachrichten

Ein arthroskopischer Eingriff bei Kniearthrose macht im Hinblick darauf, ob und wann ein Gelenkersatz fällig wird, offenbar keinen Unterschied.

Therapiestart mit Blutdrucksenkern erhöht Frakturrisiko

25.04.2024 Hypertonie Nachrichten

Beginnen ältere Männer im Pflegeheim eine Antihypertensiva-Therapie, dann ist die Frakturrate in den folgenden 30 Tagen mehr als verdoppelt. Besonders häufig stürzen Demenzkranke und Männer, die erstmals Blutdrucksenker nehmen. Dafür spricht eine Analyse unter US-Veteranen.

Ärztliche Empathie hilft gegen Rückenschmerzen

23.04.2024 Leitsymptom Rückenschmerzen Nachrichten

Personen mit chronischen Rückenschmerzen, die von einfühlsamen Ärzten und Ärztinnen betreut werden, berichten über weniger Beschwerden und eine bessere Lebensqualität.

Update Orthopädie und Unfallchirurgie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.