Neurology/original research
Interrater Reliability of 3 Simplified Neurologic Scales Applied to Adults Presenting to the Emergency Department With Altered Levels of Consciousness

https://doi.org/10.1016/j.annemergmed.2006.03.031Get rights and content

Study objective

The Simplified Motor Score was recently found to exhibit equal test performance to the Glasgow Coma Scale (GCS) when predicting 4 clinically important trauma outcomes. The present study tests the interrater reliability of the Simplified Motor Scale, the GCS and its components, and 2 other simplified neurologic scales when applied to patients presenting to the emergency department with altered levels of consciousness from any cause.

Methods

In this prospective observational study, emergency physicians independently assigned the GCS, Simplified Motor Scale, and 2 4-point scales—AVPU (Alert, responds to Verbal stimuli, responds to Painful stimuli, Unresponsive) and ACDU (Alert, Confused, Drowsy, Unresponsive)—to qualifying adult subjects. Two physicians filled out prospective data forms within 5 minutes of each other while remaining blinded to each other’s assessments. Data were pooled and analyzed for interrater reliability of all scales using simple agreement, unweighted κ, Spearman’s ρ, and Kendall’s τ-b.

Results

One hundred twenty-six subjects were enrolled, with 6 later excluded. Percentage agreements were 83% for the Simplified Motor Scale, 58% for the ACDU scale, 57% for the AVPU scale, and 42% for the Total GCS. The κ values were 0.70 for the Simplified Motor Scale, 0.43 for ACDU, 0.41 for AVPU, and 0.32 for the Total GCS. The Simplified Motor Scale also had the highest Spearman’s ρ (.85) and second highest Kendall’s τ-b (0.81).

Conclusion

The Simplified Motor Scale has the best interrater reliability for the assessment of altered level of consciousness of traumatic and nontraumatic cause among the scales tested.

Introduction

Patients presenting to the emergency department (ED) with altered levels of consciousness of traumatic and nontraumatic cause require careful assessment and continuous monitoring. Historically, a variety of neurologic scales has been used to monitor these patients, including the Glasgow Coma Scale (GCS),1 Advanced Trauma Life Support AVPU (Alert, responds to Verbal stimuli, responds to Painful stimuli, Unresponsive) scale,2 ACDU (Alert, Confused, Drowsy, Unresponsive) scale,3 and Swedish Reaction Level Scale.4 With the exception of the GCS, most of these have never undergone prospective assessment to ensure reliability and validity but have rather been adapted for use either because of institutional preference or ease of application. The GCS has recently been found to exhibit only moderate interrater reliability when tested in the ED setting in patients with altered levels of consciousness from traumatic and nontraumatic causes.5

Recently Gill et al6 proposed a statistically derived scale, the Simplified Motor Scale (SMS), as a potential replacement for the GCS. The SMS was found to perform as well as the GCS at predicting 4 clinical outcomes after traumatic brain injury in 8,000 patients. Although these findings pointed toward the potential of the SMS to replace the more complex GCS, the SMS would require prospective validation in a new sample of patients with altered levels of consciousness from traumatic and nontraumatic causes before clinical application.

In this study, we chose to test the interrater reliability of the AVPU scale, ACDU scale, and SMS prospectively when applied to patients presenting to the ED with altered levels of consciousness from traumatic and nontraumatic causes. If found to demonstrate improved interrater reliability when compared with the GCS in the clinical setting, 1 or more of these alternative scales could replace this more complex neurologic scale for the assessment of patients with altered levels of consciousness in the ED.

Section snippets

Study Design and Setting

This prospective, observational study examined the interrater reliability of the GCS, its 3 components, and the SMS, AVPU, and ACDU scales when measured by 2 attending-level physicians within 5 minutes in adults (17 years of age or older) presenting to our ED with altered levels of consciousness. Enrollment took place during a 14-month study period in our ED, which is a Level I trauma center with a census of 69,000 patients per year. Convenience sampling took place during an even distribution

Results

Enrollment was initiated in 126 patients by 20 emergency physicians, with 6 patients being excluded from the final analysis because of lack of complete data recorded on their prospective data form. We analyzed the data from the remaining 120 patients (52% men, 48% women), from whom we obtained a total of 240 paired readings. The median subject age was 58 years (range 18 to 89 years) (Table 2).

The 3 scales with the best interrater agreement in terms of percent agreements and κ values were the

Limitations

Because we chose not to exclude intubated patients from the study, it is possible that the interrater agreement of all scores were inflated. Most intubated patients had been given hypnotic and paralytic drugs as part of the rapid sequence intubation protocol, and therefore their scores could have been artificially and uniformly GCS=3, AVPU=0, ACDU=0, and SMS=0. Because there is no universal agreement about the assignment of neurologic scores to intubated patients, we chose not to instruct our

Discussion

In this study, we found that the SMS (2=obeys commands, 1=localizes pain, 0= withdrawal to pain or less response) outperformed other proposed simplified neurologic scales when tested prospectively for interrater reliability in ED patients presenting with altered mental status of nontraumatic (76%) and traumatic (23%) causes. For purposes of comparison, we also tested interrater reliability of the components of the GCS and total GCS score for each patient, and although the GCS components

References (14)

There are more references available in the full text version of this article.

Cited by (79)

  • Predictive value and interrater reliability of mental status and mobility assessment in the emergency department

    2024, Clinical Medicine, Journal of the Royal College of Physicians of London
  • Intra- and inter-observer reproducibility of shoulder laxity tests: Comparison of the drawer, modified drawer and load and shift tests

    2018, Journal of Orthopaedic Science
    Citation Excerpt :

    Utilizing the modified test with the modified Hawkins scale improved the reliability of shoulder laxity examination, but the obtained kappa values were less than 0.5, which indicates that the results are no better than those achieved by chance alone [6]. Nevertheless, the kappa values of other clinical measures such as the Glasgow coma scale, Babinski sign and digital rectal examination for prostate cancer are all less than 0.5 and unsatisfactory [25–27], which are similar to our results of shoulder laxity testing. Yet, they have been established as a diagnostic tool in clinical practice by combining with other methods and history taking.

View all citing articles on Scopus

Supervising editor: E. John Gallagher, MD

Author contributions: MG and SG conceived of the study idea. MG, KM, ELL, and AS compiled the study data. MG and SG performed the data analyses, and all authors contributed to the writing and revision of the paper. MG takes responsibility for the paper as a whole.

Funding and support: The authors report this study did not receive any outside funding or support.

Available online June 6, 2006.

Reprints not available from the authors.

View full text