Skip to main content

01.12.2017 | Research | Ausgabe 1/2017 Open Access

Perioperative Medicine 1/2017

Are physician assistant and patient airway assessments reliable compared to anesthesiologist assessments in detecting difficult airways in general surgical patients?

Perioperative Medicine > Ausgabe 1/2017
Erin Payne, Jacqueline Ragheb, Elizabeth S. Jewell, Betsy P. Huang, Angela M. Bailey, Laura M. Fritsch, Milo Engoren
Physician assistant


Airway management remains one of the most important responsibilities of anesthesiologists (2003). Prediction of difficult airway allows time for proper selection of equipment, technique, and personnel experienced in managing patients with difficult airway. It is important to identify this group of patients as unanticipated difficult airway may result in catastrophic outcomes such as brain damage or death (Cook et al. 2011). Closed claim analysis has found that the vast majority (85%) of airway-related events involve brain damage or death (Cook et al. 2010).
In major multi-site surgical facilities, expertise and equipment to accommodate patients who have difficult airway may not be available on all surgical sites. This may lead to procedure cancelations or deferrals on the morning of surgery. Late deferrals are a major cause of inefficient use of operating-room time and waste of resources and are unsatisfactory for both patients and surgical teams (Kumar and Gandhi 2012; Garg et al. 2009).
Although anesthesia preoperative assessment clinics are widely available in different institutions to identify preoperative morbidities and optimize patients for surgery, this is mostly conducted by screening patients’ medical records and phone interviews. Face to face preoperative anesthesia interviews are difficult to conduct as they necessitate patients traveling to the clinics, and, in practice, are usually conducted in the morning of the procedure by the anesthesiologist.
Patient self-assessment tools and questionnaires have been used to obtain an overall health assessment, to explore the effects of any medical problems on the everyday activity of the patients, to identify needs of medical interventions, and to guide the necessity for elective and emergency hospital admission (Wasson et al. 1999; Miyamichi et al. 2012). These assessment tools, used by patients and validated by clinicians, have been shown to identify medical conditions for treatment and reduce overcrowding of health facilities, including unnecessary clinic visits and hospital admissions.
We hypothesized that a patient assessment tool used by the patient or by a physician assistant would be useful in identifying patients with difficult airway and hence ensuring their allocation and management to a well-equipped and staffed facility. Validation of such a tool will facilitate airway examination in the preoperative setting. This will minimize same-day cancelation or deferral in surgical facilities lacking tools and equipment to deal with this patient population.


After consultation with a psychometrician, we designed a patient assessment tool (Fig. 1) with illustrations of the major difficult airway predictors and key questions to identify patients who may have a difficult airway (Sim and Wright 2005; L’Hermite et al. 2009). The tool was then shown to other anesthesiologists, who provided feedback, and the tool was then adjusted to meet their suggestions. Next, the tool was shown to non-medical laypersons for their suggestions and anesthesiologist observation of their performance using the form. Additional changes were made. This revised tool was then shown to another group of non-medical laypersons, who appeared to have good performance of an airway self-assessment and which elicited no further suggestions.
After Institutional Review Board approval and subject informed consent, a non-consecutive sample of 215 patients presenting to the perioperative clinic for history and physical examination between July 2015 and May 2016 was asked to complete the airway tool (Fig. 1) during their visit. They were chosen based on the availability of the consenters, PAs, and anesthesiologists. A subsequent independent assessment using the same tool was performed by four clinic physician assistants (PA) and separately by an anesthesiologist. The PAs were selected based on willingness to participate and provided informed consent. Each PA had various levels of experience working in our clinic, but each was given training on airway examination by an anesthesiologist prior to this study as they do not routinely include an airway exam in their surgical history and physical examination.
Adult English-speaking patients undergoing non-cardiac surgical procedures who presented to the surgical clinic on the days one of the study’s authors (EP, JR) was present were approached for consent. Patients were excluded if they were < 18 years of age, do not speak English, or were unable to consent due to guardianship. The tool and written instructions (Fig. 1) were given to the patients by the non-medical research assistant obtaining consent. The study was conducted using STROBE criteria for cohort studies (http://​www.​strobe-statement.​org/​index.​php?​id=​available-checklists); accessed November 6, 2017.

A priori power analysis

Assuming that the average proportion of positive ratings on a dichotomous question is 0.7, that the raters are unbiased, that the two-tailed null value is 0.5, and that the pairwise kappa we wish to detect is 0.7, we would need 173 subjects (Sim and Wright 2005). To correct for possible systemic biases, we planned to recruit 215 subjects.

Statistical analysis

Patients’ demographics were described using frequency and proportion for categorical data and means and standard deviations for continuous data. A comparison between the results of the three assessments made by patient, PA, and anesthesiologist was conducted on each of the five airway measures, using kappa: pairwise (Cohen) comparisons (PA-patient, PA-anesthesiologist, patient-anesthesiologist). We also computed a group (Fleiss) kappa for the three-way agreement. kappa ≥ 0.7 was considered good agreement.


Of the 215 patients, 122 (57%) were female. Most (n = 102, 47%) had a high school education, and 68 (32%) had college education, while only 45 (21%) had less than a high school education. Patients were (mean ± standard deviation) 59 ± 15 years old and weighed 86 ± 50 kg, with a maximum weight of 185 kg.
We found good agreement between observers only on question 2 “can you put three fingers in your mouth?” and poor agreement on question 1—Mallampati classification—and question 4 “Can you fit three fingers between your chin and your Adam’s Apple?”. The agreements for the other questions were mostly fair (Table 1).
Table 1
Agreement (kappa) between the groups
<  .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
< .001
5 Down
< .001
< .001
< .001
< .001
5 Up
< .001
< .001
< .001
< .001
5 Tilt
< .001
< .001
5 Left
< .001
< .001
< .001
< .001
5 Right
< .001
< .001
< .001
< .001
Item 1 is Mallampati classification, 2 is “Can you fit three fingers in your mouth when fully opened?,” 3 is “Can you place your lower teeth in front of your upper teeth?,” 4 is “Can you fit three fingers between your chin and your Adam’s apple?,” and 5 is neck motion and position (Fig. 1). P is the probability that kappa differs from zero by chance alone
Pat patient, PA physician assistant, Anes anesthesiologist
Notably, we also found that the agreements between the patients’ assessments and the anesthesiologists’ assessments were very similar to the agreements between the PA’s assessments and the anesthesiologists’ assessments. While patients and anesthesiologists agreed on the Mallampati class half the time (50%), there was no consistent bias between grader and different scores. Patients were as likely to overgrade (n = 47) as undergrade (n = 53), p = .564. Similarly, PA’s were as likely to overgrade (n = 50) as undergrade (n = 61) compared to anesthesiologists, p = .264. Of the patients whom the anesthesiologist graded the airway as Mallampati IV, patients only agreed 45% of the time and PAs only 34% (Fig. 2).


In this study, we evaluated the interrater agreement using an airway assessment tool between patients, PAs, and anesthesiologists in a preoperative clinic setting. We hypothesized that using this airway assessment tool (Fig. 1), patients would be able to reliably and accurately assess their airway compared to the anesthesiologist and thus would be useful in identifying patients with difficult airways and ensuring that this patient population is managed in the appropriate facility with sufficient resources. We included all risk predictors for a difficult airway that are routinely used and documented at the University of Michigan for an airway exam in the design of our assessment tool: Mallampati classification, inter-incisor gap, thyromental distance, and cervical flexion and extension (L’Hermite et al. 2009; Khan et al. 2009).
The Joint Commission’s standards for a hospital of surgery center to be accredited require a pre-procedure airway assessment; however, it does not specify the required elements of the assessment (https://​www.​jointcommission.​org/​standards_​information/​jcfaqdetails.​aspx?​StandardsFaqId=​869&​Programid=​46; accessed November 6, 2017). As the Joint Commission also has a “comparable care” mandate, many healthcare facilities have interpreted these standards to require either the non-anesthesiologist performing the procedure with conscious sedation or a designee, usually the procedure room registered nurse, to perform an airway assessment similar to that performed by an anesthesiologist (https://​www.​patientsafety.​va.​gov/​professionals/​onthejob/​sedation.​asp; accessed November 6, 2017). Guidelines for the performance of the airway assessment by non-anesthesiologists have been promulgated by the American Society of Anesthesiologists (American Society of Anesthesiologists Task Force on Sedation and Analgesia by Non-Anesthesiologists 2002). However, there are scant studies determining how well non-anesthesiologists perform components of an airway assessment. Kandray et al. evaluating Mallapati classification found that one dentist and 21 dental hygiene students examining a sample of 234 patients agreed 77% (95% confidence interval = 72, 82%) of the time with kappa = 0.54 (0.42, 0.64), which is higher than the kappas found for Mallampati classification (Kandray et al. 2013). A study at a university medical center found that the agreement on Mallampati classification between gastroenterologists and anesthesiologists was no better than chance: kappa = 0.103 (− 0.0126, 0.219), when gastroenterologists were compared to other gastroenterologists, the agreement was similarly poor: kappa = 0.120 (− 0.0211 to 0.260), and when compared among themselves doing the examination twice, gastroenterolgists had only moderate agreement between their first and second exam of the same patient: kappa = 0.420 (0.119, 0.722) (Lopez et al. 2014). Even the agreement among anesthesiologists is variable. A Danish study calculated kappa between two anesthesiology specialists and two anesthesiology residents on measures of an airway assessment consisting of the measurement of the mouth opening, the thyromental distance, the ability to protrude the mandible, and an evaluation of the Mallampati class and head and neck mobility in 136 patients scheduled for elective surgery. The kappa for residents was 0.28 (0.05, 0.61) and specialists was 0.39 (0.06, 0.72) for neck mobility and 0.41 (0.30, 0.52) and 0.80 (0.65, 0.95), respectively, for Mallampati classification (Rosenstock et al. 2005). Our findings are in agreement with these studies, but are novel in assessing patient self-assessment and including more components of an airway exam.
Our results showed that for most components of the airway assessment, there is less than good agreement between patients and experts, making self-assessment unreliable to identify potential difficult airways. In particular, patients were only able to identify a Mallampati IV airway, as rated by an anesthesiologist, 45% of the time (Fig. 2). These findings suggest that this tool cannot be used by patients to evaluate their airways. As it may be burdensome for some patients to physically visit an anesthesiologist for the important airway assessment, other less burdensome methods should be investigated, e.g., cellular phone videoconferencing may permit a remote anesthesiologist to conduct the airway assessment, but would need to be first studied.
Operationally, the direction of disagreement between patient and anesthesiologist or PA and anesthesiologist may be important. A patient or PA who scores a low Mallampati score and normal mouth opening, thyromental distance, and cervical mobility but has a Mallampati IV airway and decreased mouth opening, thyromental distance, and cervical mobility as assessed by the anesthesiologist and hence requires an unanticipated awake fiber-optic intubation may disrupt the scheduling and workflow at the facility or lead to a postponed case. Conversely, if the patient or PA assesses Mallampati IV and other markers of a difficult intubation and the anesthesiologist disagrees, there may be no disruption to the facility—only the fiber-optic scope is put away unused. Importantly, we found no pattern between over- and under-scoring the components of the airway assessment.
We also found disappointingly similar poor agreement with the same difficult airway predictors comparing PAs and anesthesiologists. This could reflect the difference in training or experience among providers. It could also reflect lack of standardization of airway assessment descriptors, e.g., two raters may see the identical view, but not have a standard definition of finger size in rating inter-incisor or thyromental distance, leading to different ratings. This has implications because many mid-level providers are often tasked with airway assessment in various clinical settings (https://​www.​patientsafety.​va.​gov/​professionals/​onthejob/​sedation.​asp; accessed November 6, 2017). This can lead to inappropriate patient allocation to centers with insufficient resources in managing difficult airways or the patient inappropriately receiving conscious sedation by the surgeon instead of monitored anesthesia care by the anesthesiologist. Given the importance of non-anesthesiologists performing airway assessments, research is needed on how best to improve their airway assessments.
In contrast to other studies demonstrating interrater variability in airway assessment, our study is the first to include patients using a self-assessment tool (Kandray et al. 2013; Rosenstock et al. 2005). The main strength of our study is the design and validation of the tool with a psychometrician and subsequent testing as well as revision based on feedback from both anesthesiologists and non-medical laypersons. There are several limitations to our study. First, we did not determine the agreement between the two anesthesiologists in this study. Disagreements between anesthesiologists may suggest that the airway assessment is not sufficiently standardized, either in the definitions or in the performance. We did not assess which provider (PA or anesthesiologist) or patient best assessed the airway for a difficult intubation as we did not follow patients through to intubation. Another limitation is the limited amount of training and experience given to the PAs prior to this study. Patients came from a small geographic area with a relatively homogenous population and our results may not be generalizable to different patient populations. Finally, we did not debrief the patients after the self-conducted airway assessment to determine the reasons why patients ascribed a particular value to each component of the assessment.


We found that agreements between patients and anesthesiologists on different components of an airway examination vary by component but the overall is poor to good. We also found that the agreements between PAs and anesthesiologists on the same airway examination also vary by component but the overall is poor to good. This suggests that neither the patient’s self-assessment nor the PA’s assessment of an airway can reliably and accurately replace the anesthesiologist’s examination.


We thank Yiyi Zhang (University of Michigan, Penny W. Stamps School of Art & Design) and Robert K Fraumann, MD, and JD for the artwork and photographs.


This was funded solely from departmental resources. The funding body had no role in the design of the study, in the collection, analysis, and interpretation of data, and in the writing of the manuscript.

Availability of data and materials

The datasets during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate

This study was approved by the Institutional Review Board, which required participatory consent (HUM00096044).

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://​creativecommons.​org/​licenses/​by/​4.​0/​), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://​creativecommons.​org/​publicdomain/​zero/​1.​0/​) applies to the data made available in this article, unless otherwise stated.
Über diesen Artikel

Weitere Artikel der Ausgabe 1/2017

Perioperative Medicine 1/2017 Zur Ausgabe

Neu im Fachgebiet AINS

Meistgelesene Bücher aus dem Fachgebiet AINS

  • 2014 | Buch

    Komplikationen in der Anästhesie

    Fallbeispiele Analyse Prävention

    Aus Fehlern lernen und dadurch Zwischenfälle vermeiden! Komplikationen oder Zwischenfälle in der Anästhesie können für Patienten schwerwiegende Folgen haben. Häufig sind sie eine Kombination menschlicher, organisatorischer und technischer Fehler.

    Matthias Hübler, Thea Koch
  • 2013 | Buch

    Anästhesie Fragen und Antworten

    1655 Fakten für die Facharztprüfung und das Europäische Diplom für Anästhesiologie und Intensivmedizin (DESA)

    Mit Sicherheit erfolgreich in Prüfung und Praxis! Effektiv wiederholen und im entscheidenden Moment die richtigen Antworten parat haben - dafür ist dieses beliebte Prüfungsbuch garantiert hilfreich. Anhand der Multiple-Choice-Fragen ist die optimale Vorbereitung auf das Prüfungsprinzip der D.E.A.A. gewährleistet.

    Prof. Dr. Franz Kehl, Dr. Hans-Joachim Wilke
  • 2011 | Buch

    Pharmakotherapie in der Anästhesie und Intensivmedizin

    Wie und wieso wirken vasoaktive Substanzen und wie werden sie wirksam eingesetzt Welche Substanzen eignen sich zur perioperativen Myokardprojektion? 
    Kenntnisse zur Pharmakologie und deren Anwendung sind das notwendige Rüstzeug für den Anästhesisten und Intensivmediziner. Lernen Sie von erfahrenen Anästhesisten und Pharmakologen.

    Prof. Dr. Peter H. Tonner, Prof. Dr. Lutz Hein
  • 2013 | Buch

    Anästhesie und Intensivmedizin – Prüfungswissen

    für die Fachpflege

    Fit in Theorie, Praxis und Prüfung! In diesem Arbeitsbuch werden alle Fakten der Fachweiterbildung abgebildet. So können Fachweiterbildungsteilnehmer wie auch langjährige Mitarbeiter in der Anästhesie und Intensivmedizin ihr Wissen gezielt überprüfen, vertiefen und festigen.

    Prof. Dr. Reinhard Larsen

Mail Icon II Newsletter

Bestellen Sie unseren kostenlosen Newsletter Update AINS und bleiben Sie gut informiert – ganz bequem per eMail.