Elsevier

Journal of Voice

Volume 21, Issue 1, January 2007, Pages 92-100
Journal of Voice

Reliability of Speech-Language Pathologist and Otolaryngologist Ratings of Laryngeal Signs of Reflux in an Asymptomatic Population Using the Reflux Finding Score

https://doi.org/10.1016/j.jvoice.2005.09.004Get rights and content

Summary

Objective: To determine inter- and intrajudge agreement in rating signs of laryngopharyngeal reflux (LPR) under “ideal” conditions: Experienced coworkers in a practice devoted to voice-disordered patients, raters trained in the items on a standardized scale, raters from both speech-language pathology (SLP) and otolaryngology, and raters of asymptomatic participants. Study Design: Prospective study using a scale to rate videolaryngoscopic examinations. Methods: Two SLPs and two otolaryngologists used the Reflux Finding Scale (RFS) to independently rate videotapes of endoscopic examinations for 30 participants asymptomatic of reflux. Results: Thirteen (43%) were assigned a total score >7, indicative of LPR, by at least one rater. Intraclass correlation coefficients showed a significant lack of agreement in total scores provided by the otolaryngologists and by all raters combined. One otolaryngologist and the two SLPs demonstrated good interrater agreement in total scores. McNamar's statistic and Poisson regression modeling showed differences in rater agreement for many individual items. Repeated ratings of four participants showed no significant differences, indicating good intrarater reliability. Conclusions: Level of rater agreement regarding the presence and the severity of physical findings attributed to LPR within and between otolaryngologists and SLPs differed. Given the role each profession plays in the diagnosis and treatment of LPR and related voice disturbances, higher levels of interprofessional agreement are desired. Results support the need for greater consensus among professionals regarding the discreet features of physical findings associated with LPR, a fuller understanding of normal variants, and greater emphasis on interrater reliability when rating physical findings.

Introduction

The term “laryngopharyngeal reflux” (LPR) refers to a backward flow of stomach contents to the pharynx and is differentiated from gastroesophageal reflux disease (GERD) by its complexity of symptoms, presumed pathophysiology, and physical sequelae.1 The position statement of the American Academy of Otolaryngology-Head and Neck Surgery1 reviewed primary literature on this topic. The Committee found that the most common clinical manifestation of LPR is reflux laryngitis, with or without granuloma formation. Additional laryngeal conditions frequently associated with acid irritation include posterior commissure hypertrophy, subglottic stenosis, laryngeal carcinoma, polypoid degeneration, laryngospasm, paradoxical vocal fold movement, globus pharyngeus, and vocal nodules.

Most patients with LPR deny classic symptoms of GERD, particularly heartburn.2, 3 Patients with LPR may describe symptoms such as hoarseness, vocal fatigue, chronic cough, dysphagia, globus sensation, halitosis, excessive mucus, postnasal drip, and throat clearing.1, 4, 5 Although the exact mechanisms for LPR have not been conclusively described, it has been associated with the upright position and a possible dysfunction of the upper esophageal sphincter, whereas GERD tends to occur in the supine position and involves a dysfunction of the lower esophageal sphincter. Additionally, patients with GERD have longer periods of acid exposure and dysmotility.1 Differentiating the exact mechanisms of each condition is difficult as LPR and GERD may co-occur.

Definitive diagnosis of LPR is difficult to determine. Double-probe pH monitoring has been considered the gold standard,1, 2, 6, 7 but patient resistance, expense, and difficulty quantifying intermittent episodes have led many otolaryngologists to base initial diagnosis on physical findings during endoscopic examination. Unfortunately, there is disagreement among otolaryngologists regarding the laryngeal findings that constitute this diagnosis.8 Evidence of tissue irritation, particularly near the upper esophageal sphincter, may include edema and erythema of the arytenoid joints, vocal folds and posterior larynx, posterior commissure hypertrophy, and ulcerations or granulomatous masses, including cobblestoning around the esophageal inlet.4, 9, 10 However, some of these signs have been reported in the asymptomatic, non-treatment-seeking population.9, 11, 12

An additional factor confounding the accurate diagnosis of LPR is disagreement among otolaryngologists regarding the presence and severity of laryngeal signs. Many studies provide ratings made by an otolaryngologist alone or in consensus with a speech-language pathologist. However, Branski et al13 had five otolaryngologists rate 122 videotaped laryngeal examinations of patients with dysphonia on a five-point scale of classic signs of LPR (edema and erythema at the anterior commissure, along the musculomembranous vocal folds, and on the artyenoids/interarytenoid space, and interarytenoid pachydermia). The severity of LPR findings and likelihood that it is a component of the patient's dysphonia were also rated. Interrater agreement was poor, and intrarater reliability was highly variable. The authors concluded that accurate assessment of LPR based on laryngeal findings could not be made dependably.

The difficulties inherent in subjective ratings are well known, but they become unacceptable if they lead to false diagnoses. We questioned interjudge agreement in rating signs of LPR would improve if “ideal” conditions were created. That is, would interrater agreement improve if the raters had many years of experience working together in a practice devoted exclusively to voice-disordered patients, if raters were trained in the items on a standardized scale developed to represent laryngeal signs of reflux,14, 15 if raters included both speech-language pathologists and otolaryngologists affiliated with the practice, and if they rated only asymptomatic participants? The current investigation was designed to answer the components of this question. Given the reduced range of physical findings or variations thereof anticipated in an asymptomatic population, strong inter- and intrarater agreement was expected.

Section snippets

Participants

Participants were 30 nonsmoking male and female adults (15 men, 15 women), ages 35–55 years old. They were recruited from the general, non-treatment-seeking population. Prerequisites included not more than two alcoholic drinks per day, no history of swallowing, voice or pulmonary disorders, and self-described overall health quality of good or better. Each participant had no more than rare-to-intermittent use of ibuprofen, antihistamines, and steroids. There was no history of using prescription

Total RFS

Each rater made a total of 240 ratings (30 participants × 8 items). The means, standard deviations, and ranges of the total RFS scores provided by the two speech-language pathologists and two otolaryngologists are provided in Table 1.14 The frequency of rating a participant with a total RFS score greater than 7 is also provided. The mean values were within normal limits, as defined by Belafsky et al.14 However, 43% of the participants (13 of 30) were assigned a total score >7, indicative of

Discussion

Findings from the current study underscore key issues that persist in the clinical identification and diagnosis of LPR from physical findings in persons who otherwise do not report symptoms of reflux disease. These issues include interrater reliability or consensus among professionals who make provisional diagnoses of LPR based on physical findings, lack of characterization of normal variations of the physical appearance of the larynx, and over/under-diagnosis of LPR in asymptomatic patients.

Conclusions

In the current study, healthy, asymptomatic, non–treatment-seeking persons with low RFS scores received videoendscoboscopic examination. Experienced otolaryngologists and speech pathologists rated the physical appearance of those videoendoscopic images using the RFS scoring system. Results revealed that even under ideal practice conditions where close, ongoing collaboration is standard care, variability of interpretation of presence, degree, or absence of physical findings exists. The extent of

References (19)

There are more references available in the full text version of this article.

Cited by (41)

  • Laryngopharyngeal Reflux and Atypical Gastroesophageal Reflux Disease

    2020, Gastrointestinal Endoscopy Clinics of North America
  • The Reliability of the Reflux Finding Score Among General Otolaryngologists

    2015, Journal of Voice
    Citation Excerpt :

    In contrast, Kelchner et al16 found high inter-rater agreement between two speech-language pathologists and one laryngologist for total RFSs. However, the second otolaryngologist in their study had poor agreement with the other three.16 In studies examining signs of laryngeal irritation,8,9 the inter-rater reliability is poor indicating that there is lack of consensus with the respect to endoscopic laryngeal findings.

  • Comparison of voice quality in patients with GERD-related dysphonia or chronic cough

    2014, Otolaryngologia Polska
    Citation Excerpt :

    Breathiness is reflected in the shimmer value [36]. Hoarseness results in increased jitter, shimmer, and HNR [28, 29]. Elevated levels of jitter and shimmer might be a result of reduced control of laryngeal phonation and degenerative tissue changes [36, 37].

View all citing articles on Scopus
View full text