A pilot study to assess oral health literacy by comparing a word recognition and comprehension tool

The American Dental Association (ADA) defines oral health literacy as “the degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate oral health decisions” [1]. More health literate patients are better able to navigate the healthcare systems and have better health outcomes [2]. Those with low levels of health literacy are less likely to understand and follow treatment recommendations and lack the skills needed to make informed decisions about their own health care [3, 4]. Findings from the 2003 National Assessment of Adult Literacy (NAAL) showed that 71% of adults older than age 60 had difficulty in understanding print materials in prose format. This group also had the highest proportion of persons with health literacy defined as “below basic”, i.e. having no more than the most simple and concrete literacy skills such as signing forms, or locating easily identifiable information in a short, simple text [5].

Less is known about how health literacy impacts health-related outcomes for men and women. The NAAL in 2003 also reported that women in the United States had slightly higher overall levels of health literacy compared to men, and that a higher percentage of men than women had Below Basic Levels of Health Literacy.

While there is considerable literature on health literacy in medicine, health literacy in dentistry is a relatively new area of research. Several studies have shown that oral health literacy is associated with adults’ oral health status [6‐8] as well as their children’s oral health [9‐12]. Investigators suggest that oral health literacy may contribute to oral health disparities since those with low oral health literacy are more likely to be poor, not well educated, older and with limited English language skills [13]. Others suggest that those with low literacy are unable to communicate effectively with health care providers and this gap in communication may account for their worse oral health status [14, 15].

Dentists often encounter patients with limited oral health literacy skills in clinical practice, but they are not always able to identify those that may not be able to readily understand health explanations and instructions resulting in poor oral health outcomes. There is much need for quick, easy-to use oral health literacy tools that will allow for a comfortable experience for both providers and patients in identifying those that may need special methods of communication in clinical settings [16].

In dentistry, the most commonly used comprehension-based methods of assessing oral health literacy include the Test of Functional Health Literacy in Adults (TOHFLA) [17], the Test of Functional Health Literacy in Dentistry (TOHFLiD) [18] or the Oral Health Literacy Instrument (OHLI) [19]. These instruments are comprehension and numeracy tests which measure several aspects of health literacy, but take between 20–30 minutes to administer and are not suitable for use in a fast paced clinical care setting [17, 20].

Word-recognition tests such as the Rapid Estimate of Adult Literacy in Dentistry (REALD-30) have been developed to assess ability to read and pronounce common dental words correctly [21]. Thirty dental words are arranged in order of increasing difficulty. Richmond et al. developed a 99-item REALD with better predictive validity but the time taken to administer is much more than its increased predictive validity [20]. Therefore the 30-item REALD is more user-friendly and faster to use in a clinical setting.

While the REALD-30 is brief and easy to administer, it only measures recognition and pronunciation and does not assess understanding or comprehension. A recent review of health literacy instruments [22] found that the comprehension measures, such as the Medical Test of Functional Health Literacy in Adults (TOHFLA), were correlated with word-recognition tests such as the Rapid Estimate of Adult Literacy in Medicine (REALM) demonstrating that recognition is related to comprehension [17]. However, few studies have investigated the association between word recognition and comprehension in oral health literacy. Richman et al. [23] conducted a study of 44 parents of Head Start children to investigate how word recognition, vocabulary knowledge and comprehension of 35 pediatric dentistry terms were related. The authors found that word recognition was weakly correlated with comprehension and concluded that word recognition instruments are not a good indicator of oral health literacy. However, this study was limited by small sample size and its sole focus on pediatric dentistry. The REALD-30 has been used primarily in low income settings among young parents and has not been used extensively in other settings or with other populations. Very limited work has been done with oral health literacy among older adults. A recent study assessed the effectiveness of an educational intervention on oral health literacy and oral hygiene among older adults in assisted living complex. Oral health literacy was assessed using the REALD 30 and oral hygiene was assessed by the plaque index. REALD 30 scores were relatively high with mean scores at baseline of 23.21 (SD = 4.42) which improved significantly to 25.55 (SD = 4.6; p < 0.001). Oral hygiene also improved significantly, but was not related to improvements in the REALD 30 [24].

The primary objective of this study was to pilot test three methods to measure oral health literacy among older adults using the Rapid Estimate of Literacy in Dentistry (REALD-30) tool. We assessed word recognition and enhanced the REALD-30 tool by adding a comprehension component. We also assessed comprehension of a brochure about dry mouth. We compared these methods by investigating the associations among these methods and whether demographic characteristics differentially influence the oral health literacy assessments. We also aimed to assess the relationship between these health literacy tests and gender and oral health status and behaviors.

We propose that participants will obtain higher scores on the REALD-30 word recognition than on word comprehension and that the literacy test scores will be significantly correlated to each other. We also hypothesized that females, those that are younger, more educated and with more incomes will have higher literacy scores and that higher literacy scores would be associated with better oral health behaviors.

As shown in Table 2, a total of 150 participants comprising 75 males and 75 females participated in the study. The majority of the participants were patients at the Advanced Education in General Dentistry practice (60 percent) while the others were spouses, caregivers or friends accompanying the patients. About half of the participants were less than 60, although 20.6% were over the age of 70. Most participants had some college (40%) or had completed college (33.3%); most were white (80%), married or living with someone (43%), and had family incomes of less than $50,000 (64%). Participants rated their oral health worse than their overall general health. Those who rated oral health as fair/poor (41.3%) was twice as high than those who rated their overall general health as fair/poor (20.7%). Most participants reported they had visited the dentist in the past 6 months (71%). Most said they brushed at least twice a day (70%) and flossed at least once a day (45%) and few smoked (20%). Many had some sort of dental insurance in the form of either private insurance or Medicaid.

Table 3 displays the word list and the percent correct on recognition and comprehension. Sugar, Smoking and Brush were the only three words that all participants could both pronounce and define correctly, although almost all participants could pronounce and define Floss, Braces, Extraction and Denture. In contrast, Hypoplasia, Fistula, Temporomandibular, Hyperemia and Apicoectonomy were more difficult words with 50 percent or less of the participants correctly pronouncing these words. Less than 25 percent of the participants could correctly define Bruxism, Dentition, Gingiva, Incipient, Hypoplasia, Fistula, Temporomandibular, Hyperemia and Apicoectonomy. Some of the limitations in the word recognition approach to literacy assessment are apparent in Table 3 which illustrates that participants could pronounce words but could not define them. These included, for example, caries, periodontal and sealant. Internal reliability for both scales was very positive. Chronbach’s alpha for the REALD-30 WR was 0.902 and for REALD-30 COMP alpha was 0.851.

Figure 1 shows the distribution of the REALD-Word Recognition (REALD-WR) scores which ranged from 3–30 with a mean of 22.98 (SD = 5.1). Figure 2 presents the distribution of the REALD-Comprehension (REALD-COMP) with a range of 7–26 and mean of 16.1 (SD = 4.3). It is also apparent from the two figures that the distribution of the scores is lower for comprehension. Participants scored significantly higher on word recognition compared to comprehension (p < 0.001).The data provide support for the first hypothesis that word recognition scores are higher than comprehension scores. Results for the dry mouth brochure are shown in Table 4. Most people could define dry mouth after reading the brochure. Most participants also could list at least one cause and one treatment of dry mouth, although participants tended to have higher scores on reporting treatments for dry mouth (mean = 2.3; SD = 0.86) compared to causes of dry mouth (mean =1.9; SD = 0.90). Overall, scores for the dry mouth brochure comprehension, which could range from 0 to 7, were relatively high, with a mean score of 5.1 (SD = 1.60) and median of 5 for the sample.

Tables 5 and 6 show the bivariate analyses of participant characteristics and literacy assessment. Scores for REALD-30 word recognition and comprehension were divided into quartiles. Quartiles for REALD-30 recognition were 1–19, 20–24, 25–27 and 28–30; for REALD- 30 comprehension, quartiles were 0–13, 14–15, 16–19 and 20–30. Gender, education, race and tooth brushing frequency were significantly associated with scores on both recognition and comprehension on the REALD-30. Males, those with high school or less education, non-whites and those who brushed less than once per day more likely to have low scores on both measures compared to females, those with more education, whites and those who brushed more frequently. For dry mouth brochure comprehension scores shown in Table 7, gender, education, marital status and income were significantly related to older scores. Males, those with less than a high school education, non-white, and those with an income of $20,000 were more likely to have lower dry mouth comprehension scores. Pearson correlations among the three measures are presented in Table 8. REALD-30 recognition and comprehension are significantly and highly correlated (r = 0.779; p <0.01). Comprehension of the brochure also is significantly correlated with scores on both REALD-30 measures.

The multivariate regression analyses for REALD-30 recognition, REALD-30 comprehension and the dry mouth brochure are shown in Table 9. All three regression models are highly significant (p < 0.001). Three variables were significantly related to the REALD-30 measures: females, those with higher educations and whites had higher scores on both measures compared to males, those with less than a high school education and non-whites. It is noteworthy that for REALD-30 comprehension analysis, gender had a higher significance level and larger regression coefficient than for recognition, suggesting that gender may be more important for comprehension in this sample. Results of the multivariate analysis of the dry mouth scores were similar to the results for REALD-30 except that race did not have a significant effect on comprehensions.

This study builds on the previous work of Lee and colleagues [21] developing a brief measure of oral health literacy based on word recognition by adding a comprehension component to the REALD-30. As with other studies that use the REALD-30, our participants had relatively high scores (mean 22.98; SD = 5.1) on word recognition and the distribution was skewed towards higher scores. The distribution on the comprehension scores was less skewed than the word recognition scores and the sample had somewhat lower scores (mean =16.05; SD = 4.3). REALD-30 comprehension may be a better tool to differentiate among literacy levels. The dry mouth brochure scores also were high with a highly skewed distribution. This result may be due to the 5th grade reading level of the brochure making it more accessible to those with lower literacy ability.

Several authors [23, 27] have discussed the limitations of the word recognition approach to literacy assessment because word recognition does not necessarily indicate that the person understands the meaning of the word. The findings from this study support this assertion in that we found that scores on the REALD word recognition measure were higher than word comprehension measure indicating that participants were able to pronounce more words than they could define. Often participants correctly pronounced words for which they did not know dental meanings such as pulp which they often associated with orange juice pulp or gingiva which many defined as gum disease because of their familiarity with gingivitis. That being said, REALD-30 scores on recognition and comprehension were highly correlated suggesting that REALD-30 word recognition may be useful as a screening measure of oral health literacy when time is limited in the clinical context or as an initial assessment for further follow-up. REALD-30 word-recognition and comprehension scores were also correlated with measures of comprehension of the dry mouth brochure lending additional evidence for the validity of the REALD-30 measures as oral health literacy assessment tools.

A secondary focus of this paper was to assess gender differences in oral health literacy. The effect of gender was highly significant at the bivariate and multivariate levels with females demonstrating greater oral health literacy on all measures. The effects of gender on oral health literacy has not been investigated extensively because much of the development of the REALD-30 has been done with young females [10, 11, 28] without emphasis on the analysis of gender. An early study of oral health literacy among dental patients in private practice (n = 101) which included both males and females did not find gender differences in the REALD-30 word recognition scores [29]. The participants in our study consisted of older adults (ages 50 and older) who were dental patients at a University dental practice. Oral health literacy in our sample was similar to Jones et al’s [29] sample in the dental practice which had a mean score on the REALD-30 of 23.9 (SD = 1.3). Our scores also similar to Hjertstedt et al., [24] who reported baseline scores on the REALD-30 of 23.21 (SD = 4.42) although their sample was relatively small (n = 67). Our participants had considerably higher mean scores on word recognition using the REALD-30 compared to samples of young, low income mothers (mean score of 15.8; SD = 5.1; [11]).

The bivariate and multivariate analyses provide limited support for our expectations about the REALD 30 and participant characteristrics. As expected, education was consistently related to health literacy measures. Race was significantly related to word recognition and comprehension using the REALD-30 but not to literacy assessed by the dry mouth brochure. Contrary to our expectations, self –reported oral health status was not significantly related to any of the three literacy assessments although tooth brushing was related to word recognition at the bivariate level.

There are two words in the REALD-30 floss and extraction that have higher scores on REALD-30 comprehension that word-recognition. This occurred because some adults had difficulty pronouncing certain words but could still define that same word correctly. Hence the participant received a point for comprehension but not recognition. It is possible that the participant could read the word but could not pronounce it because of language difficulties. This is another example of where word-recognition itself is limited because it measures correct pronunciation and not understanding of the word.

Our study also demonstrated that the dry mouth brochure is a useful and practical method of measuring oral health literacy. Participants found it easy and quick to read, taking an average of about 3 minutes to complete. Participants’ comprehension scores on the brochure were quite high suggesting that this tool is accessible to those with lower literacy. Overall we suggest that the dry mouth brochure is a comprehensive, multi-faceted approach to measuring literacy because it allows patients to read the brochure, synthesize the information they read and then recall it during questions asked of them. Use of such brochures to measure health literacy could have many uses in clinical practice such as in patients with high caries risk during oral hygiene instruction.

Our results are somewhat consistent with other studies in finding a relatively high level of oral health literacy measured by word recognition among adult dental patients. As with other studies, we found significant associations between oral health literacy and education, but did not find a relationship between oral health literacy, self-rated oral health or oral health behaviors. Our findings may be related to our focus on older adults or on dental patients who have been treated at a university dental practice. However, our study demonstrates the feasibility and acceptability of using the REALD-30 (both word-recognition and comprehension) in dental treatment setting among older adults.

In this study we developed two oral health literacy comprehension tools, the REALD-30 comprehension and Dry Mouth Brochure comprehension, both of which correlated with the REALD-30 word-recognition test previously developed by Lee et al. [21]. Our study shows that using brief comprehension tests along with word-recognition tests enhance oral health literacy measurements, as adults may recognize words but may not actually understand them. Such literacy tests have use both in clinical practice and research settings. However further development of these comprehension tools is needed to make them more effective for use in clinical practice.