INTRODUCTION
Health literacy has been defined as “the ability to obtain, process, or understand basic health information needed to make appropriate health care decisions
”
1. Both inadequate (i.e. very low) and marginal health literacy (HL) appear to be important factors in the causal pathway to health disparities, especially in low income patients with chronic diseases
2‐
5. Given the high prevalence (46% of the US population) of inadequate (i.e. very low) plus marginal HL, often described as ‘limited HL’
6 and limited literacy’s association with poor health outcomes
3,
7‐
12, there has been great interest in including HL assessments in epidemiologic and clinical research
13. However, because standard HL measurements require face-to-face interviews
14‐
16, take from 3 to 20 minutes, and cannot be administered by phone, they are often not feasible in large epidemiologic and public health research.
Chew and colleagues developed three self-reported HL “screening” questions and found that a single item about “confidence with completing forms” with a response cut-point of “somewhat,” may be sufficient to detect patients with inadequate HL (C-index 0.74 (0.69-0.79)), sensitivity, 0.60; specificity, 0.82), but the items did not perform as well in patients with inadequate plus marginal HL (C-index 0.72 (0.69-0.76)
17,
18. Chew also found that a scale combining the three questions offered no additional benefit to the one question about confidence with forms. A recent review article endorsed the use of the ‘confidence with forms’ item to assess HL in clinical settings
19. However, these self-reported items have only been validated among largely homogeneous English-speaking populations
17,
18,
20. The performance of the self-reported HL questions within Spanish-speaking and ethnically diverse patient subgroups has not been assessed
19.
It is important to validate these three self-reported HL items both individually and as a scale among Spanish speakers, patients with low-income, and minorities because the prevalence of limited HL is highest among these groups
6,
21. HL and limited English proficiency have a complex relationship, adding to the importance of measuring HL in languages other than English
22. However, Spanish HL assessment currently requires face-to-face, multi-item, interviewer-administered assessments[
23. Therefore, we examined the performance of three self-reported HL questions individually and as a summative scale among English and Spanish-speaking, diverse, low-income, populations with type 2 diabetes. We further explored whether the self-reported questions performed equally well across language, race/ethnicity, educational attainment, age, gender, and health status subgroups.
RESULTS
Of 296 participants, 48% were Spanish-speaking, and only 9% were white, non-Hispanic (Table
1). Limited HL was prevalent: 47% had inadequate HL as measured by the sTOFHLA and 12% had marginal literacy. For the self-reported HL questions, 57% reported being confident with forms “somewhat” or less, 45% of participants reported problems learning “sometimes” or more frequently, and 42% reported needing help reading “sometimes” or more frequently.
Table 1
Patient Characteristics (N = 296)
Age, mean (SD) | 54.9 (12.1) |
Gender |
Male | 126 (42.6) |
Female | 170 (57.4) |
Race |
White, non-Hispanic | 25 (8.5) |
White, Hispanic | 156 (52.7) |
Black | 70 (23.7) |
Asian | 40 (13.5) |
Multiracial / Other | 5 (1.7) |
Language |
English | 154 (52%) |
Spanish | 142 (48%) |
Income <$20,000 | 215 (72.6) |
Education |
Less than high school | 149 (50.3) |
High school graduate/GED | 52 (17.6) |
More than high school | 95 (32.1) |
Fair or poor health status | 223 (75.3%) |
Health literacy level by s-TOFHLA |
Inadequate (score, 0-16) | 140 (47.3) |
Marginal (score, 17-22) | 34 (11.5) |
Adequate (score, 23-36) | 122 (41.2) |
Overall, participants who reported less confidence with forms (C-index 0.82, CI 0.77-0.87), more problems learning (C-index 0.72, CI 0.67-0.78), needing more help reading (C-index 0.68, CI 0.62-0.74), and higher summative scale measures (worse HL) (C-index 0.82, CI 0.77-0.86) were consistently more likely to have inadequate HL (sTOFHLA 0-16), as demonstrated by C-indices >0.5 (range for the questions and scale 0.68-0.84). Overall, these questions also successfully differentiated those with inadequate plus marginal HL (sTOFHLA 0-22) compared to those with adequate HL (sTOFHLA 23-36) (C-indices ranging from 0.69-0.81). (“confident with forms,” C-index 0.81, 0.76-0.86; “problems learning,” C-index 0.74, 0.68-0.79; “help reading,” C-index 0.69, 0.64-0.75; scale, C-index 0.82, 0.77-0.87) The performance of the summative scale was not statistically significantly different from the “confident with forms” question (p for inadequate HL = 0.85; p for inadequate plus marginal =0.77). Both the “confident with forms” item and the summative scale performed better than the other 2 questions for both inadequate and inadequate plus marginal HL (p < 0.01 for all comparisons).
In our stratified analyses by language, for inadequate (Table
2) and inadequate plus marginal HL (Table
3) the C-indices did not significantly differ between English and Spanish speakers. However, the three questions demonstrated higher sensitivity and lower specificity at any given cut point among Spanish speakers compared to English speakers. Sensitivity, specificity, and likelihood ratios were highest for the “confident with forms” question, among English and Spanish speakers, for identifying both inadequate HL (English, C-index 0.76; Spanish, C-index 0.74) (Table
2) and inadequate plus marginal HL (English, C-index 0.70; Spanish, C-index 0.80) (Table
3). For both inadequate (Table
2) and inadequate plus marginal HL (Table
3), a cut point of “somewhat” or less confident with forms, a cut point used in prior studies
18, appeared to maximize both sensitivity and specificity for English speakers. However, for both literacy levels a cut point of “a little” or less confident with forms functioned best, among Spanish speakers (Table
2 &
3). The test characteristics for the summative scale (See Online
Appendix) demonstrate that a cut point of 9, corresponding to answers of “sometimes/somewhat” on all three questions, appeared to maximize both sensitivity and specificity for English and Spanish speakers. (See Online
Appendix)
Table 2
Test Characteristics for Health Literacy Questions Compared to sTOFHLA Scores for Inadequate Health Literacy
ENGLISH | |
aConfident with Forms | 0.76 (0.67 – 0.85) |
eExtremely | | 6 | 67 | 1.00 (0.91 – 1.00) | 0.00 (0.00 – 0.03) | 0.28 (0.13 – 0.60) |
Quite a bit | | 10 | 21 | 0.84 (0.69 – 0.92) | 0.57 (0.48 – 0.66) | 1.51 (0.78 – 2.90) |
Somewhat | | 7 | 24 | 0.57 (0.41 – 0.71) | 0.75 (0.67 – 0.82) | 0.92 (0.43 – 1.97) |
A little | | 8 | 3 | 0.38 (0.24 – 0.54) | 0.96 (0.90 – 0.98) | 8.43 (2.36 – 30.16) |
Not at all | | 6 | 2 | 0.16 (0.08 – 0.31) | 0.98 (0.94 – 1.00) | 9.49 (2.00 – 45.01) |
bProblems Learning | 0.72 (0.63 – 0.82) |
eNever | | 11 | 78 | 1.00 (0.91 – 1.00) | 0.00 (0.00 – 0.03) | 0.44 (0.27 – 0.74) |
Rarely | | 6 | 15 | 0.70 (0.54 – 0.83) | 0.67 (0.58 – 0.75) | 1.26 (0.53 – 3.02) |
Sometimes | | 9 | 19 | 0.54 (0.38 – 0.69) | 0.79 (0.71 – 0.86) | 1.50 (0.74 – 3.02) |
Often | | 3 | 3 | 0.30 (0.17 – 0.46) | 0.96 (0.90 – 0.98) | 3.16 (0.67 – 15.00) |
Always | | 8 | 2 | 0.22 (0.11 – 0.37) | 0.98 (0.94 – 1.00) | 12.65 (2.81 – 56.96) |
cHelp Reading | 0.65 (0.55 – 0.75) |
eNever | | 14 | 69 | 1.00 (0.91 – 1.00) | 0.00 (0.00 – 0.03) | 0.64 (0.41 – 1.00) |
Rarely | | 3 | 14 | 0.62 (0.46 – 0.76) | 0.59 (0.50 – 0.67) | 0.68 (0.21 – 2.23) |
Sometimes | | 10 | 25 | 0.54 (0.38 – 0.69) | 0.71 (0.62 – 0.78) | 1.26 (0.67 – 2.38) |
Often | | 1 | 7 | 0.27 (0.15 – 0.43) | 0.92 (0.86 – 0.96) | 0.45 (0.06 – 3.55) |
Always | | 9 | 2 | 0.24 (0.13 – 0.40) | 0.98 (0.94 – 1.00) | 14.23 (3.22 – 62.94) |
dSummative Scale | 0.76 (0.67 – 0.85) |
SPANISH | |
aConfident with Forms | 0.74 (0.66 – 0.83) |
eExtremely | | 4 | 4 | 1.00 (0.97 – 1.00) | 0.00 (0.00 – 0.09) | 0.38 (0.10 – 1.44) |
Quite a bit | | 8 | 8 | 0.96 (0.90 – 0.98) | 0.10 (0.04 – 0.24) | 0.38 (0.15 – 0.94) |
Somewhat | | 15 | 14 | 0.88 (0.81 – 0.93) | 0.31 (0.19 – 0.46) | 0.41 (0.22 – 0.76) |
A little | | 34 | 10 | 0.74 (0.65 – 0.81) | 0.67 (0.51 – 0.79) | 1.29 (0.71 – 2.35) |
Not at all | | 42 | 3 | 0.41 (0.32 – 0.50) | 0.92 (0.80 – 0.97) | 5.30 (1.74 – 16.11) |
bProblems Learning | 0.63 (0.54 – 0.73) |
eNever | | 26 | 12 | 1.00 (0.96 – 1.00) | 0.00 (0.00 – 0.09) | 0.80 (0.45 – 1.42) |
Rarely | | 6 | 9 | 0.75 (0.66 – 0.82) | 0.32 (0.19 – 0.47) | 0.25 (0.09 – 0.64) |
Sometimes | | 34 | 11 | 0.69 (0.59 – 0.77) | 0.55 (0.40 – 0.70) | 1.14 (0.65 – 2.01) |
Often | | 8 | 3 | 0.36 (0.27 – 0.46) | 0.84 (0.70 – 0.93) | 0.98 (0.28 – 3.52) |
Always | | 29 | 3 | 0.28 (0.20 – 0.38) | 0.92 (0.79 – 0.97) | 0.37 (1.15 – 11.02) |
cHelp Reading | 0.68 (0.60 – 0.77) |
eNever | | 30 | 19 | 1.00 (0.96 – 1.00) | 0.00 (0.00 – 0.09) | 0.58 (0.38 – 0.90) |
Rarely | | 14 | 7 | 0.71 (0.61 – 0.79) | 0.50 (0.33 – 0.64) | 0.74 (0.32 – 1.69) |
Sometimes | | 24 | 11 | 0.57 (0.48 – 0.66) | 0.68 (0.51 – 0.80) | 0.80 (0.44 – 1.48) |
Often | | 6 | 0 | 0.34 (0.26 – 0.44) | 0.97 (0.86 – 1.00) | 4.88 (0.28 – 84.51) |
Always | | 29 | 1 | 0.28 (0.20 – 0.38) | 0.97 (0.86 – 1.00) | 10.70 (1.51 – 75.84) |
dSummative Scale | 0.74 (0.66 – 0.83) |
Table 3
Test Characteristics for Health Literacy Questions Compared to sTOFHLA Scores for Inadequate + Marginal Health Literacy
ENGLISH | |
aConfident with Forms | 0.70 (0.62 – 0.78) |
eExtremely | | 17 | 56 | 1.00 (0.94 – 1.00) | 0.00 (0.00 – 0.04) | 0.48 (0.31 – 0.74) |
Quite a bit | | 14 | 17 | 0.72 (0.59 – 0.81) | 0.60 (0.49 – 0.69) | 1.29 (0.69 – 2.42) |
Somewhat | | 12 | 19 | 0.48 (0.36 – 0.61) | 0.78 (0.68 – 0.85) | 0.99 (0.52 – 1.89) |
A little | | 10 | 1 | 0.28 (0.19 – 0.41) | 0.98 (0.93 – 0.99) | 15.67 (2.06 – 119.3) |
Not at all | | 7 | 1 | 0.12 (0.06 – 0.22) | 0.99 (0.94 – 1.00) | 10.97 (1.38 – 86.92) |
bProblems Learning | 0.69 (0.61 – 0.77) |
eNever | | 24 | 65 | 1.00 (0.94 – 1.00) | 0.00 (0.00 – 0.04) | 0.58 (0.41 – 0.81) |
Rarely | | 6 | 15 | 0.60 (0.47 – 0.71) | 0.69 (0.59 – 0.78) | 0.63 (0.26 – 1.53) |
Sometimes | | 17 | 11 | 0.50 (0.38 – 0.62) | 0.85 (0.77 – 0.91) | 2.42 (1.22 – 4.81) |
Often | | 4 | 2 | 0.22 (0.13 – 0.34) | 0.97 (0.91 – 0.99) | 3.13 (0.59 – 16.58) |
Always | | 9 | 1 | 0.15 (0.08 – 0.26) | 0.99 (0.94 – 1.00) | 14.1 (1.83 –108.49) |
cHelp Reading | 0.66 (0.57 – 0.74) |
eNever | | 23 | 60 | 1.00 (0.94 – 1.00) | 0.00 (0.00 – 0.04) | 0.60 (0.42 – 0.86) |
Rarely | | 5 | 12 | 0.62 (0.49 – 0.73) | 0.64 (0.54 – 0.73) | 0.65 (0.24 – 1.76) |
Sometimes | | 21 | 14 | 0.53 (0.41 – 0.65) | 0.77 (0.67 – 0.84) | 2.35 (1.30 – 4.25) |
Often | | 2 | 6 | 0.18 (0.11 – 0.30) | 0.91 (0.84 – 0.96) | 0.52 (0.11 – 2.50) |
Always | | 9 | 2 | 0.15 (0.08 – 0.26) | 0.98 (0.93 – 0.99) | 7.05 (1.58 – 31.52) |
dSummative Scale | 0.73 (0.64 – 0.81) |
SPANISH | |
aConfident with Forms | 0.80 (0.71 – 0.88) |
eExtremely | | 4 | 4 | 1.00 (0.97 – 1.00) | 0.00 (0.00 – 0.12) | 0.25 (0.07 – 0.92) |
Quite a bit | | 9 | 7 | 0.96 (0.91 – 0.99) | 0.14 (0.06 – 0.31) | 0.32 (0.13 – 0.77) |
Somewhat | | 18 | 11 | 0.89 (0.81 – 0.93) | 0.39 (0.24 – 0.58) | 0.40 (0.21 – 0.75) |
A little | | 39 | 5 | 0.73 (0.64 – 0.80) | 0.79 (0.60 – 0.90) | 1.92 (0.83 – 4.41) |
Not at all | | 44 | 1 | 0.39 (0.30 – 0.48) | 0.96 (0.82 – 0.99) | 10.81 (1.56 – 75.09) |
bProblems Learning | 0.70 (0.61 – 0.80) |
eNever | | 27 | 11 | 1.00 (0.97 – 1.00) | 0.00 (0.00 – 0.12) | 0.61 (0.35 – 1.07) |
Rarely | | 7 | 8 | 0.76 (0.67 – 0.83) | 0.39 (0.24 – 0.58) | 0.21 (0.09 – 0.54) |
Sometimes | | 38 | 7 | 0.70 (0.61 – 0.78) | 0.68 (0.49 – 0.82) | 1.35 (0.67 – 2.69) |
Often | | 10 | 1 | 0.36 (0.28 – 0.45) | 0.93 (0.77 – 0.98) | 2.46 (0.33 – 18.40) |
Always | | 31 | 1 | 0.27 (0.20 – 0.36) | 0.96 (0.82 – 0.99) | 7.68 (1.10 – 53.88) |
cHelp Reading | 0.71 (0.63 – 0.80) |
eNever | | 33 | 16 | 1.00 (0.97 – 1.00) | 0.00 (0.00 – 0.12) | 0.51 (0.33 – 0.79) |
Rarely | | 16 | 5 | 0.71 (0.62 – 0.78) | 0.57 (0.39 – 0.73) | 0.79 (0.32 – 1.98) |
Sometimes | | 28 | 7 | 0.57 (0.47 – 0.65) | 0.75 (0.57 – 0.87) | 0.99 (0.48 – 2.03) |
Often | | 6 | 0 | 0.32 (0.24 – 0.41) | 1.00 (0.88 – 1.00) | 3.31 (0.19 – 57.02) |
Always | | 30 | 0 | 0.27 (0.19 – 0.35) | 1.00 (0.88 – 1.00) | 15.52 (0.98 – 246.3) |
dSummative Scale | 0.82 (0.75 – 0.89) |
In stratified analyses, after adjustment for multiple comparisons, we found that the self-reported questions performed well and consistently across age, gender, educational attainment, health status, and race/ethnicity participant subgroups for identifying inadequate HL. For inadequate plus marginal HL there was slightly more variation between groups, but none of these differences were statistically significant (See Online
Appendix, all P > 0.01).
DISCUSSION
Because of its well-established role in health outcomes and health disparities, HL is an important factor to study in public health and epidemiological research
13. To our knowledge, this is the first study to test the performance of self-reported HL questions among an ethnically diverse, English and Spanish-speaking population, and to compare the performance of the questions between language and other patient characteristic subgroups. We found that three self-reported HL questions could identify those with inadequate, and inadequate plus marginal HL within this ethnically diverse, English and Spanish-speaking population with a moderate degree of discrimination. The “confident with forms” question performed best among the individual items and within both language and all other patient characteristic subgroups. The summative scale performed similarly to the individual “confident with forms” question.
Our findings build on previous studies of the three self-reported HL measures. As in prior studies
17,
18,
20, the “confident with forms” question performed the best out of the three questions. In contrast to prior work, we found that both the “confident with forms” question and the summative scale could discriminate moderately well between those with inadequate plus marginal vs. adequate HL, in addition to inadequate HL, for both English and Spanish speakers. For the “confident with forms question” Chew et al found a C-index of 0.72 for inadequate plus marginal HL while we found a C-index of 0.81 for the overall sample. This is important because marginal HL, in addition to inadequate HL, has been associated with poor health outcomes including mortality and health disparities
4,
12,
26. Because dose response associations have been found between HL level and poor patient outcomes,
31 some investigators may want to identify both literacy level subgroups. Our results also mirror those of prior studies in finding similar performance between the “confidence with forms” item and the summative scale
17.
In stratified analysis by language, the C-indices for the “confidence with forms” question were similar for Spanish and English speakers. However, the item seemed to have higher sensitivity but lower specificity among Spanish speakers at every cut point. The optimum cut point for the “confident with forms” question for English speakers that maximized both sensitivity and specificity was “somewhat” or less, while for Spanish speakers the optimum cutpoint was “a little” or less. These findings may be the result of cultural variation and /or Spanish-speaking participants responding to the ‘confident with forms” question for forms not only written in Spanish, but also in English. As such, researchers may want to consider different cut points for English and Spanish-speaking subgroups.
The utility of the “confident with forms” question and summative scale among the Spanish speakers in our population may also be affected by the relatively high prevalence of language concordant patient-physician dyads in this clinical setting and the ubiquitous access to Spanish transcription and translation services
22. Patient–physician language concordance has been shown to be a powerful determinant of patient satisfaction with communication and may have leveled the playing field with their English-speaking counterparts in terms of patients feeling confident with forms
22. As such, the self-reported measures in this population may have been detecting true HL deficits rather than those related to language discordance or limited English proficiency.
Because of a prior lack of brief, validated measures of HL for diverse populations, some have suggested using demographic characteristics to estimate HL
32. This approach does not permit the ability to assess the independent effects of HL beyond demographic characteristics. This is important because HL levels have been shown to vary widely within patient demographic subgroups
6. Therefore, we contend that independent measurement of HL, for example with the “confident with forms” question or summative scale, would contribute substantially to epidemiologic and clinical research. In the clinical setting, screening for limited health literacy is controversial, with the current expert recommendations against routine screening
32‐
34. However, in selected clinical situations, such as the prescribing of high-risk medications, screening for limited health literacy has been advocated, and the use of a single-item screener would be more feasible in busy clinical settings than standard literacy assessments
19.
While imperfect in their precision, the summative scale, and specifically the single “confident with forms” question, have some clear advantages over direct, longer HL measurements. They are brief and can be administered via telephone. Our group has recently field-tested these questions both individually and as a scale within a large sample of diverse diabetes patients and have demonstrated robust, independent associations with a range of outcomes, including perceived need for self-management support
35, higher rates of hypoglycemia
36, and lower patient use of electronic health records
37. While these studies did not assess performance of these items across demographic sub-groups, these associations lend support to the items’ predictive validity.
Our study has some limitations. First, we included only patients with poorly controlled diabetes, which may limit generalizability to healthier populations. Second, this study was conducted at four sites within one county health care system and may not reflect regional differences. Third, in our practice environment there is excellent access to translation services and many physicians and staff speak Spanish. Results may differ for Spanish-speaking patients in different linguistic environments. Finally, our results reflect the criterion validity of the self-reported HL questions, i.e., their relationship with a gold-standard HL measurement. Further work is needed to establish predictive validity of these questions in relation to health outcomes of interest.
In summary, although limited HL is associated with a range of health outcomes, it is often not feasible to measure directly in clinical, epidemiologic, or public health studies because standard measurement tools are lengthy and cannot be administered by telephone. Our study suggests that the single self-reported “confident with forms” question or the summative scale of the three self-reported HL questions discriminate diverse English speakers and Spanish speakers with adequate HL from those with inadequate and inadequate plus marginal HL to a degree that warrants application and further assessment in epidemiologic and clinical research involving diverse populations.
ACKNOWLEDGMENTS
We gratefully acknowledge the data analysis performed by Kathy Z. Fung, MS, (funded by the San Francisco Veterans’ Affairs Medical Center) and the biostatistical advice provided by John Boscardin, Ph.D (funded by the National Center for Research Resources KL2RR024130). Funds were provided by Agency for Healthcare Research and Quality R18 HS01726101 (to DS), a NIH Clinical and Translational Science Award ULRR024131 (to DS), the National Center for Research Resources KL2RR024130 (to US.) and Agency for Healthcare Research and Quality K08 HS017594 (to US), a Veterans Affairs Career Development Award and a Pfizer Fellowship in Clear Health Communication (RS). None of the funders had any role in study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.