Background
Methods
Results
Overview
Systematic review of the literature
Study | Materials Evaluated |
n Articles Evaluated | Medium | Evaluation Methods | Results | Conclusions |
---|---|---|---|---|---|---|
Barbosa and Martins (2007) [26] | Internet information about floaters and light flashes found by two search engines, MetaCrawler and MSN. | 49 | Online | FKGL and quality component scoring system | Mean FKGL = 9.9 | “It is important for ophthalmologists not only to help to develop good-quality websites but also to direct their patients to sites that provide accurate information.” |
Brown, et al. (2004) [25] | Cataract information leaflets from 12 ophthalmology departments in England | 12 | Print | SMOG, criteria for obtaining informed consent | Mean SMOG = 10 | SMOG scores exceed recommended level of 5 or lower |
Ebrahimzadeh, et al. (1997) [24] | Educational brochures from the AAO | 22 | Print | FKGL, FRES, GFI | 32 % were at or below an 8th-grade reading level, 55 % between 8th- and 10th-grade levels, and 15 % were a 10th-grade reading level or higher | “With aid of computer programs, reading levels of materials can be analyzed and revised to reflect low health literacy” |
Edmunds, et al. (2013) [19] | The “top 10 patient-oriented websites for 16 different ophthalmic diagnoses” | 160 | Online | FKGL, FRES, SMOG, GFI | Mean FKGL = 11.3 | “[R]eadability scores were inferior to those recommended, irrespective of the measure used....we recommend the use of readability scoring when producing such resources in the future.” |
Edmunds, et al. (2014) [27] | “[O]nline literature specifically for Graves’ disease and thyroid-associated ophthalmopathy” by Google search | 50 | Online | FKGL, FRES, SMOG, GFI | Mean FKGL = 11 | “None of the web pages evaluated had readability scores in accordance with published guidelines....Screening of this online material, as well as subsequent revision, is crucial to increase future patient knowledge, satisfaction, and compliance.” |
Hansberry, et al. (2014) [28] | Patient education material on AAO website | unspecified | Online | FKGL, FRES, SMOG, GFI, Coleman-Liau Index, the New Fog Count Formula, the New Dale-Chall Readability Formula, FORCAST formula, Raygor Readability Estimate, and the Fry Graph | Mean FKGL = 11.7 | “[W]e believe revisions in line with the recommendations of the NIH may be warranted to improve patient comprehension” |
Huang, et al. (2015) [29] | Websites from 7 ophthalmologic organizations: AAO, American Association of Ophthalmic Oncologists and Pathologists, AAPOS, AGS, ASCRS, ASOPRS, American Society of Retina Specialists, American Uveitis Society, Cornea Society, and NANOS | 339 | Online | FKGL, FRES, SMOG, GFI, Coleman-Liau Index, New Fog Count, New Dale-Chall Readability Formula score, FORCAST score, Raygor Readability Estimate Graph score, and Fry Readability Graph score. | Mean FKGL ranged from 10.4 to 12.6 | “Online PEMs on major ophthalmologic association websites are written well above the recommended reading level. Consideration should be given to revision of these materials to allow greater comprehension among a wider audience” |
John, et al. (2015) [30] | First 10 PEMs to appear in search on Google search for 10 pediatric ophthalmology conditions | 100 | Online | FKGL, FRES, SMOG, GFI, Coleman-Liau Index, New Dale-Chall, FORCAST Formula, Fry Graph, Raygor Reading Estimate, and the New Fog Count | Mean FKGL = 11.75 | Only 12 % of articles were written below a 9th-grade level and only 3 % met recommended criteria. |
Khurana, et al. (2003) [23] | Various ocular medication inserts | 10 glaucoma medication inserts, 6 nonglaucoma inserts | Print | FKGL, SMOG | Mean FKGL = 12.9 (glaucoma inserts), Mean FKGL = 11.1 (nonglaucoma inserts) | All medications reviewed were written above an 8th-grade reading level |
Martins and Morse (2005) [31] | Websites about retinopathy of prematurity found by two search engines, MetaCrawler and MSN. | 40 | Online | FKGL and quality component scoring system | Mean FKGL = 10.83 | “In the majority of the sites (62.5 %) the ROP information was fair or poor.” |
Muir and Lee [2] (2010) | Educational brochures from the AAO (Revised in 2008), NIH, NEI, AGS, the Glaucoma Research Foundation, and Prevent Blindness America | 49 | Print | FKGL | Mean FKGL = 8.3 (AAO), 9.7 (non-AAO) | “Unfortunately, there is still a dearth of written ophthalmic educational materials available from any agency for the least literate patients, precisely those who are at the greatest risk of blindness.” |
Williams, et al.a
| Duke Eye Center Glaucoma Guide (patient education material about various glaucoma topics written by Duke faculty) | 12 | Print | FKGL, FRES, SAM, word count | Mean FKGL = 10.0 (before revision); Mean FKGL = 6.4 (after revision) | Revisions of patient education materials using existing guidelines significantly improved their readability and suitability for a low-health-literacy population |
Zaidi and Jones (2009) [32] | Websites that appear in the first 5 pages of a Google search for three terms related to blepharoplasty | 101 | Online | Objective quality of each website was measured using JAMA criteria: declaration of authorship, attribution of sources, disclosure of conflict of interests, and provision of date content was posted or updated (1 point for each, maximum 4 points total) | “Most sites scored low for quality—40 % scored zero for objective quality; 41 % scored just one point; 10 % scored two points; 6.5 % of sites scored three points; only 2.5 % of sites scored favourably on all four criteria.” | “This study identifies the poor quality of information on oculoplastic surgery, which is available to patients using the internet” |
The Duke experience: Improving suitability and readability of patient education materials
General Content |
• Focus on 2-3 key concepts. |
• Limit content to what patients really need to know. |
• Use only words that are well known to individuals without medical training. |
• Make certain content is appropriate for age and culture of the target audience. |
• Identify action steps. State in beginning and repeat in the end of the document. |
Text Construction |
• Keep within a range of about a 6th to 8th grade reading level. |
• Use one- to two-syllable words. |
• Use short paragraphs. |
• Use active voice. |
• Use a clear topic sentence at the beginning of each paragraph. Follow the topic sentence with details and examples. |
• Examples and stories may help engage readers. |
• Use words like “you” instead of “the patient.” |
• Structure the material logically, but include your most important points at the beginning of the document. |
• You need to grab the reader’s attention at the beginning. People often do not read all the text and may miss your key point if you save the best for last. |
• Some users prefer step-by-step instructions. Others may find concepts arranged from the general to the specific easier to understand. |
• Use bulleted lists instead of blocks of text to make information more readable. |
• Include specific actions the reader may or should take. Your document’s purpose should not be solely to inform but also to get the reader to take an action. |
• Avoid abstract words in instructions for actions. |
• Be consistent with terms. |
• Emphasize the benefits of the desired behavior. |
• Do not make assumptions about people who read at a low level. Don’t talk down to the reader. Maintain an adult perspective. |
Visual Presentation |
• Use colors that are appealing to your target audience. |
• Use illustrations and photos with concise captions. Keep captions close to photos and illustrations. |
• Avoid graphs and charts unless they actually help understanding. If you do use them, make sure they are simple and clear. |
• Balance the use of text, graphics, and white space. Try for 40-50 % white space. |
• Avoid using all capital letters. Upper and lower case are easier to read. To show emphasis, use bold, larger type size or different fonts. |
• Avoid italics of more than a few words at a time. |
• Make print large enough for your target audience. For most readers text the equivalent of Times New Roman 12 point is adequate. For seniors, consider using 14 point. |
• Use easy-to-read fonts, such as Times New Roman, Arial, Tahoma and Helvetica. |
• Use bolded headings and subheadings to separate and highlight document sections. |
• When possible, use graphics or spell out fractions and percentages. |
• Only justify the left margin. This means the left margin should be straight and the right margin should be “ragged.” |
• Do not print text on top of shaded backgrounds, photos, or patterns. |
SAM Criteria | SAM Score | |||
---|---|---|---|---|
Original Handouts (n = 12) | Revised Handouts (n = 12) |
p value | ||
Content | (a) Purpose is evident | 1.29 | 1.63 | 0.011 |
(b) Content about behaviors | 1.25 | 1.71 | 0.033 | |
(c) Scope is limited | 1.88 | 1.92 | 0.341 | |
(d) Summary or review included | 0.63 | 1.38 | 0.003 | |
Literacy demand | (a) Reading grade level | 0.33 | 1.25 | <0.001 |
(b) Writing style, active voice | 1.13 | 1.88 | 0.001 | |
(c) Vocabulary | 1.04 | 1.96 | <0.001 | |
(d) Context is given first | 1.42 | 1.71 | 0.271 | |
(e) Advance organizers | 1.75 | 2.00 | 0.167 | |
Layout and typography | (a) Layout factors | 0.88 | 1.96 | <0.001 |
(b) Typography | 1.63 | 2.00 | 0.026 | |
(c) Subheadings (“chunking”) used | 1.33 | 2.00 | 0.003 | |
Learning stimulation and motivation | (a) Interaction used | 0.25 | 0.55 | 0.081 |
(b) Behaviors are modeled and specific | 1.58 | 1.83 | 0.082 | |
(c) Motivation--self-efficacy | 1.55 | 1.92 | 0.015 | |
Cultural appropriateness | (a) Match in logic, language, experience | 1.25 | 1.96 | 0.001 |
(b) Cultural image and examples | N/A | N/A | N/A | |
Total SAM Score (%) (mean ± SD): | 60 ± 7 | 88 ± 4 | <0.001 |
SAM Score | Word Count | Flesch-Kincaid Grade Level | Flesch Reading Ease Score | |||||
---|---|---|---|---|---|---|---|---|
Handout Topic | Original | Revised | Original | Revised | Original | Revised | Original | Revised |
Acute Glaucoma | 51 % | 91 % | 251 | 212 | 8.6 | 5 | 60 | 75 |
Advice for Family Members | 63 % | 93 % | 583 | 326 | 11.4 | 6.3 | 46.2 | 66.5 |
Chronic Angle Closure Glaucoma | 54 % | 83 % | 358 | 223 | 10.8 | 4.6 | 53.4 | 79.1 |
Cataract Surgery Glaucoma Patients | 55 % | 89 % | 844 | 632 | 9.5 | 7.3 | 55.8 | 63.7 |
Glaucoma Medications | 63 % | 88 % | 833 | 550 | 10.4 | 6.5 | 52.8 | 70.1 |
Overview of the Glaucoma Team | 50 % | 84 % | 689 | 559 | 12 | 8.2 | 42.2 | 59.1 |
What You Should Know Before Glaucoma Surgery | 59 % | 88 % | 733 | 643 | 11.7 | 7.2 | 43.6 | 64.7 |
What You Need to Know After Glaucoma Surgery | 70 % | 89 % | 775 | 507 | 9.3 | 4.8 | 58.2 | 76.4 |
What to Expect from Glaucoma Surgery | 72 % | 86 % | 654 | 608 | 9.3 | 6.9 | 56 | 65.9 |
“Top Ten” for Glaucoma Patients | 69 % | 81 % | 494 | 489 | 6.6 | 5.8 | 69.4 | 72.4 |
What to Expect on Your Visit | 54 % | 94 % | 694 | 717 | 11.8 | 8.3 | 44.3 | 59.2 |
When to Call Your Eye Doctor | 57 % | 89 % | 335 | 391 | 8.9 | 6.4 | 58.2 | 67 |
Average: | 60 % | 88 % | 604 | 488 | 10 | 6.4 | 53 | 68 |
Standard Deviation: | 7 % | 4 % | 201 | 166 | 1.6 | 1.2 | 8 | 6 |
Paired t-Test: |
p < 0.001 |
p = 0.006 |
p < 0.001 |
p < 0.001 |
Subject ID | Age | Gender | Race/Ethnicity | Eye Conditions | History of Eye Surgery | Highest Degree | First Language | REALM-SF Score |
---|---|---|---|---|---|---|---|---|
S1 | 47 | Male | Latino/Chicano | Glaucoma, diabetic retinopathy | Yes | Some high school | Spanish | 4th-6th grade level |
S2 | 70 | Male | White/European-American | Glaucoma, cataracts | No | College degree | English | ≥ 9th grade level |
S3 | 31 | Male | White/European-American | Ocular hypertension | No | College degree | English | ≥ 9th grade level |
S4 | 66 | Female | White/European-American | Glaucoma, cataracts | No | Not reported | English | ≥ 9th grade level |
S5 | 75 | Female | White/European-American | Glaucoma, cataracts | Yes | Some college | English | ≥ 9th grade level |
CIRF Item | Score Mean (SD) |
---|---|
How easy or hard is the handout to read? | 3.9 (0.9) |
How easy or hard is the handout to understand? | 4.0 (0.5) |
How easy or hard is the handout to remember? | 3.7 (0.7) |
How easy or hard is the handout to locate information? | 3.9 (0.6) |
How easy or hard is the handout to keep for future reference? | 4.1 (0.7) |
How organized is the handout? | 4.6 (0.7) |
How attractive is the handout? | 4.2 (0.8) |
How is the print size? | 4.3 (0.7) |
How is the tone of the handout? | 4.1 (0.8) |
How helpful is the handout? | 4.7 (0.5) |
How is the spacing between lines? | 4.4 (0.7) |
General Evaluation of the Handouts |
• Is this handout helpful? |
• What catches your eye? |
• Who do you think this handout is for? |
• What do you think about the handouts? |
• What do you like about the handouts? |
• What would you want to change about the handouts? |
• Is there information that you don’t need? |
• Is there anything that you don’t like? |
• Which handout is the best? Why? |
• Which handout is the worst? Why? |
Comprehension |
• Tell me in your own words what this is trying to say. |
Readability |
• Do you see any words that you think some people might have a hard time understanding? |
• Is anything confusing? |
Past Experiences with Patient Education Handouts |
• Do you get handouts like these from your doctor? |
• Do you like educational handouts? |
• Do you usually read them? |
• Do you save them to reference later? |
• What information do you like to see in handouts? |
• What don’t you like about patient education handouts? |
Graphics |
• What would you like to see in a picture? |
• Would a picture be helpful? |
• What are your thoughts on having a video to go along with the handouts? |
Theme | Supporting Quotations |
---|---|
Emphasize that we are available to help | S1: “If you forgot something and need some help, you know who to call about the drops.” |
S3: “I like that at the top it says in bold ‘the short answer is that if you’re worried, call.’ I think that leaves one with the idea that you’re not going to be a nightmare patient if you’re calling. I don’t think anyone wants to feel like that, like they’re the patient that’s calling too much or being a little anxious for no reason.” | |
Be concrete | S2: “You miss two or three appointments, you might lose your eyes. So missing appointments is pretty vague, but the repercussions are very costly.” |
S2: “Doing them both at the same time [cataract surgery and glaucoma surgery] will lower your eye pressure. My question is, what does that do for you? If you lower the eye pressure, I assume that’s good. If it’s high it’s not good.” | |
S2: “There are a couple of sentences that were sort of confusing to me. Like this one: ‘Make sure you understand how much vision loss you have [from glaucoma].’ How are you supposed to know that?” | |
S3: “I think it’s good that it [the post-operative handout] says, ‘if you have pain that’s not improved by Tylenol.’ You’re going to have pain [after surgery], but if it’s not going away with a light pain reliever, then you should call.” | |
S5: “I liked the idea that it was stressing that by missing any drops…how that can be detrimental.” | |
Give practical information | [Interviewer: What stood out most to you about this one?] |
S1: The reminders. Like setting your iPhone to remember [when to take your drops]. | |
S3: “It’s got good practical information....About remembering to take a refill with you when you’re traveling, that’s a good reminder. Having people write a schedule is a practical piece of advice, and, you know, it’s good that it stresses the importance of the consistency of the appointments.” | |
Keep it simple | S3: “It keeps it pretty simple. I think someone’s family member could read this and be like, okay, I’m on the same page with my loved one here.” |
S5: “Absolutely, [I] prefer one page [for the length of the handout].” | |
Highlight key points | S2: “Some people that are real busy would look at the highlights, skim through it. See what seems important....You could highlight a few things like exercise or whatever that’s important, that type of thing. It’s the things I look at whenever I read these types of things, the things that I focus in at. People don’t read more than 2 or 3 pages at a time.” |
S3: “The emergency number is at the bottom. I would maybe move that to the top.” | |
Give a picture of the road ahead | S2: “It seems like when I get involved [have the need for cataract surgery], this would be a good starting point for me, to answer questions for me.” |
S3: “I don’t have it [glaucoma] yet, you know, I guess I’m a candidate because of the pressure, but I think with that in mind its got good information.” | |
S3: “If you’re new to this as I’m going to be when I have the drops, it’s good for someone who’s younger like me who has a risk, I don’t think it’s bad to mention to have them somewhere convenient, by your bathroom, by your bedside, in the kitchen, with other medications.” | |
S3: “It’s saying you’ve got to keep an eye on this stuff [post-operative complications] for a long time and be careful about observing it.” | |
Maintain a positive tone | S3: “It’s caring, sensitive, helpful, and it’s positive. You stay on top of this and everything should be, you know, that will be the best for you. So it’s encouraging in the sense of, just stay on top of this condition, and, you’ll be in a better place as a patient.” |
S3: “It encourages you to educate yourself, and once you’ve met with one of you all, you kind of educate yourself about the risk factors; in my case, the eye pressure. So you just kind of leave knowing that you just have to take your drops and just kind of stay on top of it.” | |
Provide source for more information | S3: “I like the encouragement to learn about it [referring to the website to go to for more information on glaucoma], I think that lends to the positive nature about the handout, you know, go learn more about it, educate yourself. I guess doctors say an educated patient is an empowered patient.” |
S4: “I like this website. Because I do try to read everything I can find on glaucoma.” | |
Illustrate | S1: Would like a picture of “what glaucoma looks like in the eye.” |
S2: “Definitely pictures are better. I’m not too graphic, but color would be a good if it’s a picture of the eye. Help people understand better.” | |
S3: Desires “an image about the optic nerve and where that is. Like an eye diagram.” | |
S5: “Perhaps there could be a little bit more of the demonstration or more of a verbal detailed aspect of actually putting the drop in your eye because that was traumatic for me.” |