Readability of Dutch online patient-directed health information on breast reconstruction

The Netherlands was the first European country after the USA to obtain internet access in November 1988 [1]. Both the Netherlands and Belgium began offering commercial internet access in 1993 [1, 2]. Currently, the Internet is one of the primary sources of health information. It has been reported that online health information can help increase medical knowledge, engagement, and competence in health decision-making strategies [3]. Moreover, patients are now capable of receiving healthcare information and support while staying in a trusted environment with the use of patient-centered web sites, blogs, and support communities [4]. On the contrary, online health information can be commercially biased and patient information can be therefore misleading. Furthermore, literature has shown that patient’s frequency of searching the internet correlates with health anxiety and can trigger unnecessary health concerns [5, 6].

Quite importantly, information should be written at a suitable readability level that is comprehensible by the largest proportion of society. Readability tools can be very useful as they measure the complexity of written text. It can thus be utilized to estimate how difficult content is to comprehend. The Dutch government states on its website that level B1 on the Common European Framework of Reference for Languages (CEFR) is recognized as most appropriate for the general population [7]. Other measures have been taken, such as the implementation of the Web Content Accessibility Guidelines (WCAG) in order to make the internet more accessible by improving natural information (text, images, and sounds) and code or markup that defines structure, presentation, etc. [8]. As of late 2016, all government web sites of European Union member states are required to conform to the latest version.

The European statistical office, Eurostat, found that 73% of the Dutch population and 56% of the Belgian had queried the internet searching for health-related information with a minimum of once in the past year [9, 10]. The primary reason for patients to search the Internet is because it is believed to be the fastest way of obtaining information [11]. The Internet could be considered an important factor in the periclinical phase, providing patients with information, support, autonomy, and anonymity. The medical influence of the Internet is likely to grow, not necessarily in combination with clinicians’ expertise.

The global incidence of breast cancer is rising, of which Belgium has the highest incidence followed by the Netherlands, ranking fourth [12]. Previous literature has concluded incomprehensive patient-directed information after analyzing readability of English online healthcare information on a variety of surgical topics including the operative treatment of breast cancer [13], mastectomy, lumpectomy [14], lymphedema [15], and breast augmentation [16]. Breast reconstruction is becoming an increasingly conventional factor for women undergoing mastectomy, a variety of reconstructive techniques exist utilizing patients’ own tissue or alloplastic material [17, 18]. Intelligible online health information is becoming more important due to the easier accessibility for patients querying the Internet for background information. Studies analyzing English online health information on breast reconstruction reported that the readability level was too high for the general population despite the argument by the American Medical Association (AMA) that patient information written above the recommended sixth-grade reading level would be too difficult [19‐23]. A Dutch study on online health information found that 80% of their included texts from a wide variety of medical specialties was too advanced [24]. However, Dutch online health information on breast reconstruction has not been analyzed yet.

Therefore, the aim of this study was to analyze the readability level of online health information on breast reconstruction in the Dutch Language.

A web search for “borstreconstructie” was performed using the largest Internet search engine, Google (Google Inc., Mountain View, California), on September 13, 2017 to simulate an online patient search for information on breast reconstruction. We disabled location information and excluded sponsored content to avoid inadvertent bias in our results. The 20 most popular web sites with Dutch written content were included: 10 top-ranking from Belgium and 10 from the Netherlands (Table 1). Relevant articles (defined as ≥ 100-word count and within one click away from the original link) were gathered and formatted into plain text in separate Microsoft Word 2017 documents (Microsoft Corp., Redmond, Washington). Saved articles were edited to exclude videos, images, advertisements and links, and organized by original web site.

The CEFR scale was introduced by the Council of Europe and used as measurement tool for learning, teaching, and assessing European languages. The scale includes six levels: A1 (Breakthrough), A2 (Waystage), B1 (Threshold), B2 (Vantage), C1 (Effective Operational Proficiency), and C2 (Mastery). Scale A (A1 and A2) are “basic users” of the language, B (B1 and B2) “independent users,” and C (C1 and C2) are “proficient users” [25]. Maximum readability and the biggest comprehending group are achieved at level B1 for medium knowledge of the language [7]. Examples can be observed in the attachment included with this article and were derived from those by the Council of Europe [25].

Readability was analyzed using four tools: Accessibility.nl (Accessibility Foundation, Utrecht), Klinkende Taal (Klinkende Taal B.V., Amsterdam), Texamen (BureauTaal, Beusichem), and WizeNote (WizeNoze B.V., Amsterdam) (Table 2). Accessibility.nl is available online for free, while the remaining are paid services. These were the only tools available and known to evaluate readability of Dutch written content. All four tools utilize the CEFR scale.

Articles were individually analyzed with all four readability tools. The six readability levels each were numerically coded (A1 = 1 to C2 = 6) according to the ordinal scale to calculate averages (Table 3) [26]. Readability level per web site was calculated using the average across all articles.

The following examples were derived from the Council of Europe [25]:

The level per article was calculated from the mean average of the four readability tools. Descriptive statistics were collected using Excel (Microsoft Corp., Redmond, Washington). Statistical analyses were performed using IBM SPSS software v24.0 (IBM Corp., Armonk, N.Y.). t test was used for group differences with continuous data. A value of p < 0.05 was defined as being significant.

The top 20 highest ranked websites were Borstkanker.nl, Kanker.nl, UZLeuven.be, NVPC.nl, BeautifulABC.​com, Plastischchirurg​en.​com, Think-pink.be, ZOL.be, Isala.nl, Allesoverkanker.be, UMCUtrecht.nl, Chirurgenoperatie.nl, Natrelle.nl, ErasmusMC.nl, BorstcentrumGent.be, Kiesbeter.nl, DrColpaert.be, ZiekenhuisGeel.be, DeVoorzorg.be, and JessaZH.be (Table 1). In total, 84 articles were analyzed of which 47 articles were Dutch and 37 articles were Belgian (Table 1). Five percent were government web sites, 10% industry, 35% association, foundation, or NGO (non-governmental organization), and 50% were health care providers (Fig. 1 and Table 1). Dutch written health information on breast reconstruction was written in CEFR level B2/C1 (average 4.48) (Table 4).

When sub-analyzing by readability tool, Klinkende Taal was found to have the lowest average CEFR level at B2 (average 4.01), ranging from 2.40 (Kiesbeter.nl) to 5.00 (ZOL.be, BeautifulABC.​com and Chirurgenoperatie.nl). WizeNote (average 4.57) and Accesibility.nl (average 4.58) both had CEFR level B2/C1 and respectively ranged from 3.00 (kiesbeter.nl) to 5.00 (UMCUtrecht.nl, Isala.nl, Plastischchirurg​en.​com, ZiekenhuisGeel.be, Think-pink.be, BorstcentrumGent.be, Allesoverkanker.be, ZOL.be, BeautifulABC.​com, Chirurgenoperatie.nl) and from 3.40 (Kiesbeter.nl) to 5.50 (JessaZH.be and Chirurgenoperatie.nl). Texamen scored the highest average at C1 (average 4.77) with a range from 3.60 (Kiesbeter.nl) to 5.25 (BeautifulABC.​com) (Table 4).

Average CEFR levels of both nations were estimated at B2/C1. No significant difference (p = 0.200) was noted in readability scores between the Netherlands (4.34 SD ± 0.57) and Belgium (4.63 SD ± 0.39). A web site from the Netherlands, Kiesbeter.nl, had the lowest overall average CEFR level at B1 (3.10), followed by Kanker.nl with B1/B2 (average 3.67). DeVoorzorg.be had a B2 CEFR level (average 3.80) which was the lowest for Belgium and third overall. The other 17 web sites were estimated at B2/C1 (50%) and C1 (35%) (Fig. 2).

In total, 27 (32.1%) individual web site articles were found to have a CEFR level B1 or lower, analyzed using at least one readability tool.

Health information is more easily accessible due to the increasing influence of the Internet and mobile devices [27]. The extended availability of educative materials might have benefits; well-informed patients may become capable and willing of participating in treatment decision-making [28]. In 2014, more than half of the population of the Netherlands and Belgium searched for online health information [9, 10]. However, negative influences on patients cannot be excluded due to the unverified accuracy of data, misleading information and choosing self-treatment instead of consulting a healthcare professional [6]. The worldwide incidence of cancer is increasing of which breast cancer is the most frequent malignancy in women [12]. Mastectomy remains an essential part of breast cancer treatment or prophylaxis [29]. Literature shows that over two-thirds of breast cancer patients have queried the internet for information on breast reconstruction [28]. It seems realistic to argue that the number of patients seeking health information online is likely to grow in the future. In this study, we analyzed the readability of Dutch patient-directed health information on breast reconstruction available on the Internet as it is essential for patients to have access to good quality information, and readability plays a key role in this. Our results show that online health information readability, or reading comprehension, level on breast reconstruction is often too high for the average population. Of the 20 included web sites, one was found to comply with the recommended CEFR level B1. Fifty percent of the web sites were estimated at level B2/C1. This study is the first readability assessment of online health information on breast reconstruction in the Dutch speaking population.

The Dutch government recommends the use of CEFR level B1, comprehendible for 95% of the population [7]. Dutch immigration and integration laws state that one is required to understand, speak, and read on CEFR level A2 in order to become a Dutch citizen [30]. The company behind “Texamen,” “BureauTaal,” reported that Level C1 on the CEFR scale is comparable to an academic grade language which implies that approximately 60% of the population of the Netherlands and Belgium will have difficulty comprehending information [31]. The Netherlands and Belgium are frequently considered to be countries with excellent educational systems, both ranking in the highest quartile of the Human Development Index by the United Nations [32]. Despite that standing, literacy difficulties affect nearly 12% of the population in the Netherlands and 15% in Belgium, and this topic should not be underestimated or disregarded [33]. Due to the easy access and increasing availability of the internet, suitable online health information readability is crucial. Readability tools can help analyze and fine-tune web sites content. These tools have been introduced in other languages; results of previous literature on the readability of English written information on breast reconstruction were in line with ours [19‐22].

In 2014, Vargas et al. studied the top 10 web sites and reported that English online health information on breast reconstruction exceeded the sixth-grade reading level advised by the AMA. The authors found an average overall reading grade level of 11.5 [19]. After 1 year, the same authors reevaluated readability, alongside with complexity and suitability and determined that all three factor levels were still too advanced [20]. On the contrary, a study from the UK by Light et al. included a simple readability analysis of patient-direction information on breast reconstruction and listed the majority as “easy to read” on a three level scale (easy, moderate, or difficult): Only 7.0% of all included web sites was categorized as “difficult to read” [34]. However, the methods and the tools used for readability assessment were unclear. Recently, Lynch et al. analyzed readability, accessibility, and quality of the top 75 web sites on breast reconstruction; readability and quality were found to be below standard [22]. Nonetheless, sufficient accessibility was met by 31 web sites (42.0%) based on the “Web Content Accessibility Guidelines (WCAG).” We did not include these measures due to the fact that there was nothing yet known on readability of patient-directed health information in Dutch. Deviating from the primary aim was therefore unsuitable.

We found readability levels of information on breast reconstruction from the Netherlands and Belgium to be nearly identical. This finding is consistent with previous Eurostat data showing minimal differences in their report; slightly more Belgians compared to Dutch (87 versus 82%, respectively) considered online health information easy to comprehend [9, 10]. However, in the group of dissatisfied internet users, a higher percentage of Belgians considered the health content to be too advanced or too unclear (48 versus 41%) [9, 10]. Of note, these data did not make a clear distinction between Dutch written or international content, although the latter was likely included. One of the reasons might be that health information was not available in the participants’ spoken language. This was the case in 5 % of the Dutch and 8% of the Belgian participants. We decided to include web sites from both the Netherlands and Belgium to broaden the patient search for breast reconstruction health information. Information used in the studies by Eurostat was not specifically about breast reconstruction which may have affected their findings differently than ours. Moreover, we performed an analysis on the readability of the text in order to try to predict and foresee complexities. This does not necessarily mean that patients will actually find it difficult.

Average web site readability levels were calculated, but no individual statistical analyses between numerical differences were performed. Nevertheless, it is interesting that only one (5%) web site, Kiesbeter.nl, was estimated at B1 level. It is a government web site run by the Dutch National Health Institute. Furthermore, this web site has been approved by Stichting Makkelijk Lezen (Easy Reading Foundation). The Dutch government has been actively trying to innovate their websites to an acceptable reading level [7]. Investments were made in making websites more accessible and monitoring them due to quality improvement using readability tools such as Texamen [35]. In 2006, the company behind Texamen, BureauTaal, concluded that the majority of communication, including government, was at level C1 [36]. This finding is in line with our study results. In 2015, a study examined readability levels of Dutch municipality web sites with the Accessibility.nl tool and reported an average of B2 (62%) [37]. However, 25% of these municipality websites were measured at B1. Progression has been made over 10 years when the average was measured at C1 [37]. In 2016, 46% of all Dutch municipalities had an accessibility statement on their web site based on the Dutch web site guidelines [38].

We also calculated the average readability level per tool even though this was not the primary aim of this study. This was done in order to account for any possible differences in scoring between the tools and make the reader aware of this by describing our findings per tool. Limited recourses exist and only four readability tools for the Dutch language were known to us. Kraf et al. evaluated three of these tools (Accessibility.nl, Klinkende Taal and Texamen) and found a discrepancy in scoring [26]. Our results are in line with the findings of Kraf et al. as Texamen also scored higher. Therefore, we did not want to limit this analysis to only one tool and incorporated all available four. Unfortunately, the algorithm of each tool is a trade secret of the company. This makes it hard for us to pinpoint why in some cases the same text scores differently. We are aware of most factors that are being taken into account by the tools as shown in Table 2. Yet, this is not enough to provide an accurate answer on the discrepancy in scores as we simply do not have sufficient information.

Finally, technology is becoming increasingly important. Changing technology may improve digital media information delivery and improve patient communication. Dutch research has insinuated that spoken animations could improve comprehension in case of low health literacy [39]. Nonetheless, preoperative counseling is essential combined with receiving appropriate and understandable information. Surgeons should be capable of guiding patients and assessing their needs in order to tailor them with suitable information. Appropriate readability may be rewarding in improving satisfaction and health outcomes [40, 41]. However, critics fear that oversimplification of text may in turn lead to inaccurate information and miscommunication [23]. European guidelines exist to improve readability and accessibility of medicinal product labels and leaflets that focus on text size, fonts, headings, and layout [42]. As mentioned before, the WCAG guidelines were created in order to improve accessibility [8]. The introduction and implementation of similar guidelines for (online) health information may help improve readability in particular, expanding the target group.

There are several limitations concerning this study. The readability software used for the study is the only available software for the Dutch language. Other individual readability formulas exist for the Dutch language; however, there are only few and do not use CEFR as a unit for calculating readability (e.g., Cloze score or Douma score, the Dutch version of the Flesch-Kincaid Grade Level). However, modern Dutch readability tools utilized in our study analyze and use similar formula specifics (e.g., word and sentence length) and additionally translate readability scores into a standardized European scale. Even though we used the four available CEFR readability software tools, we recognize that the limited number could be perceived as a limitation. Furthermore, limitations of Accessibility.nl, Klinkende Taal, and Texamen as identified by Kraf et al. and Jansen et al. are known. Kraf et al. analyzed Accessibility.nl, Klinkende Taal, and Texamen in 2011 and reported inconsistent readability scores which differ per tool [26]. In 2013, Jansen et al. revaluated readability assessments of the three tools by calculating and comparing them to a different readability unit; Accessibility.nl and Klinkende Taal were found to correlate significantly but Texamen was not [43]. Therefore, readability scores should be interpreted critically. More readability, availability, and accessibility tools reviewing the Dutch language should be developed for future improvement and standardization of online healthcare information. Furthermore, research should focus on identifying geographical differences between countries and analyzing to possibility of an online healthcare algorithm for global use.

Breast reconstruction is becoming an increasingly frequent procedure for women with breast cancer or those desiring prophylactic mastectomy. Our study showed that Dutch written online health information on breast reconstruction was too difficult to understand for the general population. Therefore, it may hold benefit for patient-directed (online) health information to be analyzed and revised in order to improve patient quality of care regarding breast reconstruction.