The evolution of health literacy assessment tools: a systematic review

The review was completed by using PubMed, the Educational Resources Information Center (ERIC), the Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Knowledge databases. Additionally references in already published reviews and other publications were screened and a manual search on websites and print sources dealing with health literacy measurement was conducted. The search strategies encompassed key words as well as MESH terms depending on the database and were supplemented by synonyms and thesaurus terms as described in Additional file 2. The search was performed from January 2009 to 24^th April, 2013 and was limited to fully available English language publications developing and validating (testing, evaluating) generic instruments to measure health literacy. We decided on the specific period of time to cover literature left out in previous reviews on health literacy measurement [13, 20]. The search was limited to instruments targeting adolescents and adults. Translations of instruments originally published before the search period were excluded.

Articles were included by screening titles and abstracts of all unique publications and formulating full-text reports of all records passing the title/abstract screen by two independent reviewers. All health literacy instruments were categorized according to their characteristics including their purpose, applied health literacy taxonomy and construct, instrument design, availability, scoring method, validation-study sample characteristics as well as psychometric properties such as reliability and validity of the instrument. Additionally, a quality assessment according to the specifications of the reporting guidelines for survey research (SURGE) was performed. It encompasses reporting items in eight subdomains: article background, methods used, sample selection criteria, research tool characteristics, response rate as well as presentation of results, interpretation and discussions of findings and requirements of ethics and disclosure [21]. Categories within the framework were adapted when relevant for the appraisal of health literacy indices. The accurate reporting on the development and validation of instruments assessing patient reported outcomes such as health literacy is important in terms of an objective assessment of applied methods and identified psychometric properties of instruments and therefore the generalizability of study results. Especially during the research process transparency concerning methodological issues of indices can help to enhance overall study quality by allowing refinements on the instrument. The SURGE is an adequate instrument to appraise the reporting quality in surveys including detailed information on the characteristics of the used survey instruments. Therefore, it served as an appropriate instrument to appraise the reporting quality of health literacy indices.

After extracting the instrument characteristics these were entered into an evidence table and critically assessed for reporting quality by two independent raters, followed by a third rater checking the extracted information for accuracy. Disagreements were resolved by a consensus process between the three raters.

Among the 17 included publications on the development/validation of a generic health literacy measurement tools certain patterns can be identified. As depicted in Table 1, about one third of the instruments use either a direct test of an individual’s abilities (objective measurement) or the elicitation of self-reported abilities (subjective measurement). In studies using the objective measurement approach, patient abilities are assessed by solving tasks dealing with print literacy, numeracy or oral literacy whereas the self-report approach is characterized by the self-report of perceived abilities in multiple domains. Moreover according to Table 1 the combination of both measurement modes can be found in 41,2% of all identified instruments, respectively.

The generic instruments identified here consider the multi-dimensional measurement approach by applying print literacy in all instruments and measuring quantitative abilities in almost half of the identified tools. In contrast, only three instruments are considering communication skills of individuals when measuring an individual’s health literacy. Following the theoretical framework of health literacy being a multidimensional, dynamic construct [36] with an comprehensive approach, 76,5% of all identified instruments are based on a multidimensional construct of health literacy as shown in Table 1. Therefore multiple domains of health literacy are addressed such as healthcare information seeking, communication in the patient-provider encounter, interaction with the health care system and awareness of rights and responsibilities [37]. Additionally the utilization of a multidimensional measurement approach is pursued in almost all instruments mostly assessing print and quantitative literacy.

The direct testing of competencies related to the health literacy construct is used frequently in the academic literature and five novel instruments were published in the search period. The Medical Term Recognition Test (METER) developed in the United States is a brief self-administered screening tool (2 min administration time) for the clinical setting and includes 40 medical words and 40 words without an actual meaning (non-words) while aiming the identification of the medical words [38]. The format of the tool includes many words from the Rapid Estimate of Adult Literacy in Medicine (REALM)[39]. Thus, there is a high correlation (r = 0.74) between the instruments [38]. The Short Assessment of Health Literacy in Spanish and English populations (SAHL-S&E) also uses a word recognition approach as applied in the REALM and combines these with a comprehension test using multiple choice questions designed by an expert panel [40]. To guaranty word recognition as well as comprehension the examinees read aloud 18 medical terms and associate each term with another word similar in meaning. The English as well as the Spanish version of the test demonstrate high correlations to other health literacy indices, display high reliability values and are particularly suitable to screen individuals with low health literacy [40]. One instrument developed to measure health and financial literacy addresses the link between literacy and decision making in the context of health related and financial factors. It examines health literacy by using 9 items dealing with health knowledge regarding health insurance, burden of disease as well as medication skills [41]. The test to measure critical health competencies (CHC-Test) consists of 72 items presented in 4 scenarios dealing with skills such as the understanding of medical concepts, searching literature, basic statistics and the design of experiments and samples [42]. The bilingual health literacy assessment (Talking Touchscreen) focuses on building a novel item pool in accordance with items used in the Test of Functional Health Literacy in Adults (TOFHLA). It measures prose, document and quantitative literacy in the field of certain lifestyle diseases as well as insurance related issues and patient rights administering these items with a multimedia gadget [43, 44]. A detailed description of the characteristics of instruments using an objective measurement approach is, presented in Table 2.

All identified instruments measuring health literacy by a self-report use a multidimensional concept of health literacy by integrating several domains and factors associated with health literacy. The self-report approach was applied in five instruments published in the search period. The Multidimensional Measure of Adolescent Health Literacy (MAHL) assesses health literacy as a dynamic construct by addressing several domains: patient-provider encounter, interaction with the health care system, rights and responsibilities and health information. These are developed by analyzing items of numerous already existing instruments, identifying relevant items and modifying as well as supplementing them by new items [37]. The Health Literacy Management Scale (HELMS) consists of 8 scales with 4–5 items and aims to assess health literacy by using a comprehensive approach. It encompasses multiple domains such as patient attitudes towards health and their proactivity as well as access, understanding and use of health information and access and communication with healthcare professionals [16]. The 127 item Swiss Health Literacy Survey (HLS-CH) also addresses numerous domains such as information and (critical) decision making, cognitive and interpersonal skills as well as problem solving. In this regard health literacy is rather a package of competencies interacting with each other [45]. The All Aspects of Health Literacy (AAHLS) measures health literacy based on the framework developed by Nutbeam [46] and measures functional, communicative and critical literacy by using 14 items derived from an analysis of already existing scales in the field of health as well as media literacy [47]. Seemingly relevant items from numerous sources were adopted, partially modified, and supplemented resulting in an adequate overall reliability of Cronbach’s alpha = 0.74 whilst weak consistency among the subscales. The 63 item Health Literacy Scale developed in Taiwan (MHLS) also captures health literacy as a multi-domain construct encompassing obtaining, understanding and processing health related information related to health promotion, disease symptoms, diagnosis, and treatment and using them in decision making [48]. A further detailed description of the characteristics of instruments applying a subjective measurement approach is, presented in Table 2.

The combination of a direct testing and a self-report of health literacy skills is practiced frequently among indices, thus seven instruments identified in the search period use this approach. It enables to combine the methodological advantages of both approaches by diminishing possible straits [49]. The Health Literacy Skills instrument (HLSI) as well as the short form (HLSI-SF) are 25/10 item tools that use real life health stimuli to assess an individual’s health literacy addressing print, oral, quantitative and internet based information seeking skills. The short form is derived by analyzing the psychometric properties of the HLSI and selecting best performing items. Additionally an 8 item self-report of the perceived performance among the skills addressed in the direct assessment of health literacy is conducted. Both approaches assess print literacy, numeracy and oral literacy as well as media literacy in a different manner demonstrating an acceptable internal consistency reliability of a Cronbach’s alpha of 0.86 for the HLSI and 0.70 for the HLSI-SF [36, 50]. The European Health Literacy Survey (HLS-EU) carried out in eight European countries (Germany (NRW), Bulgaria, Austria, Greece, Spain, Ireland, Netherlands, Poland) also uses a mixed assessment approach measuring functional health literacy with the Newest Vital Sign (NVS) and using a self-report survey with 47 items. It defines health literacy in three domains (health care, disease prevention, health promotion) and 4 modes (access, understand, evaluate and apply health information). Though the HLS-EU demonstrates a robust reliability of a Cronbach’s alpha of 0.97 for general health literacy the Spearman’s rho correlation between the NVS and HLS-EU with r = .245 is comparatively low indicating different constructs of health literacy [8, 49, 51]. Similar findings are apparent in the Canadian explanatory study aiming to define a health literacy measure by combining nine self-report items dealing with the access, understanding and appraisal of health information as well as communication skills in the patient provider encounter. Additionally, nine task performance (objective) items focus on understanding health related skills. A correlation between the measurement approaches could not be demonstrated [52]. A further Canadian study developing an instrument for measuring the health literacy of Canadian high school students focuses on skills to understand and evaluate health information. It uses 11 health related passages from several sources (internet, heath centers, health education and media materials) and develops 47 items examining the comprehension and interpretation of the presented information in the passages. A self-rating of health literacy skills is also included. Despite of a satisfactory overall reliability value of a Cronbach’s alpha of 0.92, bivariate correlations of r = 0.256 between the self-rating and the direct testing doesn’t indicate a strong coherence [53]. The brief subjective measure of numeracy (SNS) and general health literacy (SLS) is an 11 item instrument combining a subjective measurement of functional literacy by using the SBSQ [54] and the subjective numeracy scale (SNS) [55] with numerous previously developed objective indices to scale down bias of self-reports demonstrating a robust internal reliability [56]. The health literacy measurement applied in the special diabetes program for Indians (SDPI-HH-PL) follows a similar approach by combining items of the SBSQ to measure document literacy by a self-report and items of previously published instruments to measure numeracy by directly testing quantitative skills [54, 57‐59]. Though the mixed measurement approach broadens the health literacy framework some studies indicate an absence of coherence between the underlying constructs subsequently detecting missing correlations between the measurement approaches [8, 52, 53]. A further detailed description of the characteristics of instruments applying a mixed measurement approach is, presented in Table 2.

The application of reporting guidelines is a useful way to facilitate transparency and gauge the reliability of an instrument used in a survey. However the compliance with reporting guidelines such as the “reporting guideline for survey research” recently compiled by Bennett and colleges [21] is limited among papers reporting on the development and validation of health literacy indices as depicted in Table 3. Among the 17 identified publications, about a third does not report on the significant reporting features specified in the guideline. The reporting frequency varies across different domains of the guideline. Study objectives, presentation of the results as well as interpretation and discussion of the findings are appropriately described in all publications. Article parts related to methodological issues such as data replication and verification (58,8%), the procedures of sample selection such as sample size calculation (23,5%), and representativeness of the sample (41,2%) are reported noticeably less as described in Table 3. Furthermore, the description of the characteristics of health literacy indices is limited among features such as the instrument pretesting, reported reliability and validity as well as the scoring method, not described in 52,9%, 23,5% and 64,7%, respectively, of all publications. Additionally, 58,8% (n = 10) of the articles do not present items of the instrument entirely making it difficult to perform an appraisal as presented in Table 2. Though reflection of non-response is central among the analysis of quantitative data, only two third of the publications do report these and 82,4% do not discuss the role of non-response rates among the performed study as listed in Table 3. Similar findings apply to the handling of missings, which are not described in more than two third of the publications. However several checklists provide guidance on the reporting of survey research and instrument development and could be used in order to report on study results adequately [60, 61].