Background
Health information and misinformation are readily accessible to the general public, particularly through mass media and the Internet [
1‐
7]. Due to the ease with which large amounts of health information can now be accessed, people are playing a more active and autonomous role in their health [
1,
8]. The health information that people access can affect their health decisions and behaviours—for example, from how they maintain their health and cope with a chronic condition to decisions made about how to treat an illness or whether to consult a health professional [
9,
10].
As well as websites and traditional information sources such as magazines, radio and television, health information is also available on social media such as Facebook [
11], YouTube health channels and Twitter [
7,
12]. Regardless of the type of media, health information remains essentially unregulated, and problems and concerns about its quality have been noted [
7,
13‐
15]. Consequently, people are exposed to and have to navigate vast amounts of health information, often on their own, typically with little knowledge about how to evaluate it or the need to do so [
16].
Health interventions (an umbrella term for anything that can potentially change a health outcome, such as medical treatments, food and drinks, beauty products, exercise or therapy and behaviour changes) are one of the most commonly researched health topics [
9,
10], yet the quality of health information is variable [
5]. Some health interventions are promoted using phrases such as ‘evidence-based’ and ‘clinically proven’. Phrases such as these are intended to convince people of a health intervention’s effectiveness, so when the claims are unwarranted, they can be misleading. Some of the other complexities that can influence people’s decision-making about health interventions are that some people tend to rely on anecdotes rather than information derived from research [
17,
18]. These can overrate the trustworthiness of the health information they find, and most overestimate the benefits and underestimate the harms of health interventions [
19]. Belief in false or unsubstantiated claims about health interventions may result in people receiving inappropriate health interventions that are at best ineffective, at worst harmful, as well as wasteful of healthcare resources. Conversely, not believing claims that are based on reliable research evidence about beneficial health interventions can also cause harm, for example, through inappropriate treatments or delays in seeking appropriate health care [
20‐
24] or choosing ineffective treatments over effective ones [
25].
Health information may be misleading, misinterpreted or leave people confused [
13,
26]. People require basic skills to assess the quality of both general health information and information about health interventions and their effects. Without education about key concepts relevant to evaluating the effects of health interventions and how to interpret research results, people are, irrespective of their level of education, vulnerable to believing untrustworthy treatment claims and may make health decisions based upon information that is inaccurate, incomplete or even harmful [
21,
25]. Because of its nature and extensive reach, health information on the Internet is impossible to regulate. However, providing people with knowledge about key concepts in evaluating information about health interventions may help them evaluate the trustworthiness of health intervention claims and to make informed decisions. The general public is typically not trained to evaluate the accuracy and completeness of information about health interventions [
16].
Most of the existing research on helping people to understand health information has focused on the traditional skills associated with health literacy, such as reading, numeracy and understanding speech. Limitations in these skills can impact upon people’s ability to navigate the health system and are associated with poorer outcomes and decreased uptake of health services [
27]. Previous systematic reviews have examined the effectiveness of related educational interventions such as teaching online health literacy to the general public [
27], critical appraisal skills to health professionals [
28] and the effects of educational interventions on critical appraisal abilities in school students [
29]. As far as we are aware, however, there is no systematic review of studies of educational interventions designed to improve critical appraisal abilities in the general public. The aim of the current systematic review is to assess the effectiveness of educational interventions designed to improve people’s understanding of key concepts (described below) when evaluating claims about the effects of health interventions.
Discussion
This review found 24 eligible studies (14 randomised trials and 10 other eligible study types) which had examined the effects of educational interventions about key concepts for assessing claims about health interventions. Measures of the primary outcomes (knowledge, knowledge and skills) were better in educational intervention group participants than in comparison group participants, with statistically significant differences for the majority. The effect of the educational interventions on secondary outcomes (such as confidence, perception of knowledge and/or abilities, attitude, behaviour and satisfaction with the educational intervention) was less consistent. Some studies found statistically significant between-group differences favouring the educational intervention group, some found differences favouring the educational intervention group but which were not statistically significant, while a few reported results which either favoured neither group or favoured the comparison group.
Across the studies, most of the outcomes were measured immediately following the conclusion of the educational intervention, with only a few studies measuring outcomes a short time (typically 2 to 6 weeks) later. Hence, whether the effects of the educational interventions are sustained in the long term is unknown.
Apart from two trials which were reported in the same article [
39,
40], none evaluated the same educational intervention as any of the other included studies. Some variation was expected due to the inclusion of different target learners (e.g. children and adults) and different settings (e.g. formal education settings, community groups), but even beyond this, there was substantial variation in the content taught, the intensity and duration of the educational interventions, the educational format and the educational intervention provider.
The situation was similar for outcome measures, with no two studies using the same outcome measures except for the two trials that were performed and reported together [
39,
40] and three other trials that were associated [
37,
38,
48]. One consequence of this wide range of measures is that synthesis of results from multiple studies is hampered. Few of the measures used were validated, with most developed for each study. A set of flexible evaluation tools (CLAIM Evaluation Tools) that can be used to measure people’s ability to assess claims about treatment effects has recently been developed and validated [
62]. This outcome assessment was used within three included recent studies [
37,
38,
48]. The use of common outcome measures in future studies of such educational interventions would facilitate comparison of results and synthesis.
Risk of bias in the studies was variable, with the risk high in at least one domain for all randomised studies. There were a few good-quality randomised trials and mostly high or uncertain risk of bias in the other trials. Given the heterogeneity and risk of bias in the studies included in our review, it is difficult to draw firm overall conclusions about the effects the educational interventions on the outcomes of interest. Neither is it possible to make recommendations about the characteristics of educational interventions (such as particular content, duration and format) that are essential. As there are very few head-to-head comparisons of different interventions, currently, there is no reliable evidence that one type of intervention is more effective than another.
That said, educational interventions evaluated in randomised trials with low risk of bias which yielded statistically significant better effects than control and provided a description of educational intervention content and delivery to enable similar projects and potentially easy to implement are promising. Some of these interventions address a wide range of topics, others specific topics, but all of them have only been evaluated in a single trial and need replication.
Promising educational interventions for adults covering a wide range of relevant topics are those evaluated in the trials by Semakula and colleagues [
48] and Welch and colleagues [
50]. The educational intervention in the Semakula trial was a series of podcasts (which aimed to improve the ability to assess claims about treatment effects; see link in Additional file
1: Table S1). The Welch trial was a web-based module (which covered the basics of the evidence-based process, including literature searching and critical appraisal). Although critical appraisal skills were taught, the Welch study assessed only knowledge, with the impact on skills remaining unknown. A promising educational intervention for adults with a focus on specific topics is the 80-page booklet about understanding disease risk and health intervention benefits and harms (see link in Additional file
1: Table S1 to access) evaluated by Woloshin [
39,
40].
For children, promising educational interventions covering a range of topics are those evaluated by Nsangi et al. 2017a [
37] and Tait et al. [
49] The educational intervention evaluated by Tait [
49] was a digitally interactive, individually completed program delivered via an iPad and focused on improving knowledge about clinical trials, including research basics, protocols, randomisation, placebo and blinding. The educational intervention evaluated by Nsangi [
37] requires more investment of time (80 min per week over 9 weeks) and used a teacher-delivered program and set of learning resources (including a textbook in comic book format, teacher’s guide, exercises books, poster, activity guides—see link in Additional file
1: Table S1 to access).
To the best of our knowledge, this is the first systematic review designed to identify all the educational interventions developed to improve people’s knowledge and abilities related to the key concepts of critical appraisal of health claims. A strength of the review is our extensive search for and rigorous assessment of the available studies. Our results expand upon the review reported by Nordheim et al. [
29], who synthesised similar educational interventions but only those specifically targeted to child learners with school-based educational interventions. Their review included five studies, and the main findings were similar to ours, namely, that the studies showed that educational interventions can improve short-term knowledge and skills (Kirkpatrick level 2), with effects on other outcomes unclear and conclusions limited by the low quality of studies [
29]. All but one of the studies in Nordheim’s review were conducted in the USA, whereas there were ten countries represented in the current review, about half of which were conducted in the USA.
Limitations of study
This review’s limitations mostly arise from the heterogeneity of the included studies and the risk of bias in many of the eligible studies identified. Despite using a comprehensive search strategy, it is possible that some relevant studies have not been identified because of the complex question addressed by our review and also the potential for non-English studies to have been missed if they lacked abstracts/titles in English.
Conclusion
Considering that people now enjoy increased access to health information and are more involved in making decisions about their health than previous generations, it has become increasingly important that they have the knowledge and skills to assess which information is trustworthy. This review has shown that educational interventions can improve knowledge and skills, at least in the short term, and drawn attention to educational interventions that have been shown to have this effect. The longer-term effects and effects on behaviour, attitudes and confidence remain uncertain.
More certain estimates of the effects of educational interventions on critical assessment of health claims require well-designed educational interventions, validated outcome measures (including measures of skills and longer-term follow-up), rigorous study designs, such as pragmatic randomised trials, and assessment in a variety of populations (considering ethnicity, socioeconomic status and education levels).
Acknowledgements
There was no specific funding provided for this systematic review. IC acknowledges the salary support from the National Institute of Health Research, UK, through a grant to the James Lind Initiative. We thank Justin Clark for his vital role in the development of the search strategies, Sandy Oliver for the appreciated feedback on the protocol, Melanie Vermeulen and Laura Desha for assisting with the study translations and Andy Oxman and Astrid Austvoll-Dahlgren for their valued contributions to the project. We would also like to thank Daniel Semakula, Allen Nsangi, Anke Steckelberg, Ingrid Mühlhauser, Lena Victoria Nordheim, Marcus Gillespie, Samkelo Nsibande, Ames Dhai, Seth Kalichman, Richard Osborne, Alan Tait, Davina Mill, Wei Du, Helen Cahill and Tom Grey for their helpful correspondence and Sophie Hill and members of the Cochrane Consumers and Communication Group for their initial comments on the review’s protocol and for sharing the group’s protocol template.