Evidence-based campaigning

The concept of evidence-based medicine has become a talisman in contemporary medical practice [1, 2]. The idea is that medical interventions should be based on research and not rely primarily on tradition or professional judgement. The evidence can be of various types; most sought after are double-blind randomised clinical trials. Although many medical procedures continue to be undertaken without strong evidence of this sort, the demand to provide evidence has become an important tool for promoting change. However, evidence-based medicine can be subverted, so care is needed to balance the imperatives of evidence with professional judgement [3].

In contrast to the role of evidence within medical practice, campaigning for public health interventions relies much more on intuition and experience, including whether methods appeal to campaigners. A classic case is the anti-smoking movement. The evidence base against smoking was overwhelming to all except tobacco-company apologists, but the evidence base for comparing diverse ways of intervening against smoking was largely absent. Methods used have included publicising research findings, making authoritative announcements (most prominently by the US Surgeon General), restricting cigarette advertisements, banning smoking in specified places, taxing cigarettes, culture jamming (refacing billboards advertising cigarettes), legal actions against tobacco companies, producing anti-smoking advertisements, requiring plain packaging and changing cultural attitudes to smoking [4]. It might be argued that any and all of these interventions are justified by the enormous harm caused by smoking, but it still can be asked, which interventions have been most effective in terms of cost and effort? This is not easy to answer, because interventions seldom can be assessed separately. For example, raising taxes on cigarettes requires information campaigns and lobbying to convince government officials to increase the taxes. Furthermore, interventions were fiercely countered by the tobacco industry.

Effective campaigning can make a huge difference to social welfare. Consider, for example, the issue of crime. Criminologists have produced findings showing that tougher penalties do not reduce the crime rate [5], yet politicians in some countries have ignored this evidence and campaigned on platforms of being “tough on crime.” In the US, the result has been incarceration of millions of people at an expenditure of hundreds of billions of dollars [6]. Another example is the so-called war on drugs, namely enforcing prohibitions against illicit drugs, costing vast amounts of money and stimulating criminal activity apparently without significant impacts on drug use [7‐9]. The war on drugs was launched without strong evidence that it was more effective in reducing drug harms than alternatives [10].

There is a fair bit of evidence about the effectiveness of social interventions, for example to reduce crime or improve student performance; some popular interventions turn out to be counterproductive [11]. Evidence-based campaigning sounds worthwhile, but developing a sound evidence base is far more complex than when testing drugs, medical devices or even social interventions. There is a considerable body of writing about campaigning for social change (e.g., [12‐15]), but most recommendations are based on experience, not systematic testing [16].

At least five types of complexities need to be addressed.

Both the difficulties and importance of evaluating campaigning methods are shown by the long-standing debate between campaigners seeking to overthrow repressive governments. Some challengers use the method of armed struggle whereas others use methods of nonviolent action such as rallies, strikes, boycotts and sit-ins. Disagreement between advocates of these two approaches has persisted since at least the days of Gandhi’s early campaigns in India in the 1920s, when his leadership of the independence movement was questioned by Marxists such as M. N. Roy. Proponents of armed struggle and of nonviolent struggle used selected examples to argue their respective cases, but it was not until recently that systematic comparisons were undertaken. Chenoweth and Stephan [18] analysed 323 conflicts over a century and found that nonviolent anti-regime struggles were far more likely to succeed than armed ones. Their research will not end the debate, but for those subscribing to evidence-based campaigning, nonviolence should be the current recommendation for anti-regime struggles. In this case, the implications in terms of lives and freedom are enormous.

To help provide insight into the difficulties of evaluating the effectiveness of different approaches to health campaigning, in the following section ten ideal-type approaches are proposed. In the subsequent sections, application of these ideal types is illustrated using a case study from promotion of vaccination in Australia. Three main approaches to recent vaccination promotion are identified, each involving several ideal types, illustrated using submissions to a government inquiry. Then some of the arguments and evidence for each of the three approaches are examined, illustrating the difficulties in making a definitive judgement about their comparative effectiveness. A key conclusion from this analysis is that evaluating approaches to campaigning is far more challenging than evaluating medical interventions, and therefore scepticism about the claims of advocates is warranted.

The three campaigning approaches for promoting vaccination in Australia can be used to illustrate ideal types. Each approach potentially incorporates several ideal types, as indicated in Table 2.

Table 2 illustrates that each of the three campaigning approaches is likely to involve more than one ideal type. Table 2 lists only the dominant ideal types expected for each approach, omitting types used less frequently. For example, SAVNers often cite evidence in favour of vaccination.

To illustrate a manifestation of the ideal types, it is convenient to examine submissions to a 2015 Australian Senate committee inquiry into proposed legislation to remove religious and conscientious objections to children’s vaccination for parents to receive certain welfare benefits [34]. Plans for this so-called “No jab, no pay” legislation triggered considerable public debate, and there were thousands of submissions to the inquiry. Of these, just three are scrutinised here: the Australian Medical Association [35], in the category of health authorities; SAVN [36]; and Leask and Wiley [37], in the category of researchers.

These three submissions are the best representatives of the three categories treated here. A comment is in order about related submissions. In the category of health authorities, Australian government health departments apparently did not make extensive submissions: the New South Wales Health Department made a one-page submission (#345) and the Royal Australasian College of Physicians made a short submission (#344). Other submissions in the same approach category as SAVN were those by the Australian Skeptics (#264), Friends of Science in Medicine (#316), and Northern Rivers Vaccination Supporters (#263). In the category of researchers, the submission by the Public Health Association of Australia (#317) is largely researcher-oriented.

When, in one of the three submissions chosen for analysis [35‐37], a significant statement or an extended discussion reflecting a particular ideal campaigning type was found, a quote is used in Tables 3, 4 and 5 to illustrate the presence of the type.

Although Tables 3, 4 and 5 are not a complete representation of the themes in the three submissions, they do highlight distinct differences in approach. Most obviously, the SAVN submission [36] is dominated by negative approaches (Table 4) whereas the submission by researchers Leask and Wiley [37] is dominated by positive approaches (Table 5).

These tables are illustrative only, as several qualifications apply. First, submissions made to the inquiry may not fully represent methods used by health authorities, SAVN or researchers. Second, the submissions chosen for examination illustrating each approach are not necessarily typical of the overall approach. Third, submissions to an inquiry are a constrained type of health promotion; for example, they are less likely to contain publicity materials. Fourth, submissions may not articulate methods actually used in practice. For example, Dunlop and Stokes [36] do not mention SAVN’s efforts to censor the AVN.

Examination of Tables 3, 4 and 5 shows that the distribution of expected types itemised in Table 2 only partly overlaps with the distribution found in the submissions. This can be explained by the qualifications noted above; a more comprehensive survey of materials and actions would enable a better assessment of whether the expected types in Table 2 are appropriate.

Returning to Table 2, it would be possible to develop a more elaborate classification. Each principal group relies on others, who might be called agents, to implement its approach. For example, health departments and SAVN use the media to pursue their agendas; SAVN seeks to enrol government agencies as agents to attack critics; researchers aim to influence doctors and nurses to use their method of encouraging parents to vaccinate, while relying on government health departments to provide the context for their own efforts.

The tables can at most capture a snapshot of an ongoing process of campaigning. For example, the availability of the method of endorsement depends on the current state of play, in particular the willingness of experts to make a stand. In general terms, the campaigning methods depend on the current phase of a controversy, including the scientific evidence, key players and resources available.

Noting these limitations of the classification of approaches to campaigning illustrates a key point: it is difficult to analyse the effectiveness of an approach to campaigning, because there are so many complications and overlaps.

The promotion of vaccination in Australia overall has been highly effective, and promoters claim credit for the relatively low numbers of deaths and disabilities from vaccine-preventable diseases. However, this observation does nothing to distinguish the effectiveness of different approaches to campaigning.

Government health departments seem to imply that their current mix of methods to promote vaccination is optimal. For example, vaccination is free, which is an incentive, but vaccination is not mandatory except for some military personnel and health sector workers. However, health departments do not cite any rigorous studies to justify their choice of methods. For example, they do not give any empirical evidence that mandatory vaccination for health workers improves health outcomes.

SAVN commentators claim they have been effective, saying that their activities have curtailed the influence of the AVN by reducing its income and its credibility in media stories [38‐40]. However, SAVN, despite a massive investment of effort in its campaign against the AVN, has never presented strong evidence that its campaign has increased vaccination rates or reduced the incidence or impact of vaccine-preventable illness.

Researchers into effectively promoting vaccination to parents can point to the results of various studies. However, whether widespread adoption of their approach by doctors and nurses would significantly improve vaccination rates and population health remains to be demonstrated.

There is ample evidence that vaccination can reduce morbidity and mortality for specific diseases [41, 42]. Citing such evidence is standard in vaccination campaigning, fitting within the category of ‘arguments for’. However, comparatively speaking, there is a dearth of evidence for the effectiveness of different approaches to promoting vaccination.

SAVN’s campaigning has been based on the assumption that the AVN’s activities had negatively affected vaccination rates and that discrediting and silencing the AVN would lead to improved health outcomes. A contrary view is that vaccine-critical groups have had little effect on vaccination rates, but rather are a response to concerns about vaccination that arise for other reasons, such as perceived adverse reactions of children to vaccinations [43]. This is compatible with the findings of a survey of AVN members showing that very few initially developed vaccine-critical views as a result of AVN materials; more commonly, members had concerns about vaccination and were attracted to the AVN because it provided a forum for these concerns [44]. From this perspective, trying to stifle vaccination critics is likely to have little impact on vaccination rates.

A possible proxy for the effectiveness of SAVN, in terms of its goal of promoting vaccination by discrediting and silencing critics, is the level of conscientious objection to vaccination. According to figures [45], the percentage of children whose parents sought conscientious objection increased every year from 2000 to 2014. The figure for 2015 is anomalous because the conscientious objection option was removed on 1 January 2016, reducing the incentive to apply in 2015. See Table 6. The increase from 2000 through 2014 may reflect parents’ increased awareness of the provision for conscientious objection [35, 37, 46]. The point here is that this trend predated the formation of SAVN and the furious struggle between SAVN and the AVN; the rate of increase, in percentage points per year, seems not to have changed substantially after SAVN became active beginning in 2009. Although this does not prove that SAVN has been ineffective, it is compatible with the view that vaccine-critical groups are more a product of parental concerns than a cause.

The aim here is not to assess the effectiveness of different approaches to promoting vaccination, but rather to illustrate the difficulties of assessing campaigning methods. It is useful here to return to the five types of complexities of campaigning noted in the introduction that make such assessments difficult and see how they are played out in the Australian vaccination struggle.

The argument here is that campaigners on many health issues proceed with their preferred approach without much solid evidence that it is superior to alternative approaches. There is a certain irony in championing interventions to support evidence-based medicine while not having strong evidence to support the choice of how to promote the intervention. This seems inevitable due to the great difficulty in obtaining evidence at the level of approaches.

To help clarify campaigning options, ten ideal types were presented. In practice, approaches to campaigning, as the term “approaches” is used here, typically encompass several ideal types. Within an approach, it is more feasible to assess the effectiveness of campaigning methods. For example, if compliance is sought by rewards, then it is feasible to compare the effect of different rewards. However, when comparing different ideal-type methods or mixtures of methods, comparisons are more difficult.

Several reasons were presented as to why comparisons of campaigning approaches are more difficult than assessment of medical interventions. These include absence of controls, lack of independence of those doing assessments and the resistance of some subjects to interventions. A double-blind study of campaigning methods is hard to conceive.

An implication of this examination of campaigning methods is that claims about the effectiveness of particular strategies should be treated sceptically, especially when the claims are made by partisans involved in campaigning. Determining the effectiveness of campaigning approaches is vitally important, but it is wise to recognise the limitations of current knowledge.