Vaccine preferences driving vaccine-decision making of different target groups: a systematic review of choice-based experiments

A standardized form was used to extract and review data from each included study. This form was in correspondence with templates used by previous reviews [19, 30] and covered topics related to the: a) study characteristics, b) choice task and experimental design, c) conduct, d) analysis, and e) journal and funding. As previous research [19‐21] indicated that choice-based experiments use various definitions/terminology for similar attributes, attributes were first divided into four overarching categories: outcomes, process, cost and other. ‘Outcomes’ referred to the results or consequences of administering vaccines. ‘Process’ incorporated the activities related to the delivery and administration of vaccines and ‘cost’ covered the (financial) costs of vaccines. Attributes that could not be grouped under the former three were classified into the category ‘other’. Within all four categories, attributes with shared features were then grouped according to their underlying concept. These groups of attributes were called ‘domains’ and can be regarded as subgroups which allowed a more comprehensive synthesis of results. Data was extracted from full text articles and corresponding supplementary material that was available online. The search, data extraction, review of study characteristics, quality assessment and data comparison were conducted by one reviewer (MD). Eligibility of ambiguous studies and study characteristics were discussed with a second researcher (IvdP). Atlas.ti (version 8.4.4) and spreadsheets of MS Excel were used for the data extraction and quality assessment.

Prior to the data comparison, the methodological quality of included studies was critically appraised by using the 13-criteria-checklist of Mandeville Lagarde & Hanson [30], which incorporates all key stages DCEs: choice task design, experimental design, conduct and analysis. Full texts of included studies were appraised by allocating scores to each criterion of the checklist. Three answer options (scores) were possible and depended on the presence of items. A score of 0 was assigned to items that were not satisfied, absent or not reported, 0.5 to items that were partly present or satisfied and 1 to items that were present or satisfied [19, 30]. The maximum score was 13 and was associated with a high methodological quality. Accordingly, the minimum score of 0 indicated a low methodological quality. As recommended by the developers of the checklist [30], the quality threshold of 75% was used. A quality score of at least 10 (out of 13) was considered sufficient to be included in the data comparison.

A descriptive synthesis was used to indicate the relative importance of individual vaccine attributes for specific target groups. The amount of times that particular vaccine attributes (domains) were reported in studies were counted as well as the amount of times domains were reported statistically significant by the authors (incl. p-value threshold or alpha). Main models and overall results were used if available. Subgroup results were used when outcomes were reported for subgroups/classes only (i.e. no overall data). If a study included multiple attributes related to the same domain (e.g. ‘vaccine side effects’ and ‘risk of dosing’ both targeting the domain ‘vaccine risk’), statistical significance was reported for each of the attributes separately. This implies that a study could report statistical significance for a single domain more than once. To ensure a more accurate reflection of domains driving decision-making, the amount of studies reporting statistical significance, for a particular domain, were also stated. If no p-value threshold was reported but p-values were given, the commonly used threshold of p < 0.05 was used [31]. The overall frequency as well as the classification of statistically significant attributes/domains were presented in a tabular summary and were reported for each target group [32].

To determine whether exclusion of lower-quality studies changed findings for any of the target groups (i.e. inferred selection bias), results of only including high-quality studies (quality score ≥ 10) were compared to results of including all eligible studies (quality score 0–13). Face validity was used to determine if and to what extend results were in accordance with each other.

Before the start of the study, a review protocol was submitted on PROSPERO (ID: 178245). The review was executed as planned/described. No ethical approval of a Medical Review Ethics Committee was needed [33].

General characteristics of the 42 included studies can be found in Tables 1 and 2. Most choice-based experiments applied a DCE or CA format (resp. 73.8 and 23.8%). The majority of the studies (n = 37) were published after 2010: nineteen (45.2%) between 2011 and 2015 and eighteen (42.9%) between 2016 and September 2020 (Table 1). Included studies were conducted in twenty countries mainly spread across Europe (n = 25), Asia (n = 11) and North America (n = 8) (Table 2). More than 80% was conducted in HICs (n = 34). Choice experiments were least performed upper and Low-Middle-Income Countries (LMICs). Fourteen existing vaccines or vaccine programs were studied, most commonly HPV and influenza vaccines (Table 2). Vaccine programs referred either to the administration of a course of vaccines (e.g. all childhood vaccines) or to combination vaccines (e.g. Tdap). Preferences of representatives were most often studied (47.6%, Table 1). This target group usually referred to (expectant) parents, guardians or caregivers (n = 19), in particular mothers of children aged below 5 (n = 9). Vaccinees were targeted in a third of the studies (35.7%). They either focused on the (general) adult population (n = 7) or children/adolescents (n = 6), especially teenage girls. Preferences of health advisors such as paediatricians were least captured among included studies (9.5%, Table 1). The variety of objectives reported in studies is presented in Table 1.

The number of choice tasks ranged from four to 36, with most studies (40.5%) including less than ten choice tasks (Table 3). Ten studies (23.8%) used one method to identify appropriate attributes and levels, while 31 (73.8%) used more than one method listed in Table 3. Literature reviews and qualitative research such as focus groups were most popular (Table 3). Nearly all studies (92.9%) presented two or more vaccine scenarios per choice task and used a multinomial choice structure. Among these studies (n = 39), D-efficiency was most often used to pair and group choice profiles (35.9%). In addition, choice questions were mainly unforced and an option to remain undecided (opt-out) was provided (51.3%) (Table 3). Multiple descriptions were used to indicate the opt-out alternative (e.g. ‘no vaccination’, ‘neither’). Sixteen studies (41.1%) forced respondents to choose between two vaccine scenarios, the majority of them (n = 12) provided an opt-out in second instance (two-stage choice). Respondents were for instance asked if they would make the same choice in real life [34]. Two studies [35, 36] reported different formats in main texts and example questions.

An overview of the experimental designs used across studies is presented in Table 4. A fractional factorial design was used in 35 studies (83.3%). Only one study (2.4%) used all possible combinations (full factorial design). Among the studies reporting their type of design (n = 36), a range of software packages was used, with Ngene being most popular (25.0%). Eight studies (22.2%) used approaches other than software such as a catalog or a manual approach. Studies that included interaction terms along with main effects (n = 23) generally used more choice tasks than the studies analysing main effects only (n = 2). Although seventeen studies (40.5%) did not provide details on their design plan in the main text, primary analyses of thirteen studies showed that it was restricted to main effects (Table 4).

In total 226 vaccine attributes were included in the 42 choice experiments. The number of attributes per study ranged from three to eight, the number of levels per attribute ranged from two to seven. With regard to the overarching categories, 38.9% of the attributes were categorized as outcome, 24.8% as process, 23.0% as other and 13.3% as cost. Overall, eighteen domains were identified (3 outcome, 8 process, 1 cost, 7 other). Details on the categories and domains are delineated in the data comparison sections (Tables 7 and 8) and in Additional file 3.

More than 80% of the studies reported a pilot and/or soft launch (Table 4). The size differed from four [37] to three hundred respondents [38] and from a single-stage [35] to multiple-stage procedures (e.g. combination of pre-pilot, pilot and soft launches) [39, 40]. The majority of self-administered surveys was completed online (25 studies). Sample sizes ranged from fifty [41] to 2505 respondents [42]. Most studies included between two and four hundred respondents (Table 4). Larger sample sizes were not necessarily accompanied by the use of stricter thresholds (e.g. p < 0.001). The rule of thumb proposed by Orme [43] was most often used to justify sample sizes of CAs. Half of the studies did not justify their sample size nor included sample size calculations (Table 4). A third of the studies (35.7%) compensated respondents in cash, vouchers or a physical gift, the value varied from £1–2 to $55 (Table 4).

A summary of the approaches used to analyse data is presented in Table 5. Half of the studies applied mixed or random parameter logit models (MXL/RPL). Random or mixed effects logit models were most often used to analyse forced choices. Nearly all studies (97.6%) accounted for variation in preferences across groups. Subgroup analyses were either performed by using separate models for different groups or by incorporating interaction terms into the model. The methods used to distinguish subgroups are outlined in Table 5. Methodological aims drove the subgroup analyses of 29 studies. These studies used for instance different cost ranges for subgroups or compared groups who passed and failed the consistency/dominance tests and groups with and without preference to opt-out. With regard to the outcome measures, welfare measures such as WTP were most frequently used (45.2%), followed by probability or uptake analyses (42.9%). Least reported measures were market simulations, willingness-to-accept and positive or predictive value (see ‘other measures’ Table 5). Most studies (83.3%) used software to analyse the data. These packages were not necessarily the same as the ones used to construct experimental designs (e.g. Ngene vs. Nlogit in Hofman et al. [44]).

The majority of the studies (59.5%) were published in clinical journals (Table 5). Of the 39 studies reporting their source of funding, approximately a quarter was funded or supported by a pharmaceutical or manufacturing company producing the vaccine under study (n = 11). The remaining studies, except for Ngorsuraches et al. [45], received a research grant of governmental bodies, non-profit organizations or research/education institutes (Table 5).

Studies capturing preferences of vaccinees used 48 attributes, of which 50% were classified as outcome, 16.7% as process, 10.4% as cost and the remaining 22.9% as other. Thirteen domains were identified in total (three outcome, four process, one cost and five other). Figure 2 presents the total amount of attributes incorporated in each category and domain. Importance rankings derived from this figure are outlined in Additional file 6. Most frequently used outcome measures were vaccine effectiveness and vaccine risk (both 18.8%), followed by protection duration (12.5%) and cost (10.4%). Vaccine effectiveness referred to the level of protection that a vaccine provided against a disease or to the deaths/illnesses prevented over a certain time span. Vaccine risk referred to the frequency of (mild or serious) side effects after vaccination. Context, information, other disease related factors, vaccine advice/support were also reported, but could not be grouped in any of the three categories. They are classified as other. ‘Context’ referred to vaccine coverage rates at local and population level, ‘information’ to the media coverage or attention about the vaccine and ‘vaccine advice/support’ to recommendations of family, friends, doctors, governmental bodies and international organizations. Process-related domains were least reported. Vaccine accessibility was for instance only included in the study of Verelst, Willem, Kessels & Beutels (2.1%) [74]. Statistical significance was reported in all nine studies. The average sample size was 1113 and three p-value thresholds were used to determine if attributes were statistically significant (Table 7). Vaccine risk and vaccine effectiveness, both outcome measures, were most often statistically significant (resp. 15 and 14 times). However, the latter was not found to be significant for one class of the study of Determann et al. [50]. Domains that were also commonly statistically significant included: cost, protection duration, dosing & visits, information and vaccine advice or support (Table 7). The importance of the former three was also confirmed in the frequency of reporting (Fig. 2). Factors other than disease risk, such as the spread of the disease and availability of curative treatments, were grouped under the domain ‘other disease related factors’ (other). This domain as well as the domain ‘time’ (process) were not statistically significant in any of the studies (Table 7).

In total, 48 attributes were identified in nine studies capturing vaccine preferences of representatives (Fig. 2). Of these attributes, 37.5% were outcome-related, 31.3% were process-related and 16.7% were cost-related. The remaining 11.6% of the attributes were categorized as other. Overall, fifteen domains were identified (three outcome, seven process, one cost, four other). In contrast to vaccinees, attributes regarding vaccine advice or support were not reported. However, three process-related domains were added: service delivery, target group and vaccine administration. ‘Service delivery’ covered practical aspects such as vaccine location and availability of appointments, while ‘vaccine administration’ referred to the mode of administration (e.g. injection). Vaccine effectiveness was most frequently reported (18.8%), followed by cost (16.7%), dosing & visits (12.5%), vaccine risk (10.4%) and protection duration (8.3%). In line with studies targeting vaccinees, vaccine accessibility was least reported. Two studies [65, 66] did not report statistical significance or did not provide a legend (description). For the study of Shono & Kondo [71], statistical significance could be determined based on reported p-values. Therefore, seven studies were included in Table 8. On average 1037 respondents were included and among the four thresholds used, p < 0.05 was most commonly applied. All attributes included in high-quality studies were statistically significant in one or more study/studies (Table 8). In line with the frequency of reporting, vaccine effectiveness and cost were most commonly statistically significant, followed by vaccine risk and protection duration. Domains that were least used were also least reported statistically significant (Table 8). These were categorized as process and other. Varying results were reported for information (other) and service delivery (process) (Table 8). The professional administering vaccines was for instance statistically significant, while the availability of appointments and location were not [39]. Information was found to be statistically significant half of the times, as information on benefits and risks were statistically significant, but the format was not [39].

A total of 96 attributes were identified in high-quality studies. Attributes were most commonly classified as outcome (43.8%), followed by process (24.0%), other (18.8%) and cost (13.5%). Attributes covered sixteen domains. Figure 2 showed that the same domains were valued by vaccinees and representatives. The outcome measure vaccine effectiveness was most often preferred regardless target group. The order of the remaining domains showed slight differences (Additional file 6). Vaccine risk as well as duration of protection (both outcome-related), were for instance more important for vaccinees compared to representatives, while representatives valued costs of vaccines and dosing & visits more than vaccinees (cost and process-related). For both target groups, vaccine accessibility (process) was least preferred. Statistical significance was reported (or could be determined) in sixteen studies and was most often defined by using p < 0.01. The average sample size was 1080. Studies with sample sizes above 500 usually applied multiple thresholds to determine at which point attributes were (not) statistically significant (e.g. p < 0.01, p < 0.05 and p < 0.10). Although studies capturing preferences of vaccinees reported statistical significance more often, overall results showed that for both target groups outcome and cost-related domains were most frequently significant. The domain other disease related factors (other) was not statistically significant and vaccine accessibility (process) was an equal amount of times significant (at p < 0.05) and insignificant (Additional file 7).

The quality assessment showed that the choice task, experimental design and elements of conduct received less attention compared to analysis. Studies conducted in LMICs particularly reported inappropriate experimental designs, showed conceptual overlap and failed to pilot test the survey. Although Michaels-Igbokwe et al. [19] observed similar methodological patterns in 2017, choice experiments in vaccination have not yet structurally improved their designs and conduct. The time lag between the conduct and publication of results could play a role, as high-quality studies were on average published three years after its conduct. Improvements in choice design and conduct are notwithstanding crucial to ensure reliable estimates of vaccine preferences. As recommended by Soekhai et al. [18] this might be facilitated by formulation of guidelines to report choice experiments. Furthermore, future research could broaden the approaches used to measure SP (e.g. add contingent valuation) to adequately capture preferences of health professionals. Literature [9, 93, 94] showed that decision strategies particularly differ for medical professionals and vaccinees. In this review only a limited amount of studies on preferences of health professionals could be included. Additional research could also focus on target groups other than those distinguished in this review (e.g. policy makers, based on gender) or on vaccine preferences in low-resource settings. A combination would also be interesting as qualitative research [95, 96] indicated that national decision-makers in LMICs particularly preferred simplified delivery mechanisms, thermostability and an extended shelf-life. In light of the current corona pandemic, it would also be worthwhile to assess preferences for (future) vaccines against epidemic infections, such as COVID-19 and SARS.

In contrast to previous qualitative studies [9, 11, 12], this review demonstrates that vaccine preferences show similar patterns for vaccinees and representatives. Broadly the same strategies could be adopted to promote and optimize vaccination behaviour. Strategies should focus on outcomes, for instance by providing proper and understandable information about the effectiveness of vaccines, duration of effectiveness and risks associated with vaccine administration, dosing and handling. Insight into the latter is particularly important for vaccinees. Effective pricing strategies should be applied (if applicable) when introducing or continuing the use of vaccines. Particularly for vaccine decisions that involve representatives (e.g. childhood vaccines), the element of cost is important. The robustness analysis indicates that disease risk is important for vaccinees and representatives as well. Therefore, information strategies should not only cover vaccine-related aspects, but should also inform target groups about the severity and probability of diseases. Across included studies, individuals value a reduced number of doses/visits when deciding for themselves and others. Vaccine programs that currently include multiple injections (such as HPV and COVID-19 vaccines), should hence try to minimize the amount of dosages as much as possible.