COVID-19 Vaccine Preferences in China: A Comparison of Discrete Choice Experiment and Profile Case Best–Worst Scaling

The COVID-19 outbreak has resulted in various virus strains causing physical and mental damage worldwide [1]. Although the World Health Organization (WHO) declared that COVID-19 no longer constitutes a Public Health Emergency of International Concern (PHEIC) as of May 2023, it remains an ongoing global health crisis that threatens healthcare and socioeconomic systems worldwide [2]. Vaccines remain a priority for epidemic prevention and control, and a significant amount of research has been accelerated globally [3]. According to the WHO [4], there are currently over 375 COVID-19 vaccine projects in different stages of development worldwide, featuring varying mechanisms (e.g. inactivated, recombinant protein, adenovirus vector) and administration methods (e.g. inhalation, oral, and intramuscular injection), providing diverse options for individuals. While this diversity expands vaccination possibilities, it also calls for a deeper understanding of individual preferences for specific vaccine attributes. Numerous studies [5‐7] have highlighted the persistence of vaccine hesitancy, which remains one of the ten threats to global health [8]. Thus, assessing preferences for COVID-19 vaccine characteristics is critical to tailoring strategies that promote vaccine uptake effectively.

The two most commonly used methods for measuring stated preferences are the discrete choice experiment (DCE) and best–worst scaling (BWS). DCE presents respondents with a series of reality-based hypothetical choice sets and asks them to choose between two or more alternatives, each containing different combinations of attributes and levels [8]. In a BWS task, respondents are asked to choose the best and worst (or most and least preferred) options from several alternatives [9]. The options vary according to three task types. In the object case BWS (also referred to as BWS-1), the choice is between different whole objects; in the profile case (BWS-2), the choice is between individual attribute/level pairs within a single profile; and in the multi-profile case (BWS-3), the choice is between whole profiles, where each profile is described by number of attributes and levels.

In prior global research, numerous DCE studies have been conducted on the preference for COVID-19 vaccines across various age groups, including children [10], adolescents [11], adults [12‐16], and older adults [17]. Additionally, some studies focus on specific populations, such as university workers [18], college students [18‐20], health professionals [21], ICU clinicians [22], immune-mediated inflammatory disease (IMID) patients [23], and immigrants [24]. While the importance of attributes may vary, ‘efficacy’, ‘risk of adverse effects’, and ‘duration of protection’ are usually ranked as the top three attributes. Moreover, ‘effectiveness’ is the attribute that is most valued in most studies. A comparative study [25] between China and the United States shows that ‘effectiveness’ is valued the most by Americans, while ‘cost’ is more important to the Chinese people. Research [11, 24] conducted on teenagers and immigrants reveals that ‘brand’ is the most valued attribute.

The application of BWS in COVID-19 vaccination applications is limited. A study [26] conducted in China using the BWS-1 method examined the preferences of Chinese residents aged 18–59 years for COVID-19 vaccine. It found that residents were more concerned about ‘adverse reactions’, while they paid less attention to ‘injection times’ and ‘vaccination distance’. In contrast, a Canadian study [27] using the BWS-2 method identified ‘physical distancing’ as the most preferred immune-specific attribute, followed by ‘vaccination status of other people’ and ‘vaccine dosing’. Another study [28] utilized BWS-1 to investigate barriers preventing African Americans from receiving COVID-19 vaccination. It found that the most significant barriers included ‘safety concerns of COVID-19 vaccines’, ‘rapid mutation of COVID-19’, ‘ingredients of COVID-19 vaccines’, ‘Emergency Use Authorization (fast-track approval) of COVID-19 vaccines’, and ‘inconsistent information of COVID-19 vaccine’. Conversely, ‘religious reasons’ were deemed the least significant barrier.

Despite extensive research estimating preferences for COVID-19 vaccines based on the DCE and BWS methods, to date, there has been no comparative analysis of the results provided by these two tools for assessing COVID-19 vaccine selection preference. A review [29] about healthcare preference concludes that limited evidence suggests the preferences derived from DCE and profile-case BWS may not be concordant, regardless of the decision context, and indicates the need for further study. Given these differences, our hypothesis is that the results obtained from DCE and BWS may not be equivalent, and thus, they should not be used interchangeably. We need to be cautious in understanding the findings and explore these differences further—why they occur, how they are influenced, and their implications for decision making—in order to understand the relative merits and choices between the DCE and BWS methods.

In addition, cultural and policy differences may influence respondents’ preferences, and significant gaps remain in our understanding of these influences within specific sub-populations and the Chinese cultural context, especially after the downgrade of COVID-19 from a PHEIC. Therefore, we are interested in understanding the differences between these two methods in measuring Chinese COVID-19 vaccine preference under the new context. Moreover, although the development and diversification of COVID-19 vaccines have expanded the range of vaccination methods, ‘vaccination method’ has not been considered as an attribute in prior studies.

To address the aforementioned research gap, this study aims to quantify the Chinese preferences for COVID-19 vaccination, focusing on various vaccine attributes, using both the DCE and profile case BWS, and to explore the differences between these two tools for measuring preference for COVID-19 vaccine selection in China. Specifically, within a priority-setting context, this study sought to provide insights to (1) determine the preference of Chinese residents for the COVID-19 vaccine; (2) evaluate the concordance between the preferences estimated by DCE and BWS, examining aspects such as the direction of preference and the rank order of preference, as well as the correlation between rescaled preference weights from both methods; and (3) identify variations in preferences across different subgroups. The findings of this study are expected to offer crucial information for policymaking and the development of a feasible, effective, and sustainable vaccination strategy.

According to the consensus [29] issued by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) on conjoint analysis, we developed and conducted our experiment using the following steps: establishing attributes and levels, designing choice sets (scenarios), collecting data, and analyzing data.

To the best of our knowledge, this is the first study to concurrently use DCE and BWS to reveal Chinese residents’ preferences for COVID-19 vaccination and to compare the differences in the results of these two methods. Our findings indicate that Chinese residents prefer COVID-19 vaccines that are domestically branded, have higher effectiveness, longer duration of protection, lower risk of severe adverse events, and are administered orally. In addition, our results indicate a similar pattern in the DCE and BWS methods, with the respondents having a strong preference for 90% vaccine effectiveness. However, the methods diverged in other preferences. When comparing the results with existing research, the discrepancy suggests that DCE is more suitable for capturing preferences in our context.

Understanding Chinese residents’ preferences for COVID-19 vaccines could significantly enhance public health strategies by informing targeted vaccine communication and distribution plans, ultimately increasing vaccination uptake and contributing to improved public health outcomes. In terms of brand, our finding shows the domestic vaccine is preferred, which is consistent with a previous study [39]. It is believed that the government’s promotion and advocacy of domestic vaccines during the COVID-19 outbreak has increased public trust in them [40, 41]. Moreover, Chinese people are willing to pay 200/400 CNY for the vaccines, which shows their understanding of the importance of vaccination and their purchasing power to some extent. It is important to note that the hypothetical scenarios differ from the fact that the COVID-19 vaccine in China is provided for free by the government and health insurance, which may introduce a potential bias in the study.

The results of the comparison between DCE and BWS indicate that the overall consistency is poor, supporting the hypothesis that these methods may not yield equivalent findings. In other studies of stated preferences, the findings from Whitty et al. [42], Soekhai et al. [43], and Armeni et al. [44] also indicate a relatively low level of consistency between the two methods. Specifically, in terms of attribute ranking of the whole population, the attribute ‘effectiveness’ ranks second in the DCE and last in BWS. This is noteworthy because previous studies of COVID-19 vaccines [10, 15, 16, 18, 23, 45‐47], and even studies of other vaccines like those for influenza [48] and HPV [49], have identified effectiveness as one of the top three most important attributes. Our study found that even with various methods, it is still impossible to achieve complete orthogonality when the number of attributes and levels is large, resulting in a trade-off phenomenon. In other words, in the selection scenario, when the valued attribute is present, it may be traded off with other attribute levels, where a less attractive level of the valued attribute will be chosen to be the least important, ultimately affecting the overall ranking of the relative importance of the valued attribute. For instance, in this study, even though effectiveness is considered the most important attribute, when a 60% effectiveness occurs more frequently, this level is highly likely to be selected as the least important, thus affecting the overall relative importance score ranking of the attribute ‘efficiency’. In other words, this is related to the fact that BWS makes it easier to identify and express extreme preferences, which also leads to the possibility that it may not adequately capture participants’ more complex trade-offs at the attribute level, resulting in bias between its preference estimates and DCE [50]. This discrepancy suggests that DCE may offer a more practical and suitable method for capturing preference for COVID-19 vaccine selection. Besides, it is suggested that BWS is suitable for simpler situations where properties and levels are limited and can be completely orthogonal, while DCE is a better choice in more complex product selections. Our research results are in line with studies by Soekhai et al. [43], Flynn et al. [51]. and Himmler et al. [52], in which the authors also reported that DCE may result in less cognitive burden than BWS-2, making it more appropriate for decision-making contexts that require the evaluation of multiple attributes and levels.

Another possible reason for the lack of concordance between the two methods is the differences in preference construct. DCE and profile-case BWS are grounded in distinct theoretical frameworks and psychological models. This may lead to variations in how participants express their preferences, resulting in different characteristics and patterns within the same research context. Consequently, there may be fundamental discrepancies between the two methods regarding the concordance of preference information acquisition, as noted in the literature by Whitty and Oliveira Gonçalves [53]. However, it is not conclusively established, and further research is warranted to explore these findings in greater depth. In addition, the results of the BWS analysis revealed that both the vaccine hesitancy group and the vaccine recipient group assigned the same relative importance score ranking to various attributes. We believe this might be due to the limitations of the BWS method, which may not effectively capture and distinguish preference differences between these groups.

Further, we found there is preference heterogeneity for COVID-19 vaccination among different groups. Both the vaccine-hesitant group and the high-risk group prefer oral vaccination, which indicates that the availability and convenience of vaccines are also crucial selection conditions [54]. This may be related to the misconception that the traditional intramuscular injection has more adverse reactions after vaccination and might seriously affect daily life [55]. Another latent reason might be the fear of needles, which can produce nervousness, anxiety, and other uneasy emotions [56]. Moreover, traditional intramuscular injection vaccination needs to be carried out by medical professionals, and a lack of convenient access, unfamiliarity, long waiting times, and other obstacles may lead to the rejection of intramuscular injection of COVID-19 vaccine [55]. Additionally, oral vaccination is an ideal and easy-to-accept delivery route, which is highly accepted by residents, and the oral method is convenient, allows self-management, and is suitable for people of all ages [56, 57]. The preference for oral vaccines has significant implications for improving targeted vaccination strategies and reducing vaccine hesitancy.

In addition, we found that for the vaccine recipient groups, emphasizing the ‘added’ properties of vaccines has an important role. For example, the promotion of vaccination methods and vaccine brands should be highlighted because this group usually already has a high level of identification and trust in the basic attributes of vaccines (such as effectiveness, duration of protection, etc.). For the vaccine-hesitant group, however, the picture is different. In addition to promoting vaccination methods to reduce psychological barriers, it is necessary to carry out targeted publicity on the core attributes of vaccines, especially emphasizing the effectiveness of vaccines and the low risk of adverse reactions in order to enhance the acceptance of vaccines in this group.

This study has several notable advantages. Firstly, the attributes and levels of the DCE and BWS were established based on a thorough literature review and two rounds of Delphi consultations, providing a solid foundation for the experiment. Secondly, while both DCE and BWS are widely used methods to estimate preferences, they are grounded in different theoretical frameworks and involve distinct choice architectures. This study explored the potential discrepancies of DCE and BWS measurements in the field of preference for COVID-19 vaccination, providing evidence for improving the accuracy of preference assessments. Thirdly, vaccination methods like oral, injection, and aerosol inhalation, which were relatively rare in previous studies, have been included. In addition, our subgroup analysis provides additional insights for preferences of different groups.

This study has some limitations that need to be acknowledged. Firstly, it only focused on the first six attributes that experts considered most important in using the DCE and BWS methods. It was impossible to include a more comprehensive combination of attributes and levels, which may have led to other factors that could affect vaccination preference being overlooked. Secondly, the online survey was conducted through convenience sampling, which may not have accurately represented the characteristics of the sample population. Certain demographic groups, such as the elderly, who may not be as inclined to fill in an online questionnaire, might have been excluded, possibly resulting in sample selection bias that could influence our analysis. Thirdly, to maintain consistency with BWS settings and ensure comparability of results, we did not include an opt-out option when designing the DCE. While this approach aligns with our objective of evaluating participants’ preferences for vaccine attributes, it may have constrained our ability to fully capture the preferences of individuals who would have chosen not to vaccinate. However, subgroup analysis in this study provided additional insights by revealing significant differences in preferences between the vaccination and vaccine hesitancy groups. Through this, we partially address the potential limitation by identifying the preferences of the vaccination hesitancy group, providing valuable information that might otherwise have been missed. Nonetheless, we acknowledge that not including an opt-out option might have influenced some participants’ decision-making process, and this should be considered when interpreting the results. Moreover, differences in how the vaccination and vaccine hesitancy groups understood the selection tasks and expressed their preferences may have introduced bias, potentially deviating from an ideal comparative framework. Future research could address these limitations by incorporating opt-out options into the experimental design. This adjustment would better reflect real-world scenarios, particularly in contexts where vaccination is not mandatory or strongly encouraged, thereby enhancing the external validity of the findings. Finally, this study is subject to hypothetical bias, as respondents are required to make choices between hypothetical service options rather than objective reality. Despite these limitations, this study provides a novel perspective to understanding COVID-19 vaccine preference comparing the DCE and BWS, offering valuable insights that have not been previously explored.

In this study, we not only identified Chinese residents’ preferences for COVID-19 vaccine, but also compared the results of two different preference measurement methods. We concluded that using either a DCE or BWS leads to different preference weights as well as relative importance values. A potential reason lies in the way BWS processes multiple complex profiles at once, which leads to tradeoffs and imposes a certain level of cognitive burden on respondents. Thus, DCE is the recommended method for researching COVID-19 vaccine selection preferences, and the BWS method cannot serve as a substitute for DCE. In addition, our study underscores the importance of prioritizing vaccine effectiveness in public health communications and ensuring the availability of preferred administration methods, which could significantly enhance uptake among key demographic groups.