Taking the Shortcut: Simplifying Heuristics in Discrete Choice Experiments

Discrete choice experiment (DCE) is globally recognised as a valuable instrument to measure preferences [1‐4]. Health-related DCEs can be designed to inform decision making on the development, authorisation, reimbursement and marketing of drugs and devices as well as treatments in clinical practice [5‐11]. This stated preference (SP) method allows researchers to quantify the importance of certain characteristics (attributes) of particular alternatives, calculate willingness-to-accept risks or willingness-to-pay for benefits, and forecast potential participation rates/market shares. Discrete choice experiments are based on random utility theory (RUT) [12‐14], which imposes strong assumptions on respondent choice behaviour. Respondents are expected to use complex and rational decision-making processes when completing choice tasks [4, 15]. It is assumed that respondents evaluate all alternatives and their corresponding attributes and attribute levels in each choice task and choose the alternative that maximises their utility. These behavioural premises directly or implicitly presume that respondents have limitless cognitive resources to gather and process information, to correctly understand and interpret the choice situation and make choices in line with several axioms used to describe normative rationality (i.e., monotonicity, continuity, completeness, transitivity) [4, 16, 17].

However, decades of research in behavioural economics and psychology has shown that the assumptions which underlie RUT are often violated, particularly when respondents are faced with complex choice problems [18, 19]. Specifically, respondents may not always maximise utility, but instead use choice processes that do not adhere to the normative rationality assumptions by applying simplifying decision rules that are more selective in the amount and type of processed information (i.e., simplifying heuristics) [18, 20‐22]. This loosely corresponds to the concept of ‘bounded rationality,’ which states that respondents’ cognitive capacity limits the degree to which they can maximise utility and that some choices require the use of simplifying strategies [23, 24]. In face of time, attention, resource, information, and/or mental processing limits, respondents defer to simplifying heuristics, thereby partially or fully violating RUT assumptions [20].

Discrete choice experiments in health still largely ignore the possibility of simplifying heuristics and are generally designed and analysed under strict RUT assumptions. In health applications, DCEs are particularly vulnerable to the use of simplifying heuristics as they generally include complex choices and (multiple) benefit and risk attributes. Health risks particularly are hard to evaluate [25, 26], since these often-small risks can have a significant downside for patients when they eventuate, and their evaluation is loaded with emotional respondent reactions and fraught decisional contexts. To support health policy development based on DCEs, the choice model should emulate what happens in practice [27, 28]. Hence, using choice models in health care under strict RUT assumptions likely compromises the realistic representation of decision makers’ choice decisions, thereby negatively impacting the external validity of DCE outcomes [27]. As a consequence, ignoring the possibility of heuristics in choice models, or even simply deleting ‘irrational’ responses from the DCE dataset, will likely lead to biased preference estimates [22, 29‐31], incorrect claims on preferences heterogeneity [32] and, ultimately, to bias in DCE outcome measures. Therefore, it is important that researchers be aware of factors that can induce heuristics during the design phase of a DCE and model their DCE data in a way that allows for heuristic choice processes. However, a search of the literature demonstrates the lack of an overview of commonly applied simplifying heuristics in (health-related) DCEs. It is unclear, therefore, how to identify and deal with these heuristics in practice; more specifically, it leaves unclear how researchers might alter DCE design and modelling strategies to accommodate for the effects of heuristics.

Hence, the aim of this paper is three-fold; (1) to provide an overview of common simplifying heuristics in health-related DCEs, (2) to describe how choice task design and context as well as target population selection might impact the use of heuristics, (3) to specify DCE design strategies that recognise the use of simplifying heuristics and modelling strategies to demonstrate the use and impact of simplifying heuristics in DCE study outcomes. Therefore, we conducted an extensive scoping review drawing on literature (i.e., published papers and books) from various fields including health sciences, psychology, econometrics, management sciences, marketing research and transportation science.

Completing a DCE requires cognitive effort on the part of respondents. It follows that we should therefore consider what might occur when the cognitive demand we impose via the DCE exceeds respondents’ limited cognitive capacity, since DCE respondents may employ methods or strategies that reduce the effort expended to choose an alternative among several substitutes [33]. These methods or strategies which allow people to make choices more quickly, and/or frugally than classical utility-maximising strategies, are termed ‘simplifying heuristics’ [18, 20‐22, 34]. The use of heuristics can be considered adaptive (i.e., responsive to decision context), or even rational, depending on the context in which decisions must be made. It has been shown that even when ignoring information, certain heuristics do allow respondents to arrive at an optimal decision without allocating full effort [20]. Yet, although heuristics help individuals solve complex decision making in real life, their use may result in choices made in partial or total conflict with the RUT assumptions. Hence, the focus of this manuscript is to describe a subset of heuristics known and studied in psychology that has relevance to DCEs [18].

Since respondents’ usage of simplifying heuristics can be triggered by the degree to which DCEs are cognitively or emotionally demanding, it can be useful to minimise factors that impose such demands when planning a DCE study (Sect. 3). For this, it is important to be aware of DCE characteristics that are likely to induce heuristics. These characteristics of DCEs can generally be classified into factors relating to task complexity and context effects [20]. Researchers should be aware that choice task simplifications can threaten the DCE’s external validity by abstracting choice tasks in such a way that they no longer represent decision problems in a realistic way [28]. Hence, DCE choice tasks should be presented in a manner compatible with real-world level of complexity, when applicable. Additionally, researchers should in general expect that all respondents may be inclined to use simplifying heuristics in their decision making, regardless of their sociodemographic characteristics. However, certain social subgroups more often/quickly tend to experience choice tasks as cognitively demanding and are more likely to resort to heuristic decision making [35‐37] (Sect. 4). Notably, respondent’s literacy and numeracy, as well as their age and education, can influence the perceived cognitive burden of choice tasks.

Task complexity increases as the hypotheticality, number of choice tasks, attributes and alternatives of the choice tasks increase [38, 39]. The concept of hypothetical bias is well known in DCEs; the absence of constraints (i.e., real-life consequences) likely impacts study outcomes [40, 41] and may lower respondents’ engagement in the choice process. Without constraints, the importance of choosing the option with the highest utility may diminish, which in turn might make respondents inclined to decrease their cognitive effort and adapt heuristic strategies to make a choice [42, 43]. Additionally, the more hypothetical a DCE is for a given patient population, the more difficult it may be for respondents to anticipate a certain state or scenario, and consequently, to form a preference and make a choice (e.g., elicit preferences of the general population for cancer care) [44, 45]. Lengthy choice experiments add to task complexity and can cause respondents to increasingly rely on simplifying heuristics as boredom and fatigue set in [46, 47]. In addition to the complexity of each individual choice task, the cumulative cognitive burden over the course of the DCE should be considered by the researcher [46]. Increasing the number of attributes increases the amount of information which needs to be processed; this in turn increases the risks of cognitive overload and can thereby induce the use of heuristics [20]. While a higher number of attributes has been found to increase respondent’s confidence in their choice [20] and omitting relevant attributes may cause respondents to ‘fill in the blanks’ or infer missing information [48‐51], the DCE design should include alternatives that are sufficiently characterised without cognitively overloading the respondent. Task complexity can further increase as the number of alternatives per choice task increases. Discrete choice experiments in health generally include two to three alternatives per choice task [4, 10], and it has been found that respondents are most likely to trade attributes across alternatives when presented with two alternatives per choice task (in unlabelled experiments) [20]. Balancing choice complexity and providing respondents with a sufficient range of alternatives, accurately capturing real-life choice contexts [52], enables respondents to make considered choices that capture trade-off behaviour [53].

Context effects pertain to information and values in the choice task that influence respondents’ decision-making process [20]. These include the similarity of alternatives, reference points, framing effects and unfamiliarity with the topic of choice. If respondents are faced with similar alternatives encompassing approximately equal overall utility scores, choice complexity increases and so may respondents’ reliance on heuristics [20]. Since this is more likely to occur if the DCE entails restricted attribute level ranges [22, 54], sufficiently wide attribute level ranges that are reflective of the population’s preference range in the DCE should be considered [55]. Respondent’s perception of the adequacy of attribute level ranges may also be related to their unobserved reference points. Given that respondents may pick and set reference points based on any information or experience prior to the experiment [56], reference points may vary greatly between respondents and cause the use of heuristics [57]. Providing respondents with information on a reference point can keep the influence of individual reference points somewhat constant [58]. In addition to pre-set reference points, individuals can also apply individual risk perceptions to attributes that determine how likely they consider themselves to fall within a certain risk category, which might be driven by cognitive biases, such as overconfidence, availability, or illusion of control [59]. Similarly, positive versus negative framing of attributes (e.g., survival vs mortality or effectiveness vs failure rate) can induce loss aversion despite conveying the same information and have been shown to induce the use of heuristics as well as different study outcomes [49, 60]. Finally, empirical evidence shows that cognitive burden tends to increase in DCEs with unfamiliar choice contexts [38], attributes [61] and attribute levels [22]. This effect can be accelerated in emotionally laden [62] and high-risk choice contexts [38], which provoke affective reactions and respondents’ refusal to imagine and trade certain attributes [63]. In such situations, respondents might instinctively exclude certain alternatives by attaching negative emotions to them based on the choice context and not necessarily the attribute (levels) (affect heuristic). This could happen, for instance, when evaluating the chance of hospitalisation after suffering from severe side effects due to treatment, which affects emotions and feelings associated with hospitalisation.

Overall, we recommend designing DCE choice tasks that are compatible with real-world level complexity (i.e., also mirroring the potential use of heuristics in real life) and avoiding the promotion of the use of heuristics as a result of solely choice task complexity. Researchers should upfront identify and pre-test elements in their DCE that might promote the use of heuristics and a priori plan on how to avoid this in their design (see Sect. 7.1) or how to accommodate for the likely impact in their modelling strategy (see Sect. 7.2).

Older age and lower education attainment have been associated with, on average, lower health literacy skills (i.e., a person’s ability to understand and apply health information in health-related decisions [64]) in The Netherlands [65]. Respondents with a low education and low health literacy have indicated that they consider fewer DCE attributes in their decision making [49]. In addition, respondent’s numeracy has been linked to their understanding of numeric attributes (e.g., risk attributes) [66, 67]. Previous studies suggest that high numeracy skills improve choice consistency [53, 68], decrease respondent’s susceptibility to framing effects [69] and are associated with elaborate processing techniques of numeric information in choice tasks, such as considering and combining numeric information [70]. In addition to conventional numeracy and health literacy, the related concept of risk literacy affects respondents’ ability to evaluate and understand risk attributes [71]. For age, a bell-shaped distribution was found, indicating that, on average, younger and older respondents showed the most inconsistent choice behaviour in a DCE conducted in Scotland [48], although younger adults seem to resort to different types of heuristics compared to older adults [72]. In older adults, cognitive aging increases the likelihood of applying simplifying strategies [73, 74]. In addition, digital competencies of elderly (i.e., e-literacy [75]) may also impact the perceived complexity of the DCE and, hence, induce heuristic strategies. Inconsistent choice behaviour in young adults, in contrast, may be driven by self-imposed time-pressure, impatience and rushing through the DCE [76, 77]. Choice based on the use of heuristics may further be exacerbated when recruiting participants through online panels that incentivise rapid survey completions (respondents 'speeding’ through the survey) [77]. Finally, language barriers for non-native speakers can challenge respondents’ understanding of the attributes, consequently increasing perceptions of complexity, and leading to heuristic decision making.

For the reasons mentioned above, researchers should always (a) collect respondents’ socio-demographic information (including literacy and numeracy) along with their choices, () be particularly alert if the aforementioned social groups are (over)represented in their sample, and (c) plan for accommodating the likely impact of the sample recruitment in their modelling strategy (see Sect. 7.2).

Although heuristics can be characterised in many ways, in this paper we describe them by the amount of information they require respondents to use and the decision process that respondents apply [20, 33]. Some heuristics do not require any of the information presented in the DCE tasks, with respondents deferring to a random-choice strategy or making position-based decisions. Other heuristics enable participants to ignore part of the information presented in choice tasks (either alternatives and/or attributes). Finally, some heuristics may require processing all information presented but impact the decision process that respondents use to make a choice.

The data generation process respondents use to evaluate the information is described by five characteristics in this paper. First, heuristics differ in the way that respondents structure the evaluation of the information presented, which may be alternative-based, attribute-based or a combination. Second, although the usual empirical representation associated with RUT applications adopts a linear-in-parameters utility function, which implies compensatory evaluation, respondents may employ simplifying heuristics based on non- or semi-compensatory behaviour (e.g., excellent levels of some attributes are not [completely] compensated for by poor levels of other attributes [20]). Third, the use of certain heuristics may imply explicit exclusion of one or more attributes from the evaluation process. Fourth, the attribute weighting strategy employed by respondents may differ across heuristics. While in some cases respondents may use attribute weights to make a choice, with other heuristics this may not be the case: e.g., a simpler strategy such as equal weighting of attributes, ordinally prioritising the attributes from ‘most important’ to ‘least important’, or no strategy at all may be used. Fifth, the strategy applied to evaluate attribute (levels) differs across heuristics and can be primarily qualitative or quantitative in nature. Respondents might convert attribute levels into unobserved (latent) categories of levels, which might be qualitative or quantitative irrespective of whether the attribute levels are metric or categorical. For instance, when respondents evaluate a risk of 5%, and they only evaluate this as being lower than a threshold of 10%, the evaluation process is actually qualitative in nature, as no value is associated with the numbers per se.

Table 1 provides an overview of heuristics most common in (health-related) DCEs. As noted earlier, this overview is a subset of heuristics known and studied in psychology [18]. The exclusion of a specific heuristic in no way implies that it does not occur empirically, only that in our judgement it is less likely to do so in health applications. All heuristics are characterised and described by means of specifications described in Sect. 5.

Four heuristics have been identified that have respondents using the full information provided in the choice task, but in the decision strategy they are assumed to employ processes that are not in line with RUT. First, reference point effects, also known in the health preference literature as anchoring and adjustment, refers to a choice process where attribute values in the first alternative encountered influence preferences in later choice tasks [78, 79]. In other words, the levels of some or all attribute(s) of the first alternative respondents encounter serve as reference points for the evaluation of the attribute levels in all following alternatives. Although respondents evaluate numerical information under this heuristic, they merely compare a numeric level as to whether it is under or over a reference point, making the evaluation strategy qualitative in nature. Second, ordinal recoding (also known in the literature as categorisation), refers to the process by which respondents construct categories and attach qualitative labels to metric attribute levels [8, 80, 81]. This means that respondents no longer interpret the actual numerical information of attributes (e.g., risk of side effects expressed as 5, 10, 20%) provided in the choice task, but instead defer to the categorisation of the numerical values (e.g., low, medium, high risk of side effects). Third, when using the conjunctive or disjunctive heuristic, respondents establish a minimal acceptable cut-off point for all or key attributes, respectively [82]. If an alternative in a choice task falls below the cut-off (or above depending on cut-off definition) on any attribute (i.e., qualitative evaluation strategy) or particular key attributes, it is excluded as a possible option. Fourth, tallying implies that respondents compare all the attributes across the available alternatives without appreciating their relative importance [83]. Respondents simply select the alternative with the most ‘best’ and/or the least ‘worst’ attribute levels. This heuristic has several well-known variations; (1) the majority of confirming dimensions [20, 84], (2) equal weights [85‐87] and (3) frequency of good and bad features [88]. These variations of tallying differ slightly in the process respondents use to compare attribute levels across alternatives. For the frequency of good and bad features, respondents can decide to focus on only the number of good features within an alternative, but in contrast to the ‘majority of confirming dimensions’ they can also decide based only the number of bad features, or both. Both these heuristics are different from the equal weights heuristic as they are based on categorising attribute levels into ‘good/best’ and ‘bad’ but do not require respondents to place any value on these attribute levels.

We consider seven heuristics that enable participants to ignore part of the information presented in choice task. First, attribute non-attendance is a heuristic that results in respondents completely ignoring a certain attribute or attributes when evaluating the alternatives in a choice task [89‐91], while making trade-offs across other attributes between alternatives. Second, attribute level non-attendance is an elaborated specification of the attribute non-attendance heuristic, which suggests that respondents ignore a certain attribute or certain attributes whenever their level falls within a predefined range [92]. For instance, a respondent might disregard the risk of side-effect attribute, unless it shows the highest level of side effects. Third, whenever respondents employ choice set formation (i.e., alternative screening), they eliminate entire choice alternatives based on certain threshold levels of particular attributes (often benefit, cost and risk attributes) [29, 93, 94]. To illustrate, if the risk of side effects in one alternative is considered unacceptably high, the full alternative may not be considered at all within its choice set, and hence cannot be chosen. After such a screening stage, respondents trade across the other attributes between surviving alternatives. Fourth, elimination-by-aspect refers to a choice process where respondents exclude an alternative based on predefined cut-off values of selected attributes [95], which in essence makes this a combined screening and selection process. This process starts with the most important attribute and continues for as many other attributes as needed to achieve a singular alternative. This is different from the choice set formation heuristic where only a particular (set of) attribute(s) that violate the respondent’s cut-off values results in the full alternative being eliminated. Fifth, when respondents show lexicographic preferences (also referred to in literature as take-the-best [96]), they select alternatives based entirely on their superiority on one most important attribute [97, 98]. If the levels of this attribute overlap in the choice task, the second most important attribute is then evaluated (i.e., qualitative evaluation of attributes on ordinal scale). This is an iterative process that is repeated until only one alternative remains. Sixth, dominant decision-making behaviour refers to the choice process where a respondent selects one attribute to be the most important [99, 100]. Based on the level of only this one attribute, an alternative is selected. If the levels of this attribute are equal across alternatives, a random choice is made. This behaviour is nothing more than a truncated lexicographic decision rule employing only one attribute (which may vary from one respondent to another). Seventh, the final heuristic in this group of heuristics is satisficing. This means that an attribute’s aspiration levels are defined, and the first alternative encountered that surpasses these aspiration levels is chosen [23]. For instance, if the left-most alternative shows satisfactory levels for all attributes, this alternative would be chosen without considering the other alternative(s) within the same choice task (although other alternative(s) might have more favourable attribute levels).

Finally, two heuristics are identified that do not require respondents to use any information from the choice tasks. First, task non-attendance refers to choice processes when respondents know nothing about the influence of the attributes in his/her utility. This behaviour may be activated if respondents do not care about their responses or do not pay attention to the DCE instructions and choice task. Two variations exist within this heuristic: (1) complete ignorance, wherein respondents make choices randomly (i.e., all alternatives are equally likely) [101] and (2) flatlining, wherein respondents consistently choose an alternative placed at a certain spot of the choice task (e.g., only the alternative on the left or right side of the presentation format), regardless of the attribute levels shown in this alternative [102, 103]. Second, when employing the reading order heuristic, also known as left-right bias, respondents display a preference for alternatives that are placed at a certain spot dependent on where people start reading the presentation format (e.g., left side for respondents familiar with Latin-based writing and right side for respondents familiar with Arabic-based writing), regardless of the attribute levels [102, 103]. However, in contrast to flatlining, respondents do not consistently choose this option, instead they are merely more inclined to choose this particular alternative.

For DCEs to generate useful and valid outcomes, they should mimic real-life choice situations as much as possible [2, 106]. Making choices about health (care) is generally considered complex, both in real life and in DCEs. Therefore, preference elicitation techniques—including DCEs—are susceptible to the use of heuristics. This is likely not problematic unless researchers design their DCE in a way that induces heuristics in measurement that differ from heuristics used in real-life choices and/or ignore the use of heuristics in modelling procedures and assume choices were made under RUT assumptions. To date, the potential use and impact of heuristics in applied health-related DCEs generally is only alluded to in papers when discussing results and potential limitations. However, this manuscript shows that researchers are not as helpless as they might think regarding the impact heuristics might have on their study and study outcomes. Researchers can (1) plan and design their study to reduce the use of task-induced heuristics, (2) detect the use of heuristics by means of a priori qualitative work and a posteriori data analyses and (3) estimate the effects of the use of heuristics on their study outcomes. Future DCE studies should report on the likelihood of heuristics being employed in their experiment and the potential impact of this on their study outcomes. Additionally, methodological research is needed to generate (1) further insights on how best to design a DCE and train respondents to prevent increased use of heuristics beyond their use in real-life decision making and (2) define appropriate and accessible modelling strategies to estimate the impact of the use of heuristics on study outcomes. In conclusion, it seems that a lot (more) can be done regarding heuristics in DCEs, researchers should never take shortcuts in modelling choice behaviour.