Trends in Cognitive Sciences
ReviewFeature ReviewThe neural mechanisms of inter-temporal decision-making: understanding variability
Section snippets
The subjective nature of preferences
Experience tells us that preferences are subjective. Some people are patient, others impatient, some take risks and others tend to avoid them. In recent years, many areas of decision science, including psychology, behavioural economics, psychiatry and cognitive and systems neuroscience, have adopted approaches that focus on subjective aspects of choice and valuation. One area of research that has been extremely fruitful in the study of subjective choice and valuation is intertemporal decision
Standard models
Computational models of DD aim to develop functions that capture the relationship between temporal proximity and subjective value (Figure 1a). Two models have dominated the field of temporal discounting for the last decades and both include a single free parameter, the discount rate k. The decay of subjective value (SV) over time is modelled as an exponential (i.e. SV=Ae–kD [2]) or hyperbolic (i.e. SV=A/(1+kD) [3]) function of delay, where A is the objective amount of the reward, D is the delay
The trait–state distinction
An important question that arises in decision research is whether certain differences observed in choice and valuation (e.g. between different individuals or experimental conditions) are caused by state or trait differences. In particular with respect to DD, it has recently been suggested that both factors might affect decision-making [21]. In this view, preferences are flexible and dependent on the decision context or current requirements of the organism. These factors might thus induce a
The cognitive neuroscience of DD: trait effects
Recent theoretical accounts distinguish between at least two general processing stages in decision-making [49]: valuation, which is the neural computation and representation of the subjective values of available decision options, and choice, which comprises processes leading to and supporting action selection. Following this taxonomy, we first focus on the neural mechanisms underlying reward valuation processes in DD. We then address the role of processes related to conflict monitoring and
Domain-general and domain-specific valuation
Overwhelming evidence implicates the ventral striatum (VS) and orbitofrontal cortex (OFC), in particular its ventromedial part (often referred to as ventromedial prefrontal cortex, vmPFC, or medial OFC) in the representation of the incentive value of a broad range of different classes of rewards 49, 50, 51, 52. The VS is a projection region of dopaminergic neurons in the substantia nigra (SN) and the ventral tegmental area (VTA), which have been implicated in reinforcement learning and reward
Conflict monitoring, strategy adaptation and the anterior cingulate cortex
In addition to effects of subjective reward value, decision-making is affected by another important variable, decision conflict. Decisions are difficult when options are of similar value, whereas decisions are easier when option values are clearly different. The degree of decision conflict in such situations is correlated with activity in prefrontal control regions, in particular the anterior cingulate cortex (ACC) [77]. In intertemporal choice, this effect is typically examined by comparing
MTL contributions to DD
Two regions in the MTL, the hippocampus and the amygdala, have repeatedly been implicated in DD, although their precise contributions are poorly understood. Damage to the hippocampus increases DD in rats 88, 89, 90, although it is unclear whether this constitutes a selective impairment in DD or generalizes to other forms of cost–benefit decision-making. Amygdala damage in rodents, conversely, is known to impair performance in a range of cost–benefit choice tasks, including probability
Interim conclusions
We have shown that at least three distinct networks contribute to intertemporal decision-making. A valuation network comprising vmPFC, mOFC, ventral striatum and PCC represents the subjective discounted value of future rewards. The lateral PFC and ACC are involved in DD, predominantly through their role in cognitive control, conflict monitoring and strategy adaptation. Finally, although still poorly understood, MTL regions including the hippocampus might contribute to DD through representing
Contextual modulations in DD: state effects
In light of the consistent association of DD with substance abuse and addiction, it is of high clinical relevance to identify mechanisms or interventions that reduce impulsive discounting. At the same time, a better understanding of modulating factors could enhance our understanding of the psychological (and neural) processes underlying DD. We address three basic types of state modulation that provide insight into the psychological and neural mechanisms of DD, as well as into the potential for
Interim conclusions
The findings summarized in the preceding sections show that DD is a considerably stable trait within individuals, but one that is also subject to modulation on various levels. Behavioural framing and context effects suggest an important role for mental representation of the decision problem: how subjects represent delay and/or outcome seems to be a crucial factor in the valuation of that outcome. The overall decision context might also activate learned behavioural patterns, which might play a
Concluding remarks
A basic model of the functional neuroanatomy of intertemporal choice is beginning to take shape. This model includes neural circuits that support different aspects of intertemporal decisions: vmPFC, VS and PCC are involved in the representation of subjective discounted value [49], and hyposensitivity of these regions might contribute to impulsive discounting in psychiatric conditions. PFC and ACC are part of a network that exerts cognitive control during decision-making, and in DD might bias
Acknowledgements
We thank Markus Staudinger and Sebastian Gluth for helpful comments on a previous version of this manuscript.
Glossary
- Delay (or temporal) discounting
- the phenomenon that agents typically devalue rewards as a function of the delay to their delivery.
- Dynamic inconsistency
- Consistent delay discounting entails that adding a common constant delay to the available options does not change which option is preferred by an individual. The fact that human preferences are typically not consistent in this fashion is referred to as dynamic inconsistency. For example, an individual may prefer 20€ in 1 week over 25€ in 2 weeks,
References (163)
A comparison of four models of delay discounting in humans
Behav. Process
(2009)Psychophysics of time perception and intertemporal choice models
Phys. A
(2008)- et al.
Dissociable neural representations of future reward magnitude and delay during temporal discounting
Neuroimage
(2009) Prefrontal coding of temporally discounted values during intertemporal choice
Neuron
(2008)- et al.
Episodic future thinking reduces reward delay discounting through an enhancement of prefrontal–mediotemporal interactions
Neuron
(2010) A comparison of delay discounting among smokers, substance abusers, and non-dependent controls
Drug Alcohol Depend.
(2010)Cigarette smokers discount past and future rewards symmetrically and more than controls: is discounting a measure of impulsivity?
Drug Alcohol Depend.
(2008)Does delay discounting play an etiological role in smoking or is it a consequence of smoking?
Drug Alcohol Depend.
(2009)- et al.
Pathological gambling is associated with impulsivity in a delay discounting procedure
Behav. Process.
(2003) - et al.
Excessive discounting of delayed rewards in substance abusers with gambling problems
Drug Alcohol Depend.
(1999)
Novelty seeking, risk taking, and related constructs as predictors of adolescent substance use: an application of Cloninger's theory
J. Subst. Abuse.
The neurobiology of decision: consensus and controversy
Neuron
Neural representations of subjective reward value
Behav. Brain Res.
The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology
Prog. Neurobiol.
Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala
Neuron
Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task
Curr. Biol.
The neural substrates of probabilistic and intertemporal decision making
Brain Res.
Functional neuroimaging of reward processing and decision-making: a review of aberrant motivational and affective processing in addiction and mood disorders
Brain Res. Rev.
Nicotine dependence is characterized by disordered reward processing in a network driving motivation
Biol. Psychiatry
Brain activations reflect individual discount rates in intertemporal choice
Brain Res.
The effects of delaying reward on choice preference in rats with hippocampal or selective septal lesions
Behav. Brain Res.
Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task
Neuron
A discounting framework for choice with delayed and probabilistic rewards
Psychol. Bull.
A note on measurement of utility
Rev. Econ. Stud.
An adjusting procedure for studying delayed reinforcement
Area under the curve as a measure of discounting
J. Exp. Anal. Behav.
Separate neural systems value immediate and delayed monetary rewards
Science
Discounting of delayed rewards: models of individual choice
J. Exp. Anal. Behav.
Notes on discounting
J. Exp. Anal. Behav.
Discounting time and time discounting: subjective time perception and intertemporal preferences
J. Marketing Res.
The neural correlates of subjective value during intertemporal choice
Nat. Neurosci.
Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making
J. Neurosci.
Three-month stability of delay and probability discounting measures
Exp. Clin. Psychopharmacol.
Heritability of delay discounting in adolescence: a longitudinal twin study
Behav. Genet.
One-year temporal stability of delay-discount rates
Psychon. Bull. Rev.
Contextual control of delay discounting by pathological gamblers
J. Appl. Behav. Anal.
An “as soon as possible” effect in human intertemporal decision making: behavioral evidence and neural mechanisms
J. Neurophysiol.
Behavioral and neuroeconomics of drug addiction: competing neural systems and temporal discounting processes
Drug Alcohol Depend.
Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls
J. Exp. Psychol. Gen.
Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: drug and monetary rewards
Exp. Clin. Psychopharmacol.
Frontoparietal cortical activity of methamphetamine-dependent and comparison subjects performing a delay discounting task
Hum. Brain Mapp.
Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers
Psychopharmacology (Berl.)
Measures of impulsivity in cigarette smokers and non-smokers
Psychopharmacology (Berl.)
Lower ventral striatal activation during reward anticipation in adolescent smokers
Am. J. Psychiatry
Moderate drug use and delay discounting: a comparison of heavy, light, and never smokers
Exp. Clin. Psychopharmacol.
Delay discounting of money and alcohol in actively using alcoholics, currently abstinent alcoholics, and controls
Psychopharmacology (Berl.)
Impulsive responding in alcoholics
Alcohol Clin. Exp. Res.
Pathological gamblers, with and without substance use disorders, discount delayed rewards at high rates
J. Abnorm. Psychol.
Delay discounting by pathological gamblers
J. Appl. Behav. Anal.
Pathological gambling is linked to reduced activation of the mesolimbic reward system
Nat. Neurosci.
Cited by (463)
Neuroanatomical and functional substrates of the hypomanic personality trait and its prediction on aggression
2023, International Journal of Clinical and Health Psychology