Skip to main content
Erschienen in: Journal of Neural Transmission 3/2018

Open Access 26.10.2017 | Neurology and Preclinical Neurological Studies - Review Article

Reward and value coding by dopamine neurons in non-human primates

verfasst von: Aydin Alikaya, Mackenzie Rack-Wildner, William R. Stauffer

Erschienen in: Journal of Neural Transmission | Ausgabe 3/2018

Abstract

Rewards are fundamental to everyday life. They confer pleasure, support learning, and mediate decisions. Dopamine-releasing neurons in the midbrain are critical for reward processing. These neurons receive input from more than 30 brain areas and send widespread projections to the basal ganglia and frontal cortex. Their phasic responses are tuned to rewards. Specifically, dopamine signals code reward prediction error, the difference between received and predicted rewards. Decades of research in awake, behaving non-human primates (NHP), have shown the importance of these neural signals for learning and decision making. In this review, we will provide an overview of the bedrock findings that support the reward prediction error hypothesis and examine evidence that this signal plays a role in learning and decision making. In addition, we will highlight some of the conceptual challenges in dopamine neurophysiology and identify future areas of research to address these challenges. Keeping with the theme of this special issue, we will focus on the role of NHP studies in understanding dopamine neurophysiology and make the argument that primate models are essential to this line of research.

Introduction

Rewards are a central feature of everyday life that promote learning, incentivize value-based decisions, and provide a currency for social exchanges. Dopamine neurons respond to rewarding events. Specifically, phasic dopamine activity reflects the difference between received and predicted rewards, i.e., reward prediction errors. Systems neuroscience studies in non-human primates (NHP) have been critical to understanding this coding scheme, as well as investigating the behavioral implications of reward prediction error coding.
Right now, we stand at the threshold of a new neuroscientific age. This age will be defined by big data and molecular control of neural information processing. From this vantage point, this review endeavors to provide a brief introduction to dopamine neurons, examine the critical findings revealed by NHP studies that have shaped our understanding of dopamine function, and evaluate what role this valuable species should have in future investigations. We will attempt to identify outstanding challenges to our current understanding of this critical brain system, propose areas for future research, and reinforce the need to preserve and technologically advance behavioral neurophysiology studies in NHP. Keeping with the theme of this special issue on NHP studies of basal ganglia function, this review does not attempt to be comprehensive and passes over many of the recent advances observed in rodent models. In addition, we do not discuss the role of dopamine cell loss in Parkinson’s disease, nor the valuable contribution of the monkey MPTP Parkinson’s disease model (Langston et al. 1983).
The majority of dopamine neuron cell bodies are located in the midbrain, in cell groups designated A8, A9, and A10 (Dahlstroem and Fuxe 1964). These groups broadly overlap with the retrorubral field (RRF), substantia nigra pars compacta (SNc), and ventral tegmental area (VTA), respectively. (Dahlstroem and Fuxe 1964; German and Manaye 1993) (Fig. 1a). Dopamine neurons express the enzyme Tyrosine Hydroxylase (TH) that converts the amino acid tyrosine to l-3,4-dihydroxyphenylalanine (l-DOPA). l-DOPA is converted to dopamine via aromatic l-amino acid decarboxylase (AAAD) and packaged into vesicles by the vesicular monoamine transporter (VMAT) (Lovenberg et al. 1962; Yelin and Schuldiner 1995).
Dopamine neurons are few in number, less than 200,000 in Rhesus macaque monkeys (Stark and Pakkenberg 2004). Despite this, dopamine terminals are found throughout the striatum (Lynd-Balta and Haber 1994a, b) (Fig. 1a) and primate frontal cortex (Smiley et al. 1994; Smith et al. 2014; Williams and Goldman-Rakic 1993, 1998). Dopamine neuron action potentials (AP) are recognized according to their distinct electrophysiological profile that features broad AP waveforms as well as low and irregular baseline impulse rates (Guyenet and Aghajanian 1978). Classical identification of dopamine neurons was based on the tight correlation between these distinctive waveform characteristics and apomorphine sensitivity (Bunney et al. 1973; Guyenet and Aghajanian 1978; Schultz 1986; Schultz and Romo 1987). A partial survey of the literature that used apomorphine injections to identify dopamine neurons revealed that 77 of 85 putative dopamine neurons were suppressed by apomorphine, whereas none of 39 putative non-dopamine neurons were inhibited by apomorphine (Aebischer and Schultz 1984; Bunney et al. 1973; Guyenet and Aghajanian 1978; Schultz 1986; Schultz and Romo 1987; Studer and Schultz 1987). New alternative techniques, such as optogenetic photo-identification and juxtacellular labeling, promise similar reliability (Brischoux et al. 2009; Cohen et al. 2012; Eshel et al. 2015; Stauffer et al. 2016; Ungless and Grace 2012). Photo-identification is especially promising, as it uses dopamine neuron selective expression of optogenetic channels coupled with optical stimulation to unambiguously identify many dopamine neurons in each animal (Cohen et al. 2012; Eshel et al. 2015; Lammel et al. 2012; Stauffer et al. 2016), but see (Lammel et al. 2015). This strategy promises a less-subjective criterion for dopamine neuron identification, compared to identification via waveform characteristics, and will be critical to achieve an unbiased picture of dopamine neuron diversity.
Dopamine neuron activity is traditionally divided into irregular, slow (0.3–8 imp/s) tonic activity and phasic (burst firing) activity when the impulse rate can briefly reach 20–30 imp/s (Grace and Bunney 1983). Phasic bursts of dopamine neurons are the most efficient way to change the dopamine concentration in their target structures (Gonon 1988). These phasic bursts are thought to respond to rewards, though there is significant disagreement about the degree of functional heterogeneity (see section “Challenges and future research directions”).

Dopamine neurons code reward prediction error

Early evidence that dopamine neurons are involved in reward came from self-stimulation studies in maze-running rats (Olds and Milner 1954). However, behavioral neuroscience experiments in awake, head-fixed monkeys revealed the fundamental insight that phasic dopamine responses reflect reward prediction errors (Bayer and Glimcher 2005; Bayer et al. 2007; Bromberg-Martin et al. 2010; Enomoto et al. 2011; Fiorillo 2013; Fiorillo et al. 2003, 2008, 2013a, b; Hollerman and Schultz 1998; Kobayashi and Schultz 2008; Lak et al. 2014, 2016; Ljungberg et al. 1992; Matsumoto and Hikosaka 2009; Mirenowicz and Schultz 1994, 1996; Nakahara et al. 2004; Nomoto et al. 2010; Schultz et al. 1993, 1997; Stauffer et al. 2014; Tobler et al. 2005; Waelti et al. 2001). Reward prediction errors are defined in animal learning theory and machine learning as the differences between received and predicted rewards (Sutton and Barto 1998). A central tenet of animal learning theory is that temporal contiguity between a conditioned stimulus (CS) and reward (unconditioned stimulus, US) is not sufficient to drive learning. Rather, the reward must be unexpected; it must evoke a prediction error (Rescorla and Wagner 1972). Accordingly, reward prediction errors are teaching signals (Sutton and Barto 1998), and the reward prediction error nature of phasic dopamine responses strongly implicates these responses in learning.
Phasic dopamine responses are dependent on learning, and are time locked to unpredicted rewards and stimuli that elicit behavioral reactions (Mirenowicz and Schultz 1994, 1996; Romo and Schultz 1990; Schultz 1986; Schultz and Romo 1990). Early on in learning, when the associative strength between CS and reward is low, reward delivery strongly activates dopamine neurons (Ljungberg et al. 1992; Schultz et al. 1993) (Fig. 1b). Later, as the rewards become well predicted by the CS, dopamine neurons respond more strongly to the CS and less strongly to rewards (Ljungberg et al. 1992; Schultz et al. 1993) (Fig. 1c). With enough training, even higher order reward predictors (higher order CS) can activate dopamine neurons (Pan et al. 2005; Schultz et al. 1993) (Fig. 1d). Together, these studies provide overwhelming evidence that the phasic activity of dopamine neurons encodes reward prediction errors.
Dopamine reward prediction error responses are an ideal neural mechanism to mediate behavioral reinforcement learning, because they indicate both the occurrence of prediction errors and the proper direction to updating predictions. Rewards that are better than predicted activate dopamine neurons (positive prediction error responses), whereas rewards that are worse than predicted inhibit dopamine activity (negative prediction error responses). Modeling studies demonstrated that the prediction error term in popular reinforcement learning (RL) algorithms closely resembles the phasic dopamine signal (Montague et al. 1996; Schultz et al. 1997).
Using RL models, many studies have shown that dopamine responses conform to key principles of learning theory. For instance, when a US is predicted by a CS, a second CS presented at the same time or later than the first CS is ‘blocked’ from forming an association with the already predicted US, and dopamine neurons consistently fail to develop a response to the secondary, blocked CS (Steinberg et al. 2013; Waelti et al. 2001). Similarly, dopamine neurons are sensitive to temporal jittering of reward delivery. Early delivery of a predicted reward causes dopamine activation, whereas later than, predicted delivery leads to a diminished response (Hollerman and Schultz 1998). Moreover, dopamine responses reflect the discounting of future rewards, as suggested by reinforcement learning and economic theory (Enomoto et al. 2011; Fiorillo et al. 2008; Kobayashi and Schultz 2008). Trial-by-trial dopamine responses reflect the reinforcement history, a weighted average of past outcomes, in simple learning contexts (Bayer and Glimcher 2005). When the experimental context involves more complicated inter-trial task structure, dopamine neurons use this task structure to quickly update their responses on one trial, for instance, during reversal learning (Bromberg-Martin et al. 2010). Together, these results demonstrate the fidelity of dopamine responses to predictions made by learning theory, and they provide compelling evidence that phasic dopamine responses play a role in learning.

Dopamine activity reflects economic value

The magnitudes of dopamine prediction error responses scale positively with reward parameters that increase value, including reward size (Bayer and Glimcher 2005; Tobler et al. 2005), and probability (Fiorillo et al. 2003; Lak et al. 2016; Nakahara et al. 2004; Nomoto et al. 2010), and negatively with reward parameters that decrease value, including delays (Fiorillo et al. 2008; Kobayashi and Schultz 2008) and bitter substances (Fiorillo et al. 2013b). Moreover, when monkeys indicate preference rankings between goods that have the same reward magnitude, expected value, and delay, dopamine responses vigorously reflect the preference rankings for reward type (Lak et al. 2014) and information content (Bromberg-Martin and Hikosaka 2009). These results indicate that dopamine reward prediction error responses reflect subjective value.
To demonstrate the functional relationship between subjective value and dopamine activity, it is necessary to measure a psychometric function of subjective value. Economic theory demonstrates that choices between risky options reveal subjective value (utility) as a function of physical value (Debreu 1959; von Neumann et al. 1944). Risk-avoiding individuals display concave utility functions, where the potential loss is greater than potential gain (Fig. 2a). In contrast, risk seekers have convex utility functions, where the potential utility gain outweighs the potential utility loss (Fig. 2b). A psychometric utility function with a consequential shape—a shape that can be meaningfully correlated with a neurometric function—can, therefore, be measured from choices under risk (Caraco et al. 1980; Machina 1987; Stauffer et al. 2014). Choices between risky rewards show that monkeys are risk seeking for small rewards (McCoy and Platt 2005; O’Neill and Schultz 2010; Yamada et al. 2013), but become more risk avoiding as rewards get larger (Genest et al. 2016; Stauffer et al. 2014). This behavioral pattern, risk seeking for small rewards and risk avoiding for large rewards, translates into a convex then concave utility function (Fig. 2c) (Genest et al. 2016; Stauffer et al. 2014). The magnitudes of dopamine responses to unpredicted rewards are highly correlated with the shapes of the measured utility functions (Fig. 2d). In addition, when more reward than delivered is predicted, the prediction error response of dopamine neurons depends on the local slope of the utility function (Stauffer et al. 2014). These results demonstrate that the fundamental variable coded by dopamine prediction error responses is the same variable used to make decisions.
An outstanding question related to economic value coding is whether economic costs reduce the magnitude of dopamine responses. In one study, increasing the effort required to get a reward resulted in reduced responses of some dopamine neurons, but not others (Pasquereau and Turner 2013). The behavioral measures used to gauge effort, however—reaction time and error rate—do not map linearly onto economic value. Thus, the true economic costs remained unknown. More studies are required to determine whether dopamine neurons code a net utility signal that accounts for the economic costs associated with effort.
When behavioral decisions are made, dopamine responses reflect the chosen value, which is a post-decision variable (Lak et al. 2016; Morris et al. 2006). The current data suggest that dopamine neurons do not play a direct role in selecting options for a particular choice. Nevertheless, the close correlation between dopamine response to rewards and reward utility suggests that dopamine-teaching signals play a fundamental role in the choices we make over time. This functional role in value-based decisions was demonstrated by a recent study using optogenetic stimulation of dopamine neurons in a macaque monkey. Reward predicting objects that were followed by optogenetic activation of dopamine neurons were chosen more frequently than identical objects that were not followed by optogenetic activation (Stauffer et al. 2016) (Fig. 3). Thus, the likely role of the dopamine prediction error response is to train downstream brain structures about value.

Challenges and future research directions

Despite the overwhelming evidence for the reward prediction error hypothesis of dopamine function, several outstanding challenges remain unaddressed. Here, we will attempt to define and outline three critical challenges deserving of attention. These include (1) determining whether dopamine neurons are functionally homogenous, (2) elucidating the relationship between behavioral measures and striatal dopamine release, and (3) defining the functional role of dopamine signals in the cortex.

Functional diversity in dopamine neurons

Several studies have reported that a fraction of dopamine neurons are excited by novel or aversive stimuli and outcomes (Brischoux et al. 2009; Cohen et al. 2012; Fiorillo 2013; Fiorillo et al. 2013b; Lak et al. 2016; Matsumoto and Hikosaka 2009; Schultz and Romo 1987). Although there is significant anatomical diversity in the input–output pathways between medial and lateral dopamine neurons (Lynd-Balta and Haber 1994a, b, c; Watabe-Uchida et al. 2012), it remains unclear whether these non-reward-related activations represent the activity of distinct dopamine neuron circuits.
Multiple aspects of dopamine signaling may contribute to the observed complexity of dopamine responses, including context dependency and complex temporal dynamics. Dopamine neurons are exquisitely sensitive to the experimental context. For instance, when visual stimuli predict both appetitive and aversive stimuli, approximately 40% of dopamine neurons respond to the stimulus predicting the aversive outcome (Matsumoto and Hikosaka 2009; Mirenowicz and Schultz 1996). When the sensory stimuli are more perceptually distinct, as in when reward is predicted by an auditory cue and aversive outcome predicted by a visual cue, the number of dopamine neurons that respond to the aversive cue drops dramatically (Mirenowicz and Schultz 1996). This shows that stimulus context influences the activity of dopamine neurons. Likewise, the distribution of outcomes also alters dopamine responding. Highly rewarding contexts, such as behavioral situations with high reward probability, increase dopamine activations to neutral cues (Kobayashi and Schultz 2014; Matsumoto et al. 2016) and cues that predict aversive outcomes (Matsumoto et al. 2016). Even trial-by-trial behavioral measures in mice and monkeys predict whether dopamine neurons will respond to the current behavioral stimuli (Lak et al. 2017; Matsumoto et al. 2016). Such context-, and even trial-, specific effects seem to demand a systems neuroscience perspective that places focus on the behavior of the animal and not just the underlying neural circuits. Well-controlled experiments will be critical to understand the behavioral consequences of dopamine activity in real-world environments with complex emotional contexts.
The temporal dynamics of the dopamine response can further complicate the interpretation of these responses. Short latency activations can occur 50–90 ms following behavioral events that reflect physical impact, novelty, and stimulus generalization (Fiorillo 2013; Fiorillo et al. 2013b; Lak et al. 2014; Matsumoto and Hikosaka 2009; Nomoto et al. 2010). These short latency responses are not modulated by value, whereas later response components are (Fiorillo et al. 2013a; Lak et al. 2016). Likewise, robust rebound activations are often observed following negative prediction error responses when dopamine neurons can be silent for 200–500 ms (Bayer et al. 2007; Fiorillo et al. 2013b). Together, these complex dynamics, such as short latency and rebound activations, can complicate the interpretation of negative prediction error responses. It is important to note that an abundance of caution should be exercised with the interpretation of this neuronal behavior. Despite the various conclusions that can be reached by applying statistics to selected time windows, it is unclear how an aversive outcome-predicting stimulus that evokes a short latency activation, a long pause in firing, and then a rebound activation would influence dopamine release in the striatum.

The relationship between behavior and striatal dopamine release

Classic studies have repeatedly shown that, even during operant paradigms, dopamine responses are time-locked to external reward predictors, rather than to the onset of well controlled, single joint movements or associated EMG activity (Fig. 4a) (Ljungberg et al. 1992; Schultz et al. 1993; Schultz and Romo 1990). In contrast, larger, multi-muscle movements in monkeys (Schultz et al. 1983) or whole-body movements in rodents (Dodson et al. 2016; Howe and Dombeck 2016; Howe et al. 2013) are correlated with increased dopamine activity in midbrain cell bodies and striatal dopamine release sites. For instance, a recent study in freely behaving rodents found that phasic dopamine release was time-locked to behavior as well as stimuli (Fig. 2d) (Hamid et al. 2016). Investigating this discrepancy and exploring the larger question of how information processing in the striatum is modulated by incoming dopamine signals are of critical importance. Local striatal neurons (cholinergic interneurons) and afferent connections can influence dopamine release at dopamine terminals in the striatum (Cachope and Cheer 2014; Threlfell et al. 2012). This local influence might be especially significant in primates, because the basal ganglia are spatially organized according to cortical inputs (Alexander and DeLong 1985a, b; Alexander et al. 1986). The activity of dopamine neurons and cholinergic neurons is correlated in the NHP basal ganglia, but it is unclear how the behavioral variables coded by cholinergic neurons influence dopamine release (Morris et al. 2004). It is, therefore, important to characterize dopamine release in different functional regions of the striatum and observe the relationship between release and well-controlled behaviors. Recently, cyclic voltammetry was used to monitor dopamine reward responses in NHP striatum (Min et al. 2016; Schluter et al. 2014; Yoshimi et al. 2015), and this technique can shed light on how local network effects influence dopamine release and whether release reflects behavioral parameters other than reward, such as movements and actions.

The role of fast dopamine signals in the frontal cortex

It is widely believed that phasic dopamine signals update action values in the striatum (Shen et al. 2008), but it is less well known what role these signals have in the frontal cortex. There was an extensive expansion of frontal cortex dopamine projections that accompanied the evolution of higher cognitive abilities in primates (Lewis et al. 1987; Smiley et al. 1994; Williams and Goldman-Rakic 1993, 1998). Accordingly, several studies have examined the role of dopamine in attention, working memory, and associative learning (Jacob et al. 2013, 2016; Noudoost and Moore 2011a; Puig and Miller 2012; Vijayraghavan et al. 2007; Williams and Goldman-Rakic 1995). These studies and others like them have relied upon the (relatively) slow process of agonist or antagonist infusion. Even when done with high spatial and temporal precision (Noudoost and Moore 2011b), these manipulations cannot approximate the natural dynamics of phasic dopamine signals. Instead, optogenetics can be employed for millisecond timescale control of dopamine release (Boyden et al. 2005; Tsai et al. 2009). Recently, a dual virus injection was shown to selectively label wild-type NHP dopamine neurons. Optical stimulation of cell bodies positively modulated behavioral read-outs of value. It was not known from that study whether the opsin was expressed in neuron terminals (Stauffer et al. 2016), but future research using next generation molecular tools in NHPs will permit projection specific recording and perturbation of neural activity.

Conclusions

Electrophysiological recordings from dopamine cell bodies in the midbrain have demonstrated that phasic dopamine responses code reward prediction error, the difference between received and predicted reward. Studies in awake, behaving NHP have been critical to this endeavor, because they are highly trainable and can provide a wealth of data through in-depth exploration of single unit dopamine activity. NHP possess a rich and complex behavioral repertoire which has led to advanced understanding of the role of dopamine in learning, movement, and decision making. Not discussed here but worth mentioning, the MPTP monkey model has been critical to the study of neuronal and behavioral deficits associated with Parkinson’s disease. In short, the unique properties of NHP models have made them essential to understanding midbrain dopamine function and dysfunction.
The findings reviewed here demonstrate that, even for studying a relatively simple and evolutionarily old neural structure like dopamine neurons, there are significant advantages to using NHP models. Non-human primates possess behavioral and anatomical characteristics that are more similar to humans than any other experimental animal model. From a behavioral standpoint, the cognitive capability and choice flexibility reviewed here and demonstrated elsewhere (Eiselt and Nieder 2013; Stauffer et al. 2015) resembles human choice behavior. From an anatomical perspective, NHP dopamine projections to the striatum and frontal cortex are most analogous to those in humans. The NHP striatum contains the densest concentration of dopamine terminals (Lynd-Balta and Haber 1994a, b) and is functionally organized according to cortical inputs (Alexander and DeLong 1985a, b; Alexander et al. 1986). Likewise, the dopamine projections to the frontal cortex are massively expanded in NHP, where they primarily target executive and motor regions (Smiley et al. 1994; Williams and Goldman-Rakic 1993, 1998). For these reasons, and because of the clinical relevance of dopamine to numerous movement and mental health disorders including but not limited to Parkinson’s disease, dystonia, ADHD, OCD, psychosis, depression, and schizophrenia, it is critical to maintain and advance behavioral neurophysiology in awake, behaving primates.
A new generation of molecular tools—including optogenetics and in vivo single cell imaging—has revolutionized how we ask questions and even what questions we can ask. These technologies, however, have not been widely incorporated into monkey neurophysiology studies. Although progress is being made, as reviewed elsewhere in this issue (Galvan et al. 2017), there are many technical challenges impeding easy implementation of next generation molecular tools in NHP. Efficient light delivery, large-scale viral infection, and the lack of genetically modified NHP lines all pose significant challenges. Nevertheless, recent developments, including red shifted opsins and improved optical fibers (Acker et al. 2016), better recording/stimulating devices (Yazdan-Shahmorad et al. 2016), and virally mediated cell-type-specific ChR2 expression (El-Shamayleh et al. 2017; Klein et al. 2016; Stauffer et al. 2016), point towards a promising future for optogenetic NHP studies. These new technologies coupled with the enormous and oft-demonstrated utility of the NHP model should ensure continued focus on and research in this most evolutionarily relevant experimental species.

Acknowledgements

The authors would like to thank Andreea Bostan, Jing He, and Amber Torrise for comments and discussion. This work was supported by the University of Pittsburgh Brain Institute and by the National Institutes of Health through the NIH Director’s New Innovator Award 1DP2MH113095.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://​creativecommons.​org/​licenses/​by/​4.​0/​), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Unsere Produktempfehlungen

e.Med Interdisziplinär

Kombi-Abonnement

Für Ihren Erfolg in Klinik und Praxis - Die beste Hilfe in Ihrem Arbeitsalltag

Mit e.Med Interdisziplinär erhalten Sie Zugang zu allen CME-Fortbildungen und Fachzeitschriften auf SpringerMedizin.de.

e.Med Neurologie & Psychiatrie

Kombi-Abonnement

Mit e.Med Neurologie & Psychiatrie erhalten Sie Zugang zu CME-Fortbildungen der Fachgebiete, den Premium-Inhalten der dazugehörigen Fachzeitschriften, inklusive einer gedruckten Zeitschrift Ihrer Wahl.

e.Med Neurologie

Kombi-Abonnement

Mit e.Med Neurologie erhalten Sie Zugang zu CME-Fortbildungen des Fachgebietes, den Premium-Inhalten der neurologischen Fachzeitschriften, inklusive einer gedruckten Neurologie-Zeitschrift Ihrer Wahl.

Literatur
Zurück zum Zitat Aebischer P, Schultz W (1984) The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett 50:25–29PubMedCrossRef Aebischer P, Schultz W (1984) The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett 50:25–29PubMedCrossRef
Zurück zum Zitat Alexander GE, DeLong MR (1985a) Microstimulation of the primate neostriatum. I. Physiological properties of striatal microexcitable zones. J Neurophysiol 53:1401–1416PubMedCrossRef Alexander GE, DeLong MR (1985a) Microstimulation of the primate neostriatum. I. Physiological properties of striatal microexcitable zones. J Neurophysiol 53:1401–1416PubMedCrossRef
Zurück zum Zitat Alexander GE, DeLong MR (1985b) Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. J Neurophysiol 53:1417–1430PubMedCrossRef Alexander GE, DeLong MR (1985b) Microstimulation of the primate neostriatum. II. Somatotopic organization of striatal microexcitable zones and their relation to neuronal response properties. J Neurophysiol 53:1417–1430PubMedCrossRef
Zurück zum Zitat Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381PubMedCrossRef Alexander GE, DeLong MR, Strick PL (1986) Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9:357–381PubMedCrossRef
Zurück zum Zitat Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol 98:1428–1439PubMedCrossRef Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophysiol 98:1428–1439PubMedCrossRef
Zurück zum Zitat Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K (2005) Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci 8:1263–1268PubMedCrossRef Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K (2005) Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci 8:1263–1268PubMedCrossRef
Zurück zum Zitat Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci 106:4894–4899PubMedPubMedCentralCrossRef Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci 106:4894–4899PubMedPubMedCentralCrossRef
Zurück zum Zitat Bromberg-Martin ES, Hikosaka O (2009) Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63:119–126PubMedPubMedCentralCrossRef Bromberg-Martin ES, Hikosaka O (2009) Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63:119–126PubMedPubMedCentralCrossRef
Zurück zum Zitat Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef
Zurück zum Zitat Bunney BS, Aghajanian GK, Roth RH (1973) Comparison of effects of l-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nature 245:123–125 Bunney BS, Aghajanian GK, Roth RH (1973) Comparison of effects of l-dopa, amphetamine and apomorphine on firing rate of rat dopaminergic neurones. Nature 245:123–125
Zurück zum Zitat Caraco T, Martindale S, Whittam TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef Caraco T, Martindale S, Whittam TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef
Zurück zum Zitat Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef
Zurück zum Zitat Dahlstroem A, Fuxe K (1964) Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol Scand Suppl 232:231–255 Dahlstroem A, Fuxe K (1964) Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol Scand Suppl 232:231–255
Zurück zum Zitat Debreu G (1959) Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud 26:174–177CrossRef Debreu G (1959) Cardinal utility for even-chance mixtures of pairs of sure prospects. Rev Econ Stud 26:174–177CrossRef
Zurück zum Zitat Dodson PD, Dreyer JK, Jennings KA, Syed ECJ, Wade-Martins R, Cragg SJ, Bolam JP, Magill PJ (2016) Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc Natl Acad Sci 113:E2180–E2188PubMedPubMedCentralCrossRef Dodson PD, Dreyer JK, Jennings KA, Syed ECJ, Wade-Martins R, Cragg SJ, Bolam JP, Magill PJ (2016) Representation of spontaneous movement by dopaminergic neurons is cell-type selective and disrupted in parkinsonism. Proc Natl Acad Sci 113:E2180–E2188PubMedPubMedCentralCrossRef
Zurück zum Zitat Eiselt AK, Nieder A (2013) Representation of abstract quantitative rules applied to spatial and numerical magnitudes in primate prefrontal cortex. J Neurosci 33:7526–7534PubMedCrossRef Eiselt AK, Nieder A (2013) Representation of abstract quantitative rules applied to spatial and numerical magnitudes in primate prefrontal cortex. J Neurosci 33:7526–7534PubMedCrossRef
Zurück zum Zitat El-Shamayleh Y, Kojima Y, Soetedjo R, Horwitz GD (2017) Selective optogenetic control of Purkinje cells in monkey cerebellum. Neuron 95:51–62PubMedCrossRef El-Shamayleh Y, Kojima Y, Soetedjo R, Horwitz GD (2017) Selective optogenetic control of Purkinje cells in monkey cerebellum. Neuron 95:51–62PubMedCrossRef
Zurück zum Zitat Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef
Zurück zum Zitat Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N (2015) Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525:243–246PubMedPubMedCentralCrossRef Eshel N, Bukwich M, Rao V, Hemmelder V, Tian J, Uchida N (2015) Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525:243–246PubMedPubMedCentralCrossRef
Zurück zum Zitat Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Song MR, Yun SR (2013a) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef Fiorillo CD, Song MR, Yun SR (2013a) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Yun SR, Song MR (2013b) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef Fiorillo CD, Yun SR, Song MR (2013b) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef
Zurück zum Zitat Galvan A, Caiola MJ, Albaugh DL (2017) Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates. J Neural Transm (Vienna). doi:10.1007/s00702-017-1697-8 Galvan A, Caiola MJ, Albaugh DL (2017) Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates. J Neural Transm (Vienna). doi:10.​1007/​s00702-017-1697-8
Zurück zum Zitat Genest W, Stauffer WR, Schultz W (2016) Utility functions predict variance and skewness risk preferences in monkeys. Proc Natl Acad Sci USA 113:8402–8407PubMedPubMedCentralCrossRef Genest W, Stauffer WR, Schultz W (2016) Utility functions predict variance and skewness risk preferences in monkeys. Proc Natl Acad Sci USA 113:8402–8407PubMedPubMedCentralCrossRef
Zurück zum Zitat German DC, Manaye KF (1993) Midbrain dopaminergic neurons (nuclei A8, A9, and A10): three-dimensional reconstruction in the rat. J Comp Neurol 331:297–309PubMedCrossRef German DC, Manaye KF (1993) Midbrain dopaminergic neurons (nuclei A8, A9, and A10): three-dimensional reconstruction in the rat. J Comp Neurol 331:297–309PubMedCrossRef
Zurück zum Zitat Gonon FG (1988) Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 24:19–28PubMedCrossRef Gonon FG (1988) Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 24:19–28PubMedCrossRef
Zurück zum Zitat Grace AA, Bunney BS (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization. Neuroscience 10:301–315PubMedCrossRef Grace AA, Bunney BS (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons—1. Identification and characterization. Neuroscience 10:301–315PubMedCrossRef
Zurück zum Zitat Guyenet PG, Aghajanian GK (1978) Antidromic identification of dopaminergic and other output neurons of the rat substantia nigra. Brain Res 150:69–84PubMedCrossRef Guyenet PG, Aghajanian GK (1978) Antidromic identification of dopaminergic and other output neurons of the rat substantia nigra. Brain Res 150:69–84PubMedCrossRef
Zurück zum Zitat Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2016) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedCrossRef Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2016) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedCrossRef
Zurück zum Zitat Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef
Zurück zum Zitat Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef Howe MW, Tierney PL, Sandberg SG, Phillips PE, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef
Zurück zum Zitat Jacob SN, Ott T, Nieder A (2013) Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals. J Neurosci 33:13724–13734PubMedCrossRef Jacob SN, Ott T, Nieder A (2013) Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals. J Neurosci 33:13724–13734PubMedCrossRef
Zurück zum Zitat Jacob SN, Stalter M, Nieder A (2016) Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex. Nat Commun 7:13218PubMedPubMedCentralCrossRef Jacob SN, Stalter M, Nieder A (2016) Cell-type-specific modulation of targets and distractors by dopamine D1 receptors in primate prefrontal cortex. Nat Commun 7:13218PubMedPubMedCentralCrossRef
Zurück zum Zitat Klein C, Evrard HC, Shapcott KA, Haverkamp S, Logothetis NK, Schmid MC (2016) Cell-targeted optogenetics and electrical microstimulation reveal the primate koniocellular projection to supra-granular visual cortex. Neuron 90:143–151PubMedCrossRef Klein C, Evrard HC, Shapcott KA, Haverkamp S, Logothetis NK, Schmid MC (2016) Cell-targeted optogenetics and electrical microstimulation reveal the primate koniocellular projection to supra-granular visual cortex. Neuron 90:143–151PubMedCrossRef
Zurück zum Zitat Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef
Zurück zum Zitat Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A (2017) Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Curr Biol 27:821–832PubMedCrossRef Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A (2017) Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision. Curr Biol 27:821–832PubMedCrossRef
Zurück zum Zitat Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature 491:212–217PubMedPubMedCentralCrossRef Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC (2012) Input-specific control of reward and aversion in the ventral tegmental area. Nature 491:212–217PubMedPubMedCentralCrossRef
Zurück zum Zitat Lammel S, Steinberg EE, Foldy C, Wall NR, Beier K, Luo L, Malenka RC (2015) Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85:429–438PubMedPubMedCentralCrossRef Lammel S, Steinberg EE, Foldy C, Wall NR, Beier K, Luo L, Malenka RC (2015) Diversity of transgenic mouse models for selective targeting of midbrain dopamine neurons. Neuron 85:429–438PubMedPubMedCentralCrossRef
Zurück zum Zitat Langston JW, Ballard P, Tetrud JW, Irwin I (1983) Chronic Parkinsonism in humans due to a product of meperidine-analog synthesis. Science 219:979–980PubMedCrossRef Langston JW, Ballard P, Tetrud JW, Irwin I (1983) Chronic Parkinsonism in humans due to a product of meperidine-analog synthesis. Science 219:979–980PubMedCrossRef
Zurück zum Zitat Lewis DA, Campbell MJ, Foote SL, Goldstein M, Morrison JH (1987) The distribution of tyrosine hydroxylase-immunoreactive fibers in primate neocortex is widespread but regionally specific. J Neurosci 7:279–290PubMed Lewis DA, Campbell MJ, Foote SL, Goldstein M, Morrison JH (1987) The distribution of tyrosine hydroxylase-immunoreactive fibers in primate neocortex is widespread but regionally specific. J Neurosci 7:279–290PubMed
Zurück zum Zitat Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMedCrossRef Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMedCrossRef
Zurück zum Zitat Lovenberg W, Weissbach H, Udenfriend S (1962) Aromatic l-amino acid decarboxylase. J Biol Chem 237:89–93PubMed Lovenberg W, Weissbach H, Udenfriend S (1962) Aromatic l-amino acid decarboxylase. J Biol Chem 237:89–93PubMed
Zurück zum Zitat Lynd-Balta E, Haber SN (1994a) The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59:625–640PubMedCrossRef Lynd-Balta E, Haber SN (1994a) The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience 59:625–640PubMedCrossRef
Zurück zum Zitat Lynd-Balta E, Haber SN (1994b) The organization of midbrain projections to the ventral striatum in the primate. Neuroscience 59:609–623PubMedCrossRef Lynd-Balta E, Haber SN (1994b) The organization of midbrain projections to the ventral striatum in the primate. Neuroscience 59:609–623PubMedCrossRef
Zurück zum Zitat Lynd-Balta E, Haber SN (1994c) Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol 345:562–578PubMedCrossRef Lynd-Balta E, Haber SN (1994c) Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J Comp Neurol 345:562–578PubMedCrossRef
Zurück zum Zitat Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef
Zurück zum Zitat Matsumoto H, Tian J, Uchida N, Watabe-Uchida M (2016) Midbrain dopamine neurons signal aversion in a reward-context-dependent manner. Elife 5:e17328PubMedPubMedCentralCrossRef Matsumoto H, Tian J, Uchida N, Watabe-Uchida M (2016) Midbrain dopamine neurons signal aversion in a reward-context-dependent manner. Elife 5:e17328PubMedPubMedCentralCrossRef
Zurück zum Zitat McCoy AN, Platt ML (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8:1220–1227PubMedCrossRef McCoy AN, Platt ML (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8:1220–1227PubMedCrossRef
Zurück zum Zitat Min HK, Ross EK, Jo HJ, Cho S, Settell ML, Jeong JH, Duffy PS, Chang SY, Bennet KE, Blaha CD et al (2016) Dopamine release in the nonhuman primate caudate and putamen depends upon site of stimulation in the subthalamic nucleus. J Neurosci 36:6022–6029PubMedPubMedCentralCrossRef Min HK, Ross EK, Jo HJ, Cho S, Settell ML, Jeong JH, Duffy PS, Chang SY, Bennet KE, Blaha CD et al (2016) Dopamine release in the nonhuman primate caudate and putamen depends upon site of stimulation in the subthalamic nucleus. J Neurosci 36:6022–6029PubMedPubMedCentralCrossRef
Zurück zum Zitat Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMedCrossRef Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMedCrossRef
Zurück zum Zitat Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef
Zurück zum Zitat Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed
Zurück zum Zitat Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef
Zurück zum Zitat Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef
Zurück zum Zitat Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef
Zurück zum Zitat Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef
Zurück zum Zitat Noudoost B, Moore T (2011b) A reliable microinjectrode system for use in behaving monkeys. J Neurosci Methods 194:218–223PubMedCrossRef Noudoost B, Moore T (2011b) A reliable microinjectrode system for use in behaving monkeys. J Neurosci Methods 194:218–223PubMedCrossRef
Zurück zum Zitat Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427PubMedCrossRef Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427PubMedCrossRef
Zurück zum Zitat O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789–800PubMedCrossRef O’Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789–800PubMedCrossRef
Zurück zum Zitat Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef
Zurück zum Zitat Puig MV, Miller EK (2012) The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 74:874–886PubMedCrossRef Puig MV, Miller EK (2012) The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 74:874–886PubMedCrossRef
Zurück zum Zitat Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99 Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non reinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 64–99
Zurück zum Zitat Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMedCrossRef Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMedCrossRef
Zurück zum Zitat Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1461PubMedCrossRef Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1461PubMedCrossRef
Zurück zum Zitat Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201–217PubMedCrossRef Schultz W, Romo R (1987) Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57:201–217PubMedCrossRef
Zurück zum Zitat Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMedCrossRef Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMedCrossRef
Zurück zum Zitat Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef
Zurück zum Zitat Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed
Zurück zum Zitat Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef
Zurück zum Zitat Smiley JF, Levey AI, Ciliax BJ, Goldman-Rakic PS (1994) D1 dopamine receptor immunoreactivity in human and monkey cerebral cortex: predominant and extrasynaptic localization in dendritic spines. Proc Natl Acad Sci USA 91:5720–5724PubMedPubMedCentralCrossRef Smiley JF, Levey AI, Ciliax BJ, Goldman-Rakic PS (1994) D1 dopamine receptor immunoreactivity in human and monkey cerebral cortex: predominant and extrasynaptic localization in dendritic spines. Proc Natl Acad Sci USA 91:5720–5724PubMedPubMedCentralCrossRef
Zurück zum Zitat Smith Y, Wichmann T, DeLong MR (2014) Corticostriatal and mesocortical dopamine systems: do species differences matter? Nat Rev Neurosci 15:63PubMedCrossRef Smith Y, Wichmann T, DeLong MR (2014) Corticostriatal and mesocortical dopamine systems: do species differences matter? Nat Rev Neurosci 15:63PubMedCrossRef
Zurück zum Zitat Stark AK, Pakkenberg B (2004) Histological changes of the dopaminergic nigrostriatal system in aging. Cell Tissue Res 318:81–92PubMedCrossRef Stark AK, Pakkenberg B (2004) Histological changes of the dopaminergic nigrostriatal system in aging. Cell Tissue Res 318:81–92PubMedCrossRef
Zurück zum Zitat Stauffer WR, Lak A, Yang A, Borel M, Paulsen O, Boyden ES, Schultz W (2016) Dopamine neuron-specific optogenetic stimulation in rhesus macaques. Cell 166(1564–1571):e1566 Stauffer WR, Lak A, Yang A, Borel M, Paulsen O, Boyden ES, Schultz W (2016) Dopamine neuron-specific optogenetic stimulation in rhesus macaques. Cell 166(1564–1571):e1566
Zurück zum Zitat Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef
Zurück zum Zitat Studer A, Schultz W (1987) The catecholamine uptake inhibitor nomifensine depresses impulse activity of dopamine neurons in mouse substantia nigra. Neurosci Lett 80:207–212PubMedCrossRef Studer A, Schultz W (1987) The catecholamine uptake inhibitor nomifensine depresses impulse activity of dopamine neurons in mouse substantia nigra. Neurosci Lett 80:207–212PubMedCrossRef
Zurück zum Zitat Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Zurück zum Zitat Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ (2012) Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75:58–64PubMedCrossRef Threlfell S, Lalic T, Platt NJ, Jennings KA, Deisseroth K, Cragg SJ (2012) Striatal dopamine release is triggered by synchronized activity in cholinergic interneurons. Neuron 75:58–64PubMedCrossRef
Zurück zum Zitat Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef
Zurück zum Zitat Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef
Zurück zum Zitat Ungless MA, Grace AA (2012) Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35:422–430PubMedPubMedCentralCrossRef Ungless MA, Grace AA (2012) Are you or aren’t you? Challenges associated with physiologically identifying dopamine neurons. Trends Neurosci 35:422–430PubMedPubMedCentralCrossRef
Zurück zum Zitat Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AF (2007) Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci 10:376–384PubMedCrossRef Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AF (2007) Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci 10:376–384PubMedCrossRef
Zurück zum Zitat von Neumann J, Morgenstern O, Kuhn HW, Rubinstein A (1944) Theory of games and economic behavior (60th anniversary commemorative edition). Princeton University Press, Princeton von Neumann J, Morgenstern O, Kuhn HW, Rubinstein A (1944) Theory of games and economic behavior (60th anniversary commemorative edition). Princeton University Press, Princeton
Zurück zum Zitat Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef
Zurück zum Zitat Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef
Zurück zum Zitat Williams SM, Goldman-Rakic PS (1993) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex 3:199–222PubMedCrossRef Williams SM, Goldman-Rakic PS (1993) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb Cortex 3:199–222PubMedCrossRef
Zurück zum Zitat Williams GV, Goldman-Rakic PS (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376:572–575PubMedCrossRef Williams GV, Goldman-Rakic PS (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376:572–575PubMedCrossRef
Zurück zum Zitat Williams SM, Goldman-Rakic PS (1998) Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex 8:321–345PubMedCrossRef Williams SM, Goldman-Rakic PS (1998) Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex 8:321–345PubMedCrossRef
Zurück zum Zitat Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793PubMedPubMedCentralCrossRef Yamada H, Tymula A, Louie K, Glimcher PW (2013) Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110:15788–15793PubMedPubMedCentralCrossRef
Zurück zum Zitat Yazdan-Shahmorad A, Diaz-Botia C, Hanson TL, Kharazia V, Ledochowitsch P, Maharbiz MM, Sabes PN (2016) A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron 89:927–939PubMedCrossRef Yazdan-Shahmorad A, Diaz-Botia C, Hanson TL, Kharazia V, Ledochowitsch P, Maharbiz MM, Sabes PN (2016) A large-scale interface for optogenetic stimulation and recording in nonhuman primates. Neuron 89:927–939PubMedCrossRef
Zurück zum Zitat Yelin R, Schuldiner S (1995) The pharmacological profile of the vesicular monoamine transporter resembles that of multidrug transporters. FEBS Lett 377:201–207PubMedCrossRef Yelin R, Schuldiner S (1995) The pharmacological profile of the vesicular monoamine transporter resembles that of multidrug transporters. FEBS Lett 377:201–207PubMedCrossRef
Zurück zum Zitat Yoshimi K, Kumada S, Weitemier A, Jo T, Inoue M (2015) Reward-induced phasic dopamine release in the monkey ventral striatum and putamen. PLoS One 10:e0130443PubMedPubMedCentralCrossRef Yoshimi K, Kumada S, Weitemier A, Jo T, Inoue M (2015) Reward-induced phasic dopamine release in the monkey ventral striatum and putamen. PLoS One 10:e0130443PubMedPubMedCentralCrossRef
Metadaten
Titel
Reward and value coding by dopamine neurons in non-human primates
verfasst von
Aydin Alikaya
Mackenzie Rack-Wildner
William R. Stauffer
Publikationsdatum
26.10.2017
Verlag
Springer Vienna
Erschienen in
Journal of Neural Transmission / Ausgabe 3/2018
Print ISSN: 0300-9564
Elektronische ISSN: 1435-1463
DOI
https://doi.org/10.1007/s00702-017-1793-9

Weitere Artikel der Ausgabe 3/2018

Journal of Neural Transmission 3/2018 Zur Ausgabe

Neurology and Preclinical Neurological Studies - Review Article

α-Synuclein nonhuman primate models of Parkinson’s disease

Neurology and Preclinical Neurological Studies - Review Article

Advances in optogenetic and chemogenetic methods to study brain circuits in non-human primates

Neurology and Preclinical Neurological Studies - Review Article

Non-human primate models of PD to test novel therapies

Leitlinien kompakt für die Neurologie

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

Update Neurologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.