Skip to main content
Erschienen in: Journal of Neural Transmission 7/2016

Open Access 02.02.2016 | Neurology and Preclinical Neurological Studies - Review Article

Reward functions of the basal ganglia

verfasst von: Wolfram Schultz

Erschienen in: Journal of Neural Transmission | Ausgabe 7/2016

Einloggen, um Zugang zu erhalten

Abstract

Besides their fundamental movement function evidenced by Parkinsonian deficits, the basal ganglia are involved in processing closely linked non-motor, cognitive and reward information. This review describes the reward functions of three brain structures that are major components of the basal ganglia or are closely associated with the basal ganglia, namely midbrain dopamine neurons, pedunculopontine nucleus, and striatum (caudate nucleus, putamen, nucleus accumbens). Rewards are involved in learning (positive reinforcement), approach behavior, economic choices and positive emotions. The response of dopamine neurons to rewards consists of an early detection component and a subsequent reward component that reflects a prediction error in economic utility, but is unrelated to movement. Dopamine activations to non-rewarded or aversive stimuli reflect physical impact, but not punishment. Neurons in pedunculopontine nucleus project their axons to dopamine neurons and process sensory stimuli, movements and rewards and reward-predicting stimuli without coding outright reward prediction errors. Neurons in striatum, besides their pronounced movement relationships, process rewards irrespective of sensory and motor aspects, integrate reward information into movement activity, code the reward value of individual actions, change their reward-related activity during learning, and code own reward in social situations depending on whose action produces the reward. These data demonstrate a variety of well-characterized reward processes in specific basal ganglia nuclei consistent with an important function in non-motor aspects of motivated behavior.
Literatur
Zurück zum Zitat Adamantidis AR, Tsai H-C, Boutrel B, Zhang F, Stuber GD, Budygin EA, Touriño C, Bonci A, Deisseroth K, de Lecea L (2011) Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J Neurosci 31:10829–10835PubMedPubMedCentralCrossRef Adamantidis AR, Tsai H-C, Boutrel B, Zhang F, Stuber GD, Budygin EA, Touriño C, Bonci A, Deisseroth K, de Lecea L (2011) Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J Neurosci 31:10829–10835PubMedPubMedCentralCrossRef
Zurück zum Zitat Adler A, Katabi S, Finkes I, Prut Y, Bergman H (2013) Different correlation patterns of cholinergic and GABAergic interneurons with striatal projection neurons. Front Syst Neurosci 7:47PubMedPubMedCentralCrossRef Adler A, Katabi S, Finkes I, Prut Y, Bergman H (2013) Different correlation patterns of cholinergic and GABAergic interneurons with striatal projection neurons. Front Syst Neurosci 7:47PubMedPubMedCentralCrossRef
Zurück zum Zitat Ambroggi F, Ishikawa A, Fields HL, Nicola SM (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59:648–661PubMedPubMedCentralCrossRef Ambroggi F, Ishikawa A, Fields HL, Nicola SM (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59:648–661PubMedPubMedCentralCrossRef
Zurück zum Zitat Aosaki T, Tsubokawa H, Ishida A, Watanabe K, Graybiel AM, Kimura M (1994) Responses of tonically active neurons in the primate’s striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci 14:3969–3984PubMed Aosaki T, Tsubokawa H, Ishida A, Watanabe K, Graybiel AM, Kimura M (1994) Responses of tonically active neurons in the primate’s striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci 14:3969–3984PubMed
Zurück zum Zitat Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85:491–500PubMedCrossRef Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85:491–500PubMedCrossRef
Zurück zum Zitat Apicella P, Scarnati E, Ljungberg T, Schultz W (1992) Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J Neurophysiol 68:945–960PubMed Apicella P, Scarnati E, Ljungberg T, Schultz W (1992) Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J Neurophysiol 68:945–960PubMed
Zurück zum Zitat Apicella P, Deffains M, Ravel S, Legallet E (2009) Tonically active neurons in the striatum differentiate between delivery and omission of expected reward in a probabilistic task context. Eur J Neurosci 30:515–526PubMedCrossRef Apicella P, Deffains M, Ravel S, Legallet E (2009) Tonically active neurons in the striatum differentiate between delivery and omission of expected reward in a probabilistic task context. Eur J Neurosci 30:515–526PubMedCrossRef
Zurück zum Zitat Apicella P, Ravel S, Deffains M, Legallet E (2011) The role of striatal tonically active neurons in reward prediction error signaling during instrumental task performance. J Neurosci 31:1507–1515PubMedCrossRef Apicella P, Ravel S, Deffains M, Legallet E (2011) The role of striatal tonically active neurons in reward prediction error signaling during instrumental task performance. J Neurosci 31:1507–1515PubMedCrossRef
Zurück zum Zitat Asaad WF, Eskandar EN (2011) Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus. J Neurosci 31:17772–17787PubMedPubMedCentralCrossRef Asaad WF, Eskandar EN (2011) Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus. J Neurosci 31:17772–17787PubMedPubMedCentralCrossRef
Zurück zum Zitat Báez-Mendoza R, Harris C, Schultz W (2013) Activity of striatal neurons reflects social action and own reward. Proc Natl Acad Sci USA 110:16634–16639PubMedPubMedCentralCrossRef Báez-Mendoza R, Harris C, Schultz W (2013) Activity of striatal neurons reflects social action and own reward. Proc Natl Acad Sci USA 110:16634–16639PubMedPubMedCentralCrossRef
Zurück zum Zitat Bowman EM, Aigner TG, Richmond BJ (1996) Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75:1061–1073PubMed Bowman EM, Aigner TG, Richmond BJ (1996) Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75:1061–1073PubMed
Zurück zum Zitat Bredfeldt CE, Ringach DL (2002) Dynamics of spatial frequency tuning in macaque V1. J Neurosci 22:1976–1984PubMed Bredfeldt CE, Ringach DL (2002) Dynamics of spatial frequency tuning in macaque V1. J Neurosci 22:1976–1984PubMed
Zurück zum Zitat Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106:4894–4899PubMedPubMedCentralCrossRef Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106:4894–4899PubMedPubMedCentralCrossRef
Zurück zum Zitat Bromberg-Martin ES, Matsumoto M, Hon S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef Bromberg-Martin ES, Matsumoto M, Hon S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068–1076PubMedPubMedCentralCrossRef
Zurück zum Zitat Caraco T, Martindale S, Whitham TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef Caraco T, Martindale S, Whitham TS (1980) An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28:820–830CrossRef
Zurück zum Zitat Chang SWC, Gariépy J-F, Platt ML (2013) Neuronal reference frames for social decisions in primate frontal cortex. Nat Neurosci 16:243–250PubMedCrossRef Chang SWC, Gariépy J-F, Platt ML (2013) Neuronal reference frames for social decisions in primate frontal cortex. Nat Neurosci 16:243–250PubMedCrossRef
Zurück zum Zitat Chelazzi L, Estocinova J, Calletti R, Gerfo EL, Sani I, Della Libera C, Santandrea E (2014) Altering spatial priority maps via reward-based learning. J Neurosci 34:8594–8604PubMedCrossRef Chelazzi L, Estocinova J, Calletti R, Gerfo EL, Sani I, Della Libera C, Santandrea E (2014) Altering spatial priority maps via reward-based learning. J Neurosci 34:8594–8604PubMedCrossRef
Zurück zum Zitat Cheng JJ, de Bruin JPC, Feenstra MGP (2003) Dopamine efflux in nucleus accumbens shell and core in response to appetitive classical conditioning. Eur J Neurosci 18:1306–1314PubMedCrossRef Cheng JJ, de Bruin JPC, Feenstra MGP (2003) Dopamine efflux in nucleus accumbens shell and core in response to appetitive classical conditioning. Eur J Neurosci 18:1306–1314PubMedCrossRef
Zurück zum Zitat Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N (2012) Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482:85–88PubMedPubMedCentralCrossRef
Zurück zum Zitat Corbett D, Wise RA (1980) Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable microelectrode study. Brain Res 185:1–15PubMedCrossRef Corbett D, Wise RA (1980) Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable microelectrode study. Brain Res 185:1–15PubMedCrossRef
Zurück zum Zitat Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89:2823–2838PubMedCrossRef Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89:2823–2838PubMedCrossRef
Zurück zum Zitat Cromwell HC, Hassani OK, Schultz W (2005) Relative reward processing in primate striatum. Exp Brain Res 162:520–525PubMedCrossRef Cromwell HC, Hassani OK, Schultz W (2005) Relative reward processing in primate striatum. Exp Brain Res 162:520–525PubMedCrossRef
Zurück zum Zitat Day JJ, Roitman MF, Wightman RM, Carelli RM (2007) Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10:1020–1028PubMedCrossRef Day JJ, Roitman MF, Wightman RM, Carelli RM (2007) Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10:1020–1028PubMedCrossRef
Zurück zum Zitat Day JJ, Jones JL, Wightman RM, Carelli RM (2010) Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiat 68:306–309PubMedPubMedCentralCrossRef Day JJ, Jones JL, Wightman RM, Carelli RM (2010) Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiat 68:306–309PubMedPubMedCentralCrossRef
Zurück zum Zitat Day JJ, Jones JL, Carelli RM (2011) Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur J Neurosci 33:308–321PubMedCrossRef Day JJ, Jones JL, Carelli RM (2011) Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur J Neurosci 33:308–321PubMedCrossRef
Zurück zum Zitat de Lafuente O, Romo R (2011) Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions. Proc Natl Acad Sci 49:19767–19771CrossRef de Lafuente O, Romo R (2011) Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions. Proc Natl Acad Sci 49:19767–19771CrossRef
Zurück zum Zitat DeLong MR, Crutcher MD, Georgopoulos AP (1983) Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J Neurosci 3:1599–1606PubMed DeLong MR, Crutcher MD, Georgopoulos AP (1983) Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J Neurosci 3:1599–1606PubMed
Zurück zum Zitat di Loreto S, Florio T, Scarnati E (1992) Evidence that a non-NMDA receptor is involved in the excitatory pathway from the pedunculopontine region to nigrostriatal dopaminergic neurons. Exp Brain Res 89:79–86PubMedCrossRef di Loreto S, Florio T, Scarnati E (1992) Evidence that a non-NMDA receptor is involved in the excitatory pathway from the pedunculopontine region to nigrostriatal dopaminergic neurons. Exp Brain Res 89:79–86PubMedCrossRef
Zurück zum Zitat Dickinson A, Balleine B (1994) Motivational control of goal-directed action. Anim Learn Behav 22:1–18CrossRef Dickinson A, Balleine B (1994) Motivational control of goal-directed action. Anim Learn Behav 22:1–18CrossRef
Zurück zum Zitat Dormont JF, Conde H, Farin D (1998) The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Exp Brain Res 121:401–410PubMedCrossRef Dormont JF, Conde H, Farin D (1998) The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Exp Brain Res 121:401–410PubMedCrossRef
Zurück zum Zitat Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M (2011) Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108:15462–15467PubMedPubMedCentralCrossRef
Zurück zum Zitat Fibiger HC, LePiane FG, Jakubovic A, Phillips AG (1987) The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J Neurosci 7:3888–3896PubMed Fibiger HC, LePiane FG, Jakubovic A, Phillips AG (1987) The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J Neurosci 7:3888–3896PubMed
Zurück zum Zitat Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef Fiorillo CD (2013) Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341:546–549PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef Fiorillo CD, Newsome WT, Schultz W (2008) The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11:966–973PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Yun SR, Song MR (2013a) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef Fiorillo CD, Yun SR, Song MR (2013a) Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33:4693–4709PubMedCrossRef
Zurück zum Zitat Fiorillo CD, Song MR, Yun SR (2013b) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef Fiorillo CD, Song MR, Yun SR (2013b) Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33:4710–4725PubMedCrossRef
Zurück zum Zitat Freund TF, Powell JF, Smith AD (1984) Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13:1189–1215PubMedCrossRef Freund TF, Powell JF, Smith AD (1984) Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13:1189–1215PubMedCrossRef
Zurück zum Zitat Futami T, Takakusaki K, Kitai ST (1995) Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta. Neurosci Res 21:331–342PubMedCrossRef Futami T, Takakusaki K, Kitai ST (1995) Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta. Neurosci Res 21:331–342PubMedCrossRef
Zurück zum Zitat Gdowski MJ, Miller LE, Parrish T, Nenonene EK, Houk JC (2001) Context dependency in the globus pallidus internal segment during targeted arm movements. J Neurophysiol 85:998–1004PubMed Gdowski MJ, Miller LE, Parrish T, Nenonene EK, Houk JC (2001) Context dependency in the globus pallidus internal segment during targeted arm movements. J Neurophysiol 85:998–1004PubMed
Zurück zum Zitat Goldman-Rakic PS, Leranth C, Williams MS, Mons N, Geffard M (1989) Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc Natl Acad Sci USA 86:9015–9019PubMedPubMedCentralCrossRef Goldman-Rakic PS, Leranth C, Williams MS, Mons N, Geffard M (1989) Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc Natl Acad Sci USA 86:9015–9019PubMedPubMedCentralCrossRef
Zurück zum Zitat Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience inmonkey parietal cortex. Nature 391:481–484PubMedCrossRef Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience inmonkey parietal cortex. Nature 391:481–484PubMedCrossRef
Zurück zum Zitat Grace AA (1991) Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41:1–24PubMedCrossRef Grace AA (1991) Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41:1–24PubMedCrossRef
Zurück zum Zitat Grace AA, Floresco SB, Goto Y, Lodge DJ (2007) Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci 30:220–227PubMedCrossRef Grace AA, Floresco SB, Goto Y, Lodge DJ (2007) Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci 30:220–227PubMedCrossRef
Zurück zum Zitat Guarraci FA, Kapp BS (1999) An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav Brain Res 99:169–179PubMedCrossRef Guarraci FA, Kapp BS (1999) An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav Brain Res 99:169–179PubMedCrossRef
Zurück zum Zitat Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2015) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedPubMedCentralCrossRef Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R, Vander Weele CM, Kennedy RT, Aragona BJ, Berke JD (2015) Mesolimbic dopamine signals the value of work. Nat Neurosci 19:117–126PubMedPubMedCentralCrossRef
Zurück zum Zitat Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477–2489PubMed Hassani OK, Cromwell HC, Schultz W (2001) Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85:2477–2489PubMed
Zurück zum Zitat Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814–832PubMed Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814–832PubMed
Zurück zum Zitat Histed MH, Pasupathy A, Miller EK (2009) Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions. Neuron 63:244–253PubMedPubMedCentralCrossRef Histed MH, Pasupathy A, Miller EK (2009) Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions. Neuron 63:244–253PubMedPubMedCentralCrossRef
Zurück zum Zitat Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304–309PubMedCrossRef
Zurück zum Zitat Hollerman JR, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947–963PubMed Hollerman JR, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947–963PubMed
Zurück zum Zitat Hong S, Hikosaka O (2014) Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor, and alerting signals to midbrain dopamine neurons. Neuroscience 282:139–155PubMedPubMedCentralCrossRef Hong S, Hikosaka O (2014) Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor, and alerting signals to midbrain dopamine neurons. Neuroscience 282:139–155PubMedPubMedCentralCrossRef
Zurück zum Zitat Horvitz JC, Stewart T, Jacobs BL (1997) Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res 759:251–258PubMedCrossRef Horvitz JC, Stewart T, Jacobs BL (1997) Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res 759:251–258PubMedCrossRef
Zurück zum Zitat Howe MW, Tierney PL, Sandberg SG, Phillips PEM, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef Howe MW, Tierney PL, Sandberg SG, Phillips PEM, Graybiel AM (2013) Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature 500:575–579PubMedPubMedCentralCrossRef
Zurück zum Zitat Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861–9874PubMedCrossRef Ito M, Doya K (2009) Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29:9861–9874PubMedCrossRef
Zurück zum Zitat Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1745–1749PubMedCrossRef Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1745–1749PubMedCrossRef
Zurück zum Zitat Joshua M, Adler A, Mitelman R, Vaadia E, Bergman H (2008) Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J Neurosci 28:11673–11684PubMedCrossRef Joshua M, Adler A, Mitelman R, Vaadia E, Bergman H (2008) Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J Neurosci 28:11673–11684PubMedCrossRef
Zurück zum Zitat Kagel JH, Battalio RC, Green L (1995) Economic choice theory: an experimental analysis of animal behavior. Cambridge University Press, CambridgeCrossRef Kagel JH, Battalio RC, Green L (1995) Economic choice theory: an experimental analysis of animal behavior. Cambridge University Press, CambridgeCrossRef
Zurück zum Zitat Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416PubMedCrossRef Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416PubMedCrossRef
Zurück zum Zitat Kelly PH, Seviour PW, Iversen SD (1975) Amphetamine and apomorphine responses in the rat following 6-OHDA lesions of the nucleus accumbens septi and corpus striatum. Brain Res 94:507–522PubMedCrossRef Kelly PH, Seviour PW, Iversen SD (1975) Amphetamine and apomorphine responses in the rat following 6-OHDA lesions of the nucleus accumbens septi and corpus striatum. Brain Res 94:507–522PubMedCrossRef
Zurück zum Zitat Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176–185PubMedCrossRef Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176–185PubMedCrossRef
Zurück zum Zitat Kim H, Sul JH, Huh N, Lee D, Jung MW (2009) Role of striatum in updating values of chosen actions. J Neurosci 29:14701–14712PubMedCrossRef Kim H, Sul JH, Huh N, Lee D, Jung MW (2009) Role of striatum in updating values of chosen actions. J Neurosci 29:14701–14712PubMedCrossRef
Zurück zum Zitat Kim KM, Baratta MV, Yang A, Lee D, Boyden ES, Fiorillo CD (2012) Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLoS One 7:e33612PubMedPubMedCentralCrossRef Kim KM, Baratta MV, Yang A, Lee D, Boyden ES, Fiorillo CD (2012) Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLoS One 7:e33612PubMedPubMedCentralCrossRef
Zurück zum Zitat Kimura M, Rajkowski J, Evarts E (1984) Tonically discharging putamen neurons exhibit set-dependent responses. Proc Natl Acad Sci USA 81:4998–5001PubMedPubMedCentralCrossRef Kimura M, Rajkowski J, Evarts E (1984) Tonically discharging putamen neurons exhibit set-dependent responses. Proc Natl Acad Sci USA 81:4998–5001PubMedPubMedCentralCrossRef
Zurück zum Zitat Kobayashi Y, Okada K-I (2007) Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann NY Acad Sci 1104:310–323PubMedCrossRef Kobayashi Y, Okada K-I (2007) Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann NY Acad Sci 1104:310–323PubMedCrossRef
Zurück zum Zitat Kobayashi Y, Inoue Y, Yamamoto M, Isa T, Aizawa H (2002) Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J Neurophysiol 88:715–731PubMed Kobayashi Y, Inoue Y, Yamamoto M, Isa T, Aizawa H (2002) Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J Neurophysiol 88:715–731PubMed
Zurück zum Zitat Lak A, Arabzadeh E, Harris JA, Diamond ME (2010) Correlated physiological and perceptual effects of noise in a tactile stimulus. Proc Natl Acad Sci USA 107:7981–7986PubMedPubMedCentralCrossRef Lak A, Arabzadeh E, Harris JA, Diamond ME (2010) Correlated physiological and perceptual effects of noise in a tactile stimulus. Proc Natl Acad Sci USA 107:7981–7986PubMedPubMedCentralCrossRef
Zurück zum Zitat Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef Lak A, Stauffer WR, Schultz W (2014) Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111:2343–2348PubMedPubMedCentralCrossRef
Zurück zum Zitat Lardeux S, Pernaud R, Paleressompoulle D, Baunez C (2009) Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus. J Neurophysiol 102:2526–2537PubMedCrossRef Lardeux S, Pernaud R, Paleressompoulle D, Baunez C (2009) Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus. J Neurophysiol 102:2526–2537PubMedCrossRef
Zurück zum Zitat Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O (2002) Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33:463–473PubMedCrossRef Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O (2002) Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33:463–473PubMedCrossRef
Zurück zum Zitat Ljungberg T, Apicella P, Schultz W (1991) Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 586:337–341CrossRef Ljungberg T, Apicella P, Schultz W (1991) Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 586:337–341CrossRef
Zurück zum Zitat Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMed Ljungberg T, Apicella P, Schultz W (1992) Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145–163PubMed
Zurück zum Zitat Lokwan SJA, Overton PG, Berry MS, Clark S (1999) Stimulation of the pedunculopontine nucleus in the rat produces burst firing in A9 dopaminergic neurons. Neuroscience 92:245–254PubMedCrossRef Lokwan SJA, Overton PG, Berry MS, Clark S (1999) Stimulation of the pedunculopontine nucleus in the rat produces burst firing in A9 dopaminergic neurons. Neuroscience 92:245–254PubMedCrossRef
Zurück zum Zitat Luce RD (1959) Individual choice behavior: a theoretical analysis. Wiley, New York Luce RD (1959) Individual choice behavior: a theoretical analysis. Wiley, New York
Zurück zum Zitat Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef Machina MJ (1987) Choice under uncertainty: problems solved and unsolved. J Econ Perspect 1:121–154CrossRef
Zurück zum Zitat Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctively convey positive and negative motivational signals. Nature 459:837–841PubMedPubMedCentralCrossRef Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctively convey positive and negative motivational signals. Nature 459:837–841PubMedPubMedCentralCrossRef
Zurück zum Zitat Matsumoto M, Takada M (2013) Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79:1011–1024PubMedCrossRef Matsumoto M, Takada M (2013) Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79:1011–1024PubMedCrossRef
Zurück zum Zitat Mena-Segovia J, Winn P, Bolam JP (2008) Cholinergic modulation of midbrain dopaminergic systems. Brtain Res Rev 58:265–271CrossRef Mena-Segovia J, Winn P, Bolam JP (2008) Cholinergic modulation of midbrain dopaminergic systems. Brtain Res Rev 58:265–271CrossRef
Zurück zum Zitat Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMed Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024–1027PubMed
Zurück zum Zitat Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449–451PubMedCrossRef
Zurück zum Zitat Mogami T, Tanaka K (2006) Reward association affects neuronal responses to visual stimuli in macaque TE and perirhinal cortices. J Neurosci 26:6761–6770PubMedCrossRef Mogami T, Tanaka K (2006) Reward association affects neuronal responses to visual stimuli in macaque TE and perirhinal cortices. J Neurosci 26:6761–6770PubMedCrossRef
Zurück zum Zitat Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947PubMed
Zurück zum Zitat Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H (2004) Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43:133–143PubMedCrossRef
Zurück zum Zitat Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057–1063PubMedCrossRef
Zurück zum Zitat Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280PubMedCrossRef
Zurück zum Zitat Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef Nomoto K, Schultz W, Watanabe T, Sakagami M (2010) Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30:10692–10702PubMedPubMedCentralCrossRef
Zurück zum Zitat Norton ABW, Jo YS, Clark EW, Taylor CA, Mizumori SJY (2011) Independent neural coding of reward and movement by pedunculopontine tegmental nucleus neurons in freely navigating rats. Eur J Neurosci 33:1885–1896PubMedPubMedCentralCrossRef Norton ABW, Jo YS, Clark EW, Taylor CA, Mizumori SJY (2011) Independent neural coding of reward and movement by pedunculopontine tegmental nucleus neurons in freely navigating rats. Eur J Neurosci 33:1885–1896PubMedPubMedCentralCrossRef
Zurück zum Zitat Okada K-I, Toyama K, Inoue Y, Isa T, Kobayashi Y (2009) Different pedunculopontine tegmental neurons signal predicted and actual task rewards. J Neurosci 29:4858–4870PubMedCrossRef Okada K-I, Toyama K, Inoue Y, Isa T, Kobayashi Y (2009) Different pedunculopontine tegmental neurons signal predicted and actual task rewards. J Neurosci 29:4858–4870PubMedCrossRef
Zurück zum Zitat O’Neill M, Schultz W (2010) Coding of reward risk distinct from reward value by orbitofrontal neurons. Neuron 68:789–800PubMedCrossRef O’Neill M, Schultz W (2010) Coding of reward risk distinct from reward value by orbitofrontal neurons. Neuron 68:789–800PubMedCrossRef
Zurück zum Zitat Oyama K, Hernádi I, Iijima T, Tsutsui K-I (2010) Reward prediction error coding in dorsal striatal neurons. J Neurosci 30:11447–11457PubMedCrossRef Oyama K, Hernádi I, Iijima T, Tsutsui K-I (2010) Reward prediction error coding in dorsal striatal neurons. J Neurosci 30:11447–11457PubMedCrossRef
Zurück zum Zitat Pan W-X, Hyland BI (2005) Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J Neurosci 25:4725–4732PubMedCrossRef Pan W-X, Hyland BI (2005) Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J Neurosci 25:4725–4732PubMedCrossRef
Zurück zum Zitat Pan W-X, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef Pan W-X, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242PubMedCrossRef
Zurück zum Zitat Pan X, Fan H, Sawa K, Tsuda I, Tsukada M, Sakagami M (2014) Reward inference by primate prefrontal and striatal neurons. J Neurosci 34:1380–1396PubMedPubMedCentralCrossRef Pan X, Fan H, Sawa K, Tsuda I, Tsukada M, Sakagami M (2014) Reward inference by primate prefrontal and striatal neurons. J Neurosci 34:1380–1396PubMedPubMedCentralCrossRef
Zurück zum Zitat Pasquereau B, Nadjar A, Arkadir D, Bezard E, Goillandeau M, Bioulac B, Gross CE, Boraud T (2007) Shaping of motor responses by incentive values through the basal ganglia. J Neurosci 27:1176–1183PubMedCrossRef Pasquereau B, Nadjar A, Arkadir D, Bezard E, Goillandeau M, Bioulac B, Gross CE, Boraud T (2007) Shaping of motor responses by incentive values through the basal ganglia. J Neurosci 27:1176–1183PubMedCrossRef
Zurück zum Zitat Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striaum. Nature 433:873–876PubMedCrossRef Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striaum. Nature 433:873–876PubMedCrossRef
Zurück zum Zitat Paton JJ, Belova MA, Morrison SE, Salzman CD (2006) The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439:865–870PubMedPubMedCentralCrossRef Paton JJ, Belova MA, Morrison SE, Salzman CD (2006) The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439:865–870PubMedPubMedCentralCrossRef
Zurück zum Zitat Pearce JM, Hall G (1980) A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552PubMedCrossRef Pearce JM, Hall G (1980) A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532–552PubMedCrossRef
Zurück zum Zitat Pooresmaeili A, Poort J, Roelfsema PR (2014) Simultaneous selection by object-based attention in visual and frontal cortex. Proc Natl Acad Sci USA 111:6467–6472PubMedPubMedCentralCrossRef Pooresmaeili A, Poort J, Roelfsema PR (2014) Simultaneous selection by object-based attention in visual and frontal cortex. Proc Natl Acad Sci USA 111:6467–6472PubMedPubMedCentralCrossRef
Zurück zum Zitat Ravel S, Legallet E, Apicella P (1999) Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli. Exp Brain Res 128:531–534 Ravel S, Legallet E, Apicella P (1999) Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli. Exp Brain Res 128:531–534
Zurück zum Zitat Redgrave P, Prescott TJ, Gurney K (1999) Is the short-latency dopamine response too short to signal reward? Trends Neurosci 22:146–151PubMedCrossRef Redgrave P, Prescott TJ, Gurney K (1999) Is the short-latency dopamine response too short to signal reward? Trends Neurosci 22:146–151PubMedCrossRef
Zurück zum Zitat Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton Century Crofts, New York, pp 64–99 Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton Century Crofts, New York, pp 64–99
Zurück zum Zitat Ringach DL, Hawken MJ, Shapley R (1997) Dynamics of orientation tuning in macaque primary visual cortex. Nature 387:281–284PubMedCrossRef Ringach DL, Hawken MJ, Shapley R (1997) Dynamics of orientation tuning in macaque primary visual cortex. Nature 387:281–284PubMedCrossRef
Zurück zum Zitat Roelfsema PR, Tolboom M, Khayat PS (2007) Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron 56:785–792PubMedCrossRef Roelfsema PR, Tolboom M, Khayat PS (2007) Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron 56:785–792PubMedCrossRef
Zurück zum Zitat Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365–13376PubMedPubMedCentralCrossRef Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G (2009) Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29:13365–13376PubMedPubMedCentralCrossRef
Zurück zum Zitat Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489PubMed Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22:9475–9489PubMed
Zurück zum Zitat Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMed Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63:592–606PubMed
Zurück zum Zitat Rothschild M, Stiglitz JE (1970) Increasing risk: I. A definition. J Econ Theory 2:225–243CrossRef Rothschild M, Stiglitz JE (1970) Increasing risk: I. A definition. J Econ Theory 2:225–243CrossRef
Zurück zum Zitat Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337–1340PubMedCrossRef Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337–1340PubMedCrossRef
Zurück zum Zitat Satoh T, Nakai S, Sato T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913–9923PubMed Satoh T, Nakai S, Sato T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913–9923PubMed
Zurück zum Zitat Scarnati E, Campana E, Pacitti C (1984) Pedunculopontine-evoked excitation of substantia nigra neurons in the rat. Brain Res 304:351–361PubMedCrossRef Scarnati E, Campana E, Pacitti C (1984) Pedunculopontine-evoked excitation of substantia nigra neurons in the rat. Brain Res 304:351–361PubMedCrossRef
Zurück zum Zitat Scarnati E, Proia A, Campana E, Pacitti C (1986) A microiontophoretic study on the nature of the putative synaptic neurotransmitter involved in the pedunculopontine-substantia nigra pars compacta excitatory pathway of the rat. Exp Brain Res 62:470–478PubMedCrossRef Scarnati E, Proia A, Campana E, Pacitti C (1986) A microiontophoretic study on the nature of the putative synaptic neurotransmitter involved in the pedunculopontine-substantia nigra pars compacta excitatory pathway of the rat. Exp Brain Res 62:470–478PubMedCrossRef
Zurück zum Zitat Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1462PubMed Schultz W (1986) Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56:1439–1462PubMed
Zurück zum Zitat Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27PubMed Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27PubMed
Zurück zum Zitat Schultz W (2007) Multiple dopamine functions at different time courses. Ann Rev Neurosci 30:259–288PubMedCrossRef Schultz W (2007) Multiple dopamine functions at different time courses. Ann Rev Neurosci 30:259–288PubMedCrossRef
Zurück zum Zitat Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMed Schultz W, Romo R (1990) Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63:607–624PubMed
Zurück zum Zitat Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef Schultz W, Ruffieux A, Aebischer P (1983) The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp Brain Res 51:377–387CrossRef
Zurück zum Zitat Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610PubMed Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12:4595–4610PubMed
Zurück zum Zitat Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13:900–913PubMed
Zurück zum Zitat Schultz W, Dayan P, Montague RR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef Schultz W, Dayan P, Montague RR (1997) A neural substrate of prediction and reward. Science 275:1593–1599PubMedCrossRef
Zurück zum Zitat Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. J Neurophysiol 86:1916–1936PubMed Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. J Neurophysiol 86:1916–1936PubMed
Zurück zum Zitat Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18:2613–2625PubMed Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18:2613–2625PubMed
Zurück zum Zitat Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G (2012) Reward prediction error signaling in posterior dorsomedial striatum is action specific. J Neurosci 32:10296–10305PubMedPubMedCentralCrossRef Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G (2012) Reward prediction error signaling in posterior dorsomedial striatum is action specific. J Neurosci 32:10296–10305PubMedPubMedCentralCrossRef
Zurück zum Zitat Stanisor L, van der Togt C, Cyriel MA, Pennartz CMA, Roelfsema PR (2013) A unified selection signal for attention and reward in primary visual cortex. Proc Natl Acad Sci USA 110:9136–9141 Stanisor L, van der Togt C, Cyriel MA, Pennartz CMA, Roelfsema PR (2013) A unified selection signal for attention and reward in primary visual cortex. Proc Natl Acad Sci USA 110:9136–9141
Zurück zum Zitat Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966–973PubMedPubMedCentralCrossRef
Zurück zum Zitat Steinfels GF, Heym J, Strecker RE, Jacobs BL (1983) Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res 258:217–228PubMedCrossRef Steinfels GF, Heym J, Strecker RE, Jacobs BL (1983) Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res 258:217–228PubMedCrossRef
Zurück zum Zitat Sugam JA, Day JJ, Wightman RM, Carelli RM (2012) Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiat 71:199–2015PubMedCrossRef Sugam JA, Day JJ, Wightman RM, Carelli RM (2012) Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiat 71:199–2015PubMedCrossRef
Zurück zum Zitat Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170PubMedCrossRef Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170PubMedCrossRef
Zurück zum Zitat Sutton RS, Barto AG (1998) Reinforcement learning. MIT Press, Cambridge Sutton RS, Barto AG (1998) Reinforcement learning. MIT Press, Cambridge
Zurück zum Zitat Tai L-H, Lee AM, Benavidez N, Bonci A, Wilbrecht L (2012) Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat Neurosci 15:1281–1289PubMedPubMedCentralCrossRef Tai L-H, Lee AM, Benavidez N, Bonci A, Wilbrecht L (2012) Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat Neurosci 15:1281–1289PubMedPubMedCentralCrossRef
Zurück zum Zitat Takikawa Y, Kawagoe R, Hikosaka O (2004) A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward maping. J Neurophysiol 92:2520–2529PubMedCrossRef Takikawa Y, Kawagoe R, Hikosaka O (2004) A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward maping. J Neurophysiol 92:2520–2529PubMedCrossRef
Zurück zum Zitat Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76:4040–4055PubMed Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76:4040–4055PubMed
Zurück zum Zitat Thompson KG, Bichot NP, Sato TR (2005) Frontal eye field activity before visual search errors reveals the integration of bottom-up and top-down salience. J Neurophysiol 93:337–351PubMedCrossRef Thompson KG, Bichot NP, Sato TR (2005) Frontal eye field activity before visual search errors reveals the integration of bottom-up and top-down salience. J Neurophysiol 93:337–351PubMedCrossRef
Zurück zum Zitat Tobler PN, Dickinson A, Schultz W (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23:10402–10410PubMed Tobler PN, Dickinson A, Schultz W (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23:10402–10410PubMed
Zurück zum Zitat Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645PubMedCrossRef
Zurück zum Zitat Tremblay L, Hollerman JR, Schultz W (1998) Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80:964–977PubMed Tremblay L, Hollerman JR, Schultz W (1998) Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80:964–977PubMed
Zurück zum Zitat Tsai H-C, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef Tsai H-C, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324:1080–1084PubMedPubMedCentralCrossRef
Zurück zum Zitat von Neumann J, Morgenstern O (1944) The theory of games and economic behavior. Princeton University Press, Princeton von Neumann J, Morgenstern O (1944) The theory of games and economic behavior. Princeton University Press, Princeton
Zurück zum Zitat Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48PubMedCrossRef
Zurück zum Zitat Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858–873PubMedCrossRef
Zurück zum Zitat Williams ZM, Eskandar EN (2006) Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat Neurosci 4:562–568CrossRef Williams ZM, Eskandar EN (2006) Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat Neurosci 4:562–568CrossRef
Zurück zum Zitat Yasuda M, Yamamoto S, Hikosaka O (2012) Robust representation of stable object values in the oculomotor basal ganglia. J Neurosci 32:16917–16932PubMedPubMedCentralCrossRef Yasuda M, Yamamoto S, Hikosaka O (2012) Robust representation of stable object values in the oculomotor basal ganglia. J Neurosci 32:16917–16932PubMedPubMedCentralCrossRef
Zurück zum Zitat Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19:181–189PubMedCrossRef Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19:181–189PubMedCrossRef
Zurück zum Zitat Yin HH, Ostlund SB, Knowlton BJ, Balleine BB (2005) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 23:513–523CrossRef Yin HH, Ostlund SB, Knowlton BJ, Balleine BB (2005) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 23:513–523CrossRef
Zurück zum Zitat Young AMJ (2004) Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Meth 138:57–63CrossRef Young AMJ (2004) Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Meth 138:57–63CrossRef
Zurück zum Zitat Young AMJ, Joseph MH, Gray JA (1992) Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: a microdialysis study. Neuroscience 48:871–876PubMedCrossRef Young AMJ, Joseph MH, Gray JA (1992) Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: a microdialysis study. Neuroscience 48:871–876PubMedCrossRef
Metadaten
Titel
Reward functions of the basal ganglia
verfasst von
Wolfram Schultz
Publikationsdatum
02.02.2016
Verlag
Springer Vienna
Erschienen in
Journal of Neural Transmission / Ausgabe 7/2016
Print ISSN: 0300-9564
Elektronische ISSN: 1435-1463
DOI
https://doi.org/10.1007/s00702-016-1510-0

Weitere Artikel der Ausgabe 7/2016

Journal of Neural Transmission 7/2016 Zur Ausgabe

Neurology and Preclinical Neurological Studies - Review Article

The rationale for deep brain stimulation in Alzheimer’s disease

Neurology and Preclinical Neurological Studies - Original Article

Understanding the human pedunculopontine nucleus in Parkinson’s disease

Neurology and Preclinical Neurological Studies - Review Article

DBS in Tourette syndrome: where are we standing now?

Leitlinien kompakt für die Neurologie

Mit medbee Pocketcards sicher entscheiden.

Seit 2022 gehört die medbee GmbH zum Springer Medizin Verlag

Niedriger diastolischer Blutdruck erhöht Risiko für schwere kardiovaskuläre Komplikationen

25.04.2024 Hypotonie Nachrichten

Wenn unter einer medikamentösen Hochdrucktherapie der diastolische Blutdruck in den Keller geht, steigt das Risiko für schwere kardiovaskuläre Ereignisse: Darauf deutet eine Sekundäranalyse der SPRINT-Studie hin.

Frühe Alzheimertherapie lohnt sich

25.04.2024 AAN-Jahrestagung 2024 Nachrichten

Ist die Tau-Last noch gering, scheint der Vorteil von Lecanemab besonders groß zu sein. Und beginnen Erkrankte verzögert mit der Behandlung, erreichen sie nicht mehr die kognitive Leistung wie bei einem früheren Start. Darauf deuten neue Analysen der Phase-3-Studie Clarity AD.

Viel Bewegung in der Parkinsonforschung

25.04.2024 Parkinson-Krankheit Nachrichten

Neue arznei- und zellbasierte Ansätze, Frühdiagnose mit Bewegungssensoren, Rückenmarkstimulation gegen Gehblockaden – in der Parkinsonforschung tut sich einiges. Auf dem Deutschen Parkinsonkongress ging es auch viel um technische Innovationen.

Demenzkranke durch Antipsychotika vielfach gefährdet

23.04.2024 Demenz Nachrichten

Wenn Demenzkranke aufgrund von Symptomen wie Agitation oder Aggressivität mit Antipsychotika behandelt werden, sind damit offenbar noch mehr Risiken verbunden als bislang angenommen.

Update Neurologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.