ReviewIdentifying the neurobiology of altered reinforcement sensitivity in ADHD: A review and research agenda
Introduction
Attention-deficit/hyperactivity disorder (ADHD) is one of the most frequently diagnosed childhood disorders and is characterized by inattention, hyperactivity and impulsivity (American Psychiatric Association, 2000). The biological basis of ADHD is still largely unknown, but there is emerging evidence that cognitive impairments in ADHD are subserved by both structural and functional brain abnormalities (Bush et al., 2005, Paloyelis et al., 2007, Seidman et al., 2005). For many years the focus of cognitive research has been on deficits in executive function (e.g., Barkley, 1997, Pennington and Ozonoff, 1996), especially inhibition. Recently, however, an increasing number of theoretical frameworks have incorporated altered reinforcement sensitivity as an important etiological factor (e.g., Barkley, 1997, Blum et al., 2000, Casey et al., 2007, Castellanos and Tannock, 2002, Douglas, 1989, Frank et al., 2007b, Haenlein and Caul, 1987, Newman and Wallace, 1993, Nigg and Casey, 2005, Quay, 1988, Sagvolden et al., 2005, Sergeant et al., 1999, Sonuga-Barke, 2002, Tripp and Wickens, 2008). The current paper focuses on the overlap and differences between seven theories incorporating aspects of altered reinforcement sensitivity. Predictions made or implied by these theories are discussed in terms of future experimental studies.
Evidence for abnormal reinforcement sensitivity in ADHD comes from research at different levels of analyses. At the behavioral level, the positive effect of reinforcement on cognitive skills is larger for children with ADHD than typically developing children (see Luman et al., 2005 for a review). In addition, individuals with ADHD often show a relatively strong preference for options that are rewarding now, but may be unfavorable in the long term (e.g., Drechsler et al., 2008) and they favor small immediate over larger delayed rewards (Sonuga-Barke et al., 1992). At a psychophysiological level, feedback-related autonomic responses (heart rate and skin conductance) of children with ADHD appear to ‘normalize’ when reward is added to feedback (e.g., Luman et al., 2007), consistent with the improvements in performance which occur when reinforcement is present. At the functional neuroimaging level, children with ADHD demonstrate reduced activity in the ventral striatum when anticipating reward (e.g., Scheres et al., 2007) which offers a possible explanation for the stronger preference for reward immediacy in ADHD compared to typically developing children. In addition, children with ADHD display reduced positive event-related potential (ERP) activity 200 ms following monetary losses (e.g., van Meel et al., 2005), indicative of a compromised categorization of motivationally relevant stimuli.
An increasing number of experimental studies on reinforcement sensitivity are appearing in the literature. However, the findings are not entirely consistent, confirming the complexity of altered reinforcement sensitivity in ADHD. For example, only some studies report disproportional improvement in performance in response to reward in ADHD groups (see Luman et al., 2005). Even the most consistent finding, namely that children with ADHD show a relatively strong preference for small immediate rewards over larger delayed rewards (Sonuga-Barke et al., 1992, Rapport et al., 1986), may depend on a number of contextual factors (Scheres et al., 2006, Scheres et al., in press). Besides the heterogeneity in the experimental findings, there seems to be a large gap between the experimental findings on the one hand and the theoretical models on the other. An analysis of five theoretical frameworks of ADHD (Douglas, 1989, Haenlein and Caul, 1987, Quay, 1988, Sergeant et al., 1999, Sonuga-Barke, 2002) showed that none of the frameworks were able to explain the experimental findings of the 21 experimental studies evaluated (Luman et al., 2005).
Luman et al. suggested that the inability of the theoretical frameworks to account for the experimental results might be due to the domain specificity of the different frameworks. For example, although the frameworks as offered by Haenlein and Caul (1987), Douglas (1989) or Sergeant et al. (1999) have been very influential in explaining reinforcement sensitivity in ADHD from a behavioral point of view, they do not offer predictions about the underlying (neurobiological) mechanisms of these behaviors. Another important issue contributing to the gap between experimental findings and theory is that current models offer relatively few testable experimental predictions. We believe if the models offered theoretically driven, and experimentally testable, predictions, researchers would be encouraged to conduct the studies necessary to test them. An additional issue is the scarcity of systematic studies ‘multi-level studies’ into the (neurobiological) mechanisms of reinforcement sensitivity in ADHD. We believe that a multi-level methodological approach, for example by combining behavioral and neuroimaging studies, will lead to increased understanding of the nature of reinforcement sensitivity in ADHD. Up to now, most studies have focused on behavioral outcomes, with a few exceptions (Plichta et al., 2009, Rubia et al., 2009, Scheres et al., 2007, Ströhle et al., 2008, van Meel et al., 2005).
The goal of the current paper is fourfold. (1) To provide an updated review of current theoretical models of reinforcement sensitivity in ADHD; (2) to identify the predictions regarding behavioral or neurobiological responses to reinforcement stimuli, either made by the model developers, or extracted from the models by the current authors; (3) to identify current research methods that are required to test these predictions; (4) to review the existing experimental evidence in support for these predictions. The key elements of these models and their predictions are summarized in Section 2. This helps identify which experimental methods are needed to test the models, which predictions are unique or shared among the models, and which predictions are supported by the available evidence. The predictions can serve as a useful guide to the systematic evaluation of altered reinforcement sensitivity in ADHD, leading to increased understanding of the phenomenon and its etiology.
Section snippets
Neuropathological models of altered sensitivity to reinforcement in ADHD
Seven models were selected for inclusion in this review, based on three important criteria. (1) The models must account for some of the behaviors associated with altered reinforcement sensitivity in ADHD; (2) the models should be neurobiologically meaningful. ADHD is assumed to have a biological basis (e.g., Castellanos and Tannock, 2002), and there is good evidence that reinforcement sensitivity can be explained by neurobiological processes (e.g., Cools et al., 2007, Knutson et al., 2001,
Research methods
In order to determine which of the proposed underlying mechanisms are responsible for altered reinforcement sensitivity in ADHD, we need to carry out studies that focus on multiple causes and mechanisms (or focus on a singe cause with multiple outcomes). What is required is a combination of technologies and/or research methods that investigate these mechanisms, rather than focusing on outcomes only. This is not an easy job, since testing the neurobiological mechanisms will be particularly
Predictions and available evidence
Table 1 provides an overview of the behavioral and neurobiological predictions from the seven models. It is clear from Table 1 that some of the predictions are shared (agreed upon by more than one model), while others are unique or contradictory (not shared by all models). Where possible, we have indicated whether the various models agree with, disagree with, or do not address each prediction. By comparing predictions with the experimental evidence we can identify: if and how the predictions
Moderating factors and related issues
When testing the predictions using ‘multi-level’ methods, researchers need to be aware of several confounding issues that may impact reinforcement sensitivity in ADHD.
Conclusion and research agenda
The goal of the current paper was to provide an updated review of seven neurobiologically meaningful models on reinforcement sensitivity in ADHD, to identify specific predictions and research methods required to test these predictions, and review the available experimental evidence. Since DA responses to reward are essential for learning processes (e.g., Frank, 2005, Tripp and Wickens, 2008) and motivation (Volkow et al., 2001), insight into reward sensitivity in ADHD can increase the efficacy
Limitations
First, this review focuses on the role of DA in reinforcement mechanisms, while other neurotransmitter systems, such as serotonin and norepinephrine (e.g., Blum et al., 2000) are not considered. Since these systems are described less extensively in relation to ADHD, they were excluded from the current review. Second, models that are not grounded in neuroscience are excluded, although these models have been very influential in explaining behavioral aspects of ADHD (e.g., Douglas, 1989, Haenlein
References (92)
Problems with spontaneously hypertensive rats (SHR) as a model of attention-deficit/hyperactivity disorder (AD/HD)
J. Neurosci. Methods
(2007)- et al.
Functional neuroimaging of attention-deficit/hyperactivity disorder: a review and suggested future directions
Biol. Psychiatry
(2005) - et al.
Atomoxetine increases extracellular levels of norepinephrine and dopamine in prefrontal cortex of rat: a potential mechanism for efficacy in attention deficit/hyperactivity disorder
Neuropsychopharmacology
(2002) - et al.
Dopamine transporter density in patients with attention deficit hyperactivity disorder
Lancet
(1999) Can Skinnerian theory explain attention deficit disorder. A reply to Barkley
- et al.
Alterations in reward-related decision making in boys with recent and future depression
Biol. Psychiatry
(2007) Perspectives on anxiety and impulsivity: a commentary
J. Res. Pers.
(1987)- et al.
Attention deficit disorder with hyperactivity: a specific hypothesis of reward dysfunction
J. Am. Acad. Child Psychiatry
(1987) - et al.
Response acquisition with delayed reinforcement in a rodent model of attention-deficit/hyperactivity disorder (ADHD)
Behav. Brain Res.
(2006) - et al.
Electrophysiological evidence of atypical motivation and reward processing in children with attention-deficit hyperactivity disorder
Neuropsychologia
(2008)
Attention-deficit/hyperactivity disorder (ADHD) behaviour explained by dysfunctioning reinforcement and extinction processes
Behav. Brain Res.
Effects of delayed reinforcers on the behavior of an animal model of attention-deficit/hyperactivity disorder (ADHD)
Behav. Brain Res.
A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related FMRI
Neuroimage
Increased striatal dopamine transporter in adult patients with attention deficit hyperactivity disorder: effects of methylphenidate as measured by single photon emission computed tomography
Neurosci. Lett.
The impact of reinforcement contingencies on AD/HD: a review and theoretical appraisal
Clin. Psychol. Rev.
Is it reward frequency or magnitude that drives reinforcement learning in ADHD
Psychiatry Res.
Diverse pathways to deficient self-regulation—implications for disinhibitory psychopathology in children
Clin. Psychol. Rev.
Neural hyporesponsiveness and hyperresponsiveness during immediate and delayed reward processing in adult attention-deficit/hyperactivity disorder
Biol. Psychiatry
Neurobiology of animal models of attention-deficit hyperactivity disorder
J. Neurosci. Methods
Frequent reward eliminates differences in activity between hyperkinetic rats and controls
Behav. Neural Biol.
Temporal and probabilistic discounting of rewards in children and adolescents: effects of age and ADHD symptoms
Neuropsychologia
Ventral striatal hyporesponsiveness during reward anticipation in attention-deficit/hyperactivity disorder
Biol. Psychiatry
Getting formal with dopamine and reward
Neuron
Methylphenidate elevates resting dopamine which lowers the impulse-triggered release of dopamine: a hypothesis
Behav. Brain Res.
Structural brain imaging of attention-deficit/hyperactivity disorder
Biol. Psychiatry
Psychological heterogeneity in AD/HD—a dual pathway model of behaviour and cognition
Behav. Brain Res.
The dual pathway model of AD/HD: an elaboration of neuro-developmental characteristics
Neurosci. Biobehav. Rev.
Executive dysfunction and delay aversion in attention deficit hyperactivity disorder: nosologic and diagnostic implications
Child Adolesc. Psychiatr. Clin.
In vivo neuroreceptor imaging in attention-deficit/hyperactivity disorder: a focus on the dopamine transporter
Biol. Psychiatry
Reward anticipation and outcomes in adult males with attention-deficit/hyperactivity disorder
Neuroimage
Sensitivity to delay of reinforcement in two animal models of attention deficit hyperactivity disorder (ADHD)
Behav. Brain Res.
Neurobiology of ADHD
Neuropharmacology
Adaptive control deficits in attention-deficit/hyperactivity disorder (ADHD): the role of error processing
Psychiatry Res.
Telling good from bad news: ADHD differentially affects processing of positive and negative feedback during guessing
Neuropsychologia
Effects of methylphenidate on reward strength in boys with attention-deficit/hyperactivity disorder
J. Am. Acad. Child Psychiatry
Infrequent, but not frequent, reinforcers produce more variable responding and deficient sustained attention in young children with attention-deficit/hyperactivity disorder (ADHD)
J. Child Psychol. Psychiatry
Diagnostic and Statistical Manual of Mental Disorders
Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD
Psychol. Bull.
Executive functioning, temporal discounting, and sense of time in adolescents with attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD)
J. Abnorm. Child Psychol.
Reward deficiency syndrome: a biogenetic model for the diagnosis and treatment of impulsive, addictive, and compulsive behaviors
J. Psychoactive Drugs
New potential leads in the biology and treatment of attention deficit-hyperactivity disorder
Curr. Opin. Neurol.
Neuroscience of attention-deficit/hyperactivity disorder: the search for endophenotypes
Nat. Rev. Neurosci.
Serotoninergic regulation of emotional and behavioural control processes
Trends Cogn. Sci.
BOLD responses reflecting dopaminergic signals in the human ventral tegmental area
Science
Effects of reward and non-reward on frustration and attention in attention deficit disorder
J. Abnorm. Child Psychiatry
Decision-making on an explicit risk-taking task in preadolescents with attention-deficit/hyperactivity disorder
J. Neural Transm.
Cited by (226)
Enhancement of visual dominance effects at the response level in children with attention-deficit/hyperactivity disorder
2024, Journal of Experimental Child PsychologyConditioned approach behavior of SHR and SD rats during Pavlovian conditioning
2023, Behavioural Brain ResearchMethylphenidate Enhances Spontaneous Fluctuations in Reward and Cognitive Control Networks in Children With Attention-Deficit/Hyperactivity Disorder
2023, Biological Psychiatry: Cognitive Neuroscience and NeuroimagingReward modality modulates striatal responses to reward anticipation in ADHD: Effects of affiliative and food stimuli
2022, Psychiatry Research - Neuroimaging