Interplay of prefrontal cortex and amygdala during extinction of drug seeking

The mPFC is a critical region for both the acquisition and the extinction of drug-seeking behaviour (Peters et al. 2009). In particular, it seems that two different subregions of the mPFC are involved in different phases of learning: the activity of PL is established to be crucial for the acquiring and maintenance of drug seeking (Di Pietro et al. 2006; Erb et al. 2000; Laughlin et al. 2011), whereas the activity of IL is necessary for the extinction process (Peters et al. 2008).

A number of findings show that the role of NA in PL and IL mirrors the role of those two structures. Depletion of NA from the mPFC (especially PL) impairs the acquisition of CPP or conditioned place aversion (CPA) (Ventura et al. 2003, 2005, 2008), stimulation of \(\beta\)-adrenergic receptors in PL facilitates the retrieval of cocaine-associated memories (Otis et al. 2013). Instead, the depletion of NA from PL during the extinction phase accelerates the extinction process (Latagliata et al. 2016), depletion of NA from IL during the extinction phase completely impairs the extinction (Latagliata et al. 2016) and activation of \(\beta\)-adrenergic receptors in IL accelerates the extinction (LaLumiere et al. 2010).

Inactivation studies (Do-Monte et al. 2015; Sierra-Mercado et al. 2011) show that the activity of IL is only crucial for learning during extinction, but it is not necessary for the expression of conditioned behaviour. These findings are also consistent with another finding on NA: the activity of LC is shown to be very high only during the first phase of extinction (Bouret and Sara 2004). This suggests that NA is very important for learning during extinction, similar to how it is important for the conditioning of highly motivational salient stimuli (Ventura et al. 2008), but it may not be necessary for the expression of extinction and conditioned behaviour.

This statement leads to another important question: what region is actually needed for the expression of extinction and conditioned behaviour? How does it interact with the mPFC? A strong candidate for this role is the Amg, a crucial region for Pavlovian conditioning (LeDoux 2007; Mirolli et al. 2010). A subregion of the Amg, the central amygdala nucleus (CEA), is indeed known to be able to trigger drug seeking (Everitt et al. 2003) and the intercalated cell mass (ITC), a small GABAergic nucleus in the Amg, seems to play a crucial role in the extinction of drug seeking through its GABAergic projections toward CEA (Likhtik et al. 2008). ITC can be divided into a dorsal (ITCd) and a ventral (ITCv) subregion (Pare and Duvarci 2012) and receives substantial glutamatergic projections from the mPFC (Vertes 2004). In addition, ITC shows glutamate-dependent plasticity, and is a strong candidate as a site of learning for extinction memory (Royer and Pare 2002). In addition, a study using fear conditioning shows that the basal amygdala (BA) can be subdivided into two populations, one active when the animal is exposed to a CS (BAf) and the other when the animal is exposed to an extinguished CS (BAe) (Herry et al. 2008). We propose that ITC and BA work together in an interconnected network to trigger a conditioned behaviour or prevent its performance after extinction.

NA in the mPFC, acting on both \(\alpha\)-1 and \(\beta\) adrenergic receptors, has an excitatory effect. In particular, it was shown that NA can cause facilitation of glutamate-evoked discharge when acting on \(\alpha\)-1 receptors (Devilbiss and Waterhouse 2000) and enhance the excitability of the mPFC to external glutamatergic inputs (Luo et al. 2014). In addition, it was recently shown that NA acting on both \(\alpha\) and \(\beta\) receptors triggers LTP (Maity et al. 2015).

In our model, the mPFC triggers plasticity events inside the Amg thanks to its projection to the basolateral amygdala (BLA) and ITC. Whether this process leads to conditioning or to extinction depends on which region of the mPFC is involved. If a glutamatergic input (in particular an input concerning the presence of an unconditioned stimulus—US) arrives to PL, it causes conditioning. Instead, if a glutamatergic input (i.e. an input concerning the absence of the US) arrives to IL it causes extinction. After conditioning or extinction, the activity of the LC and the mPFC is no more necessary to express or prevent behaviour because the Amg has already learned.

It is worth noting that most of the data on PL and IL cited here come from rodent works. The PL/IL distinction is not present as clearly in primates. Indeed, both afferent and efferent projections suggest that PL might be functionally homologous to the dorsolateral prefrontal cortex and IL to the orbitomedial cortex of primates (Vertes 2004). In addition, the structure of intercalated masses also presents some differences in primates: they are not organised into clusters but form a continuous net that is extended throughout the antero-posterior axis of the amygdala (Zikopoulos et al. 2016). Moreover, their inner circuit and function has yet to be clarified.

The architecture of the model is shown in Fig. 1. It is formed by integrate-and-fire neurons to represent neural populations of the Amg and of the mPFC. The Amg is composed of four different regions: lateral amygdala (LA), basal amygdala (BA), ITC and CEA. We consider activity in CEA as the output of our model. We further divided BA into two different populations, BAf and BAe, and ITC in ITCv and ITCd, in accordance with experimental findings (Herry et al. 2008; Pare and Duvarci 2012). We also split LA into two populations, each receiving as input an activation representing a different chamber of the simulated CPP experiment. mPFC is represented by two different regions, PL and IL. External inputs signal the presence of chamber A or chamber B and the occurrence of the US or its non-occurrence (no US). BAf and PL receive an input representing the US, whereas BAe and IL cortex receive as input the no US. These inputs are abstract representations of external signals that arrive from multiple regions. For example, stimuli like the chambers are probably signalled by sensory cortices and the hippocampus, while the US might be signalled by the thalamus and somatosensory cortices (LeDoux 2007). The no US represents an expectation-violation and is produced when the US is missing after the animal had previously learned to expect it (this is not explicitly simulated in the model). The brain regions involved in this process might be the medial frontal cortex for the reward expectation and the anterior cingulate cortex (ACC) for the detection of the erroneous prediction (Alexander and Brown 2011; Silvetti et al. 2011, 2014)

The model also simulates the neuromodulator NA injected by LC to both regions of the mPFC. In the model, NA has an excitatory effect on mPFC populations and also amplifies incoming glutamatergic inputs (Luo et al. 2014; Devilbiss and Waterhouse 2000).

The connections forming the model architecture are grounded in the existing literature and are reported in Table 1. PL has glutamatergic projections to ITCd while IL projects glutamate to BAe and ITCv (Vertes 2004). LA has a glutamatergic projection to both BAf and ITCd, and BAf in turn projects to CEA. BAe projects to ITCv and to GABAergic interneurons in PL. ITCd has a GABAergic projection to ITCv, which, in turn, inhibits CEA through a GABAergic projection (Pare and Duvarci 2012).

In the model, the production of NA was not simulated and NA was rather given to the model as input. Indeed, the production of NA by LC involves complex processes, for example the detection of novel or unexpected or rewarding events (Sara 2009), that go beyond the focus of this work centred on NA effects on target structures.

Plasticity is present in the connections from PL to ITCd, from IL to ITCv, from LA to ITCd, from BAe to ITCv and from LA to BAf. Connections can in particular be either strengthened or weakened based on Hebbian learning. In particular, we use postsynaptic gated Hebbian learning (Gerstner and Kistler 2002) for PL–ITCd and IL–ITCv and presynaptic gated Hebbian learning (Gerstner and Kistler 2002) for LA-ITCd, BAe–ITCv and LA–BAf. This reflects the hypothesis that the connections from PL to ITCd and from IL to ITCv become stronger when the mPFC has an high firing rate, a condition that happens in particular when it is reached by high levels of NA. On the other hand, connections from LA to ITCd and from BAe to ITCv are hypothesised to be strengthened when the ITC neurons are very active: also this condition is initially caused by a high firing rate in mPFC. These mechanisms implement the overall idea of the existence of an important prefrontal control on the Amg plasticity processes. Connections from LA to BAf are strengthened when both areas are strongly active, and this happens only with the contemporary experience of the CS and US. The connections so formed represent the core CS–US association of Pavlovian learning (Mirolli et al. 2010).

We also performed two additional simulations to propose new experiments and predict possible results. First, we removed NA from IL only on the first day of extinction, to mirror the experiment with prazosin in PL. In the second simulation we increased NA input to PL during the extinction phase. In both simulations, the manipulation tended to slow extinction learning (results shown below).

We performed a sensitivity analysis on all parameters of the model both to find its parameters and to check the robustness of the predictions, i.e. to check if they held under different parameter values. To this purpose, we randomly sampled 98 millions randomly generated different combinations of parameter values and evaluated the behaviour of the model under all these settings. We ran all the simulations on the Neuroscience Gateway Portal (Sivagnanam et al. 2013).

The experiments involved in the sensitivity analysis are summarised in Table 3. We used results from previous experiments [i.e. NA depletion studies, Latagliata et al. (2016), and inactivation studies, Do-Monte et al. (2015)] as constraints to define the parameter values considered as acceptable. Thus, each combination of parameters had to replicate the behaviour of rats under the control condition and also replicate the results obtained under the experimental manipulations listed in Table 3 to be considered a plausible set of values (“valid sample”—see also “Materials and Methods”).

Out of 98 millions samples, 103 samples satisfied all the experimental constraints. The small number of these samples indicates that the model has been highly constrained with respect to its degrees of freedom, so its behaviour is reliable.

In the simulation where NA was removed from PL and IL, 98 out of 103 never showed extinction (Fig. 6). Only 5 out of 103 samples extinguished the preference on the same day of the control.

In the simulations where NA was removed from IL only on the first day of extinction, 83 out of 103 samples showed a slower extinction than controls (Fig. 6). Only 20 out of 103 samples extinguished the preference on the same day of the control.

In the simulations where NA was added in PL on the first day of extinction, 96 samples extinguished slower than controls (Fig. 6). Only 7 samples extinguished the preference on the same day of the control.

In conclusion, the majority of samples that fit past experiments data exhibit a slower extinction than controls, both when adding NA in PL and when removing NA from IL during the first day of extinction. On the other hand, when removing NA from both PL and IL the majority of samples never show extinction. The other samples extinguish on the same day of controls, with no samples showing a faster extinction. The sensitivity analysis show that these predictions are robust given the empirical constraints used to find the model parameters and the empirical evidence used to build the model architecture and functioning.

The first prediction (NA depletion from PL and IL) was tested using 6-hydroxydopamine as discussed in the following section.

The second prediction (\(\alpha\)-1 blockade in IL on the first day of extinction slows extinction learning) could be tested in real animals using prazosin, thus mirroring the experiment where prazosin was injected in PL (Latagliata 2014), while the third prediction (the addition of NA in PL on the first day of extinction slows extinction learning) could be performed using an adrenergic agonist.

The apparatus used for the CPP consists of a three-compartment chamber with the outer compartments that have different characteristics. The experiment begins with a pretest habituation phase in which the mouse is free to explore the new environment. Later, during conditioning training, the animal is injected with a drug and is then placed into one of the compartments for several minutes. On the following day, the mouse is injected with the drug vehicle and then placed in the opposite chamber.

The experiment used 13 mice as subjects. In the test phase the animal is placed at the central compartment and is allowed to explore the entire apparatus. The time the animal spends in each compartment is measured and a CPP is considered found if the animal spends significantly more time in the drug-paired compartment (CS) versus the vehicle-paired compartment.

Animals that are subjected to surgery undergo a further test (re-test) to check whether surgery for NE depletion has affected the place preference. The day after the re-test, the extinction procedure begins.

The extinction phase is exactly like the test phase and lasts until the mouse has extinguished for at least two consecutive days. The CPP is considered extinguished when the mouse spends a similar amount of time in both chambers of the apparatus (Cunningham et al. 2006). See “Materials and Methods” for details.

During the pretest phase all mice, randomly assigned to Sham or NA-depleted groups, spent an equal amount of time (mean ± SEM) in the two lateral chambers, thus showing that the apparatus was unbiased in terms of preferences in untreated mice (see Fig. 7). A one-way ANOVA of the pretest revealed a significant effect of the factor choice for Sham [F(2,18) = 22.865, p < 0.01] and NA-depleted mice [F(2,15) = 15.629, p < 0.01]. Post hoc comparisons confirmed that mice spent more time in the lateral chambers than in the centre (centre vs paired p < 0.01, for all groups).

Following the conditioning phase all animals showed a preference for the chamber previously paired with amphetamine. An ANOVA on the CPP test results showed a significant main effect of the factor choice for Sham [F(2,18) = 56.044, p < 0.01] and NA-depleted mice [F(2,15) = 45.989, p < 0.01]. Duncan’s post hoc analyses confirmed that during the test both Sham (p < 0.01) and NA-depleted mice (p < 0.01) spent more time in the amphetamine-paired chamber in comparison with saline-paired chamber (Fig. 7).

NA depletion in mPFC cortex performed the day after CPP test did not affect the expression of amphetamine CPP on the re-test day. Indeed, a one-way ANOVA revealed a main effect of the factor choice for Sham [F(2,18) = 41.572, p < 0.01] and NA-depleted mice [F(2,15) = 55.067, p < 0.01]. Duncan’s post hoc test confirmed that both Sham (p < 0.01) and NA-depleted groups (p < 0.01) spent more time in the amphetamine-paired chamber than in the saline-paired chamber (Fig. 7).

As shown in Fig. 7, NA-depleted mice did not extinguish the preference for the conditioned chamber after 15 days (when the experiment was interrupted), while Sham animals extinguished on days 11–12. On extinction trials 11 and 12, respectively, statistical analyses revealed a significant main effect of the factor choice for Sham animals [having, respectively, F(2, 18) = 63.412, p < 0.01 and F(2, 18) = 51.707, p < 0.01 in the two conditions]. Duncan’s test showed that on both days Sham animals spent an equal amount of time in both lateral chambers (p = ns). On the other hand, NA-depleted mice spent on both days more time in the amphetamine than in the saline-paired chamber (p < 0.05). On day 15, a one-way ANOVA revealed a main effect for the factor choice for NA-depleted mice [F(2,15) = 58.600, p < 0.01]. Duncan’s post hoc test confirmed that NA-depleted mice p < 0.01 still spent more time in the amphetamine-paired chamber than in the saline-paired chamber (Fig. 7).

Our model proposes that the behaviour observed in mPFC NA depletion experiments is caused by enhanced or impaired plasticity events triggered by mPFC in the Amg. In the model, connections from LA to ITCd and from BAe to ITCv have an opposite role on functioning and learning. In particular, LA–ITCd connections are strengthened during the conditioning phase and allow the conditioned response to be expressed thanks to the inhibition of ITCv. On the other hand, BAe-ITCv connections only grow during the extinction phase and allow the ITCv to inhibit the conditioned response triggered by CEA. Importantly, this effect is modulated by NA in the mPFC. Indeed, when NA reaches PL this region is activated, the connection from PL to ITCd is strengthened, and this causes the connection from LA to ITCd to learn as well. The exact same mechanism is proposed for the connections from IL to ITCv and from BAe to ITCv. As a consequence, depletion of NA from PL during extinction training causes an early extinction because the connection from PL to ITCd and from LA to ITCd starts to weaken earlier compared to the control group, an effect caused by PL firing at a lower rate without NA. As a consequence, both connections from IL to ITCv and from BAe to ITCv start strengthening earlier. The final effect is that CEA is inhibited earlier than the control group, therefore, leading to an anticipated extinction.

The opposite happens when NA is depleted from IL. Since IL without NA fires at a low rate, connections from IL to ITCv and from BAe to ITCv are never strengthened and CEA is never inhibited. A double depletion of NA from both regions causes the very same result. Again, connections to the ITCv are never strengthened and despite the connection from LA to ITCd being weakened, the connection from LA to BAf remains stable and this is enough to trigger a conditioned response. This result, confirmed by the behavioural experiment, shows that the effect of a NA depletion from IL cannot be reversed by a NA depletion from PL.

Furthermore, the model shows that if NA is depleted from PL only on the first day of extinction training, the extinction happens before the control, and connections inside the Amg behave on the overall as they do in the condition of complete depletion of NA from PL. This result is consistent with preliminary behavioural results from our laboratory where NA is depleted on the first day with an injection of prazosin in PL (Latagliata 2014).

The sensitivity analysis showed that with other weight dynamics the model can still reproduce all experiments. For example, in 37 out of 103 samples, the weight from LA to ITCd does not ever strengthen, while all other dynamics remain the same. When this happens the activity of ITCd is strictly dependent on the activity of PL: as soon as PL stops firing, ITCd stops inhibiting ITCv and allows the extinction. These samples, while reproducing all experiments correctly, do not capture the idea of the mPFC teaching the Amg to learn new CS–US associations, as stated in the hypothesis. Indeed, the connection from LA to ITCd is proposed by many authors (Pare and Duvarci 2012; John et al. 2013; Busti et al. 2011) to be important to trigger the activation of ITCd, but our finding suggests that PL to ITCd may be an alternative route. A further analysis, for example a disconnection of LA from ITCd, could clarify this issue.

Using the model we also proposed two possible novel experiments where, respectively, NA is removed from IL or NA is added in PL during the first day of extinction. The first experiment could be performed in real animals using prazosin, whereas the second could be performed using an adrenoreceptor agonist. The results, which hold for all parameter sets found by the sensitivity analysis, showed that in both cases the extinction is slowed in, respectively, 83 out of 103 samples and 96 out of 103 samples. In those samples, the model behaves as if the extinction training started 1 day after the control group. In the experiment where NA is removed from IL on the first day, the weights to ITCv start growing 1 day after the control group, while in the experiment were NA is added to PL the weights of connections to ITCd start weakening 1 day after the control group.

For setting the parameters and running the sensitivity analysis we used as constraints the experimental data from both appetitive and aversive conditioning studies. Indeed, neural substrates that underlie both processes seem to overlap, especially regarding the regions that we analysed (Peters et al. 2009). However, it should be noted that in fear-conditioning paradigms both the acquisition and extinction of Pavlovian conditioning are usually much quicker than in appetitive conditioning, likely due to the higher biological valence of survival in an aversive context. Notwithstanding these differences, we reproduced both paradigms using the same set of parameters because our idea is that, apart from the speed of learning, a manipulation in one of the regions of the model would have the same qualitative effect with both appetitive and aversive stimuli (see “Conclusions and future perspectives”).

As reviewed by Moorman et al. (2015) many studies in the literature show that PL and IL have distinct roles in drug seeking as well as in fear conditioning. Many findings show that PL triggers conditioned responses during both the conditioning phase (Burgos-Robles et al. 2009; Schroeder et al. 2001) and reinstatement (Capriles et al. 2003; Di Pietro et al. 2006) and identify two connections that might be especially important in these processes: the one from PL to BLA (Mashhoon et al. 2010) and the one from PL to the core of the nucleus accumbens (Ma et al. 2014). On the other hand, a number of findings implicate IL in the extinction of reward and fear-related behaviours (Peters et al. 2008; LaLumiere et al. 2012; Ovari and Leri 2008; Rhodes and Killcross 2004, 2007) thanks to its projections to ITC (Amano et al. 2010) and to the shell of the nucleus accumbens (Millan et al. 2011).

Based on all these findings, many authors have proposed a simple go/stop model for PL and IL, where PL is thought to be able to trigger a particular conditioned response and IL to cause the extinction of it (Gass and Chandler 2013; Ma et al. 2014; Peters et al. 2009; Van den Oever et al. 2010). However, such a model was criticised by Moorman et al. (2015) and considered an oversimplification because the mPFC is a complex region, involved in different functions and in different networks (Bissonette et al. 2013; Cassaday et al. 2014; Dalley et al. 2004; De Bruin et al. 2000; Kesner and Churchwell 2011; St Onge and Floresco 2010). For example, other studies report that PL inactivation fails to disrupt drug seeking (Weissenborn et al. 1997; Zavala et al. 2003) and present these results as evidence of the inaccuracy of the go/stop model.

Our model proposes a possible solution for those apparent inconsistencies. Indeed, within the model PL is not always necessary to acquire the conditioning while NA in PL is needed to acquire a conditioned response to high salience stimuli (Ventura et al. 2003, 2005, 2008, 2007). The activity of PL is definitely crucial for reinstatement (Capriles et al. 2003). In summary, when there is a complete inactivation of PL, the Amg acquires the CS–US association and is able to trigger a conditioned response. When NA is depleted from PL, this region has a low firing rate and causes the connections towards the ITCd to remain weak, leading to an impairment in the acquisition of the conditioning. On the other hand, when PL receives NA during the conditioning phase, it fires at an high firing rate, triggering plasticity events in the Amg. Those events will also be important in the reinstatement phase. A previous computational model on fear conditioning (Pendyam et al. 2013) proposed a similar hypothesis: in their model, supported by experimental findings, NA acting on \(\beta\)-receptors in PL is crucial for the acquisition of the conditioned response because it triggers a sustained activity in PL that is then able to influence the activity of BLA, leading to fear expression.

The role proposed for IL is similar but specular. NA in this region was shown to be crucial during the extinction phase (Bernardi and Lattal 2010; Latagliata et al. 2016). We propose that this neuromodulator is especially important during early extinction because it triggers synaptic plasticity events that allow the Amg to learn. Indeed, according to Mueller and Cahill (2010) and experimental evidence (Bouret and Sara 2004; Sara and Segal 1991), in late extinction NA release is no longer required as plasticity cascades have already taken place. In addition, IL activity is not necessary for the expression of behaviour, as shown by the findings of Do-Monte et al. (2015) and Sierra-Mercado et al. (2011) where an inactivation of IL after the extinction training does not impair extinction expression.

Regarding the Amg, we integrated ideas from different computational models to build a new architecture that is also constrained by physiological and anatomical findings. A main feature of the architecture is that its behaviour is controlled by the mPFC. In Carrere’s model (Carrere and Alexandre 2015) BA is split into two populations, a “fear population” and an “extinction population”, in accordance with the experimental finding by Herry et al. (2008). Indeed, both Carrere and Herry propose that extinction learning takes place in BA thanks to the extinction population. On the other hand, the model proposed by Moustafa et al. (2013) hypothesises that extinction takes place thanks to the direct connection from IL to ITC, leaving to BA a role only during the acquisition of the conditioned response. The model proposed here reconciles these two views thanks to a double projection from IL to both BA and ITCv. In agreement with a finding by Laurent et al. (2008), that reports that rats with BA lesions can indeed extinguish a fear response but do not show extinction if tested the day after, we propose BA as a site of retention of extinction and IL–ITCv connection as an alternative route that permits a temporary extinction in absence of BA. In line with this interpretation, ITCv is the ITC region that is crucial for extinction thanks to its inhibitory projection to CEA, and its activity can be triggered by IL as shown in Li et al. (2011). On the other hand, ITCd is active during the acquisition of conditioning memories, as shown in a study by Busti et al. (2011) where fear conditioning enhances expression of activity markers in this region, but not in ITCv. ITCd has an inhibitory projection to ITCv (Pare and Duvarci 2012) and is thought to be important for the reinstatement of the conditioned response as well (John et al. 2013). This double inhibition system allows to change how CEA responds to stimuli thanks to inputs coming from the mPFC. Note that in John et al. (2013) model learning rates in the ITC are hypothesised to be faster than those in the BL complex (LA and BA). This allows their model to quickly extinguish the Pavlovian conditioning without losing the CS–US associations and a rapid reinstatement when the US is present again. Although we initially used this idea in the model, the sensitivity analysis showed that in our model there is no need to hypothesise a difference in learning rates between the ITC and the BL complex. Indeed, in our model, regardless of the learning rates, the connection between LA and BA remains strong, with the flexibility of responding depending on ITC learning driven by mPFC as explained above. This happens because during the extinction LA is activated by the CS, it, therefore, activates BAf above threshold and causes the connection to remain strong (see presynaptic Hebbian rule, section 8.3). The conditioned behaviour is, however, not expressed thanks to the inhibition of CEA by ITCv.