The Functions of Dopamine in Motivation, Learning and The Reward System

Since time immemorial, animals have been seeking rewarding stimuli as it results into pleasurable sensations. In the animal kingdom, the search for rewards, such as water, food and sex amongst others, is tightly linked to survival of an individual and of the species as a whole. Stimuli that lead to such actions are intrinsic or unconditioned stimuli (US). These stimuli lead to a naturally occurring pleasurable response, also known as unconditioned response (UR). These stimuli are intrinsically present, while other stimuli have to be learnt first to result into a reward. We call these extrinsic or conditioned stimuli (CS), such as working which is to be rewarded with money (conditioned response or CR). For an organism to form this connection, a CS normally co-occurs with a US during a training phase. Scientifically speaking, reward-seeking induces specific behaviours in an organism. We can thus discern different functions for reward-seeking. Dopamine has been long implicated in motivation, learning and reward-seeking. These behavioural processes are intimately linked to one another and different brain areas have been shown to play a vital role in these processes. To further add to the complexity and confusion of the system, other pathways overlap with the dopaminergic system such as the serotonergic, cholinergic, GABAergic, orexinergic and noradrenergic system.

The interaction of anatomically distinct brain areas in the dopaminergic system give rise to different types of learning stages and motivation. Ventral tegmental area and reward prediction error The ventral tegmental area (VTA) is a key area in the mesocorticolimbic dopamine system. The VTA receives inputs from sensory systems and its dopamine neurons project to several other brain structures involved in motivation and learning such as the nucleus accumbens core and shell and the prefrontal cortex. The VTA shows strong dopaminergic activity during reward-related learning in form of phasic DA impulses. In the pioneer study from Schulz et al. (1997), it was shown that there was a significant increase in the dopamine levels in the VTA of the monkey brain when performing a learning task upon consumption of a reward initially. After the training phase, phasic dopamine impulses were already observed upon presentation of the first environmental cue (appearance of light that indicated to press the lever to obtain a reward), but the initial phasic dopamine impulses diminished upon reward consumption. On the other hand, the knockout of dopamine signalling in the VTA and NAc (knockout of D2 receptors) has been generally shown to be the basis for aversive learning.

These phasic dopamine impulses correlate strongly with reward-seeking behaviour and learning. It is generally said that this DA neuron pulsing encodes a so-called “reward prediction error” (RPE). The RPE is the actual value of a rewarding stimulus minus the expectancy of the value of a rewarding stimulus. Therefore, the RPE can be distinguished in positive RPE, neutral RPE or negative RPE. A positive RPE indicates that the final reward was more pleasurable than initially expected, which generally leads to learning, such that the environmental cues leading to that reward are identified. A positive RPE can also originate from a surprising positive reward. A neutral RPE indicates that the expected pleasurable sensation of a reward was the same as the actual outcome. In a negative RPE, the reward was disappointing so to say, that is, the result was less satisfactory.

A negative RPE induces aversive learning in order to avoid future unpleasant or painful situations. Dorsal dopaminergic neurons in the VTA usually show a decrease in firing to noxious stimuli (foot shock), while an increase in activity can be observed in ventral dopaminergic neurons in the VTA. A foot shock to the hind paw of anaesthetized rats was applied and DA neurons activity in the dorsal and ventral VTA was recorded. In the upper figure, we see an initial decrease in DA neuron firing in the dorsal VTA. A noxious stimulus inhibits that neural activity. In the lower figure, there is a sharp initial increase in DA neuron firing in the dorsal VTA. These neurons are excited by a noxious stimulus. Dopamine is not the only neurotransmitter acting the in the mesolimbic pathway. The mesolimbic dopamine pathway has been shown to be subject to cholinergic modulation and transient receptor potential vanilloid 3 in the VTA.

Dopamine in the Nucleus Accumbens Core and ShellThe Nucleus accumbens (NAc) is part of the mesolimbic pathway and receives inputs from the VTA. It is important to discern that the NAc core and NAc shell compute neural inputs for different kinds of learning. The NAc core also performs RPE computation while the NAc shell is involved in incentive salience. This can be a source of confusion, since it seems that the NAc shell activity usually shows a high activity in incentive salience, while the NAc core is best studied with reward-related theories, though it is also active in incentive salience. Measurements of the Nucleus accumbens core are represented by the bold black line, measurements of the Nucleus accumbens shell are represented by the bold grey line. DA neuron signalling was measured in the Nucleus accumbens core and shell in mice during a lever-press task. Peak values are reached when pressing the first lever (SLp) in core and shell. As the mouse proceeds with the task, the dopamine levels in the core fall quickly to baseline levels (with a smaller increase during the press of the second lever (TL0)), while dopamine signalling in the shell remains constantly high exemplifying its relevance in incentive salience. At the end of the task, dopamine signalling returned to the baseline.

Dopamine in fear learning and extinctionThe amygdala is an indispensable component in fear learning and required in the consolidation of memories associated with fear. It is also part of the mesolimbic pathway, since VTA projects to the amygdala. Proper memory establishment also involves further brain structures such as the hippocampus and the NAc. Fear conditioning is a very fast process where a CS is paired with a shock US. For example, a sound (CS) occurs at the same time as a foot shock (US). The mice will show an aversive response to the foot shock, but initially, the sound will not induce a remarkable behaviour in a mouse. After the training phase, the mouse will have made a connection between the sound and the foot shock (the sound predicts the foot shock), which will be enough to induce a fear response.

The process of fear extinction describes the behavioural change of an animal to a CS, where no fear response is the outcome. This does not mean that the initial fear memory has been deleted, but that the animal has adapted to its environmental situation (the RPE of the CS has become neutral). Much of the current research has been especially focused on D1 and D2 receptors. D1 receptors result into an excitatory response while D2 receptors result into an inhibitory response. Selective knockout of D1 receptors with antagonists have shown that such mice have impaired fear acquisition. Knockout of D2 receptor in mice lead to controversial results. Mueller et al. (2010) report an impairment in fear extinction retention while Ponnusamy et al. (2005) report enhanced fear extinction using D2 receptor antagonism. Differences in results were proposed to be due to the affinities of used antagonists and slight differences in task protocols.

Dopamine in associative learningThe prefrontal cortex has been linked to multitude of higher-order functions, such as attention, planning, evaluation, decision-making, executive functions and many more cognitive functions. Unsurprisingly, it is also a key component in associative learning as learning new associations of the environment increases the organism’s adaptability and survival.

The PFC receives dopaminergic projections from the VTA during a learning task. The PFC dopaminergic neurons show a phasic spiking behaviour, albeit it is a less frequent and slower than in the VTA or NAc core indicating that the PFC dopamine release is involved in RPE. It has been proposed that these dopamine spikes are correlated to looking for more abstract rules regarding the conditions a reward occurred. Furthermore, the PFC-BG loop is only involved in the early stages of learning. As soon as a habit has been formed, the PFC is not involved in the reward-seeking behaviour of an organism. Depletion of dopamine in the orbitofrontal cortex have shown to have no effect in reversal learning (the ability of an organism to learn that a previously CS leads to no positive reward anymore, but to nor reward or an aversive outcome). Reversal learning in the orbitofrontal cortex is heavily influenced by the serotonin, which is why dopamine depletion did not affect reversal learning in any major way. Thus, the role of dopamine in the PFC is not yet fully understood, also due to other overlapping systems.