The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond (2012)

J Exp Anal Behav. 2012 Jan;97(1):125-46. doi: 10.1901/jeab.2012.97-125.

Salamone JD1, Correa M, Nunes EJ, Randall PA, Pardo M.

Abstract

For many years, it has been suggested that drugs that interfere with dopamine (DA) transmission alter the “rewarding” impact of primary reinforcers such as food. Research and theory related to the functions of mesolimbic DA are undergoing a substantial conceptual restructuring, with the traditional emphasis on hedonia and primary reward yielding to other concepts and lines of inquiry. The present review is focused upon the involvement of nucleus accumbens DA in effort-related choice behavior. Viewed from the framework of behavioral economics, the effects of accumbens DA depletions and antagonism on food-reinforced behavior are highly dependent upon the work requirements of the instrumental task, and DA-depleted rats show a heightened sensitivity to response costs, especially ratio requirements. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related choice behavior. Rats with accumbens DA depletions or antagonism reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and show increased selection of low reinforcement/low cost options. Nucleus accumbens DA and adenosine interact in the regulation of effort-related functions, and other brain structures (anterior cingulate cortex, amygdala, ventral pallidum) also are involved. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as symptoms such as psychomotor slowing, fatigue or anergia in depression and other neurological disorders.

Keywords: dopamine, adenosine, effort, work, reinforcement, behavioral economics, review

In order to survive, organisms must gain access to significant stimuli such as food, water, sex, and other conditions. The processes involved in such behavioral activities are varied and complex, and the brain mechanism related to these processes are a subject of considerable research activity. Instrumental learning processes involving reinforcement and punishment lead to the acquisition of behaviors that regulate the probability, proximity, and availability of significant stimuli. But even when such responses are already acquired, multiple factors contribute to the selection of particular instrumental behaviors in a given environmental context. For example, in a complex environment, organisms typically have access to multiple reinforcers, which can vary with regard to their quality, quantity, and temporal characteristics. In addition, distinct instrumental actions can be associated with particular reinforcers, and these actions can vary widely in topography and in terms of the quantitative features of the response requirements. Several areas of inquiry in behavioral science, including research on response–reinforcement matching, optimal foraging theory, and behavioral economics, have emerged in order to characterize the choice behavior observed in these complex environments (Allison, 1981, 1993; Aparicio, 2001, 2007; Baum, 1974; Hengeveld, van Langevelde, Groen, & de Knegt, 2009; Hursh, Raslear, Shurtleff, Bauman, & Simmons, 1988; Madden, Bickel, & Jacobs, 2000; Madden & Kalman, 2010; Salamone, 1987; Tustin, 1995; Vuchinich and Heather, 2003; Williams, 1988). This research has provided approaches for understanding how reinforcement value, as well as response requirements, influence the relative allocation of instrumental behavior across multiple options.

This perspectives article will provide an overview of recent research on the behavioral pharmacology of a specific aspect of these broader issues. One response-related factor that profoundly influences instrumental behavior is work-related response costs (Foltin 1991; Hursh et al., 1988; Kaufman 1980; Kaufman, Collier, Hill, & Collins, 1980; Madden et al., 2000; Salamone, 1986, 1987, 1992; Staddon 1979; Tustin, 1995). The present review will focus upon the effects of drugs and neurochemical manipulations that affect dopamine (DA) transmission, and how these effects interact with the response requirements, particularly ratio requirements, imposed upon food-reinforced instrumental behavior. In addition, the article will review the literature on the role of DA in effort-related choice behavior, with a particular emphasis upon DA in a brain area known as the nucleus accumbens. Finally, the interactions between nucleus accumbens DA and other neurotransmitters and brain areas will be discussed, and the broader relevance of these findings will be considered.

HYPOTHESIZED ACTIONS OF DA ANTAGONISTS: THE DECLINE AND FALL OF THE “REWARD” HYPOTHESIS OF DA FUNCTION

There have been substantial theoretical developments in the last few years related to the hypothesized behavioral functions of DA, particularly nucleus accumbens DA. In order to consider the involvement of DA in work-related aspects of instrumental response allocation, one should place these ideas into a historical context relative to other hypothesized functions of DA. A few decades ago, it became common in the behavioral neuroscience literature to label DA as a “reward” transmitter, which was said to produce feelings of subjective pleasure or motivational appetites that mediate or drive positive reinforcement phenomena. However, it has become evident to many investigators that there are conceptual limitations and empirical problems with the traditional DA hypothesis of “reward” (Baldo & Kelley, 2007; Barbano & Cador 2007; Salamone, Correa, Farrar, & Mingote, 2007; Salamone, Correa, Farrar, Nunes, & Collins, 2010; Salamone, Correa, Mingote, & Weber, 2005; Salamone, Cousins, & Snyder, 1997; Salamone, et al., 2009), not the least of which is the use of the term “reward” itself (Cannon & Bseikri 2004; Salamone 2006; Salamone et al. 2005; Sanchis-Segura & Spanagel, 2006; Yin, Ostlund, & Balleine, 2008). The term “reward” is rarely defined by researchers when they are using it to describe a behavioral process. Some use the term as though it were a synonym for “reinforcement”, while others use it in reference to “appetite” or “primary motivation”. Still others employ this term as a thinly veiled label for “pleasure”. In many cases, the word “reward” seems to be used as a rather monolithic, all-encompassing term that refers globally to all aspects of reinforcement learning, motivation and emotion, whether conditioned or unconditioned. If used in this manner, the term reward is so broad as to be practically meaningless. It should be evident that it is difficult to test a hypothesis which maintains that a neurotransmitter mediates such an ill-defined set of functions. Thus, it has been suggested that it is advantageous to maintain the distinction between the terms reward and reinforcement; with this usage, reinforcement refers more directly to instrumental learning mechanisms (Sanchis-Segura & Spanagel, 2006; Wise 2004), while reward tends to connote the primary motivational and emotional effects of reinforcing stimuli (Everitt & Robbins, 2005; Salamone et al., 2005, 2007).

In addition to these lexicographical and conceptual issues, there also is a substantial body of empirical evidence that has been accumulated in recent years, which fails to support the various forms of the DA hypothesis of “reward”. One ironic observation is that the processes most directly linked to the use of the term reward (i.e., subjective pleasure, primary motivation) are ones that have been demonstrated to be most problematic in terms of demonstrating the involvement of DA systems (Salamone et al., 2007). For example, the idea that nucleus accumbens DA mediates the subjectively reported pleasure associated with positive reinforcers has been strongly challenged (Berridge, 2007; Berridge & Kringlebach, 2008; Salamone et al., 2007). Interference with accumbens DA transmission does not impair appetitive taste reactivity for sucrose (Berridge, 2007; Berridge & Kringlebach, 2008), which is a frequently used behavioral marker of hedonic reactivity in rodents. Human studies have reported that DA antagonists failed to blunt the subjectively rated euphoria produced by drugs of abuse (Brauer & de Wit, 1997; Gawin, 1986; Haney, Ward, Foltin, & Fischman, 2001; Nann-Vernotica, Donny, Bigelow, & Walsh, 2001; Venugopalan et al., 2011; Wachtel, Ortengren, & de Wit, 2002).

Moreover, the potential role of DA systems in instrumental behavior or learning is not limited to situations involving positive reinforcement. There is considerable evidence that striatal mechanisms in general, and nucleus accumbens DA in particular, also participate in aspects of aversive learning, punishment, and responsiveness to aversive stimuli (Blazquez, Fujii, Kojima, & Graybiel, 2002; Delgado, Li, Schiller, & Phelps, 2008; Faure, Reynolds, Richard, & Berridge, 2008; Martinez, Oliveira, Macedo, Molina, & Brandao, 2008; Munro & Kokkinidis, 1997; Salamone, 1994). Although human imaging studies are used to support the idea that nucleus accumbens mediates subjective pleasure (e.g. Sarchiapone et al., 2006), the situation is much more complicated (Pizzagalli, 2010); indeed, research employing various imaging methods has demonstrated that the human nucleus accumbens also responds to stress, aversion and hyperarousal/irritability (Delgado et al., 2008; Delgado, Jou, & Phelps, 2011; Jensen et al., 2003; Levita et al., 2009; Liberzon et al., 1999; Pavic, 2003; Phan et al., 2004; Pruessner, Champagne, Meaney, & Dagher, 2004). Neurochemical and physiological studies in animals clearly indicate that DA neuron activity is not simply tied to the delivery of primary positive reinforcers. In studies involving food reinforcement in trained animals, increases in DA release were more strongly associated with the instrumental response, or conditioned stimuli signaling reinforcer availability, rather than reinforcement delivery (Roitman, Stuber, Phillips, Wightman, & Carelli, 2004; Segovia, Correa & Salamone, 2011; Sokolowski, Conlan, & Salamone, 1998). Moreover, DA neuron activity and DA release can be activated by a number of different aversive (e.g. footshock, tailshock, tail pinch, restraint stress, aversive conditioned stimuli, aversive drugs, social defeat stress) and appetitive conditions (Anstrom & Woodward 2005; Brischoux, Chakraborty, Brierley, & Ungless, 2009; Broom & Yamamoto 2005; Guarraci & Kapp 1999; Marinelli, Pascucci, Bernardi, Puglisi-Allegra, & Mercuri, 2005; McCullough & Salamone, 1992; McCullough, Sokolowski, & Salamone, 1993; Schultz 2007a, 2007b; Young, 2004). These neurochemical changes are seen across varying time horizons (including tonic, slow phasic and fast phasic changes; Hauber 2010; Roitman et al., 2004; Salamone 1996; Salamone et al. 2007; Schultz 2007a, 2007b; Segovia et al., 2011). Studies of learning indicate that DA systems in general and nucleus accumbens in particular are not only involved in learning related to reinforcement (e.g. Wise, 2004), but also are involved in learning related to punishment (Salamone et al., 2007; Schoenbaum & Setlow, 2003). Thus, it has been suggested that the term “instrumental learning” would be more broadly applicable than “reinforcement learning” for describing the hypothesized role of DA in learning processes (Salamone et al., 2007).

If DA antagonism is actually interfering with the fundamental characteristics of reinforcing stimuli, this prompts one to inquire as to what those characteristics are. Of course, reinforcement refers to behavioral contingencies that act to strengthen a particular behavior; positive reinforcement refers to a process by which a response is followed by the presentation of stimulus that typically is contingent upon that response, and these events are followed by an increase in the probability of the occurrence of that response in the future. However, it is worthwhile to consider what properties enable a stimulus to act as a reinforcer. As is often noted, Skinner did not frequently discuss the critical characteristics of stimuli that allow them to act as reinforcers. Nevertheless, Skinner did, on occasion, consider the role of motivational variables such as food deprivation in the process of reinforcement. For example, Skinner (1953) stated “Reinforcement thus brings behavior under the control of an appropriate deprivation. After we have conditioned a pigeon to stretch its neck by reinforcing with food, the variable which controls the neck-stretching is food deprivation” (p. 149). Many other investigators have offered their own perspectives on this issue, and it has been argued that there are some common characteristics that are evident across different research areas (Salamone & Correa, 2002). A large number of investigators who have written about the fundamental characteristics of reinforcing stimuli have arrived at the conclusion that stimuli that act as positive reinforcers tend to be relatively preferred, or to elicit approach behavior, and that these effects are a fundamental aspect of positive reinforcement. For example, Tapp (1969) stated “At the simplest level, reinforcers have the capacity to direct an organism’s behavior. Those stimuli that are approached are regarded as positively reinforcing” (p. 173). Reinforcers have been described as a commodity that is in demand, or a stimulus that is being approached, self-administered, attained or preserved; they also have been described as activities that are preferred, deprived or in some way being regulated (Dickenson & Balleine, 1994; Hursh et al., 1988; Lea, 1978; Premack, 1959; Staddon & Ettinger, 1989; Timberlake, 1993; Tustin, 1995; see discussion of the “motivational corollary of the empirical law of effect” in Salamone & Correa, 2002). According to the behavioral economic analysis offered by Hursh (1993) “responding is regarded as a secondary dependent variable that is important because it is instrumental in controlling consumption” (p. 166).

For these reasons, it is important to note that low doses of DA antagonists that suppress food-reinforced instrumental behavior typically have been shown to leave behavior directed towards the acquisition and consumption of food (Salamone et al., 1991); these manipulations have little effect on food intake (Fibiger, Carter, & Phillips, 1976; Ikemoto & Panksepp, 1996; Rolls et al., 1974; Rusk & Cooper, 1994; Salamone et al., 1991), discrimination and preference based upon food reinforcement magnitude (Martin-Iverson, Wilke, & Fibiger, 1987; Salamone, Cousins, & Bucher, 1994), and simple approach responses reinforced by food delivery (Salamone 1986). Although it is well known that whole forebrain DA depletions can produce aphagia (i.e., lack of eating), it is DA depletions in sensorimotor and motor-related areas of the lateral or ventrolateral caudate/putamen that have been most conclusively linked to this effect, rather than the nucleus accumbens (Dunnett & Iversen 1982; Salamone, J.D., Mahan, K., & Rogers, S., 1993; Ungerstedt, 1971). In contrast, nucleus accumbens DA depletion and antagonism have been shown repeatedly not to substantially impair food intake (Bakshi & Kelley 1991; Baldo, Sadeghian, Basso, & Kelley, 2002; Kelley, Baldo, Pratt, & Will, 2005; Koob, Riley, Smith, & Robbins, 1978; Salamone, Mahon et al., 1993; Ungerstedt 1971). Moreover, the effects of DA antagonists or accumbens DA depletions on food-reinforced instrumental behavior do not closely resemble the effects of pre-feeding or appetite suppressant drugs (Aberman & Salamone, 1999; Salamone, Arizzi, Sandoval, Cervone, & Aberman, 2002; Salamone et al., 1991; Sink, Vemuri, Olszewska, Makriyannis, & Salamone, 2008). Thus, fundamental aspects of primary reinforcement and motivation to obtain access to the reinforcer remain intact after DA antagonism or accumbens DA depletions.

Although it has been suggested that the “reward-related” actions of low doses of DA antagonists or nucleus accumbens DA depletions should produce effects that closely resemble extinction (e.g. Beninger et al., 1987; Wise, Spindler, de Wit, & Gerberg, 1978), there are several problems with this hypothesis. Even though the within-session declines in responding induced by DA antagonists have been labeled “extinction”, similar effects are seen in the motor symptoms of parkinsonism. Haase & Janssen (1985) observed that the micrographia shown by patients with neuroleptic-induced parkinsonism is characterized by a progressive worsening within a writing session. They stated that “An increasing degree of narrowing of the writing from stanza to stanza is particularly characteristic, and in typical cases the area covered by the writing assumes the shape of an inverted pyramid” (p 43). These authors also reported that the intensity of finger tapping generally decreases within a session in patients with neuroleptic-induced parkinsonism (p. 234). Similarly, parkinsonian patients that are repeatedly compressing their hands show progressively diminishing motor output (Schwab, 1972). In rats, DA antagonists cause within-session increments in response duration (Liao & Fowler, 1990), and within session decrements in lick force (Das & Fowler, 1996) and locomotion (Pitts & Horvitz, 2000). Furthermore, repeated administration of DA antagonists to rats leads to context-specific sensitization of the catalepsy response across sessions (Amtage & Schmidt, 2003). In addition, several studies have directly compared the effects of DA antagonism and extinction, and have identified substantial differences between these conditions (Asin & Fibiger, 1984; Evenden & Robbins, 1983; Faustman & Fowler, 1981, 1982; Feldon & Winer, 1991; Gramling, Fowler, & Collins, 1984; Gramling, Fowler, & Tizzano, 1987; Rick, Horvitz, & Balsam, 2006; Salamone 1986; Salamone, Kurth, McCullough, & Sokolowski, 1995, Salamone, et al., 1997; Spivak & Amit, 1986; Willner, Chawala, Sampson, Sophokleous, & Muscat, 1988; Wirtschafter & Asin, 1985). For example, Evenden & Robbins showed that low doses of α-flupenthixol (0.33–0.66 mg/kg) that reduced response rate did not produce effects that resembled extinction in rats responding on a win–stay/lose–shift task. Rick et al. reported that extinction increased behavioral variability in rats trained on an instrumental task, while low doses of the D1 antagonist SCH 23390 or the D2 antagonist raclopride did not.

Another example from this literature is Salamone (1986), which reported that the effects of 0.1 mg/kg of the DA antagonist haloperidol differed substantially from the effects of extinction in rats responding on a fixed ratio (FR) 20 schedule of reinforcement. Under extinction, rats responded at higher rates at the beginning of the session than rats treated with haloperidol, indicating that haloperidol-treated rats did not show an “extinction burst” (see also Salamone et al., 2005, which showed that rats with accumbens DA depletions actually start out responding more slowly in the beginning of the session, in contrast to the effects of extinction). Moreover, rats exposed to extinction emitted proportionately more ratios that were faster than the previous baseline response rate when compared to haloperidol-treated animals (Salamone, 1986). An additional experiment showed that, in contrast to the effects of 0.1 mg/kg haloperidol on FR 20 responding, a dose four times that size had no effect on the reinforced response of simply being in proximity to the food dish on a fixed interval 30 sec schedule (Salamone, 1986). The lack of effect of DA antagonism on this simple food-reinforced response stands in marked contrast to the effect of extinction, which substantially suppressed the instrumental response. In this same experiment, schedule-induced locomotor activity also was recorded in parallel with the instrumental response of being in proximity to the food dish. As shown in the upper panel of Figure 1, 0.4 mg/kg haloperidol suppressed motor activity induced by scheduled presentation of food but, as shown in the lower panel, haloperidol did not affect the reinforced response. In combination with other studies, these results highlight several important features of the effects of DA antagonism. First, the effects of DA antagonism do not closely resemble the effects of extinction across a broad range of conditions (Salamone et al., 1997). Second, DA antagonism suppressed schedule-induced motor activity; behavioral studies have shown that scheduled delivery of reinforcers can have activating properties (Killeen, 1975; Killeen, Hanson, & Osborne, 1978), and considerable evidence indicates that DA antagonism and accumbens DA depletions can reduce schedule-induced activities (McCullough & Salamone, 1992; Robbins & Everitt, 2007; Robbins & Koob, 1980; Robbins, Roberts, & Koob, 1983; Salamone 1988; Wallace, Singer, Finlay, & Gibson, 1983). Finally, these results were consistent with the growing body of evidence indicating that the effects of DA antagonists on instrumental behavior interact powerfully with the instrumental response requirement, and that some reinforced behaviors are relatively unaffected by DA antagonism (Ettenberg et al., 1981; Mekarski, 1988).

Fig 1  

This figure is re-drawn based upon data from Salamone (1986). Rats were tested in a large activity chamber, and were reinforced with 45 mg food pellets on a FI-30 sec schedule for being on the floor panel in front of the food dish. Locomotor …

THE EFFECTS OF DA ANTAGONISM AND ACCUMBENS DA DEPLETION INTERACT WITH THE INSTRUMENTAL RESPONSE REQUIREMENTS

In parallel with the historical developments described above, from the 1970s to the 1990s, there was an emerging emphasis in the behavioral literature on effort, response costs or constraints, and economic models of operant behavior. Several investigators emphasized how response costs or constraints affected operant response output (Foltin 1991; Kaufman 1980; Kaufman et al. 1980; Staddon 1979; Tustin, 1995). Work requirements, such as the number of lever presses necessary for obtaining food, were shown to act as determinants of instrumental response output and also to affect food consumption (Collier & Jennings, 1969; Johnson & Collier 1987). Behavioral economic models stress how a number of factors, including not only reinforcement value, but also conditions related to the characteristics of the instrumental response, can determine behavioral output (Allison, 1981, 1993; Bickel, Marsch, & Carroll, 2000; Lea, 1978). Hursh et al. (1988) suggested that the price of food reinforcement as a commodity is a cost/benefit ratio expressed as the effort expended per unit of food value consumed.

Several lines of evidence have served to strengthen support for the hypothesis that the effects of interference with DA transmission interact powerfully with the instrumental response requirement. One of the ways of controlling work requirements in an operant schedule is to use various ratio schedules. Caul and Brindle (2001) observed that the effects of the DA antagonist haloperidol on food-reinforced behavior were dependent upon the ratio requirement, with a FR 1 schedule being less sensitive than a progressive ratio. One can deplete accumbens DA by local injections of a neurotoxic substance such as 6-hydroxydopamine, and several studies have used this approach. Aberman and Salamone (1999) employed a range of ratio schedules (FR 1, 4, 16 and 64) to assess the effects of accumbens DA depletions. While FR 1 performance was not affected by DA depletion (see also Ishiwari, Weber, Mingote, Correa, & Salamone, 2004), and FR 4 responding showed only a mild and transient suppression, the FR 16 and FR 64 schedules were much more impaired. This pattern indicated that accumbens DA depletions promoted the induction of ratio strain; that is, rats with accumbens DA depletions were much more sensitive to the size of the ratio requirement. This pattern can be described as reflecting an increase in the elasticity of demand for food reinforcement (Aberman & Salamone 1999; Salamone et al., 1997, 2009). If the ratio requirement is analogous to the price of the commodity (reinforcement pellets), it appears that rats with accumbens DA depletions are more sensitive than control animals to the price of the food reinforcers (Figure 2). Needless to say, rats do not use currency to purchase operant pellets. Instead, it has been suggested that an operant procedure is more of a barter system, in which the rat trades its work (or reductions in leisure) for a commodity (Rachlin, 2003; Tustin, 1995). Thus, rats with accumbens DA depletions are more sensitive than control animals to work-related response costs, and less likely to trade high levels of ratio output for food. In a subsequent experiment, Salamone, Wisniecki, Carlson, and Correa (2001) reported that increased sensitivity to larger ratio requirements in rats with accumbens DA depletions were observed when rats were tested across a broader range of ratio schedules (as high as FR300), even when the overall relation between lever pressing and food delivered per lever press was kept constant (i.e., a unit price of 50: FR 50 for one pellet; FR 100 for two pellets; FR 200 for four pellets; and FR 600 for six pellets). These results showed that both the magnitude and the organization of the ratio requirement appear to be critical determinants of the sensitivity of an operant schedule to the effects of accumbens DA depletions.

Fig 2  

This figure shows the effect of ratio requirement on the number of lever presses emitted and operant pellets consumed in rats with accumbens DA depletions compared to rats in the vehicle control group (based upon data from Aberman & Salamone, …

Additional experiments examined the effects of accumbens DA depletions on tandem schedules, in which a ratio requirement was attached to an interval requirement. This was done in order to ensure that the results by Aberman and Salamone (1999) and Salamone et al. (2001) reflected the influence of ratio size, as opposed to other variables such as time. Research employing tandem variable-interval (VI)/FR schedules with varying combinations (e.g. VI 30 sec/FR5, VI 60 sec/FR10, VI 120 sec/FR10) has yielded a consistent pattern; accumbens DA depletions did not suppress overall response output in rats responding on the conventional VI schedules (i.e., those requiring only one response after the interval), but did substantially reduce responding on the corresponding VI schedule with the higher ratio requirement attached (Correa, Carlson, Wisniecki, & Salamone, 2002; Mingote, Weber, Ishiwari, Correa, & Salamone, 2005). These findings are consistent with research showing that accumbens DA antagonism did not impair performance on a progressive interval task (Wakabayashi, Fields, & Nicola, 2004), and that accumbens DA depletions did not affect delay discounting (Winstanley, Theobald, Dalley, & Robbins, 2005). In addition, the DA antagonist haloperidol has been shown to increase the number of reinforced responses in rats responding on a DRL 72-sec schedule (Paterson, Balci, Campbell, Olivier, & Hanania, 2010). These results suggest that interval requirements per se do not pose a severe constraint to rats with compromised DA transmission in nucleus accumbens. Over and above any effect of intermittence or time, ratio requirements provide a work-related challenge that is very disruptive to rats with accumbens DA depletions or antagonism.

In summary, nucleus accumbens DA depletions appear to have two major effects on ratio responding: 1) they reduce the response-enhancing effects that moderate-size ratio requirements have on operant responding (i.e., the ascending limb of the inverted u-shaped function relating ratio requirement to response output), and 2) they enhance the response-suppressing effects that very large ratios have on operant responding (i.e., the descending limb of the function; enhancement of ratio strain; Salamone & Correa 2002; Salamone et al., 2001, 2007, 2009). Another important factor to consider when discussing drug effects is that the baseline rate generated the schedule of reinforcement (Barrett & Bergman, 2008; Dews, 1976; McMillan & Katz, 2002). Although baseline response rate was not a critical factor for inducing ratio strain in the Salamone et al. (2001) experiment, reductions in response rate seen across several schedules of reinforcement (various fixed and progressive ratio, FI 30 sec, VI 30 sec, and tandem VI/ FR schedules) that are produced by accumbens DA depletions do appear to be related to baseline response rate. Across these schedules, there is a linear relation between baseline rate of responding under control conditions and the degree of suppression produced by accumbens DA depletions, with the deficit being greater for schedules that generate increased response rates (Figure 3). Furthermore, molecular behavioral analyses indicate that accumbens DA depletions produce a slight reduction in the local rate of responding, as indicated by the distribution of interresponse times (Mingote et al., 2005; Salamone, Kurth, McCullough, Sokolowski, & Cousins, 1993; Salamone, Aberman, Sokolowski, & Cousins, 1999), as well as an increase in pausing (Mingote et al., 2005; Salamone, Kurth, et al., 1993; see also Nicola, 2010). Computational approaches have been used to characterize these effects of accumbens DA depletions on response rate on ratio schedules (e.g. Niv, Daw, Joel, & Dayan, 2007; Phillips, Walton, & Jhou, 2007). Phillips et al. suggested that DA release in nucleus accumbens appears to provide a window of opportunistic drive during which the threshold cost expenditure to obtain the reward is decreased.

Fig 3  

Scatterplot showing relation between baseline or control rates of responding on various interval and ratio schedules of reinforcement versus the magnitude of the suppression of response rate produced by accumbens DA depletions (expressed as mean percent …

In the context of this discussion of the effects of dopaminergic drugs on ratio performance, it is useful to consider the term “reinforcement efficacy”, which is sometimes used to describe the effects of drug manipulations on ratio performance. With progressive ratio schedules, the ratio requirement increases as successive ratios are completed, and the “break point” is said to occur at the point at which the animal stops responding. One can operationally define reinforcement efficacy in terms of the break point in a progressive ratio schedule, or by measuring ratio strain in rats responding across different FR schedules. The determination of reinforcement efficacy can be a very useful tool for characterizing the actions of drugs that are self-administered, and for comparing self-administration behavior across different substances or drug classes (e.g., Marinelli et al. 1998; Morgan, Brebner, Lynch, & Roberts, 2002; Ward, Morgan, & Roberts, 2005; Woolverton & Rinaldi, 2002). Nevertheless, given the terminological difficulties discussed above, it is useful to stress that the term “reinforcement efficacy” should not be used simply as a replacement for “reward”, and that progressive ratio breakpoints should not be viewed as necessarily providing some direct and unambiguous measure related to the subjective pleasure produced by the stimulus (Salamone, 2006; Salamone et al., 1997; 2009). Drug-induced changes in progressive ratio break points can reflect actions on several different behavioral and neurochemical processes (Arnold & Roberts, 1997; Bickel et al., 2000; Hamill, Trevitt, Nowend, Carlson, & Salamone, 1999; Killeen, 1995; Lack, Jones, & Roberts, 2008; Madden, Smethells, Ewan, & Hursh, 2007; Mobini, Chiang, Ho, Bradshaw, & Szabadi, 2000). For example, changing the response requirements by increasing the height of the lever decreased progressive ratio break points (Schmelzeis & Mittleman 1996; Skjoldager, Pierre, & Mittlman, 1993). Although some researchers have maintained that the break point provides a direct measure of the appetitive motivational characteristics of a stimulus, it is, as stated in a landmark review by Stewart (1975), more directly a measure of how much work the organism will do in order to obtain that stimulus. The animal is making a cost/benefit choice about whether or not to continue responding, based partly on factors related to the reinforcer itself, but also upon the work-related response costs and time constraints imposed by the ratio schedule. For these reasons, interpretations of the actions of drugs or lesions on progressive ratio break points should be done with caution, as should be the case for any individual task. A drug that alters the break point could do so for many different reasons. Mobini et al., (2000) analyzed the effects of several drugs on progressive ratio responding using the quantitative methods developed by Killeen (1994), who suggested that schedule performance is due to interactions between multiple variables (specific activation, coupling, and response time). Mobini et al. reported that haloperidol affected both the response time parameter, and also decreased the activation parameter, while clozapine increased the activation parameter. Recent studies have shown that the DA antagonist haloperidol can suppress food-reinforced progressive ratio responding, and lower break points, but nevertheless leave intact the consumption of a concurrently available but less preferred food source (Pardo et al., 2011; Randall, Pardo, et al., 2011). These actions of haloperidol on this task differed markedly from those produced by prefeeding and appetite suppressant drugs (Pardo et al., 2011; Randall, Pardo, et al., 2011).

DA ANTAGONISM AND NUCLEUS ACCUMBENS DA DEPLETIONS AFFECT THE RELATIVE ALLOCATION OF INSTRUMENTAL RESPONDING IN EFFORT-RELATED CHOICE TASKS

As noted above, animals must make choices in complex environments that present multiple opportunities for obtaining significant stimuli, and several paths for accessing them (Aparicio, 2001, 2007; Williams, 1988). The variables that influence these choices are complex and multidimensional, and they include not only reinforcement value, but also response-related factors. Among the most important are those factors involving cost/benefit interactions based upon effort and reinforcement value (Hursh et al., 1988; Neill & Justice, 1981; Salamone, 2010a; Salamone & Correa 2002; Salamone, Correa, Mingote, & Weber, 2003; Salamone et al., 2005, 2007; Van den Bos, van der Harst, Jonkman, Schilders, & Spruijt, 2006; Walton, Kennerley, Bannerman, Phillips, & Rushworth, 2006). Considerable evidence indicates that low systemic doses of DA antagonists, as well as local disruption of nucleus accumbens DA transmission, affect the relative allocation of behavior in animals responding on tasks that assess effort-based choice behavior (Floresco, St. Onge, Ghods-Sharifi, & Winstanley, 2008; Floresco, Tse, & Ghods-Sharifi, 2008b; Hauber & Sommer 2009; Salamone et al. 2003, 2005, 2007).

One task that has been used to assess the effects of dopaminergic manipulations on response allocation is a procedure that offers rats the option of lever pressing reinforced by delivery of a relatively preferred food (e.g. Bioserve pellets; usually obtained on a FR 5 schedule), or approaching and consuming a less preferred food (lab chow) that is concurrently available in the chamber (Salamone et al., 1991). Trained rats under baseline or control conditions get most of their food by lever pressing, and consume only small quantities of chow. Low-to-moderate doses of DA antagonists, which block either D1 or D2 family receptor subtypes (cis-flupenthixol, haloperidol, raclopride, eticlopride, SCH 23390, SKF83566, ecopipam), produce a substantial alteration of response allocation in rats performing on this task; they decrease food-reinforced lever pressing but substantially increase intake of the concurrently available chow (Cousins., Wei, & Salamone, 1994; Koch Schmid, & Scnhnitzler, 2000; Salamone et al., 2002; Salamone, Cousins, Maio, Champion, Turski, & Kovach, 1996; Salamone et al., 1991; Sink et al. 2008; Worden et al. 2009).

The use of this task for assessing effort-related choice behavior has been validated in many ways. Doses of DA antagonists that produce the shift from lever pressing to chow intake do not affect total food intake or alter preference between these two specific foods in free-feeding choice tests (Koch et al., 2000; Salamone et al., 1991). In contrast, appetite suppressants from different classes, including amphetamine (Cousins et al., 1994), fenfluramine (Salamone et al., 2002) and cannabinoid CB1 antagonists (Sink et al., 2008) failed to increase chow intake at doses that suppressed lever pressing. Similarly, prefeeding reduced both lever pressing and chow intake (Salamone et al., 1991). Furthermore, with higher ratio requirements (up to FR 20, or progressive ratios), animals that are not drug treated shift from lever pressing to chow intake (Pardo et al., 2011; Randall, Pardo, et al., 2011b; Salamone et al., 1997), indicating that this task is sensitive to work load. These results indicate that interference with DA transmission does not simply reduce food intake, but instead acts to alter response allocation between alternative sources of food that can be obtained through different instrumental responses.

The shift from lever pressing to chow intake in rats performing on this task is associated with DA depletions in nucleus accumbens; decreases in lever pressing and increases in chow intake occur as a result of accumbens DA depletions, as well as local injections of D1 or D2 family antagonists into either the core or shell subregions of nucleus accumbens (Cousins & Salamone 1994; Cousins, Sokolowski, & Salamone, 1993; Farrar et al., 2010; Koch et al. 2000; Nowend, Arizzi, Carlson, & Salamone, 2001; Salamone et al., 1991; Sokolowski & Salamone, 1998). Thus, although lever pressing is decreased by accumbens DA antagonism or depletions, these rats show a compensatory reallocation of behavior and select a new path to an alternative food source.

Salamone et al. (1994) also developed a T-maze procedure, in which the two choice arms of the maze led to different reinforcement densities (e.g., four vs. two food pellets, or four vs. zero); under some conditions, a barrier can be placed in the arm with the higher density of food reinforcement to present an effort-related challenge. When the high density arm has the barrier in place, and the arm without the barrier contains fewer reinforcers, DA depletions or antagonism decrease choice for the high density arm and increase selection of the low density arm with no barrier (Cousins, Atherton, Turner, & Salamone, 1996; Denk, Walton, Jennings, Sharp, Rushworth, & Bannerman, 2005; Mott et al., 2009; Pardo et al., submitted for publication; Salamone et al., 1994).

Like the operant concurrent choice task, this T-maze task also has undergone considerable behavioral validation and evaluation (Cousins et al., 1996; Pardo et al., submitted for publication; Salamone et al., 1994; van den Bos et al., 2006). For example, when there is no barrier in the maze, rats overwhelmingly prefer the high reinforcement density arm, and neither haloperidol nor accumbens DA depletion alters their response choice (Salamone et al., 1994). When the arm with the barrier contained four pellets, but the other arm contained no pellets, rats with accumbens DA depletions still managed to choose the high density arm, climb the barrier, and consume the pellets (Cousins et al., 1996). In a recent T-maze study with mice, while haloperidol reduced choice of the arm with the barrier, this drug had no effect on choice when both arms had a barrier in place (Pardo et al., submitted for publication). Thus, dopaminergic manipulations do not alter the preference for the high density of food reward over the low density, and did not affect discrimination, memory or instrumental learning processes related to arm preference. The results of the T-maze studies in rodents, together with the findings from the FR5/chow concurrent choice studies reviewed above, indicate that low doses of DA antagonists and accumbens DA depletions cause animals to reallocate their instrumental response selection based upon the response requirements of the task, and select lower cost alternatives for obtaining reinforcers (see reviews by Salamone et al., 2003, 2005, 2007; Floresco, St. Onge, et al., 2008).

Effort discounting procedures also have been employed to study the effects of dopaminergic manipulations. Floresco, Tse, et al. (2008) demonstrated that the DA antagonist haloperidol altered effort discounting even when the effects of time delay were controlled for (see Wade, de Wit, & Richards, 2000; and Koffarnus, Newman, Grundt, Rice, & Woods, 2011 for a discussion of the mixed findings in the literature on the effects of DA antagonists and delay discounting). Bardgett, Depenbrock, Downs, Points, & Green (2009) recently developed a T-maze effort-discounting task, in which the amount of food in the high-density arm of the maze was diminished each trial on which the rats selected that arm (i.e., an “adjusting-amount” discounting variant of the T-maze procedures, which allows for the determination of an indifference point for each rat). Effort discounting was altered by the D1 family antagonist SCH23390 and the D2 family antagonist haloperidol; these drugs made it more likely that rats would choose the low-reinforcement/low-cost arm. Increasing DA transmission by administration of amphetamine blocked the effects of SCH23390 and haloperidol, and also biased rats towards choosing the high-reinforcement/high-cost arm, which is consistent with operant choice studies using DA transporter knockdown mice (Cagniard, Balsam, Brunner, & Zhuang, 2006). Together with other results, the findings reported by Bardgett et al. and Floresco, Tse, et al. support the suggestion that, across a variety of conditions, DA transmission exerts a bidirectional influence over effort-related choice behavior.

DA INTERACTS WITH OTHER TRANSMITTERS TO INFLUENCE EFFORT-RELATED CHOICE BEHAVIOR

As reviewed above, DA antagonists and accumbens DA depletions affect instrumental response output, response allocation, and effort-related choice behavior. Obviously, no single brain area or neurotransmitter participates in a behavioral process in isolation to other structures or chemicals; for that reason it is important to review how other brain areas and neurotransmitters interact with dopaminergic mechanisms. Over the last several years, several laboratories have begun to characterize the role that multiple brain structures (e.g. amygdala, anterior cingulate cortex, ventral pallidum) and neurotransmitters (adenosine, GABA) play in effort-related choice behavior (Denk et al., 2005; Farrar et al., 2008; Floresco & Ghods-Sharifi, 2007; Floresco, St. Onge, et al., 2008; Hauber & Sommer, 2009; Mott et al. 2009; Pardo et al., submitted for publication; Schweimer & Hauber, 2006; van den Bos et al. 2006; Walton, Bannerman, Alterescu, & Rushworth, 2003; Walton, Bannerman, & Rushworth, 2002).

Within the last few years, considerable emphasis has been placed upon DA/adenosine interactions. Caffeine and other methylxanthines, which are nonselective adenosine antagonists, act as minor stimulants (Ferré et al., 2008; Randall, Nunez, et al., 2011). DA-rich brain areas, including the neostriatum and the nucleus accumbens, have a very high degree of adenosine A2A receptor expression (DeMet & Chicz-DeMet, 2002; Ferré et al., 2004; Schiffman, Jacobs, & Vanderhaeghen, 1991). There is considerable evidence of cellular interactions between DA D2 and adenosine A2A receptors (Ferré, 1997; Fink et al., 1992; Fuxe et al., 2003; Hillion et al., 2002). This interaction frequently has been studied in regard to neostriatal motor functions related to parkinsonism (Correa et al. 2004; Ferré, Fredholm, Morelli, Popoli, & Fuxe, 1997; Ferré et al., 2001; Hauber & Munkel, 1997; Hauber, Neuscheler, Nagel, & Muller, 2001; Ishiwari et al., 2007; Morelli & Pinna, 2002; Pinna, Wardas, Simola, & Morelli, 2005; Salamone, Betz, et al. 2008; Salamone, Ishiwari, et al., 2008; Svenningsson, Le Moine, Fisone, & Fredholm, 1999; Wardas, Konieczny, & Lorenc-Koci, 2001). However, several reports also have characterized aspects of adenosine A2A receptor function related to learning (Takahashi, Pamplona, & Prediger, 2008), anxiety (Correa & Font, 2008), and instrumental responding (Font et al., 2008; Mingote et al., 2008).

Drugs that act upon adenosine A2A receptors profoundly affect instrumental response output and effort-related choice behavior (Farrar et al., 2007, 2010; Font et al., 2008; Mingote et al., 2008; Mott et al., 2009; Pardo et al., submitted for publication; Worden et al., 2009). Intra-accumbens injections of the adenosine A2A agonist CGS 21680 reduced responding on a VI 60-sec schedule with a FR10 requirement attached, but did not impair performance on a conventional VI 60-sec schedule (Mingote et al., 2008), a pattern similar to that previously shown with accumbens DA depletions (Mingote et al., 2005). In rats responding on the FR5/chow concurrent choice procedure, injections of CGS 21680 into the accumbens decreased lever pressing and increased chow intake (Font et al.). These effects were site specific, because injections of CGS 21680 into a control site dorsal to the accumbens had no effect (Mingote et al., 2008; Font et al.).

It also has been demonstrated that adenosine A2A receptor antagonists can reverse the effects of systemically administered DA D2 antagonists in rats tested on the FR5/chow feeding concurrent choice task (Farrar et al., 2007; Nunes et al., 2010; Salamone et al., 2009; Worden et al., 2009). Moreover, systemic or intra-accumbens injections of the adenosine A2A antagonist MSX-3 were able to block the effects of intra-accumbens injections of the D2 antagonist eticlopride in rats responding on the FR5/chow concurrent choice task (Farrar et al., 2010). In studies using the T-maze barrier procedure, adenosine A2A antagonists have been shown to reverse the effects of DA D2 antagonism in rats (Mott et al., 2009) and mice (Pardo et al., submitted for publication). Furthermore, adenosine A2A receptor knockout mice are resistant to the effects of haloperidol on selection of the high-reinforcement/high-cost arm of the T-maze (Pardo et al.).

The pattern of effects seen in these studies depends upon which specific receptor subtypes are being acted upon by the drugs being administered. Although the adenosine A2A receptor antagonists MSX-3 and KW 6002 reliably and substantially attenuate the effects of D2 antagonists such as haloperidol and eticlopride in rats responding on the FR5/chow concurrent choice procedure (Farrar et al., 2007; Nunes et al., 2010; Salamone et al., 2009; Worden et al., 2009), they produce only a mild reversal of the effects of the D1 antagonist ecopipam (SCH 39166; Worden et al.; Nunes et al.). In addition, the highly selective adenosine A1 receptor antagonist was completely ineffective at reversing the effects of DA D1 or D2 antagonism (Salamone et al., 2009; Nunes et al.). Similar results were obtained with rats and mice responding on the T-maze barrier choice task; while MSX-3 was able to reverse the effect of the D2 antagonist haloperidol on selection of the high-reinforcement/high-cost arm, the A1 antagonists DPCPX and CPT were not (Mott et al., 2009; Pardo et al., submitted for publication). These results indicate that there is a relatively selective interaction between drugs that act upon DA D2 and adenosine A2A receptor subtypes (see Table 1). Based upon anatomical studies, it appears that this is likely to be due to the pattern of cellular localization of adenosine A1 and A2A receptors in striatal areas, including the nucleus accumbens (Ferré, 1997; Fink et al., 1992; Fuxe et al., 2003; Hillion et al., 2002; Svenningsson et al., 1999). Adenosine A2A receptors are typically co-localized on striatal and accumbens enkephalin-positive medium spiny neurons with DA D2 family receptors, and both receptors converge onto the same intracellular signaling pathways. Thus, adenosine A2A receptor antagonists may be so effective in reversing the actions of D2 antagonists because of direct interactions between DA D2 and adenosine A2A receptors located on the same neurons (Farrar et al., 2010; Salamone et al., 2009, 2010).

Table 1  

Adenosine receptor antagonists.

SUMMARY AND CONCLUSIONS: IMPLICATIONS FOR BEHAVIOR ANALYSIS AND PSYCHOPATHOLOGY

In summary, there is general agreement that nucleus accumbens DA and related brain systems participate in many functions that are important for instrumental behavior, though the specifics of that involvement are still being characterized. One conceptual limitation in this area is that global constructs such as “reward”, “reinforcement”, “learning”, “motivation” and “motor control” are too general to serve as useful descriptors of the effects of DA antagonism or depletion. These constructs actually involve several distinct processes, many of which can be dissociated from each other by brain manipulations such as drugs or lesions that severely impair one process while leaving another largely intact (Berridge & Robinson, 2003; Salamone & Correa, 2002; Salamone et al., 2007). Based upon the evidence reviewed above, interference with DA transmission does not impair “reward” in any general sense, because interference with DA transmission impairs some features of instrumental behavior while leaving fundamental aspects of primary reinforcement or motivation basically intact (e.g., reinforcement of simple instrumental responses; consumption of the reinforcer).

Another important consideration is the degree of overlap between very broad constructs such as “motivation” and “motor function”. Although one could attempt to adhere to a strict dichotomy between the motivational versus the motor functions of nucleus accumbens DA, it is not conceptually necessary to do so. It has been argued that “motor control” and “motivation”, though somewhat distinct conceptually, overlap considerably in terms of some of the specific characteristics of behavior being described and the brain circuits involved (Salamone, 1987, 1992, 2010b; Salamone & Correa 2002; Salamone et al., 2003, 2005, 2007). Consistent with this line of thinking, it is reasonable to suggest that accumbens DA performs functions that represent areas of overlap between motor and motivational processes (Salamone, 1987, 2010b; Salamone et al., 2007). Such functions would include the types of behavioral activation and effort-related processes discussed above. Nucleus accumbens DA is important for enabling animals to engage in schedule-induced activities (McCullough & Salamone, 1992; Robbins & Everitt, 2007; Robbins & Koob, 1980; Robbins et al., 1983; Salamone 1988; Wallace et al., 1983), and to respond to the work-related challenges imposed by ratio schedules (Aberman & Salamone,1999; Correa et al. 2002; Mingote et al., 2005; Salamone et al., 2002, 2003, 2005; Salamone, Correa, Mingote, Weber, & Farrar, 2006) and barriers in mazes (Cousins et al., 1996; Salamone et al., 1994). Moreover, the suggested involvement of accumbens DA in behavioral activation and effort is related to the hypothesis that nucleus accumbens is important for facilitating responsiveness to the activating properties of Pavlovian conditioned stimuli (Day, Wheeler, Roitman, & Carelli, 2006; Di Ciano, Cardinal, Cowell, Little, & Everitt, 2001; Everitt et al., 1999; Everitt & Robbins, 2005; Parkinson et al., 2002; Robbins & Everitt, 2007; Salamone et al., 2007).

Thus, despite the fact that animals with impaired transmission of accumbens DA remain directed towards the acquisition and consumption of primary reinforcers, accumbens DA does appear to be particularly important for overcoming work-related challenges presented by instrumental behaviors with high response requirements. This represents one function of accumbens DA, but certainly not the only one. As emphasized in previous papers (e.g., Salamone et al., 2007), it is unlikely that accumbens DA performs only one function, and evidence in favor of the hypothesis that DA is involved in the exertion of effort or effort-related choice behavior is not incompatible with the hypothesized involvement of this system in instrumental learning (Baldo & Kelley, 2007; Beninger & Gerdjikov, 2004; Kelley et al., 2005; Segovia et al., 2011; Wise, 2004), aspects of incentive motivation (e.g. reinforcer “wanting”; Berridge 2007; Berridge & Robinson, 2003; Wyvell & Berridge, 2001) or Pavlovian-instrumental transfer (Everitt & Robbins, 2005).

A measure derived from observations of behavior, or a parameter that is generated from curve-fitting analyses, can have many factors that contribute to it and, as noted above, pharmacological research often can dissociate between these factors, because a drug can severely affect one while leaving another basically intact. A useful example of this principle is the progressive ratio break point, which, as discussed above, is influenced by several factors (Pardo et al., 2011; Randall, Pardo, et al., 2011b). Another case in which this point is highly relevant is the measurement of intracranial self-stimulation thresholds. Such measures often are viewed as providing “rate-free” indices of “reward”, or even “hedonia”, nevertheless, they are influenced by lever pressing ratio requirements as well as the electrical current level (Fouriezos, Bielajew, & Pagotto, 1990). Recent studies with intracranial self-stimulation thresholds indicate that dopaminergic modulation of self-stimulation thresholds does not affect reward value per se, but instead alters the tendency to pay response costs (Hernandez, Breton, Conover, & Shizgal, 2010). Response-reinforcement matching also has been used in some research related to behavioral economics, reinforcer value, and the functions of DA systems (e.g. Aparicio, 2007; Heyman, Monaghan, & Clody, 1987). Matching equations have been employed to describe the results of studies with VI schedules, and parameters from matching equations (e.g., Ro) have been used to represent reinforcement value (e.g., Herrnstein 1974; Ro has been referred to as the rate of reinforcement from other sources, and is inversely related to reinforcement value of the scheduled contingencies). As noted by Killeen (1995), empirically, Ro represents a “half-life constant” for the curve fitting formula. However, used in this way, Ro does not selectively represent the reinforcement value of food per se. At best, this measure reflects the relative value of the entire activity of lever pressing for and consuming the food reinforcer compared to the reinforcing value of all other stimuli and responses available (Salamone et al., 1997, 2009; Williams, 1988). Several factors can contribute to this composite measure, and a drug or lesion manipulation could yield apparent effects on “reinforcement value” that actually reflect changes in response-related factors (Salamone, 1987; Salamone et al., 1997, 2009). Moreover, matching equations have been developed that account for deviations from matching by allowing for estimates of response preference or bias (Aparicio, 2001; Baum, 1974; Williams, 1988), which also could be affected by drugs.

In view of these points, it is useful to consider how terms such as “value” are used in behavioral economics and neuroeconomics research. The aggregate reinforcement value of an instrumental activity (e.g., lever pressing for and consuming food) should probably be viewed as a composite measure that includes both the reinforcing value of the reinforcer itself, and also any net value or costs associated with the instrumental response that is required to obtain the reinforcer. Viewed in this manner, the effects of DA antagonists or depletions on effort-related choice behavior could be described in terms of actions upon the response costs associated with the particular instrumental response, rather than the reinforcing value of the reinforcing stimulus itself. Although the effects of haloperidol on bias may be minimal when two levers that are relatively similar are used (e.g., Aparicio, 2007), they may be much larger when substantially different responses are compared (e.g., lever pressing vs. nose poking or sniffing; lever pressing vs. unrestricted access to food; barrier climbing vs. locomotion to a location containing food).

In addition to providing insights into aspects of instrumental behavior seen in the laboratory, research on effort-related choice behavior also has clinical implications. Addiction is characterized by a reorganization of the preference structure of the person, dramatic changes in the allocation of behavioral resources towards the addictive substance (Heyman, 2009; Vezina et al., 2002), and inelasticity of demand (Heyman, 2000). Typically, there is a heightened tendency to engage in drug-reinforced instrumental behavior and drug consumption, often at the expense of other behavioral activities. Addicts will go to great lengths to obtain their preferred drug, overcoming numerous obstacles and constraints. Thus, drug-reinforced instrumental behavior in humans involves many processes, including exertion of effort. Recent evidence indicates that DA synthesis inhibition induced by precursor depletion resulted in a decrease in progressive ratio breakpoints reinforced by nicotine-containing cigarettes, despite the fact that this manipulation did not affect self-reported “euphoria” or “craving” (Venugopalan et al., 2011).

As well as being related to aspects of drug taking and addiction, research on effort-related choice behavior has implications for understanding the neural basis of psychopathological symptoms such as psychomotor slowing, anergia, fatigue and apathy, which are seen in depression as well as in other psychiatric or neurological conditions (Salamone et al., 2006, 2007, 2010). These symptoms, which can have devastating behavioral manifestations (Demyttenaere, De Fruyt, & Stahl, 2005; Stahl, 2002), essentially represent impairments in aspects of instrumental behavior, exertion of effort and effort-related choice, which can lead to difficulties in the workplace, as well as limitations in terms of life function, interaction with the environment, and responsiveness to treatment. Within the last few years, there has been increasing interest in behavioral activation therapy for the treatment of depression, which is used to increase activation systematically by employing graded exercises to increase the patient’s access to reinforcement and identify processes that inhibit activation (Jacobson, Martell, & Dimidjian, 2001; Weinstock, Munroe, & Miller, 2011). Furthermore, there is considerable overlap between the neural circuitry involved in effort-related functions in animals and the brain systems that have been implicated in psychomotor slowing and anergia in depression (Salamone et al. 2006, 2007, 2009, 2010; Treadway & Zald, 2011). Thus, basic and clinical research on effort-related behavioral processes, and their neural regulation, could have substantial impact on clinical research related to addiction, depression, and other disorders.

Acknowledgments

Acknowledgements: Much of the work cited in this review was supported by a grant to JDS from the US NIH/NIMH (MH078023), and to MC from Fundació UJI/Bancaixa (P1.1B2010-43).

Merce Correa and Marta Pardo are now at Area de Psicobiol., Dept. Psic., Universitat de Jaume I, Castelló, 12071, Spain.

REFERENCES

  1. Aberman J.E, Salamone J.D. Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience. 1999;92:545–552. [PubMed]
  2. Allison J. Economics and operant conditioning. In: Harzem P, Zeiler M.D, editors. Predictability, correlation and contiguity. New York: John Wiley and Sons; 1981. pp. 321–353. (Eds.)
  3. Allison J. Response deprivation, reinforcement, and economics. Journal of the Experimental Analysis of Behavior. 1993;60:129–140. [PMC free article] [PubMed]
  4. Amtage J, Schmidt W.J. Context-dependent catalepsy intensification is due to classical conditioning and sensitization. Behavioural Pharmacology. 2003;14:563–567. [PubMed]
  5. Anstrom K.K, Woodward D.J. Restraint increases dopaminergic burst firing in awake rats. Neuropsychopharmacology. 2005;30:1832–1840. [PubMed]
  6. Aparicio C.F. Overmatching in rats: the barrier choice paradigm. Journal of the Experimental Analysis of Behavior. 2001;75:93–106. [PMC free article] [PubMed]
  7. Aparicio C.F. Haloperidol, dynamics of choice, and the parameters of the matching law. Behavioral Processes. 2007;75:206–212. [PubMed]
  8. Arnold J.M, Roberts D.C. A critique of fixed and progressive ratio schedules used to examine the neural substrates of drug reinforcement. Pharmacology, Biochemistry and Behavior. 1997;57:441–447. [PubMed]
  9. Asin K.E, Fibiger H.C. Force requirements in lever-pressing and responding after haloperidol. Pharmacology, Biochemistry and Behavior. 1984;20(3):323–326. [PubMed]
  10. Bakshi V.P, Kelley A.E. Dopaminergic regulation of feeding behavior: I. Differential effects of haloperidol microinjection in three striatal subregions. Psychobiology. 1991;19:223–232.
  11. Baldo B.A, Kelley A.E. Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology. 2007;191:439–459. [PubMed]
  12. Baldo B.A, Sadeghian K, Basso A.M, Kelley A.E. Effects of selective dopamine D1 or D2 receptor blockade within nucleus accumbens subregions on ingestive behavior and associated motor activity. Behavioural Brain Research. 2002;137:165–177. [PubMed]
  13. Barbano M.F, Cador M. Opioids for hedonic experience and dopamine to get ready for it. Psychopharmacology. 2007;191:497–506. [PubMed]
  14. Bardgett M.E, Depenbrock M, Downs N, Points M, Green L. Dopamine modulates effort-based decision making in rats. Behavioral Neuroscience. 2009;123:242–251. [PMC free article] [PubMed]
  15. Barrett J.E, Bergman J. Peter B. Dews and pharmacological studies on behavior. Journal of Pharmacology and Experimental Therapeutics. 2008;326:683–690. [PubMed]
  16. Baum W.M. On two types of deviation from the matching law: bias and undermatching. Journal of the Experimental Analysis of Behavior. 1974;22:231–242. [PMC free article] [PubMed]
  17. Beninger R.J, Cheng M, Hahn B.L, Hoffman D.C, Mazurski E.J, Morency M.A, Ramm P, Stewart R.J. Effects of extinction, pimozide, SCH 23390, and metoclopramide on food-rewarded operant responding of rats. Psychopharmacology. 1987;92:343–349. [PubMed]
  18. Beninger R.J, Gerdjikov T. The role of signaling molecules in reward-related incentive learning. Neurotoxicology Research. 2004;6:91–104. [PubMed]
  19. Berridge K.C. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharmacology. 2007;191:391–431. [PubMed]
  20. Berridge K.C, Kringlebach M.L. Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology. 2008;199:457–480. [PMC free article] [PubMed]
  21. Berridge K.C, Robinson T.E. Parsing reward. Trends in Neuroscience. 2003;26:507–513. [PubMed]
  22. Bickel W.K, Marsch L.A, Carroll M.E. Deconstructing relative reinforcing efficacy and situating the measures of pharmacological reinforcement with behavioral economics: a theoretical proposal. Psychopharmacology. 2000;153:44–56. [PubMed]
  23. Blazquez P.M, Fujii N, Kojima J, Graybiel A.M. A network representation of response probability in the striatum. Neuron. 2002;33:973–982. [PubMed]
  24. Brauer L.H, De Wit H. High dose pimozide does not block amphetamine-induced euphoria in normal volunteers. Pharmacology, Biochemistry and Behavior. 1997;56:265–272. [PubMed]
  25. Brischoux F, Chakraborty S, Brierley D.I, Ungless M.A. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proceedings of the National Academy of Sciences. 2009;106:4894–4899. [PMC free article] [PubMed]
  26. Broom S.L, Yamamoto B.K. Effects of subchronic methamphetamine exposure on basal dopamine and stress-induced dopamine release in the nucleus accumbens shell of rats. Psychopharmacology. 2005;181:467–476. [PubMed]
  27. Cagniard B, Balsam P.D, Brunner D, Zhuang X. Mice with chronically elevated dopamine exhibit enhanced motivation, but not learning, for a food reward. Neuropsychopharmacology. 2006;31:1362–1370. [PubMed]
  28. Cannon C.M, Bseikri M.R. Is dopamine required for natural reward. Physiology and Behavior. 2004;81:741–748. [PubMed]
  29. Caul W.F, Brindle N.A. Schedule-dependent effects of haloperidol and amphetamine: multiple-schedule task shows within-subject effects. Pharmacology, Biochemistry, and Behavior. 2001;68:53–63. [PubMed]
  30. Collier G.H, Jennings W. Work as a determinant of instrumental performance. Jounral of Comparative and Physiological Psychology. 1969;68:659–662.
  31. Correa M, Carlson B.B, Wisniecki A, Salamone J.D. Nucleus accumbens dopamine and work requirements on interval schedules. Behavioural Brain Research. 2002;137:179–187. [PubMed]
  32. Correa M, Font L. Is there a major role for adenosine A2A receptors in anxiety. Frontiers in Bioscience. 2008;13:4058–4070. [PubMed]
  33. Correa M, Wisniecki A, Betz A, Dobson D.R, O’Neill M.F, O’Neill M.J, Salamone J.D. The adenosine A2A antagonist KF17837 reverses the locomotor suppression and tremulous jaw movements induced by haloperidol in rats: possible relevance to parkinsonism. Behavioural Brain Research. 2004;148:47–54. [PubMed]
  34. Cousins M.S, Atherton A, Turner L, Salamone J.D. Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behavioural Brain Research. 1996;74:189–197. [PubMed]
  35. Cousins M.S, Salamone J.D. Nucleus accumbens dopamine depletions in rats affect relative response allocation in a novel cost/benefit procedure. Pharmacology, Biochemistry, and Behavior. 1994;49:85–91. [PubMed]
  36. Cousins M.S, Sokolowski J.D, Salamone J.D. Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacology, Biochemistry, and Behavior. 1993;46:943–951. [PubMed]
  37. Cousins M.S, Wei W, Salamone J.D. Pharmacological characterization of performance on a concurrent lever pressing/feeding choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology. 1994;116:529–537. [PubMed]
  38. Das S, Fowler S.C. An update of Fowler and Das: Anticholinergic reversal of haloperidol-induced, within-session decrements in rats’ lapping behavior. Pharmacology, Biochemistry and Behavior. 1996;53:853–855. [PubMed]
  39. Day J.J, Wheeler R.A, Roitman M.F, Carelli R.M. Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. European Journal of Neuroscience. 2006;23:1341–1351. [PubMed]
  40. Delgado M.R, Jou R.L, Phelps E.A. Neural systems underlying aversive conditioning in humans with primary and secondary reinforcers. Frontiers in Neuroscience. 2011;5:71. [PMC free article] [PubMed]
  41. Delgado M.R, Li J, Schiller D, Phelps E.A. The role of the striatum in aversive learning and aversive prediction errors. Philosophical Transactions of the Royal Society. 2008;363:3787–3800. [PMC free article] [PubMed]
  42. DeMet E.M, Chicz-DeMet A. Localization of adenosine A2A-receptors in rat brain with [3H]ZM-241385. Naunyn-Schmiedeberg’s Archives of Pharmacology. 2002;366:478–481. [PubMed]
  43. Demyttenaere K, De Fruyt J, Stahl S.M. The many faces of fatigue in major depressive disorder. International Journal of Neuropsychopharmacology. 2005;8:93–105. [PubMed]
  44. Denk F, Walton M.E, Jennings K.A, Sharp T, Rushworth M.F, Bannerman D.M. Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology. 2005;179:587–596. [PubMed]
  45. Dews P.B. Interactions of behavioral effects of drugs. Annals of the New York Academy of Sciences. 1976;281:50–63. [PubMed]
  46. Di Ciano P, Cardinal R.N, Cowell R.A, Little S.J, Everitt B.J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of Pavlovian approach behavior. Journal of Neuroscience. 2001;21:9471–9477. [PubMed]
  47. Dickinson A, Balleine B. Motivational control of goal-directed action. Animal Learning and Behavior. 1994;22:1–18.
  48. Dunnett S.B, Iversen S.D. Regulatory impairments following selective 6-OHDA lesions of the neostriatum. Behavioral Brain Research. 1982;4:195–202. [PubMed]
  49. Ettenberg A, Koob G.F, Bloom F.E. Response artifact in the measurement of neuroleptic-induced anhedonia. Science. 1981;213:357–359. [PubMed]
  50. Evenden J.L, Robbins T.W. Dissociable effects of d-amphetamine, chlordiazepoxide and alpha-flupenthixol on choice and rate measures of reinforcement in the rat. Psychopharmacology. 1983;79:180–86. [PubMed]
  51. Everitt B.J, Parkinson J.A, Olmstead M.C, Arroyo M, Robledo P, Robbins T.W. Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems. Annals of the New York Academy of Sciences. 1999;877:412–438. [PubMed]
  52. Everitt B.J, Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience. 2005;8:1481–1489. [PubMed]
  53. Farrar A.M, Font L, Pereira M, Mingote S.M, Bunce J.G, Chrobak J.J, Salamone J.D. Forebrain circuitry involved in effort-related choice: injections of the GABAA agonist muscimol into ventral pallidum alters response allocation in food-seeking behavior. Neuroscience. 2008;152:321–330. [PMC free article] [PubMed]
  54. Farrar A.M, Pereira M, Velasco F, Hockemeyer J, Muller C.E, Salamone J.D. Adenosine A(2A) receptor antagonism reverses the effects of dopamine receptor antagonism on instrumental output and effort-related choice in the rat: implications for studies of psychomotor slowing. Psychopharmacology. 2007;191:579–586. [PubMed]
  55. Farrar A.M, Segovia K.N, Randall P.A, Nunes E.J, Collins L.E, Stopper C.M, Port R.G, Hockemeyer J, Müller C.E, Correa M, Salamone J.D. Nucleus accumbens and effort-related functions: behavioral and neural markers of the interactions between adenosine A2A and dopamine D2 receptors. Neuroscience. 2010;166:1056–1067. [PubMed]
  56. Faure A, Reynolds S.M, Richard J.M, Berridge K.C. Mesolimbic dopamine in desire and dread: enabling motivation to be generated by localized glutamate disruptions in nucleus accumbens. Journal of Neuroscience. 2008;28:7184–7192. [PMC free article] [PubMed]
  57. Faustman W.O, Fowler S.C. Use of operant response duration to distinguish the effects of haloperidol from nonreward. Pharmacology Biochemistry and Behavior. 1981;15(2):327–329. [PubMed]
  58. Faustman W.O, Fowler S.C. An examination of methodological refinements, clozapine and fluphenazine in the anhedonia paradigm. Pharmacology Biochemistry and Behavior. 1982;17(5):987–993. [PubMed]
  59. Feldon J, Weiner I. Effects of haloperidol on the multitrial partial reinforcement extinction effect (PREE): evidence for neuroleptic drug action on nonreinforcement but not on reinforcement. Psychopharmacology. 1991;105:407–414. [PubMed]
  60. Ferré S. Adenosine-dopamine interactions in the ventral striatum. Implications for the treatment of schizophrenia. Psychopharmacology. 1997;133:107–120. [PubMed]
  61. Ferré S, Ciruela F, Borycz J, Solinas M, Quarta D, Antoniou K, Quiroz C, Justinova Z, Lluis C, Franco R, Goldberg S.R. Adenosine A1–A2A receptor heteromers: new targets for caffeine in the brain. Frontiers in Bioscience. 2008;13:2391–2399. [PubMed]
  62. Ferré S, Ciruela F, Canals M, Marcellino D, Burgueno J, Casado V, Hillion J, Torvinen M, Fanelli F, Benedetti P.P, Goldberg S.R, Bouvier M, Fuxe K, Agnati L.F, Lluis C, Franco R, Woods A. Adenosine A2A-dopamine D2 receptor-receptor heteromers. Targets for neuro-psychiatric disorders. Parkinsonism and Related Disorders. 2004;10:265–271. [PubMed]
  63. Ferré S, Fredholm B.B, Morelli M, Popoli P, Fuxe K. Adenosine-dopamine receptor–receptor interactions as an integrative mechanism in the basal ganglia. Trends in Neuroscience. 1997;20:482–487. [PubMed]
  64. Ferré S, Popoli P, Giménez-Llort L, Rimondini R, Müller C.E, Strömberg I, Ögren S.O, Fuxe K. Adenosine/dopamine interaction: implications for the treatment of Parkinson’s disease. Parkinsonism and Related Disorders. 2001;7:235–241. [PubMed]
  65. Fibiger H.C, Carter D.A, Phillips A.G. Decreased intracranial self-stimulation after neuroleptics or 6-hydroxydopamine: Evidence for mediation by reward deficits rather than by reduced reward. Psychopharmacology. 1976;47:21–27. [PubMed]
  66. Fink J.S, Weaver D.R, Rivkees S.A, Peterfreund R.A, Pollack A.E, Adler E.M, Reppert S.M. Molecular cloning of the rat A2A adenosine receptor: selective co-expression with D2 dopamine receptors in rat striatum. Molecular Brain Research. 1992;14:186–195. [PubMed]
  67. Floresco S.B, Ghods-Sharifi S. Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cerebral Cortex. 2007;17:251–260. [PubMed]
  68. Floresco S.B, St. Onge J.R, Ghods-Sharifi S, Winstanley C.A. Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cognitive Affective Behavioral Neuroscience. 2008;8:375–389. [PubMed]
  69. Floresco S.B, Tse M.T, Ghods-Sharifi S. Dopaminergic and glutamatergic regulation of effort- and delay-based decision making. Neuropsychopharmacology. 2008;33:1966–1979. [PubMed]
  70. Foltin R.W. An economic analysis of “demand” for food in baboons. Journal of the Experimental Analysis of Behavior. 1991;56:445–454. [PMC free article] [PubMed]
  71. Font L, Mingote S, Farrar A.M, Pereira M, Worden L, Stopper C, Port R.G, Salamone J.D. Intra-accumbens injections of the adenosine A(2A) agonist CGS 21680 affect effort-related choice behavior in rats. Psychopharmacology. 2008;199:515–526. [PMC free article] [PubMed]
  72. Fouriezos G, Bielajew C, Pagotto W. Task difficulty increases thresholds of rewarding brain stimulation. Behavioral Brain Research. 1990;37:1–7. [PubMed]
  73. Fuxe K, Agnati L.F, Jacobsen K, Hillion J, Canals M, Torvinen M, Tinner-Staines B, Staines W, Rosin D, Terasmaa A, Popoli P, Leo G, Vergoni V, Lluis C, Ciruela F, Franco R, Ferré S. Receptor heteromerization in adenosine A2A receptor signaling: relevance for striatal function and Parkinson’s disease. Neurology. 2003;61:S19–23. [PubMed]
  74. Gawin F.H. Neuroleptic reduction of cocaine-induced paranoia but not euphoria. Psychopharmacology. 1986;90:142–143. [PubMed]
  75. Gramling S.E, Fowler S.C, Collins K.R. Some effects of pimozide on nondeprived rats licking sucrose solutions in an anhedonia paradigm. Pharmacology Biochemistry and Behavior. 1984;21:617–624. [PubMed]
  76. Gramling S.E, Fowler S.C, Tizzano J.P. Some effects of pimozide on nondeprived rats’ lever pressing maintained by a sucrose reward in an anhedonia paradigm. Pharmacology Biochemistry and Behavior. 1987;27:67–72. [PubMed]
  77. Guarraci F.A, Kapp B.S. An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential Pavlovian fear conditioning in the awake rabbit. Behavioral. Brain Research. 1999;99:169–179. [PubMed]
  78. Haase H.J, Janssen P.A.J. The action of neuroleptic drugs. Amsterdam: Elsevier Science Publishers; 1985.
  79. Hamill S, Trevitt J.T, Nowend K.L, Carlson B.B, Salamone J.D. Nucleus accumbens dopamine depletions and time-constrained progressive ratio performance: effects of different ratio requirements. Pharmacology, Biochemistry, and Behavior. 1999;64:21–27. [PubMed]
  80. Haney M, Ward A.S, Foltin R.W, Fischman M.W. Effects of ecopipam, a selective dopamine D1 antagonist, on smoked cocaine self-administration by humans. Psychopharmacology. 2001;155:330–337. [PubMed]
  81. Hauber W. Dopamine release in the prefrontal cortex and striatum: temporal and behavioural aspects. Pharmacopsychiatry. 2010;43:S32–41. [PubMed]
  82. Hauber W, Munkel M. Motor depressant effects mediated by dopamine D2 and adenosine A2A receptors in the nucleus accumbens and the caudate-putamen. European Journal of Pharmacology. 1997;323:127–131. [PubMed]
  83. Hauber W, Neuscheler P, Nagel J, Muller C.E. Catalepsy induced by a blockade of dopamine D1 or D2 receptors was reversed by a concomitant blockade of adenosine A2A receptors in the caudate putamen of rats. European Journal of Neuroscience. 2001;14:1287–1293. [PubMed]
  84. Hauber W, Sommer S. Prefrontostriatal circuitry regulates effort-related decision making. Cerebral Cortex. 2009;10:2240–2247. [PubMed]
  85. Hengeveld G.M, van Langevelde F, Groen T.A, de Knegt H.J. Optimal foraging for multiple resources in several food species. American. Naturalist. 2009;17:102–110. [PubMed]
  86. Hernandez G, Breton Y.A, Conover K, Shizgal P. At what stage of neural processing does cocaine act to boost pursuit of rewards. PLoS One. 2010;5:e15081. [PMC free article] [PubMed]
  87. Herrnstein R.J. Formal properties of the matching law. Journal of the Experimental Analysis of Behavior. 1974;21:159–164. [PMC free article] [PubMed]
  88. Heyman G.M. An economic approach to animal models of alcoholism. Alcohol Research & Health. 2000;24:132–139. [PubMed]
  89. Heyman G.M. Addiction: A disorder of choice. Cambridge, MA: Harvard University Press; 2009.
  90. Heyman G.M, Monaghan M.M, Clody D.E. Low doses of cis-flupentixol attenuate motor performance. Psychopharmacology. 1987;93:477–482. [PubMed]
  91. Hillion J, Canals M, Torvinen M, Casado V, Scott R, Terasmaa A, Hansson A, Watson S, Olah M.E, Mallol J, Canela E.I, Zoli M, Agnati L.F, Ibañez C.F, Lluis C, Franco R, Ferré S, Fuxe K. Coaggregation, cointernalization, and codesensitization of adenosine A2A receptors and dopamine D2 receptors. The Journal of Biological Chemistry. 2002;277:18091–18097. [PubMed]
  92. Hursh S.R. Behavioral economics of drug self-administration: An Introduction. Drug and Alcohol Dependence. 1993;33:165–172. [PubMed]
  93. Hursh S.R, Raslear T.G, Shurtleff D, Bauman R, Simmons L. A cost-benefit analysis of demand for food. Journal of the Experimental Analysis of Behavior. 1988;50:419–440. [PMC free article] [PubMed]
  94. Ikemoto S, Panksepp J. Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. Behavioral Neuroscience. 1996;110:331–345. [PubMed]
  95. Ishiwari K, Madson L.J, Farrar A.M, Mingote S.M, Valenta J.P, DiGianvittorio M.D, Frank L.E, Correa M, Hockemeyer J, Muller C, Salamone J.D. Injections of the selective adenosine A2A antagonist MSX-3 into the nucleus accumbens core attenuate the locomotor suppression induced by haloperidol in rats. Behavioural Brain Research. 2007;178:190–199. [PMC free article] [PubMed]
  96. Ishiwari K, Weber S.M, Mingote S, Correa M, Salamone J.D. Accumbens dopamine and the regulation of effort in food-seeking behavior: modulation of work output by different ratio or force requirements. Behavioural Brain Research. 2004;151:83–91. [PubMed]
  97. Jacobson N.S, Martell C.R, Dimidjian S. Behavioral activation treatment for depression: returning to contextual roots. Clinical Psychology: Science & Practice. 2001;8:225–270.
  98. Jensen J, McIntosh A.R, Crawley A.P, Mikulis D.J, Remington G, Kapur S. Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron. 2003;40:1251–1257. [PubMed]
  99. Johnson D.F, Collier G.H. Caloric regulation and patterns of food choice in a patchy environment: the value and cost of alternative foods. Physiological Behavior. 1987;39:351–359. [PubMed]
  100. Kaufman L.W. Foraging costs and meal patterns in ferrets. Physiology & Behavior. 1980;25:139–141. [PubMed]
  101. Kaufman L.W, Collier G, Hill W.L, Collins K. Meal cost and meal patterns in an uncaged domestic cat. Physiological Behavior. 1980;25:135–137. [PubMed]
  102. Kelley A.E, Baldo B.A, Pratt W.E, Will M.J. Corticostriatal-hypothalamic circuitry and food motivation: integration of energy, action and reward. Physiological Behavior. 2005;86:773–795. [PubMed]
  103. Killeen P. On the temporal control of behavior. Psychological Review. 1975;82:89–115.
  104. Killeen P.R. Mathematical principles of reinforcement. Behavioral and Brain Sciences. 1994;17:105–172.
  105. Killeen P.R. Economics, ecologics and mechanics: the dynamics of responding under conditions of varying motivation. Journal of the Experimental Analysis of Behavior. 1995;64:405–431. [PMC free article] [PubMed]
  106. Killeen P.R, Hanson S.J, Osborne S.R. Arousal: Its genesis and manifestation as response rate. Psychological Review. 1978;85:571–581. [PubMed]
  107. Koch M, Schmid A, Scnhnitzler H.U. Role of nucleus accumbens dopamine D1 and D2 receptors in instrumental and Pavlovian paradigms of conditioned reward. Psychopharmacology. 2000;152:67–73. [PubMed]
  108. Koffarnus M.N, Newman A.H, Grundt P, Rice K.C, Woods J.H. Effects of selective dopaminergic compounds on a delay-discounting task. Behavioural Pharmacology. 2011;22:300–311. [PMC free article] [PubMed]
  109. Koob G.F, Riley S.J, Smith S.C, Robbins T.W. Effects of 6-hydroxydopamine lesions of the nucleus accumbens septi and olfactory tubercle on feeding, locomotor activity, and amphetamine anorexia in the rat. Journal of Comparative Physiological Psychology. 1978;92:917–927. [PubMed]
  110. Lack C.M, Jones S.R, Roberts D.C. Increased breakpoints on a progressive ratio schedule reinforced by IV cocaine are associated with reduced locomotor activation and reduced dopamine efflux in nucleus accumbens shell in rats. Psychopharmacology. 2008;195:517–525. [PubMed]
  111. Lea S.E.G. The psychology and economics of demand. Psychological Bulletin. 1978;85:441–466.
  112. Levita L, Hare T.A, Voss H.U, Glover G, Ballon D.J, Casey B.J. The bivalent side of the nucleus accumbens. Neuroimage. 2009;44:1178–1187. [PMC free article] [PubMed]
  113. Liao R.M, Fowler S.C. Haloperidol produces within-session increments in operant response duration in rats. Pharmacology, Biochemistry and Behavior. 1990;36:199–201. [PubMed]
  114. Liberzon I, Taylor S.F, Amdur R, Jung T.D, Chamberlain K.R, Minoshima S, Koeppe R.A, Fig L.M. Brain activation in PTSD in response to trauma-related stimuli. Biological Psychiatary. 1999;45:817–826. [PubMed]
  115. Madden G.J, Bickel W.K, Jacobs E.A. Three predictions of the economic concept of unit price in a choice context. Journal of the Experimental Analysis of Behavior. 2000;73:45–64. [PMC free article] [PubMed]
  116. Madden G.J, Kalman D. Effects of bupropion on simulated demand for cigarettes and the subjective effects of smoking. Nicotine & Tobacco Research. 2010;12:416–422. [PMC free article] [PubMed]
  117. Madden G.J, Smethells J.R, Ewan E.E, Hursh S.R. Tests of behavioral-economic assessments of relative reinforcer efficacy II: economic complements. Journal of the Experimental Analysis of Behavior. 2007;88:355–367. [PMC free article] [PubMed]
  118. Marinelli M, Barrot M, Simon H, Oberlander C, Dekeyne A, Le Moal M, Piazza P.V. Pharmacological stimuli decreasing nucleus accumbens dopamine can act as positive reinforcers but have a low addictive potential. Eurpeoan Journal of Neuroscience. 1998;10:3269–3275. [PubMed]
  119. Marinelli S, Pascucci T, Bernardi G, Puglisi-Allegra S, Mercuri N.B. Activation of TRPV1 in the VTA excites dopaminergic neurons and increases chemical- and noxious-induced dopamine release in the nucleus accumbens. Neuropsychopharmacology. 2005;30:864–875. [PubMed]
  120. Martinez R.C.R, Oliveira A.R, Macedo C.E, Molina V.A, Brandao M.L. Neuroscience Letters. 2008;446:112–116. [PubMed]
  121. Martin-Iverson M.T, Wilke D, Fibiger H.C. Effect of haloperidol and d-amphetamine on perceived quantity of food and tones. Psychopharamcology. 1987;93:374–381. [PubMed]
  122. McCullough L.D, Salamone J.D. Anxiogenic drugs beta-CCE and FG 7142 increase extracellular dopamine levels in nucleus accumbens. Psychopharmacology. 1992;109(3):379–382. [PubMed]
  123. McCullough L.D, Sokolowski J.D, Salamone J.D. A neurochemical and behavioral investigation of the involvement of nucleus accumbens dopamine in instrumental avoidance. Neuroscience. 1993;52(4):919–925. [PubMed]
  124. McMillan D.E, Katz J.L. Continuing implications of the early evidence against the drive-reduction hypothesis of the behavioral effects of drugs. Psychopharmacology. 2002;163:251–264. [PubMed]
  125. Mekarski J.E. Main effects of current and pimozide on prepared and learned self-stimulation behaviors are on performance not reward. Pharmacology Biochemistry and Behavior. 1988;31:845–853. [PubMed]
  126. Mingote S, Font L, Farrar A.M, Vontell R, Worden L.T, Stopper C.M, Port R.G, Sink K.S, Bunce J.G, Chrobak J.J, Salamone J.D. Nucleus accumbens adenosine A2A receptors regulate exertion of effort by acting on the ventral striatopallidal pathway. Journal of Neuroscience. 2008;28:9037–9046. [PMC free article] [PubMed]
  127. Mingote S, Weber S.M, Ishiwari K, Correa M, Salamone J.D. Ratio and time requirements on operant schedules: effort-related effects of nucleus accumbens dopamine depletions. European Journal of Neuroscience. 2005;21:1749–1757. [PubMed]
  128. Mobini S, Chiang T.J, Ho M.Y, Bradshaw C.M, Szabadi E. Comparison of the effects of clozapine, haloperidol, chlorpromazine and d-amphetamine on performance on a time-constrained progressive ratio schedule and on locomotor behaviour in the rat. Psychopharmacology. 2000;152:47–54. [PubMed]
  129. Morelli M, Pinna A. Interaction between dopamine and adenosine A2A receptors as a basis for the treatment of Parkinson’s disease. Neurological Sciences. 2002;22:71–72. [PubMed]
  130. Morgan D, Brebner K, Lynch W.J, Roberts D.C. Increases in the reinforcing efficacy of cocaine after particular histories of reinforcement. Behavioural Pharmacology. 2002;13:389–396. [PubMed]
  131. Mott A.M, Nunes E.J, Collins L.E, Port R.G, Sink K.S, Hockemeyer J, Müller C.E, Salamone J.D. The adenosine A2A antagonist MSX-3 reverses the effects of the dopamine antagonist haloperidol on effort-related decision making in a T-maze cost/benefit procedure. Psychopharmacology. 2009;204:103–112. [PMC free article] [PubMed]
  132. Munro L.J, Kokkinidis L. Infusion of quinpirole and muscimol into the ventral tegmental area inhibits fear-potentiated startle: implications for the role of dopamine in fear expression. Brain Research. 1997;746:231–238. [PubMed]
  133. Nann-Vernotica E, Donny E.C, Bigelow G.E, Walsh S.L. Repeated administration of the D1/5 antagonist ecopipam fails to attenuate the subjective effects of cocaine. Psychopharmacology. 2001;155:338–347. [PubMed]
  134. Neill D.B, Justice J.B. An hypothesis for a behavioral function of dopaminergic transmission in nucleus accumbens. In: Chronister R.B, Defrance J.F, editors. The neurobiology of nucleus accumbens. Brunswick, Canada: Huer Institute; 1981. (Eds.)
  135. Nicola S.M. The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. Journal of Neuroscience. 2010;30:16585–16600. [PMC free article] [PubMed]
  136. Niv Y, Daw N.D, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology. 2007;191:507–520. [PubMed]
  137. Nowend K.L, Arizzi M, Carlson B.B, Salamone J.D. D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacology Biochemistry and Behavior. 2001;69:373–382. [PubMed]
  138. Nunes E.J, Randall P.A, Santerre J.L, Given A.B, Sager T.N, Correa M, Salamone J.D. Differential effects of selective adenosine antagonists on the effort-related impairments induced by dopamine D1 and D2 antagonism. Neuroscience. 2010;170:268–280. [PMC free article] [PubMed]
  139. Pardo M, Lopez-Cruz L, Valverde O, Ledent C, Baqi Y, Müller C.E, Salamone J.D. (submitted for publication) Adenosine A2A receptor antagonism and genetic deletion attenuate the effects of dopamine D2 antagonism on effort-based decision making in mice. [PubMed]
  140. Pardo M, Randall P.A, Nunes E.J, Lopez-Cruz L, Janniere S, Correa M, Salamone J.D. Effect of dopamine antagonism on effort-related decision making in rats responding on a progressive ratio/chow feeding concurrent choice task. Neuroscience Meeting Planner. Washington, DC: Society for Neuroscience, Online; 2011.
  141. Parkinson J.A, Dalley J.W, Cardinal R.N, Bamford A, Fehnert B, Lachenal G, Rudarakanchana N, Halkerston K.M, Robbins T.W, Everitt B.J. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behavioural Brain Research. 2002;137:149–163. [PubMed]
  142. Paterson N.E, Balci F, Campbell U, Olivier B.E, Hanania T. The triple reuptake inhibitor DOV216,303 exhibits limited antidepressant-like properties in the differential reinforcement of low-rate 72-second responding assay, likely due to dopamine reuptake inhibition. Journal of Psychopharmacology. 2010. online. [PubMed]
  143. Pavic L. Alterations in brain activation in posttraumatic stress disorder patients with severe hyperarousal symptoms and impulsive aggressiveness. European Archives of Psychiatry and Clinical Neuroscience. 2003;253:80–83. [PubMed]
  144. Phan K.L, Taylor S.F, Welsh R.C, Ho S.H, Britton J.C, Liberzon I. Neural correlates of individual ratings of emotional salience: a trial-related fMRI study. Neuroimage. 2004;21:768–780. [PubMed]
  145. Phillips P.E, Walton M.E, Jhou T.C. Calculating utility: preclinical evidence for cost-benefit analysis by mesolimbic dopamine. Psychopharmacology. 2007;191:483–495. [PubMed]
  146. Pinna A, Wardas J, Simola N, Morelli M. New therapies for the treatment of Parkinson’s disease: adenosine A2A receptor antagonists. Life Science. 2005;77:3259–3267. [PubMed]
  147. Pitts S.M, Horvitz J.C. Similar effects of D(1)/D(2) receptor blockade on feeding and locomotor behavior. Pharmacology, Biochemistry and Behavior. 2000;65:433–438. [PubMed]
  148. Pizzagalli D.A. The “anhedonia paradox” in schizophrenia: insights from affective neuroscience. Biological Psychiatry. 2010;67:899–901. [PMC free article] [PubMed]
  149. Premack D. Toward empirical behavior laws. I: Positive reinforcement. Psychological Review. 1959;66:219–33. [PubMed]
  150. Pruessner J.C, Champagne F, Meaney M.J, Dagher A. Dopamine release in response to a psychological stress in humans and its relationship to early life maternal care: a positron emission tomography study using [11C]raclopride. Journal of Neuroscience. 2004;24:2825–2831. [PubMed]
  151. Rachlin H. Economic concepts in the behavioral study of addiction. In: Vuchinich R.E, Heather N, editors. Choice, behavioral economics and addiction. Oxford, U.K: Elsevier; 2003. pp. 129–149. (Eds.)
  152. Randall P.A, Nunes E.J, Janniere S.L, Stopper C.M, Farrar A.M, Sager T.N, Baqi Y, Hockemeyer J, Müller C.E, Salamone J.D. Stimulant effects of adenosine antagonists on operant behavior: differential actions of selective A2A and A1 antagonists. Psychopharmacology. 2011;216:173–186. [PMC free article] [PubMed]
  153. Randall P.A, Pardo M, Nunes E.J, Lopez-Cruz L, Blodgett A, Lingiah K, Leser C, Vemuri V.K, Makriyannis A, Baqi Y, Müller C.E, Correa M, Salamone J.D. Effort-related choice behavior as assessed by a progressive ratio/chow feeding task: differential effects of DA D2 antagonism, adenosine A2A antagonism, cannabinoid CB1 antgonism and pre-feeding. Neuroscience Meeting Planner. Washington, DC: Society for Neuroscience, Online; 2011.
  154. Rick J.H, Horvitz J.C, Balsam P.D. Dopamine receptor blockade and extinction differentially affect behavioral variability. Behavioral Neuroscience. 2006;120:488–492. [PubMed]
  155. Robbins T.W, Everitt B.J. A role for mesencephalic dopamine in activation: commentary on Berridge (2006) Psychopharmacology. 2007;191:433–437. [PubMed]
  156. Robbins T.W, Koob G.F. Selective disruption of displacement behaviour by lesions of the mesolimbic dopamine system. Nature. 1980;285:409–412. [PubMed]
  157. Robbins T.W, Roberts D.C, Koob G.F. Effects of d-amphetamine and apomorphine upon operant behavior and schedule-induced licking in rats with 6-hydroxydopamine-induced lesions of the nucleus accumbens. Journal of Pharmacology and Experimental Therapeutics. 1983;224:662–673. [PubMed]
  158. Roitman M.F, Stuber G.D, Phillips P.E, Wightman R.M, Carelli R.M. Dopamine operates as a subsecond modulator of food seeking. Journal of Neuroscience. 2004;24:1265–1271. [PubMed]
  159. Rolls E.T, Rolls B.J, Kelly P.H, Shaw S.G, Wood R.J, Dale R. The relative attenuation of self-stimulation, eating and drinking produced by dopamine receptor blockade. Psychopharmacology. 1974;38:219–230. [PubMed]
  160. Rusk I.N, Cooper S.J. Parametric studies of selective D1 and D2 antagonists: effects on appetitive and feeding behavior. Behavioral Pharmacology. 1994;5:615–622. [PubMed]
  161. Salamone J.D. Different effects of haloperidol and extinction on instrumental behaviours. Psychopharmacology. 1986;88:18–23. [PubMed]
  162. Salamone J.D. The actions of neuroleptic drugs on appetitive instrumental behaviors. In: Iversen L.L, Iversen S.D, Snyder S.H, editors. Handbook of psychopharmacology. New York: Plenum Press; 1987. pp. 575–608. (Eds.)
  163. Salamone J.D. Dopaminergic involvement in activational aspects of motivation: effects of haloperidol on schedule induced activity, feeding and foraging in rats. Psychobiology. 1988;16:196–206.
  164. Salamone J.D. Complex motor and sensorimotor functions of striatal and accumbens dopamine: involvement in instrumental behavior processes. Psychopharmacology. 1992;107((2–3)):160–74. [PubMed]
  165. Salamone J.D. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behavioral Brain Research. 1994;61:117–133. [PubMed]
  166. Salamone J.D. The behavioral neurochemistry of motivation: methodological and conceptual issues in studies of the dynamic activity of nucleus accumbens dopamine. Journal of Neuroscience Methods. 1996;64:137–149. [PubMed]
  167. Salamone J.D. Will the last person who uses the term ‘reward’ please turn out the lights? Comments on processes related to reinforcement, learning, motivation, and effort. Addiction Biology. 2006;11(1):43–44. [PubMed]
  168. Salamone J.D. Involvement of nucleus accumbens dopamine in behavioral activation and effort-related functions. In: Iversen L.L, Iversen S.D, Dunnett S.B, Bjorkland A, editors. Dopamine handbook. Oxford, UK: Oxford University Press; 2010a. (Eds.)
  169. Salamone J.D. Motor function and motivation. In: Koob G, Le Moal M, Thompson R.F, editors. Encyclopedia of behavioral neuroscience, Vol. 3 (pp. 267–276) Oxford: Academic Press; 2010b. (Eds.)
  170. Salamone J.D, Aberman J.E, Sokolowski J.D, Cousins M.S. Nucleus accumbens dopamine and rate of responding: Neurochemical and behavioral studies. Psychobiology. 1999;27:236–47.
  171. Salamone J.D, Arizzi M, Sandoval M.D, Cervone K.M, Aberman J.E. Dopamine antagonsts alter response allocation but do not suppress appetite for food in rats: Contrast between the effects of SKF 83566, raclopride and fenfluramine on a concurrent choice task. Psychopharmacology. 2002;160:371–380. [PubMed]
  172. Salamone J.D, Betz A.J, Ishiwari K, Felsted J, Madson L, Mirante B, Clark K, Font L, Korbey S, Sager T.N, Hockemeyer J, Muller C.E. Tremorolytic effects of adenosine A2A antagonists: implications for parkinsonism. Frontiers in Biosciences. 2008;13:3594–3605. [PubMed]
  173. Salamone J.D, Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behavioural Brain Research. 2002;137((1–2)):3–25. [PubMed]
  174. Salamone J.D, Correa M, Farrar A, Mingote S.M. Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology. 2007;191:461–482. [PubMed]
  175. Salamone J.D, Correa M, Farrar A.M, Nunes E.J, Collins L.E. Role of dopamine-adenosine interactions in the brain circuitry regulating effort-related decision making: insights into pathological aspects of motivation. Future Neurology. 2010;5:377–392.
  176. Salamone J.D, Correa M, Mingote S, Weber S.M. Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. Journal of Pharmacology and Experimental Therapeutics. 2003;305:1–8. [PubMed]
  177. Salamone J.D, Correa M, Mingote S.M, Weber S.M. Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Current Opinion in Pharmacology. 2005;5:34–41. [PubMed]
  178. Salamone J.D, Correa M, Mingote S.M, Weber S.M, Farrar A.M. Nucleus accumbens dopamine and the forebrain circuitry involved in behavioral activation and effort-related decision making: implications for understanding anergia and psychomotor slowing in depression. Current Psychiatry Reviews. 2006;2:267–280.
  179. Salamone J.D, Cousins M.S, Bucher S. Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behavioural Brain Research. 1994;65:221–229. [PubMed]
  180. Salamone J.D, Cousins M.S, Maio C, Champion M, Turski T, Kovach J. Different behavioral effects of haloperidol, clozapine, and thioridazine in a concurrent lever pressing and feeding procedure. Psychopharmacology. 1996;125:105–112. [PubMed]
  181. Salamone J.D, Cousins M.S, Snyder B.J. Behavioral functions of nucleus accumbens dopamine: empirical and conceptual problems with the anhedonia hypothesis. Neuroscience and Biobehavioral Reviews. 1997;21:341–359. [PubMed]
  182. Salamone J.D, Farrar A.M, Font L, Patel V, Schlar D.E, Nunes E.J, Collins L.E, Sager T.N. Differential actions of adenosine A1 and A2A antagonists on the effort-related effects of dopamine D2 antagonism. Behavioural Brain Research. 2009;201:216–222. [PMC free article] [PubMed]
  183. Salamone J.D, Ishiwari K, Betz A.J, Farrar A.M, Mingote S.M, Font L, Hockemeyer J, Müller C.E, Correa M. Dopamine/adenosine interactions related to locomotion and tremor in animal models: Possible relevance to parkinsonism. Parkinson’s Disease and Related Disorders. 2008;14:S130–S134. [PMC free article] [PubMed]
  184. Salamone J.D, Kurth P, McCullough L.D, Sokolowski J.D. The effects of nucleus accumbens dopamine depletions on continuously reinforced operant responding: contrasts with the effects of extinction. Pharmacology Biochemistry and Behavior. 1995;50:437–443. [PubMed]
  185. Salamone J.D, Kurth P.A, McCullough L.D, Sokolowski J.D, Cousins M.S. The role of brain dopamine in response initiation: effects of haloperidol and regionally specific dopamine depletions on the local rate of instrumental responding. Brain Research. 1993;628:218–226. [PubMed]
  186. Salamone J.D, Mahan K, Rogers S. Ventrolateral striatal dopamine depletions impair feeding and food handling in rats. Pharmacology Biochemistry and Behavior. 1993;44:605–610. [PubMed]
  187. Salamone J.D, Steinpreis R.E, McCullough L.D, Smith P, Grebel D, Mahan K. Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology. 1991;104:515–521. [PubMed]
  188. Salamone J.D, Wisniecki A, Carlson B.B, Correa M. Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience. 2001;105:863–870. [PubMed]
  189. Sanchis-Segura C, Spanagel R. Behavioural assessment of drug reinforcement and addictive features in rodents: an overview. Addiction Biology. 2006;11:2–38. [PubMed]
  190. Sarchiapone M, Carli V, Camardese G, Cuomo C, Di Guida D, Calgagni M.L, Focacci C, De Riso S. Dopamine transporter binding in depressed patients with anhedonia. Psychiatric Research: Neuroimaging. 2006;147:243–248. [PubMed]
  191. Schiffmann S.N, Jacobs O, Vanderhaeghen J.J. Striatal restricted adenosine A2A receptor (RDC8) is expressed by enkephalin but not by substance P neurons: an in situ hybridization histochemistry study. Journal of Neurochemistry. 1991;57:1062–1071. [PubMed]
  192. Schmelzeis M.C, Mittleman G. The hippocampus and reward: effects of hippocampal lesions on progressive-ratio responding. Behavioral Neuroscience. 1996;110:1049–1066. [PubMed]
  193. Schoenbaum G, Setlow B. Lesions of nucleus accumbens disrupt learning about aversive outcomes. Journal of Neuroscience. 2003;23(30):9833–9841. [PubMed]
  194. Schultz W. Multiple dopamine functions at different time courses. Annual Review of Neuroscience. 2007a;30:259–288. [PubMed]
  195. Schultz W. Behavioral dopamine signals. Trends in Neurosciences. 2007b;30:203–210. [PubMed]
  196. Schwab R.S. Akinesia paradoxica. Electroencephalography and Clinical Neurophysiology. 1972;31:87–92.
  197. Schweimer J, Hauber W. Dopamine D1 receptors in the anterior cingulate cortex regulate effort-related decision making. Learning and Memory. 2006;13:777–782. [PMC free article] [PubMed]
  198. Segovia K.N, Correa M, Salamone J.D. Slow phasic changes in nucleus accumbens dopamine release during fixed ratio acquisition: a microdialysis study. Neuroscience. 2011;196:188–198. [PubMed]
  199. Sink K.S, Vemuri V.K, Olszewska T, Makriyannis A, Salamone J.D. Cannabinoid CB1 antagonists and dopamine antagonists produce different effects on a task involving response allocation and effort-related choice in food-seeking behavior. Psychopharmacology. 2008;196:565–574. [PMC free article] [PubMed]
  200. Skinner B.F. Science and human behavior. New York: Macmillan; 1953.
  201. Skjoldager P, Pierre P.J, Mittlman G. Reinforcer magnitude and progressive ratio responding: Effects of increased effort, prefeeding and extinction. Learning and Motivation. 1993;24:303–343.
  202. Sokolowski J.D, Conlan A.N, Salamone J.D. A microdialysis study of nucleus accumbens core and shell dopamine during operant responding in the rat. Neuroscience. 1998;86:1001–1009. [PubMed]
  203. Sokolowski J.D, Salamone J.D. The role of nucleus accumbens dopamine in lever pressing and response allocation: Effects of 6-OHDA injected into core and dorsomedial shell. Pharmacology Biochemistry Behavior. 1998;59:557–566. [PubMed]
  204. Spivak K.J, Amit Z. Effects of pimozide on appetitive behavior and locomotor activity: Dissimilarity of effects when compared to extinction. Physiological Behavior. 1986;36:457–463. [PubMed]
  205. Staddon J.E.R. Operant behavior as adaptation to constraint. Journal of Experimental Psychology: General. 1979;108:48–67.
  206. Staddon J.E.R, Ettenger R.H. Learning: An introduction to the principles of adaptive behavior. New York: Harcourt Brace Jovanovitch; 1989.
  207. Stahl S.M. The psychopharmacology of energy and fatigue. Journal of Clinical Psychiatry. 2002;63:7–8. [PubMed]
  208. Stewart W.J. Progressive reinforcement schedules: A review and evaluation. Australian Journal of Psychology. 1975;27:9–22.
  209. Svenningsson P, Le Moine C, Fisone G, Fredholm B.B. Distribution, biochemistry and function of striatal adenosine A2A receptors. Progress in Neurobiology. 1999;59:355–396. [PubMed]
  210. Takahashi R.N, Pamplona F.A, Prediger R.D. Adenosine receptor antagonists for cognitive dysfunction: a review of animal studies. Frontiers in Bioscience. 2008;13:2614–2632. [PubMed]
  211. Tapp J.T. Activity, reactivity, and the behavior-directing properties of stimuli. In: Tapp J.T, editor. Reinforcement and behavior. New York: Academic Press; 1969. pp. 387–416. (Ed.)
  212. Timberlake W. Behavior systems and reinforcement: An integrative approach. Journal of the Experimental Analysis of Behavior. 1993;60:105–128. [PMC free article] [PubMed]
  213. Treadway M.T, Zald D.H. Reconsidering anhedonia in depression: Lessons from translational neuroscience. Neuroscience and Biobehavioral Reviews. 2011;35:537–555. [PMC free article] [PubMed]
  214. Tustin R.D. Assessing preference for reinforcers using demand curves, work-rate functions, and expansion paths. Journal of the Experimental Analysis of Behavior. 1995;64:313–329. [PMC free article] [PubMed]
  215. Ungerstedt U. Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiologica Scandinavia Supplementum. 1971;367:95–122. [PubMed]
  216. Van den Bos R, van der Harst J, Jonkman S, Schilders M, Spruijt B. Rats assess costs and benefits according to an internal standard. Behavioural Brain Research. 2006;171:350–354. [PubMed]
  217. Venugopalan V.V, Casey K.F, O’Hara C, O’Loughlin J, Benkelfat C, Fellows L.K, Leyton M. Acute phenylalanine/tyrosine depletion reduces motivation to smoke cigarettes across stages of addiction. Neuropsychopharmacology. 2011;36:2469–2476. [PMC free article] [PubMed]
  218. Vezina P, Lorrain D.S, Arnold G.M, Austin J.D, Suto N. Sensitization of midbrain dopamine neuron reactivity promotes the pursuit of amphetamine. The Journal of Neuroscience. 2002;22:4654–4662. [PubMed]
  219. Vuchinich R.E, Heather N. Introduction: Overview of behavioural economic perspectives on substance use and addiction. In: Vuchinich R.E, Heather N, editors. Choice, behavioral economics and addiction. Oxford, U.K: Elsevier; 2003. pp. 1–31. (Eds.)
  220. Wachtel S.R, Ortengren A, de Wit H. The effects of acute haloperidol or risperidone on subjective responses to methamphetamine in healthy volunteers. Drug and Alcohol Dependence. 2002;68:23–33. [PubMed]
  221. Wade T.R, de Wit H, Richards J.B. Effects of dopaminergic drugs on delayed reward as a measure of impulsive behavior in rats. Psychopharmacology. 2000;150:90–101. [PubMed]
  222. Wakabayashi K.T, Fields H.L, Nicola S.M. Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behavioural Brain Research. 2004;154:19–30. [PubMed]
  223. Wallace M, Singer G, Finlay J, Gibson S. The effect of 6-OHDA lesions of the nucleus accumbens septum on schedule-induced drinking, wheelrunning and corticosterone levels in the rat. Pharmacology, Biochemistry, and Behavior. 1983;18:129–136. [PubMed]
  224. Walton M.E, Bannerman D.M, Alterescu K, Rushworth M.F.S. Functional specialization within medial frontal cortex of the anterior cingulated for evaluating effort-related decisions. Journal of Neuroscience. 2003;23:6475–6479. [PubMed]
  225. Walton M.E, Bannerman D.M, Rushworth M.F. The role of rat medial frontal cortex in effort-based decision making. Journal of Neuroscience. 2002;22:10996–11003. [PubMed]
  226. Walton M.E, Kennerley S.W, Bannerman D.M, Phillips P.E, Rushworth M.F. Weighing up the benefits of work: behavioral and neural analyses of effort-related decision making. Neural Network. 2006;19:1302–1314. [PMC free article] [PubMed]
  227. Ward S.J, Morgan D, Roberts D.C. Comparison of the reinforcing effects of cocaine and cocaine/heroin combinations under progressive ratio and choice schedules in rats. Neuropsychopharmacology. 2005;30:286–295. [PubMed]
  228. Wardas J, Konieczny J, Lorenc-Koci E. SCH 58261, an A2A adenosine receptor antagonist, counteracts parkinsonian-like muscle rigidity in rats. Synapse. 2001;41:160–171. [PubMed]
  229. Weinstock L.M, Munroe M.K, Miller I.W. Behavioral activation for the treatment of atypical depression: a pilot open trial. Behavior Modification. 2011;35:403–424. [PMC free article] [PubMed]
  230. Williams B.A. Reinforcement, choice, and response strength. In: Atkinson R.C, Herrnstein R.J, Lindsey G, Luce R.D, editors. Stevens’ handbook of experimental psychology, vol. 2. New York: John Wiley and Sons; 1988. pp. 167–174. (Eds.)
  231. Willner P, Chawala K, Sampson D, Sophokleous S, Muscat R. Tests of function equivalence between pimozide pretreatment, extinction and free feeding. Psychopharmacology. 1988;95:423–426. [PubMed]
  232. Winstanley C.A, Theobald D.E.H, Dalley J.W, Robbins T.W. Interactions between serotonin and dopamine in the control of impulsive choice in rats: therapeutic implications for impulse control disorders. Neuropsychopharmacology. 2005;30:669–682. [PubMed]
  233. Wirtshafter D, Asin K.E. Haloperidol and nonreinforcement produce different patterns of response slowing in a food reinforced runway task. Pharmacology, Biochemistry, and Behavior. 1985;22:661–663. [PubMed]
  234. Wise R.A. Dopamine, learning and motivation. Nature Reviews in Neuroscience. 2004;5:483–494. [PubMed]
  235. Wise R.A, Spindler J, de Wit H, Gerberg G.J. Neuroleptic-induced “anhedonia” in rats: pimozide blocks reward quality of food. Science. 1978;201:262–264. [PubMed]
  236. Woolverton W.L, Ranaldi R. Comparison of the reinforcing efficacy of two dopamine D2-like receptor agonists in rhesus monkeys using a progressive-ratio schedule of reinforcement. Pharmacology, Biochemistry, and Behavior. 2002;72:803–809. [PubMed]
  237. Worden L.T, Shahriari M, Farrar A.M, Sink K.S, Hockemeyer J, Müller C, Salamone J.D. The adenosine A2A antagonist MSX-3 reverses the effort-related effects of dopamine blockade: differential interaction with D1 and D2 family antagonists. Psychopharmacology. 2009;203:489–499. [PMC free article] [PubMed]
  238. Wyvell C.L, Berridge K.C. Incentive sensitization by previous amphetamine exposure: increased cue-triggered “wanting” for sucrose reward. Journal of Neuroscience. 2001;21:7831–7840. [PubMed]
  239. Yin H.H, Ostlund S.B, Balleine B.W. Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. European Journal of Neuroscience. 2008;28:1437–1448. [PMC free article] [PubMed]
  240. Young A.M. Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. Journal of Neuroscience Methods. 2004;138:57–63. [PubMed]