Individual differences in the attribution of incentive salience to reward-related cues: implications for addiction (2009)

Summary

Introduction

The major clinical problem in the treatment of addiction is the tendency of addicts to relapse even long after the discontinuation of drug use. In fact, the most predictable outcome of a first diagnosis of addiction is a 90% chance of relapse (DeJong, 1994), and this is due in part to sensitivity to drug-associated stimuli. For example, when addicts encounter cues previously associated with drug administration (people, places, paraphernalia, etc.) this often instigates renewed drug-seeking and/or craving for the drug (for review see Childress et al., 1993). One reason drug-related cues are thought to have these effects is because when otherwise neutral environmental stimuli are repeatedly paired with the administration of rewards, including potentially addictive drugs, such stimuli not only act as predictors of impending reward, but through Pavlovian (stimulus-stimulus) learning they acquire incentive motivational properties—they become imbued with “incentive salience” (Berridge, 2001; Bindra, 1978; Bolles, 1972; Toates, 1986). The attribution of incentive salience is defined by Berridge (1996) as the transformation of an otherwise relatively neutral perceptual or representational event (e.g., cue) into an attractive and “wanted” incentive stimulus. That is, through Pavlovian learning sensory information about rewards and their cues (sights, sounds and smells) is transformed into “attractive, desired, riveting incentives” (Berridge and Robinson, 2003). Consequentially, conditional stimuli can become become “motivational magnets” (Berridge, 2001) eliciting approach towards them, as in the case of Pavlovian conditional approach behavior towards rewards and their signals (see below and Cardinal et al., 2002a). Tomie (1996) has argued that when such cues are embedded in the device that delivers a drug, such as specialized glassware used to consume alcohol or pipes used to smoke cocaine, these cues can promote especially compulsive patterns of behavior characteristic of addiction.

It is thought that the incentive motivational properties of Pavlovian cues, their ability to goad an individual into action, are primarily responsible for the ability of drug-associated cues (and places) both to maintain drug-seeking in the absence of the drug (Everitt and Robbins, 2000; Schindler et al., 2002) and to reinstate drug-seeking behavior or relapse during abstinence (e.g., Kruzich et al., 2001). However, we would be remiss if we did not mention that there are other ways that cues (and especially contexts) associated with drugs can influence drug-seeking behavior and relapse. For example, drug-associated cues can act as discriminative stimuli, “setting the occasion” for drug-taking behavior (Skinner, 1938). Occasion-setters are conditional cues or contexts that may not themselves elicit a response or reinforce an action, but act in a hierarchical manner to modulate either instrumental responding to obtain a reward and/or the occurrence of Pavlovian conditional responses (Holland, 1992; Rescorla, 1988; Schmajuk and Holland, 1998). Such stimuli can have a powerful influence on drug-seeking behavior. For example, (Ciccocioppo et al., 2004) reported that a discriminative stimulus associated with a single cocaine self-administration session can elicit robust cocaine-seeking behavior up to one year after the last experience with cocaine. Similarly, contexts often serve to set the occasion for renewed drug-seeking (Crombag and Shaham, 2002) and can even modulate the expression of behavioral sensitization (Anagnostaras and Robinson, 1996; Anagnostaras et al., 2002). Of course, exposure to a drug itself (a drug prime) may be effective in producing relapse/reinstatement in part because the interoceptive cues produced by the drug can also “set the occasion” for further drug-seeking, although this begs the question as to what psychological processes led to the first drink, puff or injection. The extent to which these “higher-order” conditional stimuli act as incentive stimuli is not well understood.

Many theories of addiction have recognized the role of Pavlovian conditional motivation in attracting humans towards drug-associated stimuli and sources of addictive drugs, and in relapse (Di Chiara, 1998; Everitt and Robbins, 2005; Robinson and Berridge, 1993; Stewart et al., 1984; Tomie, 1996; Tomie et al., 2007), yet there have been very few attempts to directly study this process. In one example See and colleagues (Kruzich et al., 2001) reported that classically conditioned cues can reinstate drug-seeking behavior. But in most pre-clinical studies on how cues influence drug-seeking and -taking, the initial association between a cue and drug delivery occurs in an instrumental learning setting, where both the cue and drug are presented contingent upon an action. In these studies, for example, an animal will be trained to lever press in order to receive a drug infusion, which is accompanied by the presentation of a cue (e.g., light). Cue-induced reinstatement is later studied under extinction conditions (when the drug is no longer available), and after responding subsides the ability of the drug-associated cue (e.g., light) to reinstate drug-seeking behavior is examined. Thus, during tests of reinstatement the action of lever pressing behavior, that previously produced the drug and light, now results in presentation of the light alone (for review see Shaham et al., 2003). The cue, therefore, can act as a conditional (secondary) reinforcer, reinforcing the action that produces it. In humans, however, environmental cues associated with injection of a drug are usually present prior to injecting the drug—they do not suddenly appear as a consequence of taking the drug. Moreover, in addicts, cues that produce relapse or craving generally precede actions to acquire the drug, they do not follow them; i.e., they act as incentive stimuli goading the individual to action rather than reinforcing an action that has already been emitted (Stewart et al., 1984).

There is, therefore, a dearth of information on the effects of Pavlovian conditional cues on drug-seeking or drug-taking behavior. Most studies in the field involve procedures that make it very difficult to determine the extent to which behavior is controlled by instrumental reinforcement (reinforcing an action that has already occurred) vs. Pavlovian incentives that instigate actions. This distinction is important because the psychological and neurobiological processes underlying instrumental learning may be very different from Pavlovian processes that confer incentive value upon drug-associated cues and places (Cardinal et al., 2002a; Tomie et al., 2007). Thus, in trying to understand how drug cues can maintain drug-seeking behavior for long periods of time even in the absence of the drug, and how they can precipitate relapse, we need to parse out the basic psychological processes by which such stimuli acquire incentive salience (Berridge and Robinson, 2003), and in turn delineate the underlying neural substrates. For this reason we have recently begun to explore the ability of Pavlovian cues to control behavior (Flagel et al., 2008; Flagel et al., 2007; Uslaner et al., 2006), and especially individual differences in how such cues influence behavior. This work is preliminary, but we hope it will eventually provide insights into the psychological and neurobiological mechanisms by which drug-related cues gain the ability to control behavior in addiction. In the following we first discuss some of the historical literature on how researchers have studied the attribution of incentive salience to reward-related cues, and how this is revealed in behavior, and then our recent studies on individual differences.

Sign-tracking

In classical Pavlovian conditioning the presentation of a cue (conditional stimulus, CS) is associated with the presentation of a reward (the unconditional stimulus, US), and with repeated pairings the CS comes to elicit a conditional response (CR). In Pavlov’s original studies using a food reward his dogs were typically restrained and the response to the CS often measured was salivation, giving the appearance that the CS evoked a simple reflexive response similar to the unconditional response (UR) produced by the US. However, when animals were freed from their restraints and their behavior observed, it became obvious that what was learned was not just a simple reflexive response, because in this situation the CS evoked complex patterns of behavior, including food-begging behavior (H. Liddell, unpublished, cited in Timberlake and Grant, 1975). Many subsequent studies have shown that in addition to simple responses, Pavlovian CSs evoke complex emotional and motivational states that can be manifest in a variety of ways (Rescorla, 1988). Thus, when animals are free to move in their environment it is often observed that the CR consists of initial orientation to the CS (the cue or “sign”) followed by approach towards it, and frequently engagement with it (Brown and Jenkins, 1968; Hearst and Jenkins, 1974). Collectively, these CS-directed responses were called “sign tracking”, because behaviors were directed towards the cue, or “sign”, that predicted reward. Remarkably, sign tracking develops even though no action is required for the animals to receive the reward; that is, no actions are reinforced. (Note that the term “autoshaping” is often used to describe the procedure that produces this form of Pavlovian conditional approach behavior, although “autoshaping” is really a misnomer in that in the Pavlovian procedure no response is reinforced or “shaped”; Hearst and Jenkins, 1974).

With autoshaping the exact topography of the CR depends on the species, and the nature of both the CS and the US. It was originally noted that there is often a striking similarity between the behavioral patterns involved in consuming a reward and those directed towards the CS. In his studies with dogs and a food reward Pavlov (1932) stated, “…the animal may lick the electric lamp [CS], or appear to take the air into its mouth, or to eat the sound…licking his lips and making the noise of chewing with his teeth as though it were a matter of having the food itself” (pg. 95). Similarly, if presentation of a lever is immediately followed by the response-independent delivery of a food pellet, some rats will approach and often grasp and gnaw the lever as if it were itself food (Davey and Cleland, 1982). And when pigeons are exposed to a key-light (CS) that has been paired with presentation of water (US) they exhibit a drinking-specific motor pattern (complete with gullet movement) directed at the key-light. This pattern of behavior is distinct from the response that emerges in pigeons following pairings with food reward (Jenkins and Moore, 1973; see also http://go.owu.edu/~deswartz/introduction.html). Remarkably, when a CS is paired with the opportunity to copulate with a female (US), male Japanese quail (Coturnix japonica) will, under some conditions, come to approach and copulate with the inanimate object CS (Koksal et al., 2004, see also Burns and Domjan, 1996; Burns and Domjan, 2001).

The fact that presentation of the CS often leads to a CR that resembles behavior elicited by the reward itself (e.g., consummatory behavior) led to the notion that the CS acts as a surrogate for the US—“stimulus-substitution” (Pavlov, 1927; Staddon and Simmelhag, 1971). Although this is consistent with the theory that the CS takes on the incentive properties of the reward, in the past some have argued that such behavior is a mere reflection of sensorimotor conditioning in the absence of any motivational processes (see Berridge, 2001). However, we now know that CSs that evoke these consummatory reactions acquire the three fundamental properties of incentive stimuli (Berridge, 2001; Cardinal et al., 2002a): (1) they have to ability to elicit approach towards them as indicated by Pavlovian conditional approach or sign-tracking behavior (e.g., Flagel et al., 2008; Hearst and Jenkins, 1974; Peterson et al., 1972); (2) they can energize ongoing instrumental actions, as indicated by the Pavlovian-to-instrumental transfer effect (e.g., Dickinson et al., 2000; Lovibond, 1983; Wyvell and Berridge, 2000); and (3) they can reinforce the learning of new instrumental actions; that is, they can act as conditional reinforcers (e.g., Di Ciano and Everitt, 2004; Williams and Dunn, 1991). Moreover, there are many examples illustrating that the form of the CR is influenced by the nature of the CS (see Holland, 1977), suggesting that sign-tracking behavior is not due to simple stimulus-substitution. For example, the form of the CR to a CS that predicts food is very different if the CS is a lever, or a live rat, or a block of wood (Timberlake and Grant, 1975). Thus, the emergent CR is thought to reflect the activation of complex motivational processes that are under the control of a number of factors, including the nature of the US and the CS (Buzsaki, 1982; Davey et al., 1984; Jenkins and Moore, 1973; Moore, 1973; Timberlake and Lucas, 1985).

The phenomenon of sign-tracking has been well characterized with natural rewards used as the US (for review see Boakes, 1977; Hearst and Jenkins, 1974; Tomie et al., 1989) and there are many examples where reward- related cues become so irresistibly attractive that they engender seemingly maladaptive and arguably compulsive behavior (Boakes et al., 1978; Breland and Breland, 1961; Hearst and Jenkins, 1974; Williams and Williams, 1969). In a classic example, termed the “misbehavior of organisms” (Breland and Breland, 1961, 1966), raccoons were trained to deposit a wooden coin through a slot in order to obtain a food reward. The raccoons initially performed this task without hesitation, but with further training seemed unable to let go of the coin, spending several minutes compulsively handling it with their forepaws—chewing, licking, rubbing and washing the coin—as if they were trying to clean a morsel of food —and repeatedly putting the coin into the slot but then pulling it back out without releasing it. The coin itself appeared to have great incentive value, as the raccoons were very reluctant to give it up, even though holding onto it delayed or even prevented receipt of actual food. Another interesting example, termed “negative automaintenance”, was originally described by Hearst and Jenkins (1974), who paired illumination of a key light at one end of a long box with subsequent delivery of a food reward at the other end. Although no response was required to receive the food, pigeons began to approach and repeatedly peck the key light, even though doing so prevented them from retrieving the food, which was available at the other end of the box for only a limited amount of time. This behavior persisted despite the fact that the subjects were often quite hungry. One would expect that if the pigeons were able to quit responding to the distant CS (and consequentially retrieve and consume the reward) they would do so. This apparently compulsive pattern of responding is similar to that seen in addiction in that it is triggered by an impulse determined by past experience and seems to be relatively independent of volition (Tomie, 1996). Moreover, the lack of inhibitory control (i.e., maladaptive behavior) apparent in animals that sign-track is a defining attribute of impulsivity and further links this behavior with addictive behaviors and other impulse control disorders (Tomie, 1996; Tomie et al., 1998).

The studies described above are in agreement with a number of others that have shown that sign-tracking is not due to “accidental reinforcement” of the response and that is persists even when it leads to loss of reinforcement (see Gamzu and Williams, 1971; Killeen, 2003; Lajoie and Bindra, 1976; Timberlake and Lucas, 1985). Despite these findings, some researchers have questioned the role of Pavlovian (stimulus-stimulus) processes in sign-tracking behavior and have claimed that such behavior may be due to response reinforcement (e.g., see Farwell and Ayres, 1979; Locurto et al., 1976; Locurto, 1981; Myerson et al., 1979; Sanabria et al., 2006; Wessels, 1974). For example, after autoshaping training whereby rats developed a sign-tracking CR, Locurto and colleagues (1976) transferred the animals to an omission schedule. On this schedule if the animals contacted the CS-lever the food reward was withheld. Previous studies suggested that in this situation pigeons continue to contact a CS-key light, although at a lower rate, which was taken as evidence that the behavior was not controlled by response reinforcement (e.g., Williams and Williams, 1969). In contrast, Locurto et al. (1976) reported that under these conditions rats stopped contacting the CS-lever, which they took to indicate that the behavior was mediated by response reinforcement. However, even in this latter study there is clear evidence that the CS-lever maintained its Pavlovian conditional motivational properties because Locurto and colleagues (1976) also reported that the topography of the conditional response differed depending on the schedule of reinforcement (i.e., autoshaping vs. omission). Prior to instituting the omission schedule the CR consisted of approach to the lever and contact with it (i.e., gnawing and biting it). During the omission schedule the animals continued to approach and exhibit investigatory behavior (i.e., sniffing) directed towards the CS-lever, but simply stopped physically contacting it. These findings suggest that even under omission conditions the CS-lever continued to have incentive properties—the rats continued to approach and direct their attention to it, and what they learned was to avoid directly contacting it. They withheld only the terminal link in the chain of behaviors that comprised the Pavlovian CR (see also Stiers and Silberberg, 1974). Thus, even these data are consistent with the conclusion that a sign-tracking CR (approach) is dependent upon Pavlovian learning and not response reinforcement, and further illustrate the ability of Pavlovian cues to maintain behavior even in the absence of response reinforcement.

Sign-tracking to drug-associated cues

Tomie (1996) was amongst the first to describe the similarities between Pavlovian sign-tracking behavior and symptoms of drug abuse, and in recent years drug abuse researchers have begun to further explore this relationship (Flagel et al., 2008; Flagel et al., 2006; Newlin, 2002; Tomie et al., 2007). There are, however, still very few studies on the extent to which drug-associated cues support a sign-tracking CR (Cunningham and Patel, 2007; Kearns et al., 2006; Krank et al., 2008; Uslaner et al., 2006). Indeed, in a 2005 review Everitt and Robbins duly noted (pg. 1482), “it might logically be thought that Pavlovian approach is involved in maladaptively attracting humans toward sources of addictive drug reinforcers…as emphasized in the incentive salience theory of addiction. However…approach to a CS predictive of a drug…has [not] been clearly demonstrated in laboratory studies…although… [it] is readily seen in animals responding for natural rewards. It may be…that the behavioral influence of CSs associated with drugs and natural reinforcers differ fundamentally in this regard”. This idea was initially supported by a report that rats do not approach discrete cues paired with i.v. cocaine delivery (Kearns and Weiss, 2004). If it is true that “drugs and natural reinforcers differ fundamentally in this regard” a number of theories of addiction would require serious revision.

However, there have now been a number of reports that animals do approach discrete cues that have been associated with drug delivery (Cunningham and Patel, 2007; Krank et al., 2008; Uslaner et al., 2006). For example, a US consisting of an orally consumed ethanol/saccharin (Krank, 2003; Tomie, 2001; Tomie et al., 2003) or amphetamine/saccharin solution (Tomie, 2001) can support sign-tracking behavior. One concern with these studies is the possibility that it was the sweet solution rather than the alcohol that supported approach, although there were attempts to control for this variable. More recently, Krank and colleagues (2008) unambiguously demonstrated the ability of an ethanol-paired cue (light) to elicit sign-tracking behavior in rats using unsweetened ethanol solution as the US. In addition, Pavlovian conditional approach to a visual cue associated with intraperitoneal injections of ethanol has recently been demonstrated in mice using a modified conditioned place preference procedure thought to reflect sign-tracking behavior (Cunningham and Patel, 2007). Finally, an ethanol-associated cue has also been shown to produce Pavlovian-instrumental transfer effects (Corbit and Janak, 2007).

To our knowledge the first study to demonstrate that the i.v. administration of a drug could support sign-tracking to a discrete cue was by Uslaner et al. (2006). In this experiment an 8 sec presentation of an illuminated retractable lever (CS) was paired with the response-independent delivery of an intravenous injection of cocaine (US). Eight CS-US pairings were presented on a schedule with a randomly varying inter-trial interval with a mean of 900 sec. With repeated pairings of the lever and cocaine rats began to approach the lever more reliably and more rapidly, whereas rats that received pseudorandom (i.e., unpaired) CS-US presentations did not. These findings are in contrast to those reported by Kearns and Weiss (2004), and we speculate that these disparate results are due to methodological differences (for details see Uslaner et al., 2006). Kearns and Weiss (2004) used relatively short inter-trial intervals, with CS-US presentations occurring on average every 90 sec. In contrast to food reward, neurobiological and interoceptive effects of cocaine persist for longer than 90 sec. It is therefore possible that the effects of the previous drug infusion were still being experienced upon subsequent CS-US pairings, making it difficult for the rats to associate these events. Indeed, Uslaner and colleagues (2006) successfully used the longer inter-trial interval of 900 sec only after failing to observe sign-tracking behavior with shorter inter-trial intervals.

In an independent (unpublished) study using selectively-bred rat lines (Stead et al., 2006), we have recently found that animals that sign-track to a CS associated with food reward also sign-track to a cocaine-paired CS; whereas goal-trackers do not. These findings suggest that individual differences in the control over behavior by cues predictive of food rewards may also apply to drug rewards. It is important to note, however, that the topography of the CR that emerges is quite different when cocaine vs. food is used as the CS. When cocaine is used as the US animals rarely come into contact with the lever, but exhibit clear sign-tracking behavior consisting of orientation to the lever followed by approach and exploration in the immediate vicinity of the extended lever (Uslaner et al., 2006), similar to the CR seen when intracranial electrical stimulation is used as the US (Peterson, 1975; Peterson et al., 1972; Phillips et al., 1981; Wilkie and McDonald, 1978).

Individual differences in the attribution of incentive salience to reward-related cues: sign-tracking vs. goal-tracking

As mentioned above, when a discrete cue (CS) is repeatedly presented in association with delivery of a reward (US) a number of different responses can emerge that are conditional upon this relationship (CRs), and the topography of the CR is dependent on a number of factors, including the species or strain of the animal (Boakes, 1977; Kearns et al., 2006; Kemenes and Benjamin, 1989; Nilsson et al., 2008; Purdy et al., 1999), the nature of the CS and US (Burns and Domjan, 1996; Davey and Cleland, 1982; Davey et al., 1984; Domjan et al., 1988; Holland, 1977; Jenkins and Moore, 1973; Peterson et al., 1972; Schwam and Gamzu, 1975; Timberlake and Grant, 1975; Wasserman, 1973), the spatial and temporal contingencies between the CS and US (Brown et al., 1993; Costa and Boakes, 2007; Holland, 1980a; Silva et al., 1992; Timberlake and Lucas, 1985), prior experience with the CS or US (Boughner and Papini, 2003; Engberg et al., 1972; Gamzu and Williams, 1971) and the internal drive state of the animal (Berridge, 2001; Boakes, 1977; Davey and Cleland, 1982; Toates, 1986; Zamble et al., 1985; Zener, 1937). Taken together, all of these factors contribute to the incentive motivational state of the animal and thus affect the degree to which the CS acquires incentive motivational properties (Lajoie and Bindra, 1976).

However, even when trained under identical conditions there can be large individual differences in the nature of the CR. To our knowledge, Zener (1937) was the first to systematically describe such individual differences when animals were trained using classic Pavlovian conditioning procedures. Zener (1937) paired ringing a bell with food delivery, but in contrast to most of Pavlov’s (1932) original experiments the dogs in Zener’s studies were not restrained. This not only revealed the complexity of the CRs that develop, but also large individual differences. Zener (1937) reported that after training some dogs responded to the CS with “an initial glance at the bell” followed by “a constant fixation…to the food pan…”, whereas others exhibited a “small but definite movement of approach toward the conditioned stimulus…followed by a backing up later to a position to eat”, a sequence described by Zener as a “striking phenomenon” (pg. 391). Still other dogs vacillated, looking back and forth between the bell and the food pan.

Individual variation in the topography of CRs evoked by a stimulus predictive of food did not receive much further attention until 1977 when Boakes described similar individual differences in rats. Boakes (1977) used a standard autoshaping procedure consisting of brief illumination of a lever immediately followed by the non-contingent delivery of a food pellet. He reported that during CS presentation some rats approached and contacted the lever and went to the food tray after CS offset, similar to what had been described previously (Hearst and Jenkins, 1974). However, upon CS presentation other rats immediately went to the place where food would be delivered, and during the CS period repeatedly operated the tray flap. Boakes (1977), like his predecessors (Hearst and Jenkins, 1974), called CS-evoked approach to the cue (or sign) “sign-tracking” and, for the sake of symmetry, referred to CS-evoked approach to the location where the food would be delivered “goal-tracking,” and we will continue to use this terminology here. Interestingly, the probability of either sign-tracking or goal-tracking was conditional upon the presence of the illuminated lever (CS) and decreased when food (US) was removed or was presented randomly with respect to the CS (Boakes, 1977). Boakes (1977) also found that changes in the probability of reinforcement could significantly affect which behavioral response emerged. Delivering food on only half of the trials was more effective in producing a sign-tracking CR than if food was delivered on every trial, which increased the goal-tracking CR. Altering the level of food deprivation of the subjects had a lesser, but measurable effect (Boakes, 1977).

Given the importance ascribed to Pavlovian conditional processes in controlling behavior, and in addiction—the topic has been the subject of numerous studies and reviews (e.g., Cardinal et al., 2002a; Day and Carelli, 2007; Tomie, 1996; Tomie et al., 2007)—it is surprising there have been very few studies directly comparing sign-tracking vs. goal-tracking CRs since the initial studies by Zener (1937) and Boakes (1977). One explanation for this may be the fact that in most studies of Pavlovian conditional approach behavior the conditions used either do not allow the simultaneous assessment of sign-tracking and goal-tracking, or favor the development of only one of the two CRs (for exceptions see Burns and Domjan, 1996; Nilsson et al., 2008). There have, however, been a few studies on the conditions that favor a sign-tracking vs. goal-tracking CR. For example, using rats, Davey, Cleland and colleagues (Cleland and Davey, 1983; Davey and Cleland, 1982; Davey et al., 1981) reported that sign-tracking occurs to a discrete visual stimulus paired with food reward, but not to a localizable auditory CS. The predominant conditional response to an auditory stimulus is goal-tracking, with little approach behavior directed towards the CS. It has also been established that increasing the temporal or spatial distance between the CS and US results in a decline in the sign-tracking response and an increase in the goal-tracking response (Brown et al., 1993; Costa and Boakes, 2007; Holland, 1980a, b; Silva et al., 1992).

Although there has been some research on the conditions that lead to the development of a sign-tracking vs. a goal-tracking CR, there has been almost no attention paid to individual differences in the propensity to develop one or the other CR. We think this may be an important area for study because it provides a way to evaluate individual differences in the propensity to attribute incentive value to cues that are predictive of rewards. Tomie et al. (2000) has reported individual differences in lever press CR performance (i.e., propensity to sign-track) using an autoshaping paradigm and found associated changes in stress-induced corticosterone release and mesolimbic levels of monoamines. Specifically, Tomie (2000) found those individuals that exhibited an increased sign-tracking response also had elevated levels of post-session corticosterone (see also Figure 4 below) and higher tissue levels of dopamine and DOPAC in the nucleus accumbens, lower levels of DOPAC/DA turnover in the caudate putamen, and lower levels of 5-HIAA and 5-HIAA/5-HT turnover in the ventral tegmental area (VTA). Tomie and colleagues (2000) concluded that the neurochemical profile evident in rats that were more likely to sign-track shared some features of vulnerability to drug abuse. Tomie (1998) has also reported that rats that perform more vigorous sign-tracking behavior are more likely to be impulsive, as measured by the tendency to choose small immediate rewards over larger delayed rewards.

 

Figure 4

Corticosterone response before and after the first autoshaping session. Data are expressed as mean corticosterone (Cort) levels (ng/ml) ± S.E.M. prior to training (Pre; or basal levels) and immediately after the first training session (Post). …

Tomie’s work on sign-tracking to food and ethanol-related cues, and how this behavior may contribute to the development of compulsive drug use, prompted us to begin to explore individual differences in this behavior. Our initial studies focused on developing procedures to study conditional approach to a cue paired with the intravenous delivery of cocaine (Uslaner et al., 2006). However, in doing so we also conducted studies using food as a reward, and in these studies we observed large individual differences in the CR that emerged after pairing presentation of a lever with delivery of a food pellet (Flagel et al., 2008; Flagel et al., 2007), as had been described by Boakes (1977). Briefly, in our studies an illuminated lever (CS) is presented for 8 sec, and immediately after it is retracted a banana-flavored food pellet (US) is delivered into a nearby receptacle located approximately 2.5 cm from the location of the CS-lever. The delivery of the reward is independent of the animal’s behavior (i.e., no explicit action is reinforced). A single session consists of 25 CS-US pairings presented on a random interval 90 sec schedule, and one session (≈40 min) is conducted per day. Rats are not food deprived yet they all consume all of the food pellets that are delivered. Under these conditions, using a sample of commercially available Sprague-Dawley rats, we have found that animals can be divided roughly into three groups based on the conditional response that emerges. Approximately one third of the rats (sign-trackers) come to preferentially approach and vigorously engage the reward-predictive cue (CS-lever; see Figures 1A, 2A, 2C and ​and3).3). One index of this sign-tracking response is the number of times a rat depresses the lever (Figure 2C) which it typically does by grasping and gnawing it (Figure 1A). When the lever is retracted these animals immediately go and retrieve the food pellet. Approximately another third of the rats (goal-trackers) very rarely approach the reward-predictive CS (Figure 3), but instead, upon presentation of the CS-lever they learn to quickly go to the food receptacle and they come to repeatedly put their nose into it while awaiting delivery of the reward (see Figures 1B, 2B and 2D). The remaining one third of rats (intermediate group) show neither clear sign-tracking nor goal-tracking, but vacillate between the cue and the goal (see Figures 2 and ​and3;3; Flagel et al., 2008; see also Boakes, 1977; Zener, 1937). All of the animals, regardless of their CR, retrieve and eat all of the food pellets and their behavior during the intertrial intervals is virtually identical. Thus, both sign-tracking and goal-tracking are conditional upon presentation of the lever (the CS). Furthermore, if CS-lever presentation is explicitly not paired with food delivery, but occurs randomly, neither the sign-tracking nor the goal-tracking CR develop (unpublished data).

 

Figure 1

Representative frames from a videotape of a rat engaged in (A) a sign-tracking CR or (B) a goal-tracking CR. The sign-tracking CR is directed toward the CS-lever (on the left of the food receptacle) and consists of grasping and gnawing on the lever. The …

 

Figure 2

Sign-tracking and goal-tracking behavior. Rats were classified into three groups based on their mean number of ‘lever presses’ across the16 daily training sessions. The black circles represent the one third of animals that made the most …

 

Figure 3

Difference in the probability of approach to the lever vs. approach to the food cup (i.e., lever-food cup difference; see Boakes, 1977; Flagel et al., 2008). As in Figure 2, black circles represent sign-trackers (ST; n=14), open circles represent goal-trackers …

Although individual differences are apparent even after the first day of training, it is important to emphasize that both sign-tracking and goal-tracking are learned responses that are conditional upon repeated CS-US pairings, and both CRs are evoked by the same cue; that is, the same conditional stimulus (the lever; see Figures 2 and ​and3).3). For both sign-trackers and goal-trackers repeated CS-US pairings increase the probability of approach during the CS period to either the cue or the goal, respectively (Figures 2 and ​and3),3), the vigor with which they engage the cue or the goal (Figure 2C and 2D), and the rapidity with which they approach the cue or the goal (Figure 2E and 2F). Furthermore, the tendency of sign-trackers to approach the reward-predictive lever and for the goal-trackers to approach the food cup (goal) during the CS period remains stable, even with prolonged training. In our published reports (Flagel et al., 2008; Flagel et al., 2007) we show data from only 5 days of training, but in Figure 3 we show previously unpublished data in which animals were trained for 16 consecutive days and it is clear that once developed, the sign-tracking and goal-tracking CRs remained stable. In contrast, with extended training, rats in the intermediate group tend to start to sign-track, although still not at levels equivalent to the animals initially classified as sign-trackers. The initial preference for the food cup in all animals (or lack of initial preference for the lever in sign-trackers) is not surprising given that all of the animals were “pre-trained” to retrieve pellets from the food cup (see also Flagel et al., 2008).

Based on these data it is clear that for both sign-trackers and goal-trackers the reward-predictive cue (CS) provides the information necessary to support learning. In both sign-trackers and goal-trackers repeated CS-US pairings result in the development of a learned response that is conditional upon presentation of the CS. That is, the CS is equally predictive of reward for both groups, and both groups develop a conditional response, but what differs is the direction of the CR. Sign-trackers approach and manipulate the reward-predictive cue (lever) itself, even though it is located away from the location where reward will be delivered (food cup). In contrast, upon presentation of the CS-lever goal-trackers do not approach it, but treat it as a signal for impending reward delivery and approach the location where the reward will be delivered (i.e., food cup). Thus, a cue that acts as a predictive, “informational” CS and supports learning in all of the animals becomes “attractive” and elicits approach towards itself in some but not others.

Although the underlying mechanisms have not yet been delineated, it may be useful to consider these findings using the theoretical framework of incentive motivation (Berridge, 2001; Bindra, 1978; Bolles, 1972). As mentioned above, CSs that acquire incentive motivational properties, or using the terminology of Berridge (2001), stimuli that are attributed with incentive salience, become attractive, can energize ongoing behavior and can act as conditional reinforcers (also see Cardinal et al., 2002a). In sign-trackers the CS does become attractive, as indicated by the fact that they approach it. Importantly, we have also found in a series of unpublished studies that for sign-trackers the CS-lever acts as an effective conditional reinforcer, reinforcing the learning of a new instrumental response. Thus, for sign-trackers the CS itself comes to acquire at least two of the defining properties of an incentive stimulus, suggesting that in these animals the CS-lever is attributed with incentive salience. On the other hand, in goal-trackers the CS does not itself appear to be attractive, because they do not approach it, and importantly, in unpublished studies we have found that in goal-trackers the CS-lever is relatively ineffective as a conditional reinforcer. Thus, for goal-trackers the CS does not acquire these defining properties of an incentive stimulus, suggesting that in these animals the CS itself is not attributed with incentive salience.

In the context of this theoretical formulation it is interesting to think about the differences between sign-trackers and goal-trackers. It may be that they do not differ in their ability to attribute incentive salience to stimuli per se, but in where it is attributed. Thus, for goal-trackers it may be that the CS directs incentive salience away from itself—and in this case to the food cup—as goal-trackers come to vigorously engage the food cup and direct consummatory-like behaviors towards it (Mahler and Berridge 2008, personal communication). On the other hand, it is also possible that for goal-trackers the CS evokes a cognitive expectation of reward and their behavior is governed more by cognitive than by incentive processes (Toates, 1998). Whatever the case, taken together these data suggest that in sign-trackers the reward-predictive cue itself, the CS, acquires incentive motivational properties and in goal-trackers it does not. Another interesting implication of these individual differences is that they suggest it is possible to dissociate the predictive or “informational” value of a CS from its incentive motivational properties. Given that the “informational” and “incentive” properties of reward-predictive cues are dissociable psychological phenomena they are presumably mediated by dissociable neural systems. This distinction may be important in exploring the neural mechanisms by which reward-related cues come to control behavior and awaits further research.

Correlates of sign-tracking and goal-tracking behavior

Conclusion: Implications for Addiction

It is the attribution of incentive motivational properties (via Pavlovian learning) that renders reward-predictive cues attractive, desired, and “wanted” (Berridge and Robinson, 2003; Robinson and Berridge, 1993)—and the excessive or pathological attribution of incentive salience to such cues may contribute to the development of compulsive behavioral disorders (Robinson and Berridge, 1993). The idea that reward-related cues can become irresistibly attractive and gain inordinate control over behavior is not a new concept. The phenomenon was described vividly by a gambler in one of the oldest known written texts in the world, the Rig Veda, dating back to 1200 BCE (see O’Flaherty, 1981). In speaking about his dice the gambler stated, “The trembling hazelnut eardrops of the great tree … intoxicate me as they roll on the furrowed board … The dice seem to me like a drink of soma … keeping me awake and excited” … “when the brown dice raise their voice as they are thrown down, I run at once to the rendezvous with them, like a woman to her lover” … “They (the dice) are coated with honey–an irresistible power over the gambler.” Using the present terminology, these dice seem to be attributed with considerable incentive salience.

In conclusion, Pavlovian learning is thought to play a role in maladaptively attracting humans towards drug-associated stimuli and sources of addictive drugs, contributing to the propensity to relapse (Di Chiara, 1998; Everitt and Robbins, 2005; Robinson and Berridge, 1993; Stewart et al., 1984; Tomie, 1996; Tomie et al., 2007). Based on the theoretical accounts and experimental findings presented here, we suggest that individual differences in the tendency to attribute incentive salience to reward-predictive cues may confer vulnerability or resistance to compulsive behavioral disorders, including addiction. Thus, we suggest that the distinction between the sign-tracking and goal-tracking phenotype may provide a means with which to explore individual differences in the tendency to attribute incentive value to reward-predictive cues, and their ability to gain inordinate control over behavior.

Acknowledgments

Footnotes

References