Contextual interaction between novelty and reward processing within the mesolimbic system (2012)

Hum Brain Mapp. 2012 Jun; 33(6): 1309–1324.
Published online 2011 Apr 21. doi: 10.1002/hbm.21288

Nico Bunzeck,^*,¹ Christian F Doeller,^2,^3,⁴ Ray J Dolan,⁵ and Emrah Duzel^2,⁶

Author information ► Article notes ► Copyright and License information ►

This article has been cited by other articles in PMC.

Abstract

Medial temporal lobe (MTL) dependent long-term memory for novel events is modulated by a circuitry that also responds to reward and includes the ventral striatum, dopaminergic midbrain, and medial orbitofrontal cortex (mOFC). This common neural network may reflect a functional link between novelty and reward whereby novelty motivates exploration in the search for rewards; a link also termed novelty “exploration bonus.” We used fMRI in a scene encoding paradigm to investigate the interaction between novelty and reward with a focus on neural signals akin to an exploration bonus. As expected, reward related long-term memory for the scenes (after 24 hours) strongly correlated with activity of MTL, ventral striatum, and substantia nigra/ventral tegmental area (SN/VTA). Furthermore, the hippocampus showed a main effect of novelty, the striatum showed a main effect of reward, and the mOFC signalled both novelty and reward. An interaction between novelty and reward akin to an exploration bonus was found in the hippocampus. These data suggest that MTL novelty signals are interpreted in terms of their reward-predicting properties in the mOFC, which biases striatal reward responses. The striatum together with the SN/VTA then regulates MTL-dependent long-term memory formation and contextual exploration bonus signals in the hippocampus.

Keywords: novelty, reward, mesolimbic system, memory, hippocampus, substantia nigra/ventral tegmental area, ventral striatum, mOFC, exploration bonus

INTRODUCTION

Novelty is a motivationally salient learning signal that attracts attention, promotes memory encoding and modifies goal-directed behavior [Knight,¹⁹⁹⁶; Lisman and Grace, ²⁰⁰⁵; Mesulam, ¹⁹⁹⁸; Sokolov, ¹⁹⁶³]. Recent evidence from human and nonhuman primate studies raises the possibility that the motivational aspects of novelty partly relate to its shared properties with reward [Bunzeck and Duzel, ²⁰⁰⁶; Kakade and Dayan, ²⁰⁰²; Mesulam, ¹⁹⁹⁸]. This suggestion follows from observations that in animal studies the substantia nigra/ventral tegmental area (SN/VTA) of the midbrain is activated by stimuli that predict rewards as well as stimuli that are novel [Ljungberg, et al. ¹⁹⁹²]; for a review see [Lisman and Grace, ²⁰⁰⁵]. Similarly, the human SN/VTA is activated both by reward [Knutson and Cooper, ²⁰⁰⁵] and novelty [Bunzeck and Duzel, ²⁰⁰⁶; Bunzeck, et al. ²⁰⁰⁷; Wittmann, et al. ²⁰⁰⁵] as well as by cues predicting their occurrence [Knutson and Cooper, ²⁰⁰⁵; O’Doherty, et al. ²⁰⁰²; Wittmann, et al. ²⁰⁰⁵^,²⁰⁰⁷]. The neurotransmitter dopamine that is produced in the SN/VTA profoundly regulates motivational aspects of behavior [Berridge, ²⁰⁰⁷; Niv, et al. ²⁰⁰⁷].

Furthermore, there is converging evidence that the hippocampus, a medial temporal lobe (MTL) structure, which is critical for the formation of long-term episodic memories for novel events, is also implicated in various forms of reward learning [Devenport, et al.¹⁹⁸¹; Holscher, et al. ²⁰⁰³; Ploghaus, et al. ²⁰⁰⁰; Purves, et al. ¹⁹⁹⁵; Rolls and Xiang, ²⁰⁰⁵; Solomon, et al. ¹⁹⁸⁶; Tabuchi, et al. ²⁰⁰⁰; Weiner, ²⁰⁰³; Wirth, et al. ²⁰⁰⁹]. For instance, the rodent hippocampus shows increased activity in baited but not unbaited maze arms [Holscher, et al. ²⁰⁰³]; in nonhuman primates it is involved in learning place reward associations [Rolls and Xiang, ²⁰⁰⁵]; hippocampal activity follows prediction error learning rules for aversive stimuli in humans [Ploghaus, et al. ²⁰⁰⁰]; and reward increases synchronization between hippocampus and nucleus accumbens neurons [Tabuchi, et al. ²⁰⁰⁰].

A commonality in the effects of reward and novelty can be reconciled theoretically by a suggestion that novelty acts to motivate exploration of an environment to harvest rewards [Kakade and Dayan,²⁰⁰²]. According to this suggestion, a key motivational property of novelty is its potential to predict rewards, whereas familiar stimuli, if repeated in the absence of reward, gradually loose this potential. The exploration bonus hypothesis makes two types of predictions: a first one relates to the potency with which the status of being novel or familiar can predict reward and a second one relates to the contextually remote effects of this contingency on other stimuli. According to the first prediction, being a novel stimulus should be a more potent predictor of reward than being a familiar stimulus [e.g., Wittmann, et al. ²⁰⁰⁸]. That is, when novel stimuli predict reward, reward expectancy should be higher than when familiar stimuli predict rewards. The second (more indirect) prediction is that the motivationally enhancing effect of novelty on exploratory behavior should have a contextual effect on the motivational significance of other stimuli that are present in the same context. Compatible with this suggestion, Bunzeck and Duzel [ ²⁰⁰⁶] showed that in a context in which novel stimuli are present, familiar stimuli show less repetition suppression in MTL structures. This suggests that even in the absence of explicit reward, in a context in which novel stimuli are present, there is a stronger motivation to explore also the familiar stimuli in that context [Bunzeck and Duzel, ²⁰⁰⁶]. However, to date, these predictions about the relationship between novelty and reward have not been tested directly. In experimental terms, this requires manipulating the reward-predicting property of novelty such that rewards in a given context are predicted either by being novel or by being familiar. Here, we used this experimental approach to investigate the functional interaction between novelty and reward in an fMRI study.

Understanding the functional interaction between novelty and reward has profound implications for understanding how long-term plasticity for novel stimuli is regulated. A large body of physiological evidence shows that dopamine originating from the SN/VTA not only regulates motivational aspects of behavior but is critical for enhancing and stabilizing hippocampal plasticity [Frey and Morris,¹⁹⁹⁸; Li, et al. ²⁰⁰³] and hippocampus-dependent memory consolidation [O’Carroll, et al. ²⁰⁰⁶]. According to the so-called hippocampus-VTA loop model [Lisman and Grace, ²⁰⁰⁵] novelty signals are generated in the hippocampus and are conveyed to the SN/VTA through the nucleus accumbens and the ventral pallidum [Lisman and Grace, ²⁰⁰⁵]. Although the model emphasizes novelty itself as the key cognitive signal to modulate dopamine from the SN/VTA, it also explicitly raises the question how motivational factors regulate the impact of novelty on the activity of the hippocampus and the SN/VTA. The goal of this study is to approach this question from the vantage point of shared properties between novelty and reward and their functional interaction.

If novelty acts as a signal that motivates exploration to harvest rewards [Bunzeck and Duzel,²⁰⁰⁶; Kakade and Dayan, ²⁰⁰²; Wittmann, et al. ²⁰⁰⁸] parts of the hippocampus-SN/VTA loop should only show a preferential response to novelty in a context where being novel predicts rewards but not in a context where being familiar predicts reward. At the same time, the enhancement of exploration when being novel is rewarded should boost hippocampal responses to familiar stimuli that are presented in the same context, even though these would not predict rewards. In contrast, in a context in which being familiar but not being novel predicts rewards, there should be less contextual motivation to explore and consequently hippocampal activity should be low for both the novel and the familiar stimuli in that context. Hence, the hypothesis that novelty has an intrinsic property to motivate explorative behavior in the search for rewards leads to the prediction of an interaction between the novelty- and reward status of stimuli. Accordingly, the hippocampus would respond strongly to both novel and familiar stimuli when being novel predicts reward and weakly to both novel and familiar stimuli when being familiar predicts reward.

The alternative possibility is that the novelty and reward-status of information is independent. According to this possibility, there should be no functional interaction between novelty and reward. In other words, parts of the hippocampus-SN/VTA loop would only express a main effect of novelty or reward but no interaction between both.

Taken together, manipulating the contingency between novelty and rewards can help to understand the key mechanisms that drive novelty responses within the mesolimbic system. To that end, we developed a paradigm where receiving monetary reward was contingent upon the novelty status of images of scenes [Bunzeck, et al.²⁰⁰⁹]. Thus, making correct reward preference decisions (see methods) was only possible after correctly discriminating novel and familiar stimuli. Importantly, we assessed recognition memory one day after encoding and thus were able to identify to what extent components of the hippocampal-SN/VTA loop would correlate with the reward-related enhancement of long-term memory for novel and familiar stimuli.

MATERIALS AND METHODS

Two experiments were performed. While the first experiment (Experiment 1) was a behavioral experiment the second experiment (Experiment 2) involved behavioral measures and fMRI.

Subjects

In Experiment 1, 17 adults participated (13 female and four male; age range 19–33 years; mean 23.1, SD = 4.73 years) and 14 adults participated in Experiment 2 (five male and nine female; age range: 19–34 years; mean = 22.4 years; SD = 3.8 years). All subjects were healthy, right-handed and had normal or corrected-to-normal acuity. None of the participants reported a history of neurological, psychiatric, or medical disorders or any current medical problems. All experiments were run with each subject’s written informed consent and according to the local ethics clearance (University College London, UK).

Experimental Design and Task

In both experiments, three sets of (1) a familiarization phase followed by (2) a recognition memory based preference judgment task were performed. Here, new images were used for each set resulting in 120 novel and 120 familiar images being used altogether. The experimental procedures were identical for both experiments except that Experiment 1 was performed on a computer screen and Experiment 2 was performed inside an MRI scanner. (3) On day two recognition memories for all presented images was tested using the “remember/know” procedure (see below).

(1) Familiarization: Subjects were initially familiarized with a set of 40 images (20 indoor and 20 outdoor images). Here, each picture was presented twice in random order for 1.5 s with an interstimulus interval (ISI) of 3 s and subjects indicated the indoor/outdoor status using their right hand index and middle finger. (2) Recognition memory test: subsequently, subjects performed a 9 minute recognition memory based preference judgment task (session). This part (session) was further subdivided into two blocks containing each 20 images from the familiarization phase (referred to as “familiar images”) and 20 previously not presented images (referred to as “novel images”; subjects could pause for 20 s between blocks). In any given block either novel images served as CS+ and familiar images as CS− or vice versa (Fig..1). Participants were instructed to make a “preference” judgment to each image via a two-choice button press indicating “I prefer” or “I do not prefer” depending on the contingency between novelty status and reinforcement value. Importantly, the term “preferred” and “not-preferred” refers to the reward predicting status of the image (depending on the contextual contingency) rather than the aesthetic properties of the picture.

Experimental design.

The contingency was randomized and indicated on the screen prior to each run by either “Novelty will be rewarded if preferred” (in which case novel images served as CS+ and familiar images as CS−) or “Familiarity will be rewarded if preferred” (here familiar images served as CS+ and novel images as CS−). Only correct “I prefer” responses following a CS+ led to a win of £0.50 whereas (incorrect) “I prefer” responses following CS− led to a loss of £-0.10. Both correct “I do not prefer” responses following CS− and (incorrect) “I do not prefer” responses following a CS+ led to neither win nor loss. Images were presented in random order for 1 s on a gray background followed by a white fixation cross for 2 s (ISI = 3 s). To ensure that neural reward responses were limited to the presented images (i.e., reward anticipation rather than outcome) no feedback was given on a trial by trial basis. Instead subjects were informed about their overall performance after each session (containing 2 blocks with each contingency). Prior to the experiment the subjects were instructed to respond as quickly and as correctly as possible and that only 20% of all earnings would be paid.

All images were gray-scaled and normalized to a mean gray-value of 127 and a standard deviation of 75. None of the scenes depicted human beings or parts of human beings including faces in the foreground.

Training Sessions

Each subject performed two training sessions prior to the experiment. Similar to the actual experiment both training phases began with a familiarization phase, during which only 10 images were presented twice in random order (duration = 1.5 s; ISI = 3 s) and subjects indicated their indoor/outdoor status. As was the case for the main experiment, familiarization was followed by a memory based preference judgment task including familiar and novel images. For training purposes, in training session 1 a feedback was given on a trial-by-trial basis after each response. In training session 2 reward feedback was not shown immediately after each stimulus/response. Following each training session, the subject’s financial reward (maximum £1) was reported to the subject. In Experiment 2, subjects also received a brief training session containing 10 familiar and 10 novel images per response contingencies block.

One day later, subjects performed an incidental recognition memory test following the “remember/know” procedure [Tulving,¹⁹⁸⁵]. Here, in random order all 240 previously seen pictures (60 per condition) were presented together with 60 new distractor pictures on the center of a computer screen. Task: The subject first made an “old/new” decision to each individually presented picture using their right index or middle finger. Following a “new” decision, subjects were prompted to indicate whether they were confident (“certainly new”) or unsure (“guess”), again using their right index and middle finger. After an “old” decision, subjects were prompted to indicate if they were able to remember something specific about seeing the scene at study (“remember response”), just felt familiarity with the picture without any recollective experience (“familiar” response) or were merely guessing that the picture was an old one (“guess” response). The subject had 4 s to make each of both judgments and there was a break of 15 s after every 75 pictures.

fMRI Methods

We performed fMRI on a 3-Tesla Siemens Allegra magnetic resonance scanner (Siemens, Erlangen, Germany) with echo planar imaging (EPI) using a quadrature transceiver coil with a design based on the “birdcage” principle. In the functional session 48 T2*-weighted images (EPI-sequence; covering the whole head) per volume with blood oxygenation level-dependent (BOLD) contrast were obtained (matrix size: 64 × 64; 48 oblique axial slices per volume angled at −30° in the antero-posterior axis; spatial resolution: 3 × 3 × 3 mm; TR = 3120 ms; TE = 30 ms; z-shimming pre-pulse gradient moment of PP = 0 mT/m*ms; positive phase-encoding polarity). The fMRI acquisition protocol was optimized to reduce susceptibility-induced BOLD sensitivity losses in inferior frontal regions and temporal lobe regions [Deichmann, et al.²⁰⁰³; Weiskopf, et al. ²⁰⁰⁶]. For each subject functional data were acquired in three scanning sessions containing 180 volumes per session. Six additional volumes per session were acquired at the beginning of each series to allow for steady state magnetization and were subsequently discarded from further analysis. Anatomical images of each subject’s brain were collected using multi-echo 3D FLASH for mapping proton density, T1 and magnetization transfer (MT) at 1 mm resolution [Helms, et al. ²⁰⁰⁹; Weiskopf and Helms, ²⁰⁰⁸] and by T1 weighted inversion recovery prepared EPI (IR-EPI) sequences (matrix size: 64 × 64; 64 slices; spatial resolution: 3 × 3 × 3 mm). Additionally, individual field maps were recorded using a double echo FLASH sequence (matrix size = 64 × 64; 64 slices; spatial resolution = 3 × 3 × 3 mm; gap = 1 mm; short TE = 10 ms; long TE = 12.46 ms; TR = 1020 ms) for distortion correction of the acquired EPI images [Weiskopf, et al. ²⁰⁰⁶]. Using the “FieldMap toolbox” [Hutton, et al. ²⁰⁰²^,²⁰⁰⁴] field maps were estimated from the phase difference between the images acquired at the short and long TE.

The fMRI data were preprocessed and statistically analyzed using the SPM5 software package (Wellcome Trust Centre for Neuroimaging, University College London, UK) and MATLAB 7 (The MathWorks, Inc., Natick, MA). All functional images were corrected for motion artifacts by realignment to the first volume; corrected for distortions based on the field map [Hutton, et al.²⁰⁰²]; corrected for the interaction of motion and distortion using the “Unwarp toolbox” [Andersson, et al. ²⁰⁰¹; Hutton, et al. ²⁰⁰⁴]; spatially normalized to a standard T1-weighted SPM-template [Ashburner and Friston, ¹⁹⁹⁹] (care was taken that in particular midbrain regions aligned with the standard-template); re-sampled to 2 × 2 × 2 mm; and smoothed with an isotropic 4 mm full-width half-maximum Gaussian kernel. Such fine-scale spatial resolution in combination with a relatively small smoothing kernel is the basis for being able to detect small clusters of activation, for instance within the midbrain and MTL regions where differential activation patterns (i.e., novelty responses and interactions between novelty and reward) might be located in close proximity [Bunzeck, et al. ²⁰¹⁰]. The fMRI time series data were high-pass filtered (cutoff = 128 s) and whitened using an AR(1)-model. For each subject an event-related statistical model was computed by creating a “stick function” for each event onset (duration = 0 s), which was convolved with the canonical hemodynamic response function combined with time and dispersion derivatives [Friston, et al. ¹⁹⁹⁸]. Modeled conditions included novel-rewarded, novel-not-rewarded, familiar-rewarded, familiar-not-rewarded and incorrect responses. To capture residual movement-related artifacts six covariates were included (the three rigid-body translation and three rotations resulting from realignment) as regressors of no interest. Regionally specific condition effects were tested by employing linear contrasts for each subject and each condition (first-level analysis). The resulting contrast images were entered into a second-level random-effects analysis. Here, the hemodynamic effects of each condition were assessed using a 2 × 2 analyses of variance (ANOVA) with the factors “reward” (rewarding, not rewarding) and “novelty” (novel, familiar). This model allowed us to test for main effects of novelty, main effects of reward and the interaction between both. All contrasts were thresholded at P = 0.001 (uncorrected) except the regression analyses (P = 0.005, uncorrected). Both relatively liberal thresholds were chosen based on our precise a priori anatomical hypotheses within the mesolimbic system.

The anatomical localization of significant activations was assessed with reference to the standard stereotaxic atlas by superimposition of the SPM maps on one of two group templates. A T1-weighted and a MT-weighted group template were derived from averaging all subjects’ normalized T1 or MT images (spatial resolution of 1 × 1 × 1 mm). While the T1-template allows anatomical localization outside the midbrain on MT-images the SN/VTA region can be distinguished from surrounding structures as a bright stripe while the adjacent red nucleus and cerebral peduncle appear dark [Bunzeck and Duzel,²⁰⁰⁶; Bunzeck, et al. ²⁰⁰⁷; Eckert, et al. ²⁰⁰⁴].

Note that we prefer to use the term SN/VTA and consider BOLD activity from the entire SN/VTA complex for several reasons [Duzel, et al.²⁰⁰⁹]. Unlike early formulations of the VTA as an anatomical entity, different dopaminergic projection pathways are dispersed and overlapping within the SN/VTA complex. In particular, dopamine neurons that project to the limbic regions and regulate reward-motivated behavior are not confined to the VTA but they are distributed also across the SN (pars compacta) [Gasbarri, et al. ¹⁹⁹⁴^,¹⁹⁹⁷; Ikemoto, ²⁰⁰⁷; Smith and Kieval, ²⁰⁰⁰]. Functionally, this is paralleled in the fact that in humans and primates DA neuron within the SN and VTA respond to both reward and novelty [see for instance Ljungberg, et al., ¹⁹⁹² or Tobler, et al., 2003 for a depiction of recording sites].

RESULTS

All analyses (behavioral and fMRI) are based on trials with correct preference responses.

Experiment 1

Subjects discriminated between conditions in both contexts with high accuracy (Table I) and there were no statistically significant differences between conditions. Reaction time (Fig. 2A) analysis revealed that subjects responded fastest to familiar reward predicting stimuli (all P‘s < 0.007), but there was no difference between the other three conditions (novel-rewarded, novel-not-rewarded, familiar-not-rewarded; all P‘s > 0.05).

Behavioral results. (A) Reaction-times. In both experiments RTs were significantly faster for familiar rewarded images compared to all other conditions (all P < 0.01)—as indicated by the asterisk—but there was no other difference …

Behavioral results

Recognition memory performance–second day. Recognition memory analysis was based on both hits (remember responses, know responses following pictures previously seen during encoding), and false alarms ([FA]: remember, know to distractors). In a first step, we calculated the proportion of remember- and know-responses for old and new images (i.e., hit-rates and FA-rates) by dividing the number of hits (and FA, respectively) by the number of items per condition. Secondly, corrected hit-rates were obtained for remember-responses ([Rcorr], remember hit-rate minus remember FA-rate) and know-responses ([Kcorr], know hit-rate minus know FA-rate) (see Table II). In a planned comparison, we assessed the effect of reward on overall recognition memory (corrected hit-rate = Rcorr + Kcorr) for novel and familiar images. This revealed that reward significantly improved overall memory for novel images compared to novel not rewarded images (P = 0.036) but there was no such improvement of overall memory by reward for familiar images (P > 0.5; Fig. 2). Furthermore, the enhancing effect of reward on recognition memory for novel images was equally strong for recollection and familiarity as revealed by analysis of variance (ANOVA; no interaction between reward and recognition memory type [F(1,16) = 2.28, P > 0.15)].

Recognition memory

Experiment 2

As in Experiment 1, subjects discriminated between conditions in both contexts with high accuracy and no significant differences between conditions (Table I). As in Experiment 1, reaction-time (Fig. 2A) analysis showed responses were significantly faster for familiar reward predicting stimuli (all P‘s < 0.001) but there was no difference between the other three conditions (novel-rewarded, novel-not-rewarded, familiar-not-rewarded; all P‘s > 0.05).

Recognition memory performance–second day. In contrast to Experiment 1, recognition memory for novel rewarded images was not significantly improved compared to novel unrewarded images (neither overall recognition memory nor Rcorr/Kcorr; P > 0.05, Table II). Also in contrast to Experiment 1, in Experiment 2 recollection for familiar rewarded images was significantly enhanced compared to familiar not-rewarded images (P = 0.001, Table II) which resulted in enhanced overall memory (Rcorr + Kcorr) for familiar rewarded compared to familiar not-rewarded images (there was no significant difference between the corrected know-rates of familiar rewarded and familiar not-rewarded images, P > 0.05). Furthermore, data in Table II and Figure 2B shows that overall memory performance was considerably lower in Experiment 2 compared to Experiment 1, which was supported by a mixed effects ANOVA.

fMRI results−reward based recognition memory test. First, we analyzed fMRI data using a 2 × 2 ANOVA with factors “novelty” (novel, familiar) and “reward” (reward, no reward). We found a main effect of novelty in bilateral medial orbitofrontal cortex (mOFC) and the right MTL including the hippocampus and rhinal cortex, (Fig. 3; see Supporting Information Table S1 for a complete list of activated brain structures). A main effect of reward was observed within the bilateral caudate, septum/fornix, ventral striatum (ncl. accumbens), bilateral mOFC and medial prefrontal cortex (mPFC) (Fig. 4; Supporting Information Table S1). These two main effects were exclusively masked with the effects of interactions (exclusive masking, P = 0.05, uncorrected) to identify only those regions that expressed main effects in the absence of any interaction.

fMRI results Experiment 2. A main effect of novelty was observed within the right hippocampus (A), rhinal cortex (B) and medial OFC (C). Activation maps were superimposed on a T1-weighted group template (see methods), coordinates are given in MNI space …

fMRI results Experiment 2. A main effect of reward was observed within the striatum, including ncl. accumbens (A) and caudate ncl. (C), septum/fornix (B), medial PFC (C), and medial OFC (D). Activation maps were superimposed on a T1-weighted group template …

To test our two predictions regarding the exploration bonus hypothesis, we performed two additional analyses. First, within brain regions that showed a main effect of reward we analyzed, which areas also showed a stronger response for novel rewarded than familiar rewarded stimuli (i.e., conjunction). This analysis did not yield any significant results suggesting that there were no brain regions where being novel lead to a stronger reward prediction response than being familiar. Secondly, we assessed the interaction (F-contrast) between novelty and reward. Such an interaction was expressed within several brain regions including right hippocampus, inferior frontal gyrus and right OFC (Supporting Information Table S1, Fig. 5). Specifically, the hippocampus showed the expected interaction pattern with higher responses for stimuli presented in the context where being novel is rewarded (T-contrast). That is, hippocampal activity was higher for novel rewarded stimuli and familiar unrewarded stimuli (note that both of these stimuli were presented in the same context) than for novel unrewarded and familiar rewarded stimuli (again, note that both of these stimuli were presented in the same context). Planned post hoc comparison confirmed statistically significant differences between novel-rewarded vs. novel not-rewarded (P < 0.025) and familiar rewarded vs. familiar not-rewarded (P < 0.01; Fig. 5).

fMRI results Experiment 2. An interaction between novelty and reward was observed within the hippocampus and OFC. Within the hippocampus responses to familiar not-rewarded items was enhanced compared to familiar-rewarded items if presented in context …

It should be noted that the activation pattern for the interaction between novelty and reward (36, −14, −16; Fig. 5) is adjacent but not identical to the activation of a main effect of novelty, which is also located within the right hippocampus (28, −14, −20; Fig. 3). Such differential activation pattern accords to our hypotheses, cell recordings in animals and human fMRI studies. For instance, animal research has shown that different hippocampal neurons can respond to different features (such as novelty or familiarity) within the same task [Brown and Xiang,¹⁹⁹⁸]. In line with these observations, we have shown in humans that spatially distinct hippocampal activations can reflect differential properties of novelty processing, absolute novelty signals, adaptively scaled novelty signals and novelty prediction errors, ([Bunzeck, et al. ²⁰¹⁰], Supporting Information Fig. S4). Johnson et al. (²⁰⁰⁸) reported that spatially very close clusters of activation showed very different responses to novelty: one cluster showed a categorical difference between new items and old items whereas the other cluster showed a linear response decrement as a function of increased stimulus familiarity. However, to further exclude the possibility of a false positive result we applied small volume correction to both activation patterns using the right anterior hippocampus as volume. The analysis reached statistical significance (P ≤ 0.05; FWE-corrected).

Finally, we sought to link reward related memory improvement to regional brain activity patterns using regression analyses (all analyses were performed with data from Experiment 2). First, the contrast novel rewarded vs. novel not-rewarded images was entered into a second-level simple regression analysis using individual memory improvement by reward as regressor (Δ corrected hit-rate = corrected hit rate [Rcorr + Fcorr] for novel-rewarded – corrected hit-rate for novel not-rewarded). This analysis was motivated by our initial observation of improved overall memory (i.e., recollection and familiarity) for novel images by reward (Experiment 1) and previous similar findings [Adcock, et al.²⁰⁰⁶; Krebs, et al. ²⁰⁰⁹; Wittmann, et al. ²⁰⁰⁵]. This revealed a significant positive correlation between hemodynamic responses (HR) and recognition memory improvement within the SN/VTA, right anterior MTL (junction of rhinal cortex hippocampus/amygdala) and right ventral striatum (Fig. 6, Supporting Information Table S1 for all activated regions). In a second regression analysis, the same contrast for familiar images (familiar rewards vs. familiar not-reward) was correlated with individual improved recollection rate (behaviorally, recollection rate was significantly enhanced for familiar rewarded compared to not-rewarded images but there was no improvement in Fcorr). Since RTs for familiar rewarded images were significantly faster than for familiar not-rewarded images the difference between both for each subject was also entered as regressor. Here, we were only interested in those regions that showed a significant positive correlation between HR differences (familiar rewarded vs. familiar not rewarded) and increased recollection rate (familiar rewarded vs. familiar not-rewarded) but not those that also showed any correlation with RT improvement. This analysis revealed similar effects to the first regression analysis, namely, a significant correlation between HR and reward-related recollection-rate improvement within the ventral striatum (left), right hippocampus and left rhinal cortex (Fig. 7, Supporting Information Table S1), but no correlation within the SN/VTA. A statistically more sensitive post hoc analysis of the SN/VTA voxel [4, −18, −16] that showed a significant correlation for novel images also revealed no correlation between hemodynamic responses and improved recollection rate for familiar images (r = −0.07, P = 0.811).

fMRI results Experiment 2–regression analysis. A significant correlation between recognition memory improvement for novel rewarded compared to not-rewarded images (Δ corrected hit-rate) and hemodynamic response differences between novel …

fMRI results Experiment 2–regression analysis. A significant correlation between recollection rate improvement for familiar rewarded compared to familiar not-rewarded images (Δ recollection rate) and hemodynamic response differences between …

DISCUSSION

Our finding that a cluster of voxels within the MTL (including hippocampus and rhinal cortex) showed a main effect of novelty but not a main effect of reward (Fig. 3A,B), supports the idea that the hippocampus and rhinal cortex can signal novelty independent of reward-value. This finding accords with a wide range of animal and human studies suggesting that both the hippocampus and rhinal cortex are sensitive to novelty [Brown and Xiang,¹⁹⁹⁸; Dolan and Fletcher, ¹⁹⁹⁷; Knight, ¹⁹⁹⁶; Lisman and Grace, ²⁰⁰⁵; Strange, et al. ¹⁹⁹⁹; Yamaguchi, et al. ²⁰⁰⁴]. However, another region within the hippocampus also showed the hypothesized interaction of novelty and reward (Fig. 5) with significantly enhanced hemodynamic responses to familiar unrewarded images if presented in a context where being novel was rewarded.

This interaction of novelty and reward in the hippocampus provides evidence for our second prediction of a contextual effect in accordance with the exploration bonus framework (see [Sutton and Barto,¹⁹⁸¹] for a formal description of the exploration bonus within the exploration-exploitation dilemma). Based on the notion that novelty can act as an exploration bonus for reward [Kakade and Dayan, ²⁰⁰²] we predicted that in a context in which being novel is rewarded there should be enhanced exploration also of the familiar stimuli (even when they are unrewarded). Compatible with this possibility, familiar stimuli elicited stronger hippocampal activity in a context where the availability of reward was signaled by being novel as compared to a context where reward is signaled by being familiar. This contextually enhanced neural activation within the hippocampus during encoding, however, did not directly translate into long-term memory, that is, better memory for familiar items when presented in context with novel reward predicting items. Instead, recognition performance was driven by the reward predicting status of an item both for novel (Experiment 1) and familiar (Experiment 2) stimuli (see below). This suggests that, in an experimental setting in which reward prediction and contextual novelty may both influence learning, reward prediction can exert the dominance influence.

Another prediction regarding the exploration bonus framework was not confirmed. We did not find any brain regions which exhibited a main effect of reward and at the same time a significantly stronger activity for novel rewarded than familiar rewarded images. At the first glance, this negative finding seems to be at odds with previous studies [Krebs, et al.²⁰⁰⁹; Wittmann, et al. ²⁰⁰⁸]. However, in both, the Krebs et al. [ ²⁰⁰⁹] and the Wittmann et al. [ ²⁰⁰⁸] study, enhanced reward prediction for novel stimuli was found under conditions where the novelty status of stimuli was implicit and participants attended to reward contingencies. In fact, Krebs et al. reported that this enhancement was absent when participants attended to the novelty status of stimuli rather than attending to reward contingencies (note however, that in Krebs et al. novelty status per se was not predictive of reward). Hence, unlike the contextual interaction between novelty and reward (Fig. 5), this aspect of the exploration bonus may be strongly task-dependent occurring only when subjects can attend to reward contingencies without having to assess novelty. It has been suggested on the basis of rodent studies that prefrontal and hippocampal inputs compete with each other for control over the nucleus accumbens (a part of the ventral striatum) [Goto and Grace, ²⁰⁰⁸]. It is plausible that task-related attention to novelty or reward would affect such a competition.

Recognition memory scores from Experiment 1 (Fig. 2) were well compatible with the exploration bonus framework in showing a reward-related behavioral enhancement of long-term memory performance for novel but not for familiar stimuli. However, the behavioral results obtained under conditions where encoding occurred in the fMRI scanner (Experiment 2) were different in that memory for familiar stimuli did show an enhancement by reward (for novel stimuli this enhancement did not reach significance). One reason for this discrepancy may be that in Experiment 1, the encoding context and the retrieval context on the next day were identical (subjects learned and were tested in the same room) whereas for Experiment 2 they were different (subjects encoded in the fMRI and were tested in a testing room). It is well-known that changes between encoding and retrieval context can have profound influences on memory performance [Godden and Baddeley,¹⁹⁷⁵]. Compatible with this possibility, memory performance was considerably lower in Experiment 2 than in Experiment 1 (Fig. 2). Such context effects may have also led to the discrepancy in the behavioral patterns observed in Experiments 1 and 2.

The ventral striatum (Fig. 4A) and medial prefrontal cortex (Fig. 4 C,D) expressed main effects of expected reward value. In our task reward-prediction depended upon explicit novelty discrimination and thus it is apparent that regions expressing expected reward value (ventral striatum, septum/fornix) require access to information about memory for the presented picture. A likely origin of such declarative memory information is the MTL. In fact, hippocampus and rhinal cortex, as part of the MTL, not only expressed the main effect of novelty, but they are also well-known to send efferents to the ventral striatum and the medial prefrontal cortex (note that projection from rhinal cortex to the NAcc stem primarily from the entorhinal cortex [Friedman, et al.²⁰⁰²; Selden, et al. ¹⁹⁹⁸; Thierry, et al. ²⁰⁰⁰]). The precise mechanisms and computational processes, however, which may be implicated in translating novelty into reward responses, are unclear. This possibly involves the medial prefrontal cortex (including orbital parts) which–in line with previous studies [O’Doherty, et al. ²⁰⁰⁴; Ranganath and Rainer, ²⁰⁰³]–expressed both novelty and reward related activation (Fig. 3C and 4C,D).

The functional implications of our results regarding the representation of novelty and reward responses in the hippocampus, SN/VTA, ventral striatum and medial PFC are summarized in Figure8. To provide support for this model, we calculated a correlation between the activation of our regions of interest, using a Spearman correlation analysis for each subject on the deconvolved time series, to provide a group correlation coefficient R and a P-value.

Schematic illustration of the functional relationship between hippocampus, Nucleus accumbens (NAcc), medial prefrontal cortex (mPFC) and substantia nigra/ventral tegmental area (SN/VTA). To provide support for this model, we calculated a correlation between …

Since reward was contingent upon novelty and the sole region that represented both types of signals was the mPFC, this region is likely to be the source of novelty-based reward signaling (R = 0.09; P < 0.001). The hippocampus, on the other hand, is most likely the source of the novelty signal for the mPFC (R = 0.11; P < 0.001). This is plausible given that there are direct projections from the hippocampus to mPFC [Ferino, et al.¹⁹⁸⁷; Rosene and Van Hoesen, ¹⁹⁷⁷]. It is also plausible that the mPFC reward signal is then conveyed to the NAcc (R = 0.09; P < 0.001) and the SN/VTA (R = 0.03; P = 0.08). It should be noted that the SN/VTA signal only correlated with the novelty responsive mPFC (R = 0.03; P = 0.08) but not the reward responsive mPFC (R = 0.007; P > 0.6). This suggests that mOFC inputs to the SN/VTA might arise more strongly from those mPFC regions associated with novelty processing rather than reward processing. Our observation that the mPFC responds to novelty and correlates with the SN/VTA signal is also compatible with the suggestion [Lisman and Grace, ²⁰⁰⁵] that the PFC is a source of a novelty signal into dopaminergic circuitry. The role of the NAcc in novelty signaling, however, still remains unclear [Duzel, et al. ²⁰⁰⁹]. That is, although we did not observe novelty signals within the NAcc there was a strong correlation between signals in the NAcc and novelty responsive mOFC regions (R = 0.09; P < 0.001), NAcc and novelty responsive hippocampus regions (R = 0.15; P < 0.001), and the NAcc and SN/VTA (R = 0.19; P < 0.001). Finally, it should be noted that the arrows in our model indicate assumed directionality on the basis of known projections rather than quantitatively estimated causality.

Reward related improvement of recognition memory was correlated with ventral striatum, SN/VTA and MTL activation (Fig. 6). An important aspect of hippocampal learning and plasticity is a requirement for DA in the expression of the late phase LTP (long-term potentiation) but not early phase LTP [Frey and Morris,¹⁹⁹⁸; Frey, et al. ¹⁹⁹⁰; Huang and Kandel ¹⁹⁹⁵; Jay ²⁰⁰³; Morris ²⁰⁰⁶]. This supports a view that DA is required for long-term memory consolidation, which is supported by recent behavioral data in rodents [O’Carroll, et al. ²⁰⁰⁶]. Our data are compatible with this view in showing a correlation between long-term memory improvement through reward one day after encoding and activation within putative dopaminergic regions and hippocampus. In particular, we see a correlation for novel rewarded vs. not-rewarded items within SN/VTA, ventral striatum and hippocampus and a correlation for familiar rewarded vs. nonrewarded items within ventral striatum and hippocampus. Given that the ventral striatum is a primary output structure of the dopaminergic midbrain (SN/VTA) [Fields, et al. ²⁰⁰⁷] our results suggests that an ability to observe a reward-related enhancement of long-term memory through the hippocampal-SN/VTA is not limited to novel stimuli but also applies to familiar stimuli. In fact, it is likely that the degree of familiarity among the class of familiar stimuli (during encoding) was quite variable and that those stimuli whose encoding benefited most from reward were the least familiar (relatively most novel) ones. Therefore it is reasonable to assume that correlations for the novel and familiar stimulus classes were driven by the same mechanisms.

We also observed a main effect of reward in the septum/fornix (Fig. 4B), a region that is likely to harbor cholinergic neurons which project to medial temporal structures. Interestingly, animal studies show that similar to DA neurons, cholinergic neurons (in the basal forebrain) respond to novelty and habituate when stimuli become familiar [Wilson and Rolls,^1990b]. However, in tasks in which familiar stimuli predict reward, the activity of basal forebrain neurons reflect reward-prediction rather than novelty status [Wilson and Rolls, ^1990a]. Our findings (Fig. 4B) are compatible with the observation of Wilson and Rolls (^1990a) although we cannot say to what extent these activations actually involve responses of cholinergic neurons.

Taken together, we replicate recent observations that activity of the ventral striatum, SN/VTA, hippocampus and rhinal cortex correlated with reward-related memory enhancement compatible with the hippocampus-SN/VTA loop. Importantly, our findings provide new key insights into the functional properties of the components of this loop. In a task in which the novelty status of an item predicted reward the hippocampus preferentially expressed the novelty status whereas ventral striatum activity reflected the reward value independently of novelty status. The medial PFC (including orbital parts) was likely to be the site where novelty and reward signals were integrated because it expressed both novelty and reward effects and is known to be connected with the hippocampus and ventral striatum. Finally, in line with the exploration bonus theory [Kakade and Dayan,²⁰⁰²] novel reward predicting stimuli exerted contextually enhancing effects on familiar (not rewarding) items, which were expressed as enhanced neural responses within the hippocampus.

Acknowledgments

We would like to thank K. Herriot for support in data acquisition.

Additional Supporting Information may be found in the online version of this article.

Click here to view.^{(272K, doc)}

REFERENCES

Adcock RA, Thangavel A, Whitfield-Gabrieli S, Knutson B, Gabrieli JD. Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron. 2006;50:507–517. [PubMed]
Andersson JL, Hutton C, Ashburner J, Turner R, Friston K. Modeling geometric deformations in EPI time series. Neuroimage. 2001;13:903–919. [PubMed]
Ashburner J, Friston KJ. Nonlinear spatial normalization using basis functions. Hum Brain Mapp. 1999;7:254–266. [PubMed]
Berridge KC. The debate over dopamine’s role in reward: The case for incentive salience. Psychopharmacology (Berl) 2007;191:391–431. [PubMed]
Brown MW, Xiang JZ. Recognition memory: neuronal substrates of the judgement of prior occurrence. Prog Neurobiol. 1998;55:149–189. [PubMed]
Bunzeck N, Duzel E. Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA. Neuron. 2006;51:369–379. [PubMed]
Bunzeck N, Schutze H, Stallforth S, Kaufmann J, Duzel S, Heinze HJ, Duzel E. Mesolimbic novelty processing in older adults. Cereb Cortex. 2007;17:2940–2948. [PubMed]
Bunzeck N, Doeller CF, Fuentemilla L, Dolan RJ, Duzel E. Reward motivation accelerates the onset of neural novelty signals in humans to 85 milliseconds. Curr Biol. 2009;19:1294–1300. [PMC free article] [PubMed]
Bunzeck N, Dayan P, Dolan RJ, Duzel E. A common mechanism for adaptive scaling of reward and novelty. Hum Brain Mapp. 2010;31:1380–1394. [PMC free article] [PubMed]
Deichmann R, Gottfried JA, Hutton C, Turner R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage. 2003;19(2 Pt 1):430–441. [PubMed]
Devenport LD, Devenport JA, Holloway FA. Reward-induced stereotypy: Modulation by the hippocampus. Science. 1981;212:1288–1289. [PubMed]
Dolan RJ, Fletcher PC. Dissociating prefrontal and hippocampal function in episodic memory encoding. Nature. 1997;388:582–585. [PubMed]
Duzel E, Bunzeck N, Guitart-Masip M, Wittmann B, Schott BH, Tobler PN. Functional imaging of the human dopaminergic midbrain. Trends Neurosci. 2009;32:321–328. [PubMed]
Eckert T, Sailer M, Kaufmann J, Schrader C, Peschel T, Bodammer N, Heinze HJ, Schoenfeld MA. Differentiation of idiopathic Parkinson’s disease, multiple system atrophy, progressive supranuclear palsy, and healthy controls using magnetization transfer imaging. Neuroimage. 2004;21:229–235. [PubMed]
Ferino F, Thierry AM, Glowinski J. Anatomical and electrophysiological evidence for a direct projection from Ammon’s horn to the medial prefrontal cortex in the rat. Exp Brain Res. 1987;65:421–426. [PubMed]
Fields HL, Hjelmstad GO, Margolis EB, Nicola SM. Ventral Tegmental Area Neurons in Learned Appetitive Behavior and Positive Reinforcement. Annu Rev Neurosci. 2007;30:289–316. [PubMed]
Frey U, Morris RG. Synaptic tagging: implications for late maintenance of hippocampal long-term potentiation. Trends Neurosci. 1998;21:181–188. [PubMed]
Frey U, Schroeder H, Matthies H. Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices. Brain Res. 1990;522:69–75. [PubMed]
Friedman DP, Aggleton JP, Saunders RC. Comparison of hippocampal, amygdala, and perirhinal projections to the nucleus accumbens: combined anterograde and retrograde tracing study in the Macaque brain. J Comp Neurol. 2002;450:345–365. [PubMed]
Friston KJ, Fletcher P, Josephs O, Holmes A, Rugg MD, Turner R. Event-related fMRI: characterizing differential responses. Neuroimage. 1998;7:30–40. [PubMed]
Gasbarri A, Packard MG, Campana E, Pacitti C. Anterograde and retrograde tracing of projections from the ventral tegmental area to the hippocampal formation in the rat. Brain Res Bull. 1994;33:445–452. [PubMed]
Gasbarri A, Sulli A, Packard MG. The dopaminergic mesencephalic projections to the hippocampal formation in the rat. Prog Neuropsychopharmacol Biol Psychiatry. 1997;21:1–22. [PubMed]
Godden DR, Baddeley AD. Context-dependent memory in two natural environments: On land and underwater. British Journal of Psychology. 1975;66:325–331.
Goto Y, Grace AA. Limbic and cortical information processing in the nucleus accumbens. Trends Neurosci. 2008;31:552–558. [PMC free article] [PubMed]
Helms G, Draganski B, Frackowiak R, Ashburner J, Weiskopf N. Improved segmentation of deep brain grey matter structures using magnetization transfer (MT) parameter maps. Neuroimage. 2009;47:194–198. [PMC free article] [PubMed]
Holscher C, Jacob W, Mallot HA. Reward modulates neuronal activity in the hippocampus of the rat. Behav Brain Res. 2003;142:181–191. [PubMed]
Huang YY, Kandel ER. D1/D5 receptor agonists induce a protein synthesis-dependent late potentiation in the CA1 region of the hippocampus. Proc Natl Acad Sci U S A. 1995;92:2446–2450. [PMC free article] [PubMed]
Hutton C, Bork A, Josephs O, Deichmann R, Ashburner J, Turner R. Image distortion correction in fMRI: A quantitative evaluation. Neuroimage. 2002;16:217–240. [PubMed]
Hutton C, Deichmann R, Turner R, Andersson JL. 2004. Combined correction for geometric distortion and its interaction with head motion in fMRI; Proceedings of ISMRM 12, Kyoto, Japan.
Ikemoto S. Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res Rev. 2007;56:27–78. [PMC free article] [PubMed]
Jay TM. Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Prog Neurobiol. 2003;69:375–390. [PubMed]
Johnson JD, Muftuler LT, Rugg MD. Multiple repetitions reveal functionally and anatomically distinct patterns of hippocampal activity during continuous recognition memory. Hippocampus. 2008;18:975–980. [PMC free article] [PubMed]
Kakade S, Dayan P. Dopamine: Generalization and bonuses. Neural Netw. 2002;15:549–559. [PubMed]
Knight R. Contribution of human hippocampal region to novelty detection. Nature. 1996;383:256–259. [PubMed]
Knutson B, Cooper JC. Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol. 2005;18:411–417. [PubMed]
Krebs RM, Schott BH, Schutze H, Duzel E. The novelty exploration bonus and its attentional modulation. Neuropsychologia. 2009;47:2272–2281. [PubMed]
Li S, Cullen WK, Anwyl R, Rowan MJ. Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty. Nat Neurosci. 2003;6:526–531. [PubMed]
Lisman JE, Grace AA. The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory. Neuron. 2005;46:703–713. [PubMed]
Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol. 1992;67:145–163. [PubMed]
Mesulam MM. From sensation to cognition. Brain. 1998;121(Pt 6):1013–1052. [PubMed]
Morris RG. Elements of a neurobiological theory of hippocampal function: The role of synaptic plasticity, synaptic tagging and schemas. Eur J Neurosci. 2006;23:2829–2846. [PubMed]
Niv Y, Daw ND, Joel D, Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 2007;191:507–520. [PubMed]
O’Carroll CM, Martin SJ, Sandin J, Frenguelli B, Morris RG. Dopaminergic modulation of the persistence of one-trial hippocampus-dependent memory. Learn Mem. 2006;13:760–769. [PMC free article] [PubMed]
O’Doherty JP, Deichmann R, Critchley HD, Dolan RJ. Neural responses during anticipation of a primary taste reward. Neuron. 2002;33:815–826. [PubMed]
O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. [PubMed]
Ploghaus A, Tracey I, Clare S, Gati JS, Rawlins JN, Matthews PM. Learning about pain: The neural substrate of the prediction error for aversive events. Proc Natl Acad Sci U S A. 2000;97:9281–9286. [PMC free article] [PubMed]
Purves D, Bonardi C, Hall G. Enhancement of latent inhibition in rats with electrolytic lesions of the hippocampus. Behav Neurosci. 1995;109:366–370. [PubMed]
Ranganath C, Rainer G. Neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci. 2003;4:193–202. [PubMed]
Rolls ET, Xiang JZ. Reward-spatial view representations and learning in the primate hippocampus. J Neurosci. 2005;25:6167–6174. [PubMed]
Rosene DL, Van Hoesen GW. Hippocampal efferents reach widespread areas of cerebral cortex and amygdala in the rhesus monkey. Science. 1977;198:315–317. [PubMed]
Selden NR, Gitelman DR, Salamon-Murayama N, Parrish TB, Mesulam MM. Trajectories of cholinergic pathways within the cerebral hemispheres of the human brain. Brain. 1998;121(Pt 12):2249–2257. [PubMed]
Smith Y, Kieval JZ. Anatomy of the dopamine system in the basal ganglia. Trends Neurosci. 2000;23(10 Suppl):S28–S33. [PubMed]
Sokolov EN. Higher nervous functions; the orienting reflex. Annu Rev Physiol. 1963;25:545–580. [PubMed]
Solomon PR, Vander Schaaf ER, Thompson RF, Weisz DJ. Hippocampus and trace conditioning of the rabbit’s classically conditioned nictitating membrane response. Behav Neurosci. 1986;100:729–744. [PubMed]
Strange BA, Fletcher PC, Henson RN, Friston KJ, Dolan RJ. Segregating the functions of human hippocampus. Proc Natl Acad Sci U S A. 1999;96:4034–4039. [PMC free article] [PubMed]
Sutton RS, Barto AG. Toward a modern theory of adaptive networks: Expectation and prediction. Psychol Rev. 1981;88:135–170. [PubMed]
Tabuchi ET, Mulder AB, Wiener SI. Position and behavioral modulation of synchronization of hippocampal and accumbens neuronal discharges in freely moving rats. Hippocampus. 2000;10:717–728. [PubMed]
Thierry AM, Gioanni Y, Degenetais E, Glowinski J. Hippocampo-prefrontal cortex pathway: Anatomical and electrophysiological characteristics. Hippocampus. 2000;10:411–419. [PubMed]
Tulving E. Memory and consciousness. Canadian Psychology. 1985;26:1–12.
Weiner I. The “two-headed” latent inhibition model of schizophrenia: Modeling positive and negative symptoms and their treatment. Psychopharmacology (Berl) 2003;169(3–4):257–297. [PubMed]
Weiskopf N, Helms G. Multi-parameter mapping of the human brain at 1mm resolution in less than 20 minutes. ISMRM 16, Toronto, Canada: 2008.
Weiskopf N, Hutton C, Josephs O, Deichmann R. Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: A whole-brain analysis at 3 T and 1.5 T. Neuroimage. 2006;33:493–504. [PubMed]
Wilson FA, Rolls ET. Learning and memory is reflected in the responses of reinforcement-related neurons in the primate basal forebrain. J Neurosci. 1990a;10:1254–1267. [PubMed]
Wilson FA, Rolls ET. Neuronal responses related to the novelty and familarity of visual stimuli in the substantia innominata, diagonal band of Broca and periventricular region of the primate basal forebrain. Exp Brain Res. 1990b;80:104–120. [PubMed]
Wirth S, Avsar E, Chiu CC, Sharma V, Smith AC, Brown E, Suzuki WA. Trial outcome and associative learning signals in the monkey hippocampus. Neuron. 2009;61:930–940. [PMC free article] [PubMed]
Wittmann BC, Schott BH, Guderian S, Frey JU, Heinze HJ, Duzel E. Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron. 2005;45:459–467. [PubMed]
Wittmann BC, Bunzeck N, Dolan RJ, Duzel E. Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage. 2007;38:194–202. [PMC free article] [PubMed]
Wittmann BC, Daw ND, Seymour B, Dolan RJ. Striatal activity underlies novelty-based choice in humans. Neuron. 2008;58:967–973. [PMC free article] [PubMed]
Yamaguchi S, Hale LA, D’Esposito M, Knight RT. Rapid prefrontal-hippocampal habituation to novel events. J Neurosci. 2004;24:5356–5363. [PubMed]