Cracking and Packing Information about the Features of Expected Rewards in the Orbitofrontal Cortex
- National Institute on Drug Abuse Intramural Research Program, Cellular Neurobiology Research Branch, Behavioral Neurophysiology Research Section, Baltimore, Maryland 21224
-
Author contributions: A.S., Y.K.T., A.J.L., T.A.S., and G.S. designed research; A.S. performed research; A.S. and T.A.S. analyzed data; A.S., Y.K.T., T.A.S., and G.S. wrote the paper.
Abstract
The orbitofrontal cortex (OFC) is crucial for tracking various aspects of expected outcomes, thereby helping to guide choices and support learning. Our previous study showed that the effects of reward timing and size on the activity of single units in OFC were dissociable when these attributes were manipulated independently ( Roesch et al., 2006). However, in real-life decision-making scenarios, outcome features often change simultaneously, so here we investigated how OFC neurons in male rats integrate information about the timing and identity (flavor) of reward and respond to changes in these features, according to whether they were changed simultaneously or separately. We found that a substantial number of OFC neurons fired differentially to immediate versus delayed reward and to the different reward flavors. However, contrary to the previous study, selectivity for timing was strongly correlated with selectivity for identity. Taken together with the previous research, these results suggest that when reward features are correlated, OFC tends to “pack” them into unitary constructs, whereas when they are independent, OFC tends to “crack” them into separate constructs. Furthermore, we found that when both reward timing and flavor were changed, reward-responsive OFC neurons showed unique activity patterns preceding and during the omission of an expected reward. Interestingly, this OFC activity is similar and slightly preceded the ventral tegmental area dopamine (VTA DA) activity observed in a previous study ( Takahashi et al., 2023), consistent with the role of OFC in providing predictive information to VTA DA neurons.
Significant Statement
Although multiple features of outcomes can change simultaneously in real-life decision-making scenarios, how orbitofrontal cortex (OFC) neurons integrate information about the reward timing and identity and respond to changes in these features remains unexplored. Here we found that OFC neurons integrate information about reward timing and identity when the two features changed simultaneously. Combining with prior research (Roesch et al., 2006), these findings suggest that OFC tends to integrate correlated features into unitary constructs while segregating independent features into separate constructs. Additionally, we observed distinct activities in general reward-responsive neurons preceding the omission of an expected reward when both identity and timing changed. This implies OFC might convey predictions to VTA that track reward timing separately based on reward identity.
Introduction
Animals, including human beings, make decisions according to the features of expected rewards, including their size, timing, and sensory characteristics. Neurons in the orbitofrontal cortex (OFC) show activity that correlates with features of expected outcomes (Schoenbaum and Eichenbaum, 1995; Feierstein et al., 2006; Padoa-Schioppa and Assad, 2006; Roesch et al., 2006; Furuyashiki et al., 2008; Kennerley et al., 2012; Takahashi et al., 2017), and OFC activity supports learning when actual outcomes diverge from expectations through its influence on dopaminergic (DA) prediction errors (Takahashi et al., 2011). Previous work from our lab using an odor-guided choice task in which reward timing and size were manipulated independently, in separate trial blocks, demonstrated that the effects of these variables on single-unit activity in OFC could be dissociated, with distinct populations of neurons responding to one or the other variable (Roesch et al., 2006). However, while isolating variables like this in an experimental setting may be useful, it may be more typical for animals to track several features of expected reward simultaneously to make decisions, and the tracking of such correlated features may change how OFC neurons integrate information about reward expectations.
Related to this question, we recently reported that DA neurons in the ventral tegmental area (VTA) responded differently to the omission of an expected reward when only reward timing was changed as compared with when both the timing and flavor changed together (Takahashi et al., 2023). When a reward was delivered earlier than expected and the flavor was also changed, DA neurons exhibited the characteristic suppression of activity at the expected reward time, typically interpreted as signaling a negative prediction error. However, when the reward was delivered earlier than expected but the flavor remained the same, this suppression at the expected reward time was absent. These results show that neural responses to changes in at least one aspect of reward—its timing—differ based on context and other information.
`Here, we addressed two questions from this prior work: (1) how do OFC neurons integrate information about timing with reward identity, and (2) does OFC activity in response to changes in these features differ according to whether they vary together or separately? To this end, we recorded in rat OFC in the odor-based choice task used previously to characterize the activity of DA neurons (Takahashi et al., 2023). This revealed three populations of OFC neurons: those firing in response to reward timing, those firing in response to reward flavor, and those responding to reward independently of its features. Neurons responding to reward timing and flavor tracked features similarly to those reported in the previous study (Roesch et al., 2006), but in contrast to that study, under these conditions the selectivity for reward timing was strongly correlated with selectivity for reward identity. Furthermore, when both timing and flavor of rewards were changed, reward-responsive neurons showed unique activity patterns preceding and during the omission of an expected reward, which were complementary to those previously observed in VTA DA activity (Takahashi et al., 2023).
Materials and Methods
Subjects
Four male TH-cre rats (Long–Evans background) were acquired from Charles River Laboratories when they weighed ∼300 g. The rats were ∼60 d old at the start of the training period. Throughout the training and recording phases, the animals underwent water restriction. They were provided with 10 min of ad libitum water each testing day, in addition to fluid rewards earned during the behavioral task. The food was available ad libitum in the home cage. Home cage food chow was obtained from Inotiv (2018 Teklad Global 18% Protein Rodent Diets) composed of 8.6% protein, 4.0% fat, and not >5% fiber (3.1 kcal/g). The rats were tested at the NIDA-IRP in compliance with NIH guidelines set by the Animal Care and Use Committee.
Surgical procedure and histology
Surgical procedures adhered to the sterile tips technique. Electrodes consisted of a drivable bundle comprising 16 25-µm-diameter NiCR/Formvar wires (A-M Systems). These bundles were chronically implanted just dorsal to the OFC in both hemispheres, with a starting position 3.0 mm anterior to the bregma, 3.2 mm laterally, and 4.0 mm ventral to the brain surface in each rat. Two rats (AS06 and AS12) had four bundles implanted (two bundles of 16 wires in each hemisphere), while the other two animals (AS08 and AS09) had eight bundles implanted (four bundles of 16 wires in each hemisphere). After the study, the final position of the wire tips was marked by passing a 15 µA current through each electrode. Euthanasia of the rats was conducted using an isoflurane overdose, followed by perfusion. Subsequently, the brains were removed from the skulls and processed for histology using standard techniques.
Behavioral task
Recording took place in aluminum chambers with a central odor port positioned above two adjacent fluid wells on one wall. Two house lights were situated above the port. The odor port was connected to an air flow dilution olfactometer, enabling rapid delivery of olfactory cues. Odors were based on compounds obtained from International Flavors and Fragrances. The fluid wells were connected to a fluid delivery line containing water and flavored milk (Horizon ORGANIC low fat chocolate or vanilla milk, diluted by 50% with water). Control of the behavioral task was accomplished using a custom C++ program.
Each recording session comprised a series of trials structured into five trial blocks. Rats could initiate a trial by poking into the odor port while the house lights were illuminated. After a 500 ms nose poke into the odor port, one of the three odor cues was delivered for 500 ms into a small hemicylinder located behind the opening of the odor port. Following the offset of the odor, the rat could respond by moving to one of the two fluid wells within 3.0 s. After making a choice, the rat had to wait for a delay before fluid delivery commenced. If the rat exited the fluid well during the delay, no reward was presented and the trial ended (Fig. 1A). The identity of the odor determined whether the rat could obtain a reward at the left well (forced-choice left), right well (forced-choice right), or either well (free-choice). Odor cues were presented in a pseudorandom sequence, with the constraint that the free-choice odor appeared in 7 out of 20 trials, while the forced-choice odors were presented in approximately equal numbers over each trial block. Additionally, the same odor could not be presented on more than three consecutive trials.
Task design and behavior. A, Schematic of the odor-guided choice task. After the rats poked into the odor port, one of three odors was delivered for 500 ms. Following the offset of the odor, rats responded by moving to one of the two fluid wells, with the specific odor determining if reward was available at the left well (forced-choice left), right well (forced-choice right), or either well (free-choice). After making a choice, rats waited for a short (0.5 s) or long (3.0 s) delay before a single drop of flavored milk (chocolate or vanilla) was delivered according to the schedule for that well on that block. In both wells, a single drop of water was also delivered at a 7.0 s delay from well entry as a terminal reward. B, Schematic of the block sequence in one session. In the first block, the combination of the delay length (0.5 or 3.0 s) and flavored milk (chocolate or vanilla) was randomly assigned to one well, with the opposite delay and flavor scheduled for the other well. At each block transition, the delays to the flavored reward in the two wells were switched such that both delay and flavors changed (delay–flavor; first and fourth switches), or the delay was switched without changing the flavors (delay-only; second and third switches). C, The rats showed equal preference for the chocolate- and vanilla-flavored milk in a choice test (t(6) = 1.6; p = 0.17). D, The average choice rate of the short-delay side across the last five trials of the previous block (Pre), the first five trials (First), and last five trials (End) of the block after delay–flavor (left) and delay-only (right) switches. The rats successfully adjusted their behavior to the new response–delay contingencies in both delay-only and delay–flavor switches, and this preference shift was independent of milk flavor and transition condition (two-way ANOVA on choice rates across delay–flavor switches: main effect of flavor, F(1,3) = 1.1; p = 0.37; main effect of trial, F(2,6) = 87.8; p < 0.001; interaction, F(2,6) = 1.9; p = 0.24; two-way ANOVA on choice rates across delay-only switches: main effect of flavor, F(1,3) = 1.4; p = 0.32; main effect of trial, F(2,6) = 20.9; p < 0.01; interaction, F(2,6) = 0.9; p = 0.43). E, F, Average reaction time (E) and correct rate (F) of the last five trials of the previous block (Pre), the first five trials (First), and last five trials (End) of the block after delay–flavor (left) and delay-only (right) switches in short and long forced-choice trials of each flavor. The rats’ correct rates and reaction times were similar across the transitions, and milk flavor or transition condition did not affect these measures (two-way ANOVA on reaction times: long trials after delay–flavor switches, flavor, F(1,3) = 0.05; p = 0.84; trial, F(2,6) = 2.0; p = 0.21; interaction, F(2,6) = 0.08; p = 0.92; long trials after delay-only switches, flavor, F(1,3) = 1.0; p = 0.39; trial, F(2,6) = 1.5; p = 0.29; interaction, F(2,6) = 0.01; p = 0.99; short trials after delay–flavor switches, flavor, F(1,3) = 1.0; p = 0.39; trial, F(2,6) = 0.9; p = 0.44; interaction, F(2,6) = 0.4; p = 0.69; short trials after delay-only switches; flavor, F(1,3) = 2.7; p = 0.20; trial, F(2,6) = 0.5; p = 0.66; interaction, F(2,6) = 2.6; p = 0.15; two-way ANOVA on correct rates: long trials after delay–flavor switches, flavor, F(1,3) = 0.02; p = 0.90; trial, F(2,6) = 3.7; p = 0.089; interaction, F(2,6) = 2.9; p = 0.13; long trials after delay-only switches, flavor, F(1,3) = 0.00; p = 0.99; trial, F(2,6) = 2.4; p = 0.17; interaction, F(2,6) = 2.8; p = 0.14; short trials after delay–flavor switches, flavor, F(1,3) = 0.04; p = 0.86; trial, F(2,6) = 2.2; p = 0.19; interaction, F(2,6) = 2.1; p = 0.20; short trials after delay-only switches, flavor, F(1,3) = 0.3; p = 0.65; trial, F(2,6) = 0.4; p = 0.70; interaction, F(2,6) = 1.9; p = 0.23). The error bars in each figure represent SEM.
The delay preceding reward delivery differed between the fluid wells (Fig. 1A). At the start of the session, one well was randomly designated as the short-delay well and the other as the long-delay well. In the short-delay well, a drop of chocolate- or vanilla-flavored milk was delivered at 0.5 s after well entry, and a drop of water was delivered at 7.0 s after well entry (short trial). In the long-delay well, a drop of the other-flavored milk was delivered at 3.0 s after well entry, and a drop of water was again delivered at 7.0 s after well entry (long trial). The terminal reward at 7.0 s ensured that the rats remained in the well across the delays for both short- and long-trial types. On free-choice trials, the delay and flavor of reward was determined by which well the rat chose, such that it was the same as that on the forced-choice of that same well.
After the first block, there were four additional blocks such that delays and flavors were switched at each block transition. In the first and fourth transitions, both the flavor and the delay were switched to their opposites at each well. For example, if the first block had long chocolate on the left and short vanilla on the right, the second block would have short vanilla on the left and long chocolate on the right. On the second and third transitions, only the delay was changed to its opposite at both wells, with the flavors remaining the same. The length of each block was determined by selecting a random minimum number of trials, between 45 and 50, and in addition requiring that block transitions only occur when rats had chosen the short-delay well in at least 6 of the last 10 free-choice trials. If this performance criterion was not met, the block continued beyond the minimum number of trials until it was met. Importantly, this criterion was primarily used in shaping; performance in the recording sessions was much higher than this (Fig. 1). Single units were only analyzed from sessions in which rats successfully completed all five blocks.
Single-unit recording
The procedures followed those described in a previous study (Stalnaker et al., 2021). Each day, rats were placed in the training box, and neuronal activity recorded from each electrode was screened. If no single-unit activity was detected, the rats were removed from the training box, and the electrode wire assembly was advanced by either 40 or 80 µm. Otherwise, the session proceeded while neuronal activity was recorded along with timestamps provided by the behavioral program. Recordings utilized a Plexon OmniPlex system (Plexon) interfaced with the aforementioned training chamber. Signals from the electrode wires underwent amplification and filtering using standard procedures outlined in prior publications.
Data analysis
We sorted units using Plexon’s Offline Sorter software, employing a template-matching algorithm. The sorted files were subsequently processed and analyzed in MATLAB and R.
To analyze the effect of reward delay on neuronal activity, we examined the firing rate in 1,000 ms epochs following the short-delay rewards (referred to as the short epoch) and the long-delay rewards (referred to as the long epoch). We subtracted the firing rate on each trial during these epochs from the baseline activity, which was defined as the firing rate from the onset of the house light to the initiation of the odor at the beginning of the same trial. We also separately calculated these firing rates for each well. To determine if a neuron responded to specific reward delays, we conducted a one-sample t test (one-sided) to identify cases when the firing rate in particular epochs had value significantly >0 Hz. Since the firing rates were subtracted from the baseline, such a value would indicate that the given neuron responded during that epoch. Furthermore, we examined whether each neuron exhibited a side preference in its activity by conducting a t test between on firing rates in the left well and the right well during the same epoch.
Based on the results of the two t tests, we categorized the neurons as short-responsive, long-responsive, or nonspecifically reward-responsive. If a neuron demonstrated a significant increase in the short-delay epoch but not in the long-delay epoch, it was defined as a short-responsive neuron. Conversely, if a neuron displayed a significant increase in the long-delay epoch but not in the short-delay epoch, it was designated as a long-responsive neuron. Finally, if a neuron exhibited a significant increase in both the long-delay epoch and the short-delay epoch, it was classified as a reward-responsive neuron.
We also investigated the effects of reward identity by calculating the firing rate following the presentation of vanilla-flavored milk or chocolate-flavored milk. This firing rate was obtained from a 1,000 ms epoch after the short-delay reward in the short trial and after the long-delay reward in the long trial when a specific flavored milk was delivered. Similar to the previous method, this firing rate was subtracted from the baseline activity. To identify identity-responsive neurons, we conducted a one-sample t test (one-sided) to find firing rates in these epochs with values significantly higher than 0 Hz. Based on the results of the two t tests, we categorized the neurons as vanilla-responsive and chocolate-responsive neurons. If a neuron demonstrated a significant increase in the vanilla-flavored milk epoch but not in the chocolate-flavored milk epoch, it was defined as a vanilla-responsive neuron and likewise for chocolate-responsive neurons. In the t test analysis for both reward delay and reward flavor, we used neuronal activity from correct forced-choice trials across all five blocks, excluding the first five trials in each block on each side to avoid effects of the contingency switch.
To quantify the effect of reward timing and flavor selectivity, we computed delay and flavor indices using the following formulas:<mml:math display="block"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mrow><mml:mi
mathvariant="normal">Delay</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">Index</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mo>=</mml:mo><mml:mo
stretchy="false">(</mml:mo><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">short</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi><mml:mspace width=".1em"></mml:mspace></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace
width="1em"></mml:mspace><mml:mrow><mml:mtext></mml:mtext><mml:mo>–</mml:mo></mml:mrow><mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mi
mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">long</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">epoch</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>/</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace
width="1em"></mml:mspace><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi mathvariant="normal">absolute</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">value</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">of</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">short</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace
width="1em"></mml:mspace><mml:mo>+</mml:mo><mml:mrow><mml:mi mathvariant="normal">absolute</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">value</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">of</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">long</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi></mml:mrow><mml:mo
stretchy="false">)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math>DelayIndex=(averagefiringrateintheshortepoch–averagefiringrateinthelongepoch)/(absolutevalueofaveragefiringrateintheshortepoch+absolutevalueofaveragefiringrateinthelongepoch),
<mml:math display="block"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mrow><mml:mi
mathvariant="normal">Flavor</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">Index</mml:mi></mml:mrow></mml:mtd><mml:mtd><mml:mo>=</mml:mo><mml:mrow><mml:mi
mathvariant="normal">absolute</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">value</mml:mi><mml:mspace
width=".1em"></mml:mspace></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">vanilla</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi><mml:mspace
width=".1em"></mml:mspace></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace width="1em"></mml:mspace><mml:mrow><mml:mtext></mml:mtext><mml:mo>–</mml:mo></mml:mrow><mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">chocolate</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>/</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace
width="1em"></mml:mspace><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi mathvariant="normal">absolute</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">value</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">of</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">rate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">vanilla</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">epoch</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace
width="1em"></mml:mspace><mml:mo>+</mml:mo><mml:mrow><mml:mi mathvariant="normal">absolute</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">value</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">of</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">average</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">firing</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">rate</mml:mi></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd></mml:mtd><mml:mtd><mml:mspace width="2em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">in</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">the</mml:mi></mml:mrow><mml:mspace
width=".1em"></mml:mspace><mml:mrow><mml:mi mathvariant="normal">chocolate</mml:mi></mml:mrow><mml:mspace width=".1em"></mml:mspace><mml:mrow><mml:mi
mathvariant="normal">epoch</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>,</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math>FlavorIndex=absolutevalue(averagefiringrateinthevanillaepoch–averagefiringrateinthechocolateepoch)/(absolutevalueofaveragefiringrateinthevanillaepoch+absolutevalueofaveragefiringrateinthechocolateepoch),
The rationale for using the absolute value in the numerator for flavor indices is that flavor preference would not be directional
as to preference, because the two flavors were equally preferred by rats, whereas delay preference could be directional as
to preference, because rats tended to prefer the short-delay rewards. Thus, a positive delay index indicates a neuronal selectivity
for short-delay reward, while a negative value indicates a neuronal selectivity for long-delay reward. For the flavor indices,
in contrast, a value closer to one would indicate selectivity for one of the two flavored milks epoch. To test the relationship
between the neural representation of reward delay and reward identity, we conducted a correlation analysis between these indices.
To assess the impact of the delay–flavor switches and delay-only switches, we conducted a comparison of firing rates during reward delivery between delay–flavor switch blocks and the delay-only switch blocks. Neural activity was obtained from a 1,000 ms epoch corresponding to time when the reward would have been delivered on the previous block but was omitted on this block (3.0–4.0 s after well entry), on blocks and sides with short reward delivery (called short trials). We focused on this particular epoch because that is where our hypothesis predicted we would see differential activity. In addition, to assess the predictive activity, we tested neural activity during a 200 ms epoch immediately preceding the omission epoch (2.8–3.0 s on short trials). The analysis of delay–flavor and delay-only switches involved comparing neuronal activities between the first trial after the block transition and the average of the last five trials on the same block and side, using a two-way ANOVA followed with Bonferroni’s post hoc testing (p < 0.05).
To test the influence of partial information regarding the new reward–response contingency following block transitions, we partitioned the first trial following block transitions into “T1 trials” and “inference trials.” Inference trials occurred when the first trial on the short-delay side after the block transition was preceded by at least one trial after the block transition on the other side. Because of the pseudorandom ordering of trial types, this condition occurred on about half of block transitions. We called these inference trials, because we have previously found behavioral evidence that rats infer from a reward change at one well that the next reward at the other well will also be changed. T1 trials, in contrast, were those first trials after block transitions that were not preceded by trials in which the other well was rewarded after the block transition, and thus rats would have no way to infer that a change would be coming on that first trial. Note that T1 trials and inference trials occurred in different blocks and in some cases in different sessions, and thus neurons were not generally recorded in both conditions. Neuronal activities were collected from the same 1,000 ms omission epoch on short trials (3.0–4.0 s after well entry on short trial) and the same 200 ms epoch immediately before the omission epoch, again comparing the delay–flavor and delay-only switches. To explore the effect of partial insight into the new contingency on OFC activity, we compared neuronal activities between inference trials and T1 trials in both conditions. Our hypothesis posited that the firing rate on T1 trials after the delay–flavor switches would show more differential activity, compared with the end of the block and compared with inference trials. The reason for this prediction is that the differential activity is theoretically based on the block change just having happened, and thus we predicted it would be dampened on inference trials when rats have partial knowledge that the block change has already occurred. To assess this hypothesis, we performed a Student's t test (p < 0.05) comparing activity on T1 trials and inference trials in each condition and epoch. Note that we did not use an ANOVA for this test because the T1 and inference trials occurred partly on different sessions (depending on the pseudorandom ordering of trials) and thus could not be compared within the same set of neurons. In addition, to examine how the neuronal activity on T1 trials and inference trials are changed from the stable pattern of neuronal activity that developed later in the block as rats learned the new contingencies, we compared activity at each epoch on the T1 or inference trials with average neuronal activity on the last five trials of the same block using a one-sided Wilcoxon's sign ranked test (p < 0.05).
To assess the implications of partial information regarding the new reward–response contingencies on VTA DA activity, we did a parallel analysis to that described above, again comparing activity between T1 and inference trials, by reanalyzing recordings of VTA DA neurons about which an analysis was previously published (Takahashi et al., 2023).
Results
Rats (n = 4) underwent training in an odor-guided choice task identical to one used in a previous study (Takahashi et al., 2023). On each trial, they sampled one of the three different odor cues at a central port on each trial and then had to respond at either a left or a right well to receive two drops of reward (Fig. 1A). Each odor designated a different trial type, which were distributed in approximately equal proportions and ordered pseudorandomly. One odor signaled reward availability in the left well (forced-choice left), a second odor signaled reward availability in the right well (forced-choice right), and a third odor signaled availability of reward at either well (free-choice). In each block of trials, one of the two similarly preferred flavored milk solutions (chocolate or vanilla; Fig. 1C) was delivered after a short delay (0.5 s) following a correct entry into one well, and the other flavored milk was delivered after a longer delay (3 s) following a correct entry into the other well. A bolus of water was also delivered at 7 s after correct responses at either well, serving to encourage rats to wait at the well across all time periods and terminating the trials.
Rats were trained on the task before the start of recording, and then, during recording, the delay and flavor of the first reward was manipulated across four transitions between five blocks of trials (Fig. 1B). In two of these transitions (the first and the fourth), both the delay and flavor of the first drop was changed at both wells (delay–flavor switches), while in the other two transitions (the second and the third), only delay was changed, again at both wells (delay-only switches). These experimental manipulations aimed to compare the response of OFC neurons with changes in reward timing alone (delay-only switches) versus changes in reward timing combined with reward identity (delay–flavor switches). The order of the transitions was varied from session to session by randomly selecting a delay and flavor arrangement to occur on the first block. Subsequent block transitions were scheduled to occur after a randomly chosen number of trials (between 45 and 50) and additionally required rats to have chosen the immediate reward on at least 6 of the last 10 free-choice trials. Note that this was a minimum requirement to allow a block transition, and in most cases and on average, rats chose the immediate reward at higher rates by the end of the block (on average 71.5% by the last five trials of blocks). On forced-choice trials, the average correct rate was 88.1%.
Choice behavior reflected timing but not flavor or switch type
We first investigated the influence of reward delay and flavor on task-related behavioral patterns. Across transitions, rats successfully adjusted their behavior to the new response–delay contingencies in both delay-only and delay–flavor transition types, exhibiting a preference on free-choice trials for the well associated with the short delay. By the last five free-choice trials of blocks, rats were, on average, choosing the immediate, or short-delay, reward 71.5% of the time, as compared with 43.1% of the time during the first five trials of the same blocks. Importantly, this preference shift was independent of milk flavor and transition condition (Fig. 1D; statistics applicable here and in most cases throughout the Results section are listed in the figure legends). In addition, during forced-choice trials, correct rates and reaction times after delay–flavor switches and delay-only switches were similar (Fig. 1E,F). We conclude that rats responded to changes in delay but that delay-only and delay–flavor transition types did not differentially affect task-related behavioral patterns.
Single-unit activity in OFC was affected by both the timing and flavor of reward
We next assessed whether OFC neurons distinguished different reward delays and reward flavors and whether there was any relationship between neurons that tracked one versus the other. We recorded a total of 2,295 neurons during performance of full sessions of this task (the position of electrodes tips at the last recording session was illustrated in Fig. 2A). Using an approach similar to that employed in prior work (Roesch et al., 2006), we examined activity during an epoch starting at reward delivery and lasting for 1.0 s, on different delay conditions (short and long) and on different flavor conditions (chocolate and vanilla). To match the analysis in the prior paper, we conducted a one-sided t test comparing the firing rate from these epochs to baseline activity. Importantly, the task design was not fully balanced at each well location within a single session, causing one reward to more often appear as the immediate reward. Although we randomly interleaved session with both reward arrangements to eliminate this bias across sessions, this did require us to make two adjustments in the neural analysis. First, neurons whose firing was affected by the well location, defined as those that responded significantly differently after reward delivery in one well compared with the other well, were excluded from subsequent analyses. Second, we randomly sampled neurons from sessions with each of the two reward arrangements in order to equalize their numbers.
Recording sites and characterization of delay length and reward identity-selective activity. A, The colored dots indicate the approximate location of tips of electrode probes at the last recording session, with both hemispheres combined, and the same shade dots mean the recording sites from both hemispheres of the same rat. Two rats had four bundles of electrodes and thus have four dots shown, while the other two rats had eight bundles of electrodes and thus have eight dots shown. Bregma, +2.7 to +3.7 mm; scale bar, 1 mm. B, C, The colored sections represent the proportion of neurons with significant effects of delay length (B) or milk flavor (C) based on a t test performed on a firing rate in the 1,000 ms immediately after milk delivery across forced-choice trials (see Materials and Methods). The inset pie charts represent breakdowns for each category of neuron to illustrate the degree of overlap between neurons showing significant effects of flavor and significant effects of timing. D, Mean normalized population firing of short-responsive neurons (red line), long-responsive neurons (blue line), and general reward-responsive neurons (gold line) during short and long trials. Activity is aligned on well entry. The reward-responsive population responded phasically whenever reward occurred, including the trial-terminating drop of water (one-way ANOVA on a firing rate at the short reward epoch, 0.5–1.5 s: neuron classification, F(2,798) = 50.2; p < 0.001; post hoc analysis: short-responsive vs reward-responsive cells, t(798) = 9.5; p < 0.001; long-responsive vs reward-responsive cells, t(798) = 5.8; p < 0.001; one-way ANOVA on activity at the long reward epoch, 3.0–4.0 s, neuron classification, F(2,798) = 50.0; p < 0.001; post hoc analysis, short-responsive vs reward-responsive cells, t(798) = 9.3; p < 0.001; long-responsive vs reward-responsive cells, t(798) = 4.4; p < 0.001; one-way ANOVA on activity at the terminal reward epoch, 7.0–8.0 s, neuron classification, F(2,798) = 35.4; p < 0.001; post hoc analysis, short-responsive vs reward-responsive cells, t(798) = 7.5; p < 0.001; long-responsive vs reward-responsive cells, t(798) = 5.8; p < 0.001; paired t test for comparing neuronal activities between at the terminal reward epoch, 7.0–8.0 s, and before terminal reward epoch 6.0–7.0, short-responsive neurons, t(203) = 3.4; p < 0.001; long-responsive neurons, t(152) = 1.6; p = 0.12; reward-responsive, t(443) = 7.1; p < 0.001). Shaded area represents SEM. E, Relation of firing dependent on delay length to firing dependent on identity of flavored milk for significantly short-responsive neurons (red dots), significantly long-responsive neurons (blue dots), and reward-responsive neurons (gold dots). Taken together, all task-responsive neurons showed no significant correlation between delay–response and flavor–response (n = 801; r = −0.03; p = 0.35; regression line not shown). However, specific populations of delay-responsive neurons showed strong and significant correlations between delay and flavor indices, such that the stronger the response to delay, in either direction, the stronger the response to one of the two flavors (short-responsive neurons, red dashed line, n = 204; r = 0.65; p < 0.001: long-responsive neurons, blue dashed line, n = 153; r = −0.71; p < 0.001).
After removing neurons affected by well location and equally sampling the two session types, there were 1,514 neurons; 11.5% of these cells exhibited increased firing only after the short-delayed reward (short-responsive), 8.6% exhibited increased firing only after the long-delayed reward (long-responsive), and 24.7% showed increased firing after reward regardless of delay (reward-responsive; Fig. 2B). The proportion of short-responsive neurons was significantly higher than that of long-responsive neurons (χ2(1) = 9.6; p < 0.01). Figure 2D shows the normalized mean population firing rate of each subpopulation of reward-timing neurons when long-delayed and short-delayed rewards were delivered. It is notable that the reward-responsive population (shown in gold) responded with a higher firing rate than either the short- or long-responsive populations and that this population responded phasically whenever reward occurred, including the trial-terminating drop of water. Notably each in population, neural activity began to increase prior to and peaked at the time of reward delivery, consistent with the general idea that the activity reflects the expectation of reward delivery rather than reward delivery itself (Schoenbaum et al., 1998).
To identify flavor-responsive neurons, we performed a corresponding analysis across different flavor conditions (chocolate reward delivery vs vanilla reward delivery). An 8.1% of cells exhibited an increased firing rate exclusively after the delivery of vanilla-flavored milk, 8.9% of cells exhibited an increased firing rate only after the delivery of chocolate-flavored milk, and 25.3% of cells showed increased firing after reward regardless of milk flavor (Fig. 2C). The proportions of vanilla-responsive versus chocolate-responsive neurons were not significantly different (χ2(1) = 0.9; p = 0.34), and the combined proportion of specific flavor-responsive neurons was not significantly different from that of specific reward-timing neurons (χ2(1) = 2.3; p = 0.13).
We next quantified the extent to which the reward-timing populations overlapped with the flavor-responsive populations. As seen in the insets to Figure 2B, the majority of selective delay-responsive neurons also responded selectively to one of the two flavors (67% in short-responsive neurons and 62% in long-responsive neurons), and the proportions of flavor-responsive neurons in short-responsive neurons versus long-responsive neurons were not significantly different (vanilla-responsive neurons, χ2(1) = 1.3; p = 0.25; chocolate-responsive neurons χ2(1) < 0.001; p = 0.99). As seen in the insets to Figure 2C, the majority of flavor-responsive neurons also responded selectively to one of the two delay conditions (77% in vanilla-responsive neurons and 76% in chocolate-responsive neurons), and the proportions of delay-responsive neurons in vanilla-responsive neurons versus chocolate-responsive neurons were not significantly different (short-responsive neurons, χ2(1) = 0.5; p = 0.46; long-responsive neurons χ2(1) = 0.3; p = 0.58). Conversely, almost all neurons that responded nonselectively to reward regardless of delay also responded nonselectively regardless of flavor (93%) and vice versa (91%). This suggests that there were two largely separate populations of task-responsive OFC neurons: those that responded to specific flavors and specific delays and those that generally responded to reward delivery, irrespective of delay or flavor.
Finally, we examined whether the magnitude of the effect of delay on firing was correlated with the magnitude of the effect of flavor on firing. For this purpose, delay and flavor indices were computed for each neuron from the average firing rates during the epoch after the first milk reward was delivered, across all trials (see Materials and Methods). A positive delay index meant that the cell fired more strongly for rewards delivered after short delay, while a negative index meant that the cell fired more strongly for rewards delivered after long delay. Flavor indices, conversely, were nondirectional, with a greater value indicating that the cell fired more strongly for one of the flavored milk rewards than the other. As illustrated in Figure 2E, there was no correlation between delay and flavor indices across all task-responsive neurons (reward timing or flavor-responsive; n = 801; r = −0.03; p = 0.35; regression line not shown). However, neurons that responded selectively to one of the delay conditions showed a strong correlation between delay indices and flavor indices, such that the stronger the delay selectivity (in either direction), the stronger the flavor selectivity (Fig. 2E, blue and red dotted lines; short-responsive neurons, n = 204; r = 0.65; p < 0.001; long-responsive neurons, n = 153; r = −0.71; p < 0.001). This is again consistent with the idea that there were two largely separate populations, one that carried information about the delay and flavor of reward after it was delivered and a another that responded to reward without carrying specific information about the delay or flavor of that reward.
OFC single units responded differently to delay–flavor transitions versus delay-only transitions
We next sought to compare activity across delay-only and delay–flavor transitions in the population that responded nonselectively to features (general reward-responsive neurons) and in other populations that responded to specific features. We tested activity based on the results of a previous study using the same task in which we recorded DA neurons in the VTA. There we found that after delay–flavor switches, DA neurons responded to reward omission after an earlier than expected reward with a classic suppression of activity, interpreted as signaling a negative prediction error. In contrast, after delay-only transitions, they failed to show this suppression—as if reward expectations were reset by early reward, provided the flavor had not changed. Because OFC neurons have been found to convey reward expectancy information to VTA DA neurons, here we sought to test whether OFC neurons might also fire differently on these trial types.
To match the DA neuronal analysis, we examined OFC activity in both the populations thus far identified after delay-only versus delay–flavor transitions during reward omission at the later timepoint. Indeed, like DA neurons, general reward-responsive OFC neurons fired differently on the first trial after the switch compared with the end of that block, but only after delay–flavor switches and not after delay-only switches. This difference in firing began just before the reward was delivered (Fig. 3A), suggesting that it carried predictive information, as contrasted with putative error signals in VTA DA neurons, which occurred only after reward delivery on these trials.
OFC reward-responsive neurons responded differentially to an expected but omitted reward on the first trial after the flavor delivered earlier had changed (delay–flavor switches) but not when the flavor delivered earlier had remained the same (delay-only switches). A–E, Line plots in top panels show mean normalized firing rates on the first trial (yellow line) after block transitions and last five trials of the same block (black line) on short trials after delay–flavor switches (left) and delay-only switches (right), in the nonspecific reward-responsive population (A), the short-responsive population (B), the long-responsive population (C), the vanilla-responsive neurons, and the chocolate-responsive neurons (D). Gray-shaded areas indicate the time reward would be expected based on delivery in the previous block (3.0–4.0 s); blue-shaded areas indicate the time right before expected reward delivery (2.8–3.0 s). Shading around lines represents SEM. Plots in right panels show the same data averaged across the indicated epochs. Error bars represent SEM. Only reward-responsive neurons showed differential firing on the first trial after delay–flavor switches and not delay-only switches, which parallels the pattern seen in VTA DA neurons in an earlier published study, except that OFC neurons showed this pattern before the omission timing epoch in addition to during it (for reward-responsive neurons, two-way ANOVA on mean firing rates in prereward epoch, 2.8–3.0 s, switch type, F(1,443) = 4.1; p < 0.01; trial, F(1,443) = 3.0; p = 0.085; interaction, F(1,443) = 4.4; p < 0.05; post hoc analysis, switch type on the first trial, F(1,443) = 4.7; p < 0.05; switch type on last 5 trials, F(1,443) = 0.0; p = 0.87; trial on delay–flavor switches, F(1,443) = 5.6; p < 0.05; trial on delay-only switches, F(1,443) = 0.2; p = 0.62; two-way ANOVA on mean firing rates in reward-expectation epoch, 3.0–4.0 s, switch type, F(1,443) = 5.7; p < 0.05; trial, F(1,443) = 0.9; p = 0.34; interaction, F(1,443) = 10.2; p < 0.01; post hoc analysis, switch type on first trial, F(1,443) = 9.1; p < 0.01; switch type on last 5 trials, F(1,443) = 1.6; p = 0.20; trial on delay–flavor switches, F(1,443) = 4.9; p < 0.05; trial on delay-only switches, F(1,443) = 1.7; p = 0.19; for short-responsive neurons, two-way ANOVA on mean firing rates in prereward epoch, 2.8–3.0 s, switch type, F(1,203) = 1.4; p = 0.25; trial, F(1,203) = 1.2; p = 0.27; interaction, F(1,203) = 0.6; p = 0.42; two-way ANOVA on mean firing rates in reward-expectation epoch, 3.0–4.0 s, switch type, F(1,203) = 1.8; p = 0.19; trial, F(1,203) = 2.0; p = 0.16; interaction: F(1,203) = 0.7, p = 0.42; for long-responsive neurons, two-way ANOVA on mean firing rates in prereward epoch, 2.8–3.0 s, switch type, F(1,152) = 0.0; p = 0.89; trial, F(1,152) = 0.0; p = 0.98; interaction, F(1,152) = 0.0; p = 0.99; two-way ANOVA on mean firing rates in reward-expectation epoch, 3.0–4.0 s, switch type, F(1,152) = 0.7; p = 0.42; trial, F(1,152) = 0.0; p = 0.94; interaction, F(1,152) = 3.8; p = 0.054; for vanilla-responsive neurons, two-way ANOVA on mean firing rates in prereward epoch, 2.8–3.0 s, switch type, F(1,147) = 0.9; p = 0.34; trial, F(1,147) = 0.1; p = 0.82; interaction, F(1,147) = 0.4; p = 0.53; in reward-expectation epoch, 3.0–4.0 s, switch type, F(1,147) = 0.0; p = 0.99; trial, F(1,147) = 2.0; p = 0.16; interaction, F(1,147) = 1.2; p = 0.29; for chocolate-responsive neurons, prereward epoch, switch type, F(1,155) = 0.2; p = 0.67; trial, F(1,155) = 0.0, p = 0.98; interaction, F(1,203) = 0.1; p = 0.94; reward-expectation epoch, switch type, F(1,155) = 0.9; p = 0.34; trial, F(1,155) = 5.1; p < 0.05; interaction, F(1,155) = 0.2; p = 0.66).
The above pattern of results was only true of the general reward-responsive OFC population. Neither the specific short-responsive nor the long-responsive populations showed differential activity after delay-only versus delay–flavor transitions (Fig. 3B,C). The same is true of the specific flavor-responsive populations (Fig. 3D,E). These results suggest that the general reward-responsive OFC population carries separate predictions according to the combination of delay and flavor, even though this population itself does not tend to respond differentially to delay or flavor.
To further test the idea that general reward-responsive OFC neurons signal transition-specific predictive information, we took advantage of a special characteristic of the task we used, namely, that transitions at each well occurred in conjunction with transitions at the other well. As we have previously reported, changes in reward at one well can be used by rats to anticipate changes at the other well. We reasoned that the differential activity observed on the first trial after delay–flavor transitions would be especially strong when it occurred in the first well the rat experienced upon the transition, as compared with when it occurred at the other well, by which point OFC neurons could have already begun to adapt to the new delay–flavor combination. We tested this by separating delay-only and delay–flavor transitions into those on which the new short-delayed reward occurred at the first well experienced and those on which the new short-delayed reward occurred at the other well. We called these two conditions the T1 trials and the inference trials, respectively. In this analysis, we focused on neural activity during reward omission on short T1 and inference trials after both the delay-only and delay–flavor transitions. Figure 4A shows the normalized mean population firing rate of reward-responsive neurons during T1 and inference trials in both reward transition types. As shown in Figure 4B, we again found that differential activity was only observed on the first trial after delay–flavor transitions and only on the T1 trials. On the inference trials, in contrast, there was no differential activity during this epoch after delay–flavor or delay-only switches. Again, the differential activity on the T1 trial after delay–flavor switches began prior to the epoch when reward would have been expected, in contrast to DA neurons. This result provides additional evidence that OFC reward-responsive neurons signaled transition-specific predictive information.
OFC reward-responsive neurons responded differentially to an expected but omitted reward on the first trial after the flavor delivered earlier had changed (T1 trials after delay–flavor switches) but not when the flavor had changed and the change could be inferred (inference trials after delay–flavor switches), and also not on either kind of trial when the flavor had remained the same (delay-only switches). A, Line plots in top panels show mean normalized firing rates of reward-responsive neurons on T1 trials (brown line) and inference trials (green line) after delay–flavor switches (left) and delay-only switches (right). Inference trials are the first trials that occur when a preceding trial on the other side could be used to infer that the flavor and timing would change on that side; T1 trials are the first trials that occur without a preceding trial on the other side. Gray-shaded areas indicate the time reward would be expected based on delivery in the previous block (3.0–4.0 s); blue-shaded areas indicate the time right before expected reward delivery (2.8–3.0 s). Shading around lines represents SEM. B, The same data averaged across the indicated epochs and compared with that on the last five trials of the same blocks. Firing rate on T1 trials was significantly lower than that on inference trials during both the prereward epoch and the reward omission epoch after delay–flavor switches, but differential firing was not found on T1 trials versus inference trials after delay-only switches in either epoch, nor was it found on the last five trials in any condition (T1 and inference trials occurred in different sessions and thus between-subjects two-sided t tests were run comparing activity in the two conditions; delay–flavor, 1st trial at prereward epoch, 2.8–3.0 s, t(631) = −5.5; p < 0.001; 1st trial at reward-expectation epoch, 3.0–4.0 s, t(631) = −4.9; p < 0.001); delay-only, 1st trial at prereward epoch, t(670) = −0.6; p = 0.55; 1st trial at reward-expectation epoch, t(670) = 1.0; p = 0.33; delay–flavor, last 5 trials at prereward epoch, t(631) = 0.6; p = 0.65; last 5 trials at reward-expectation epoch, t(631) = 0.8; p = 0.40; delay-only, last 5 trials at prereward epoch, t(670) = −1.8; p = 0.07; last 5 trials at reward-expectation epoch, t(670) = −0.8; p = 0.40). The error bars represent SEM. C, D, Histograms show the distribution of differences between firing rates on T1 trials (C) or inference trials (D) and firing rates on the last five trials of the same blocks during the prereward epoch (2.8–3.0 s) and reward-expectation epoch (3.0–4.0 s). Histograms on the left side are from delay–flavor switches and on the right side are from delay-only switches. The numbers in each panel indicate the results of one-sided Wilcoxon's signed rank tests (p) and the average difference scores (u).
To test whether this effect of inference was also present in the VTA DA neurons, we went back and did the same separation into T1 versus inference conditions in the previously published data (Takahashi et al., 2023). Figure 5, A and B, illustrates the normalized mean population firing rate of VTA DA neurons during T1 and inference trials in both reward transition types. We found that the suppression of DA activity was also significantly stronger in the period when reward would have been expected after the transition occurred on T1 trials compared with inference trials but that activity in both conditions was suppressed compared with that in the baseline. In addition, unlike the changes in activity in the reward-responsive OFC neurons, the suppression of DA activity was not seen prior to the epoch when reward would have been expected (Fig. 5B, left panel). Thus, OFC reward-responsive activity mirrors and slightly precedes VTA DA activity that is specific to transition-type. Although these recordings were not made in the same rats, the comparison is consistent with the hypothesis that OFC conveys to VTA-predictive information that carries separate flavor-specific streams.
VTA DA neurons showed a greater negative deflection to an expected but omitted reward on the first trial after the flavor delivered earlier had changed (T1 trials after delay–flavor switches) as compared with when the omission could be inferred (inference trials after delay–flavor switches), but showed no negative deflection on either kind of trial when the flavor delivered earlier had remained the same (delay-only switches). A, Line plots in top panels show mean normalized firing rates of all reward-responsive VTA DA neurons (reanalyzed from a previous study) on T1 trials (brown line) and inference trials (green line) after delay–flavor switches (left) and delay-only switches (right). Inference trials are the first trials that occur when a preceding trial on the other side could be used to infer that the flavor and timing would change on that side; T1 trials are the first trials that occur without a preceding trial on the other side. Gray-shaded areas indicate the time reward would be expected based on delivery in the previous block (3.0–3.5 s); blue-shaded areas indicate the time right before expected reward delivery (2.8–3.0 s). B, The same data averaged across the indicated epochs and compared with that on the last five trials of the same blocks. Firing rates on T1 trial was significantly lower than that on inference trials during reward omission timing after delay–flavor switches (T1 and inference trials occurred in different sessions and thus between-subject two–sided t tests were run comparing activity in the two conditions; 1st trial at reward-expectation epoch, t(238) = −2.0; p < 0.05), but this difference was not found during the prereward epoch after delay–flavor switches (t(238) = −0.9; p = 0.36). Firing rates on T1 trials and inference trials after delay-only switches were not different in either epoch (1st trial at prereward epoch, t(238) = −1.3; p = 0.18; 1st trial at reward-expectation epoch, t(238) = −0.7; p = 0.51). On the last five trials from the same blocks after both switch types, neurons showed similar firing rates on T1 trials and inference trials (last 5 trials at prereward epoch after delay–flavor switches, t(238) = −1.4; p = 0.17; after delay-only switches, t(238) = −0.6; p = 0.53; last 5 trials at reward-expectation epoch after delay–flavor switches, t(238) = −0.9; p = 0.38; after delay-only switches, t(238) = −0.6; p = 0.56). The error bars represent SEM. C, D, Histograms show the distribution of differences between firing rates on T1 trials (C) or inference trials (D) and firing rates on last 5 trials of the same blocks, during the prereward epoch (2.8–3.0 s) and reward-expectation epoch (3.0–3.5 s). Histograms on the left side are from delay–flavor switches and on the right side are from delay-only switches. The numbers in each panel indicate the results of one-sided Wilcoxon's signed rank tests (p) and the average difference scores (u).
Discussion
In this study, we recorded single-unit activity in the lateral OFC of rats performing an odor-guided choice task in which reward timing was manipulated, sometimes alone and sometimes in conjunction with reward flavor. We found three types of task-responsive neurons categorized by their response profiles: timing-responsive, flavor-responsive, and general reward-responsive neurons. Timing-responsive and flavor-responsive neurons were similar to the neurons found in a previous report (Roesch et al., 2006) in that they were selective for specific reward features and tracked them across changes to those features. However, in contrast to the neurons identified previously, these two types of neurons showed a strongly correlated response to the two reward features, timing and flavor. That is, across these populations, the more strongly a neuron responded to a particular flavor, the more strongly it responded to a particular timing of reward. In combination with Roesch et al.'s findings, this suggests that OFC selectively integrates reward features based on how closely they are correlated in the environment. Additionally, we observed that general reward-responsive neurons showed a selective suppression of activity both before and during the omission of an expected reward on the first trial following conjunctive changes in reward timing and flavor, but not when timing alone was changed. This pattern of activity was complementary to that observed in VTA DA activity in a previous study (Takahashi et al., 2023) but emerged slightly earlier than the timing of the VTA DA activity, suggesting that it could be conveying this information to VTA to influence identity-specific prediction errors.
Cracking and packing information about the features of expected rewards in the OFC
In contrast to our prior report (Roesch et al., 2006), we observed that the same population of neurons tended to represent both manipulated reward features. The discrepancy between these results and those of the previous report may have arisen from the differences between the task designs, particularly in how the reward features were manipulated. Whereas the prior report used a task in which the reward size and timing were changed independently, here we used a task in which reward timing and flavor were often changed concurrently. Taken together, these results suggest a tendency of the OFC to “pack” correlated reward features into unitary constructs and to “crack” independent features into separate constructs.
In considering this possibility, it is particularly striking to note that the reward features manipulated in the current study are arguably more qualitatively different than in our prior work. That is, reward timing and reward number both describe variable properties of a specific reward, while identity is arguably more categorical—changing identity by definition changes the specific reward. Viewed in this light, it is all the more impressive that the current study finds correlated activity to changes in identity and timing, whereas the prior study failed to find correlated activity to changes in number and timing.
Responding differently to reward features depending on their degree of correlation may be related to the role of OFC in identifying task states, meaning the way a particular task or environment can be best structured or mapped to reflect the underlying events. OFC neurons have been found to respond not only to the sensory feature of rewards but also to a variety of other task variables, including task rule (Wallis et al., 2001; Yamada et al., 2010; Sleezer et al., 2016), spatial information (Wikenheiser et al., 2017, 2021; Yoo et al., 2018; Basu et al., 2021), action selection (Yang and Masmanidis, 2020; Yun et al., 2020), and decision confidence (Hirokawa et al., 2019; Masset et al., 2020). This diverse pattern of OFC response properties has been hypothesized to reflect how task states are represented in these different contexts, which then facilitates faster learning and more flexible responding (Wilson et al., 2014; Niv, 2019; Zhou et al., 2021). Representations of task state would be derived from the experience that certain sensory features or patterns of subsequent events were correlated within the task. This notion is consistent with the findings of this study. Thus, cracking and packing the features of expected rewards in OFC may play a role in shaping task state and facilitating flexible behavior.
Representing flavor-specific predictive information in the general reward-responsive OFC population
OFC neurons fire in anticipation of and during significant events, providing signals crucial for adaptive behavior and learning in specific contexts (Rolls, 1996; Schoenbaum et al., 2009; Wallis, 2012; Rudebeck and Murray, 2014; Stalnaker et al., 2014, 2015, 2018). Previous research showed that prediction errors signaled by VTA DA neurons are influenced by such predictive information from OFC (Takahashi et al., 2011). Prior work in this exact task has shown that DA neurons respond differently when reward is omitted after delay-only versus delay–flavor changes (Takahashi et al., 2023). Specifically, when a reward moves to an earlier time and its identity does not change, there is no suppression of firing when the same reward fails to appear later in the trial. However, when it moves and also changes identity, then the DA neurons show a characteristic suppression of firing on the first few trials of its later omission. Here our investigation found complementary signaling in the reward-responsive population in OFC, particularly in the population that was nonselective for specific reward features. These neurons fired differently in anticipation of reward omission on trials when both identity and timing changed but not when timing was changed alone. Although subtle movements in the odor port or wells could affect the activity patterns observed in both OFC and VTA, these results indicate that subpopulations of neurons in OFC are positioned to provide highly specific and context-dependent predictive information to VTA DA neurons to support their pattern of error signaling in this task.
Notably, this transition-specific predictive information was only observed in the general reward-responsive neurons and not in the flavor- or timing-responsive neurons. One possible interpretation of this result is that OFC separately represent the features of outcomes, whether packed or cracked, from the transition-specific prediction that must be based on the features and how they are related to each other in the task. However, the functional significance of representing the features of outcomes and transition-specific predictions in dissociable populations remains elusive at present. It is plausible that such segregation offers some computational advantages for flexible decision-making, and there may be anatomical characteristics supporting these distinct neuronal activities in dissociable populations. For instance, it is conceivable that general reward-responsive neurons strongly project to the VTA DA neurons, while flavor- or timing-responsive neurons mainly project to different brain areas.
Alternatively, the neurons that drive the difference in firing in the general reward-responsive population may gate firing in other neural populations that signal more specific information. Certainly, there are smaller groups of neurons within each population that show differences in activity prior to the omitted reward on the delay–flavor trials, which could be harnessed by the appropriate connectivity. Since this is just a matter of speculation, further investigations are required to determine the functional significance and potential advantages of representing the feature of outcome and transition-specific prediction in dissociable neuronal population.
Footnotes
- Received April 16, 2024.
- Revision received July 29, 2024.
- Accepted August 2, 2024.
-
We thank the National Institute on Drug Abuse (NIDA) Intramural Research Program (IRP) histology core for the technical assistance in histology. This work was supported by the IRP at the NIDA (ZIA-DA000587) and the Japan Society for the Promotion of Science (JSPS Overseas Research Fellowships: 020157, A.S.) and in part by the IRP at the National Institute of Mental Health (ZIA-MH002983, A.J.L). The opinions expressed in this article are the authors’ own and do not reflect the view of the NIH/DHHS.
-
The authors declare no competing financial interests.
- Correspondence should be addressed to Akihiro Shimbo at akihiro.shimbo{at}nih.gov or Geoffrey Schoenbaum at geoffrey.schoenbaum{at}nih.gov.
- Copyright © 2024 the authors
