Neuroscience/Psychology Q&A
paying4help
article.pdf
AR245-NE28-16 ARI 19 May 2005 12:10
An Integrative Theory of Locus Coeruleus-Norepinephrine Function: Adaptive Gain and Optimal Performance Gary Aston-Jones1,∗ and Jonathan D. Cohen2,∗ 1 Laboratory of Neuromodulation and Behavior, Department of Psychiatry, University of Pennsylvania, Philadelphia, Pennsylvania 19104; email: [email protected] 2 Department of Psychology, Center for the Study of Brain, Mind and Behavior, Princeton University, Princeton, New Jersey 08540, and Department of Psychiatry Western Psychiatric Institute and Clinic University of Pittsburgh, Pittsburgh, Pennsylvania 15260; email: [email protected]
Annu. Rev. Neurosci. 2005. 28:403–50
doi: 10.1146/ annurev.neuro.28.061604.135709
Copyright c© 2005 by Annual Reviews. All rights reserved
∗The order of authorship is alphabetical. The authors contributed equally to the research reported in this review.
0147-006X/05/0721- 0403$20.00
Key Words
neuromodulation, decision making, utility, optimization, orbitofrontal cortex, anterior cingulate cortex
Abstract Historically, the locus coeruleus-norepinephrine (LC-NE) system has been implicated in arousal, but recent findings suggest that this system plays a more complex and specific role in the control of be- havior than investigators previously thought. We review neurophys- iological and modeling studies in monkey that support a new the- ory of LC-NE function. LC neurons exhibit two modes of activity, phasic and tonic. Phasic LC activation is driven by the outcome of task-related decision processes and is proposed to facilitate ensu- ing behaviors and to help optimize task performance (exploitation). When utility in the task wanes, LC neurons exhibit a tonic activity mode, associated with disengagement from the current task and a search for alternative behaviors (exploration). Monkey LC receives prominent, direct inputs from the anterior cingulate (ACC) and or- bitofrontal cortices (OFC), both of which are thought to monitor task-related utility. We propose that these frontal areas produce the above patterns of LC activity to optimize utility on both short and long timescales.
403
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Contents
INTRODUCTION AND OVERVIEW . . . . . . . . . . . . . . . . . . . . 404 Neuromodulatory Systems and the
Regulation of Behavior: A Historical Perspective . . . . . . . . . 405
A Modern View of the LC-NE System: Optimization of Performance . . . . . . . . . . . . . . . . . . 407
EMPIRICAL AND MODELING STUDIES REVEAL ROLES FOR THE LC-NE SYSTEM IN COGNITIVE PROCESSES . . . . . 408 Classical Findings Concerning the
Neurobiology of the LC-NE System Suggested a General Role in Regulating Neural Processing and Behavior . . . . . . . 408
Recent Findings Concerning the Physiology of LC-NE Neurons Reveal Two Modes of Operation . . . . . . . . . . . . . . . . . . 410
Neural Network Models of LC-NE Function Relate Physiological Mechanisms to Behavioral Effects . . . . . . . . . . . . . 415
AN INTEGRATIVE MODEL OF LC FUNCTION: THE LC PRODUCES ADAPTIVE ADJUSTMENTS IN GAIN THAT SERVE TO OPTIMIZE PERFORMANCE . . . . . . . . . . . . . . . 417 A Simple Mathematical Model Can
Be Used to Describe Decision
Processes and Analyze Them for Optimality . . . . . . . . . . . . . . . . . 417
The LC Phasic Mode Produces Adaptive Ajustments of Gain that Optimize Performance Within a Task (Temporal Filtering) . . . . . . . . . . . . . . . . . . . . . 418
The LC Tonic Mode Produces Adaptive Adjustments of Gain that Optimize Performance Across Tasks (Exploration Versus Exploitation) . . . . . . . . . . . 420
Utility Assessment in Frontal Cortex Regulates LC Mode . . . . 423
GENERAL DISCUSSION . . . . . . . . . . 430 Descending Influences Play an
Important Role in LC-NE Function . . . . . . . . . . . . . . . . . . . . . . 431
LC Activity is Plastic and May Play a Role in Learning . . . . . . . . . . . . . 431
Interactions Between the LC-NE and DA Systems are Important for Normal and Disordered Cognition . . . . . . . . . . . . . . . . . . . . . 432
How do Specific Effects Arise from a “Nonspecific” System? . . . . . . . 433
Optimization is Critical in a Competitive Context . . . . . . . . . . 433
Relation to Earlier Studies of the LC-NE System . . . . . . . . . 434
SUMMARY AND CONCLUSIONS . . . . . . . . . . . . . . . . 438
INTRODUCTION AND OVERVIEW
Adaptive behavior in a diverse and changing world requires a trade-off between exploiting known sources of reward and exploring the environment for other, potentially more valuable or stable opportunities. The capacity to support such adaptive behavior introduces another trade-off, between complexity of
mechanisms required to sustain a broad and flexible repertoire of behaviors and the efficiency of function that comes with simpler designs. Both of these trade-offs—between exploitation and exploration, and between complexity and efficiency—are well recog- nized by engineers and computer scientists. The evolution of the brain has likely faced similar pressures. In this review, we propose
404 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
that the LC-NE system serves to adjudicate these trade-offs and thereby contributes to the optimization of behavioral performance. This proposal contrasts with traditional views of the LC-NE system.
Neuromodulatory Systems and the Regulation of Behavior: A Historical Perspective
The LC-NE system is one of several brain- stem neuromodulatory nuclei with widely distributed, ascending projections to the neocortex (see Figure 1); others include the dopaminergic, serotonergic, and cholinergic systems. These neurons play critical roles in regulating cortical function, and disturbances in these systems are central to major psychi- atric disorders, such as schizophrenia, de- pression, and bipolar disorder. Traditionally investigators have assumed that these neu- rotransmitters serve relatively simple and basic functions. For example, many have thought that dopamine (DA) release signals reward or motivation, and NE mediates arousal (Berridge & Waterhouse 2003, Jouvet 1969, Robinson & Berridge 1993, Wise & Rompre 1989). Such functions seemed to accord well with the characteristic anatomy of these systems (widely distributed projections throughout the forebrain), and it is easy to understand how disturbances in such basic and pervasive functions would have profound disruptive effects on cognition, emotion, and behavior such as those associated with psychiatric disorders. Furthermore, although NE, DA, serotonin, and acetylcholine are sometimes referred to as “classical neuro- transmitters” (presumably because of their early discovery and their effects in peripheral systems), an equally and perhaps more im- portant function of these substances in cortex is neuromodulation. That is, rather than pro- ducing direct excitatory or inhibitory effects on postsynaptic neurons, they modulate such effects produced by other neurotransmitters such as glutamate and gamma amino butyric acid (GABA). Neuromodulatory actions,
Figure 1 Illustration of projections of the LC system. Saggital view of a monkey brain showing LC neurons located in the pons with efferent projections throughout the central nervous system. Note that only few areas do not receive LC innervation (e.g., hypothalamus and caudate-putamen).
especially when they are distributed over a wide area, seemed well suited to basic, pervasive functions such as the signaling of reward and the mediation of arousal.
Whereas functions such as reward and arousal have intuitive appeal, they have often escaped precise characterization, at both the neural and the computational levels. Recently, however, this has begun to change. For exam- ple, considerable progress has been made in developing a formal theory of the role of DA in reinforcement learning. According to this theory, DA does not signal reward per se but rather mediates a learning signal that allows the system to predict better when rewards are likely to occur and thereby contribute to the optimization of reward-seeking behaviors (Montague et al. 1996, 2004). This represents a significant refinement in understanding of the relationship of DA to reward and the role that this neuromodulatory system plays in the regulation of cognition and behavior. In this review, we propose a theory that offers a similar refinement to our understanding of LC-NE function and its relationship to arousal, and how this in turn relates to the optimization of reward-seeking behaviors.
Arousal reflects a fundamental property of behavior that has proven difficult to define
www.annualreviews.org • LC and Optimization 405
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Figure 2 Inverted-U relationship between LC activity and performance on tasks that require focused attention. Performance is poor at very low levels of LC tonic discharge because animals are drowsy and nonalert. Performance is optimal with moderate LC tonic activity and prominent phasic LC activation following goal-relevant stimuli (phasic LC mode). Performance is poor at high levels of tonic LC activity (tonic mode, lacking phasic LC activity). This resembles the classical Yerkes-Dodson relationship between arousal and performance. From Aston-Jones et al. 1999.
or to explain precisely with neurobiological mechanisms. The importance of arousal is undeniable: It is closely related to other phenomena such as sleep, attention, anxiety, stress, and motivation. Dampened arousal leads to drowsiness and, in the limit, sleep. Heightened arousal (brought on by the sud- den appearance of an environmentally salient event or a strongly motivating memory) can facilitate behavior but in the limit can also lead to distractibility and anxiety. Traditional theories of LC-NE function, which have tied this structure to arousal, have not described specific mechanisms by which this system produces changes in arousal and have left important unanswered questions about the re- lationship between arousal and behavior. For
example, performance on most tasks is best with an intermediate level of arousal and is worse with too little or too much arousal. This inverted U-shaped relationship is described by the classic Yerkes-Dodson curve (see Figure 2). As we discuss below, a similar relationship has been observed between performance and LC-NE activity. This relationship could be interpreted as consis- tent with the view that the LC-NE system mediates arousal. In this review, however, we propose a theory of LC-NE function that, rather than addressing arousal per se, specifies a role for the LC-NE system in optimizing behavioral performance, which in turn may explain effects conventionally interpreted in terms of arousal.
406 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
A Modern View of the LC-NE System: Optimization of Performance
Some theories of LC function suggested that the LC-NE system has its primary effects on sensory processing and in so doing serves to regulate arousal (Berridge & Waterhouse 2003). This was motivated largely by the con- sistent observation that highly salient and arousing stimuli elicit a phasic activation of LC neurons (Aston-Jones & Bloom 1981b, Grant et al. 1988, Herve-Minvielle & Sara 1995, Rasmussen et al. 1986) and concomitant NE release (Abercrombie et al. 1988, Brun et al. 1993). In addition, NE was found to augment the throughput of signals in sen- sory brain areas (Devilbiss & Waterhouse, 2000, 2004; Hurley et al. 2004; Waterhouse & Woodward 1980; Waterhouse et al. 1980, 1998). This observation led some to think of LC as the brain’s analog of the adrenal gland, orienting the system to and augment- ing the processing of motivationally relevant stimuli. Over the past decade, however, neu- ronal recordings from the primate LC during performance of simple decision-making tasks, coupled with new anatomic studies, have sug- gested a revision of traditional views of LC- NE function (Aston-Jones et al. 1994, 1997; Clayton et al. 2004; Rajkowski et al. 2004; Usher et al. 1999). Specifically, these record- ings indicate that in the waking state there are at least two distinguishable modes of LC function. In a phasic mode, bursts of LC ac- tivity are observed in association with the out- come of task-related decision processes and are closely coupled with behavioral responses that are generally highly accurate. In a tonic mode, LC baseline activity is elevated but phasic bursts of activity are absent, and be- havior is more distractible. Moreover, strong projections to the LC found from the OFC and ACC (Aston-Jones et al. 2002; M. Iba, W. Lu, Y. Zhu, J. Rajkowski, R. Morecraft & G. Aston-Jones, manuscript in prepara- tion; Rajkowski et al. 2000), and the func- tions of these frontal areas in evaluating re-
wards and costs, suggest that these regions are important in generating these patterns of LC activity.
Here we review these findings and describe a theory of LC-NE function that seeks to in- tegrate them with an emerging understanding of the neural mechanisms underlying perfor- mance in simple decision-making tasks. We propose that within the context of a given task, phasic activity of the LC-NE system facili- tates behavioral responses to the outcome of task-specific decision processes, filtering re- sponses to irrelevant events.1 By selectively fa- cilitating responses to task-relevant processes, the LC-NE phasic response serves to opti- mize the trade-off between system complex- ity (which can support a broad range of func- tions) and efficiency of function (optimizing performance in the current task). We fur- ther propose that the LC-NE system is re- sponsive to ongoing evaluations of task util- ity (that is, the costs and benefits associated with performance), provided by input from frontal structures. When utility persistently wanes, changes in LC-NE tonic activity with- draw support for task performance, facilitat- ing other forms of behaviors that serve to explore alternative sources of reward. These functions are accomplished by the neuromod- ulatory effects of NE release at cortical tar- get sites, modulating the gain (responsivity) of processing in cortical circuits responsible for task performance. The different modes of LC activity adaptively adjust the gain of these cortical circuits, both phasically and ton- ically, facilitating or disengaging task-specific
1 By decision processes, we mean those processes responsi- ble for mapping task-relevant stimuli onto the correspond- ing response. As we discuss further below, there is growing evidence that, for simple tasks, such processes may be im- plemented relatively early in the processing stream, distinct from and preceding those responsible for response execu- tion by as much as 100–200 ms. Furthermore, whereas in this review we focus on tasks involving motoric re- sponses (which are most readily accessible to measure- ment and therefore have yielded the most data), our the- ory is intended to apply equally to tasks involving internal “responses” such as the encoding of information into long- term memory.
www.annualreviews.org • LC and Optimization 407
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
processes. This adjustment of modes serves to optimize the trade-off between exploitation and exploration of opportunities for reward and thereby maximizes utility. This adap- tive gain theory integrates and explains new findings concerning LC physiology and its re- lationship to behavior, as well as newly discov- ered projections to LC from key frontal struc- tures involved in utility assessment. It also suggests specific ways in which LC-NE func- tion interacts with the proposed role that DA plays in reinforcement learning, providing a formal framework within which to explore dy- namic interactions between these systems.
EMPIRICAL AND MODELING STUDIES REVEAL ROLES FOR THE LC-NE SYSTEM IN COGNITIVE PROCESSES
We begin with a brief summary of the ba- sic neurobiology of the LC-NE system (for more detailed reviews, see Foote et al. 1983, Berridge & Waterhouse 2003, and Moore & Bloom 1979). We follow with a consid- eration of recent neurophysiological findings that suggest the need for revising how we think about this system. We then review com- putational models developed to explain these findings. We conclude by presenting our the- ory for a role of the LC-NE system in op- timizing task performance, which relates the functioning of this system to cortical mecha- nisms involved in the evaluation of costs and benefits associated with task performance, and the trade-off between exploiting task-related sources of reward and exploring other possi- ble rewards.
Classical Findings Concerning the Neurobiology of the LC-NE System Suggested a General Role in Regulating Neural Processing and Behavior
“Locus coeruleus” means blue spot in Latin, reflecting the pigmented nature of LC neu- rons in human. The LC nucleus is a small
collection of noradrenergic neurons (about 16,000 per hemisphere in the human), lo- cated just behind the periaqueductal gray in the dorsorostral pons. Although few in num- ber, LC-NE neurons give rise to highly di- vergent and extensive efferents in rats, mon- keys, and humans (Dahlstrom & Fuxe 1964, Morrison et al. 1982). These cells provide the bulk of the brain’s NE and are the sole source of NE to the cerebral, cerebellar, and hip- pocampal cortices (Aston-Jones 2004, Aston- Jones et al. 1984, Moore & Bloom 1979). LC projections are unmyelinated and therefore slowly conducting (typically <1 m/s; Aston- Jones et al. 1985). Early studies also reported that LC terminals have nonsynaptic release sites that may provide a paracrine-type of neu- rotransmission (Beaudet & Descarries 1978, Seguela et al. 1990).
NE can have different effects on target neurons, depending on the receptor that is ac- tivated (reviewed in Berridge & Waterhouse 2003 and Foote et al. 1983). Thus, alpha1 adrenoceptor activation is often associated with excitation, and alpha2 adrenoceptor ac- tivation (the dominant type within LC it- self) is associated with inhibition (Rogawski & Aghajanian 1982, Williams et al. 1985). How- ever, modulatory effects that do not evoke simple excitatory or inhibitory effects are also frequently described. For example, NE in- creased the ratio of synaptically evoked activ- ity to spontaneous activity in target neurons in early studies (Foote et al. 1975, Segal & Bloom 1976). Later studies found that in many tar- get areas NE augments evoked responses (either excitatory or inhibitory) while decreas- ing spontaneous activity of the same neuron (Waterhouse et al. 1980, 1984; Waterhouse & Woodward 1980). Thus, modulation of neu- ronal responses to other inputs is a prominent effect of NE actions on target cells.
This modulatory action was captured in an early computational model of NE effects as an increase in the gain of the activation func- tion of neural network units (Figure 3), which was shown to mimic many of the physiologic effects of NE and could explain patterns of
408 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Figure 3 Effect of gain modulation on nonlinear activation function. The activation (or transfer) function relates the net input of a unit to its activity state (e.g., the firing rate of a single neuron or the mean firing rate of a population). The function illustrated here is given by
activation = 1 1 + e−(gain∗ net input)
.
An increase in gain (dotted line) increases the activity of units receiving excitatory input (upward arrow on right) and decreases the activity of units receiving inhibitory input (downward arrow on left), thus increasing the contrast between activated and inhibited units and driving them toward more binary function. Adapted from Servan-Schreiber et al. 1990.
behavior associated with manipulations of NE (Servan-Schreiber et al. 1990). This compu- tational model of NE’s modulatory effects set the stage for further studies using more elab- orate models involving LC neurons and their targets, as described in more detail below.
The above properties—widespread slowly conducting projections and neuromodulatory action—suggested that LC may play a gen- eral role in regulating neural processing and behavior. Commensurate with this view, tonic impulse activity of LC-NE neurons strongly covaries with stages of the sleep-waking cy- cle. These neurons fire most rapidly during waking, slowly during drowsiness and slow-
wave/non-REM sleep, and become virtually silent during REM/paradoxical sleep (Aston- Jones & Bloom 1981a, Hobson et al. 1975, Rajkowski et al. 1998, Rasmussen et al. 1986). LC activity may in fact be a primary factor that differentiates REM sleep (when other systems, including the neocortex, exhibit signs of heightened arousal) from wakefulness (Steriade et al. 1993). These and related find- ings support the view that low levels of LC ac- tivity facilitate sleep and disengagement from the environment.
Further supporting the view that the LC- NE system plays a role in general arousal and environmental responsiveness, LC neurons in rats and monkeys activate robustly following salient stimuli in many modalities that elicit behavioral responses (Aston-Jones & Bloom 1981b, Foote et al. 1980, Grant et al. 1988). For example, tapping the cage door around feeding time elicits LC activation accompa- nied by a behavioral orienting response and increased physiological signs of arousal. Con- versely, stimuli that elicit no behavioral re- sponse typically do not evoke an LC response.
The classical observations described above suggest that the LC-NE system has a rel- atively broad, nonspecific effect on cortical information processing. However, other find- ings indicate that substantial specificity exists in the LC-NE system in several domains. For example, although they are widespread, LC projections exhibit substantial regional and laminar specificity (Morrison et al. 1982). Notably, brain areas thought to be involved in attentional processing (e.g., parietal cortex, pulvinar nucleus, superior colliculus) as well as motor responding (e.g., primary motor cortex) receive a particularly dense LC-NE innervation (Foote & Morrison 1987). Also, LC terminals make conventional synapse-like appositions with postsynaptic specializations on target neurons (Olschowka et al. 1981; Papadopoulos et al. 1987, Papadopoulos et al. 1989), in addition to having possible nonsynaptic release sites.
Recent neurophysiological findings also indicate that LC may play a specific role in
www.annualreviews.org • LC and Optimization 409
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
information processing and that it may in- teract closely with top-down influences from cortical systems. These findings have led to the development of mechanistically explicit computational models that describe the
Figure 4 Phasic activation of monkey LC neurons in a signal-detection task. Peri-event time histograms (PETHs) for a typical individual LC neuron in response to various events during performance of the signal-detection task. PETHs are each accumulated for 100 sweeps of activity in this neuron synchronized with (A) target stimuli, (B ) nontarget stimuli, (C ) fix spot presentation, (D) juice solenoid activation, or (E ) bar press and release performed outside of the task, as indicated. Note the selective activation following target stimuli ( panel A ). The small tendency for a response in (C ) may reflect activation after target stimuli that occur at short but somewhat variable times after fix spots. Similarly, the activation seen before reward presentation (D) is due to activation following target cues. From Aston-Jones et al. 1994.
physiological mechanisms governing LC-NE function and their interaction with cortical mechanisms responsible for the execution and evaluation of behavior. Below, we review the recent findings concerning LC-NE phys- iology and anatomy in monkey that suggest a more specific role in information processing, and computational models of LC-NE func- tion that provide a formal description of the proposed mechanisms involved.
Recent Findings Concerning the Physiology of LC-NE Neurons Reveal Two Modes of Operation
LC phasic mode involves phasic activation of LC neurons following task-relevant processes. A number of studies have re- vealed that during accurate task performance reflecting focused attentiveness, LC neurons fire tonically at a moderate rate and respond phasically and selectively shortly following task-relevant target stimuli, but not after dis- tractors that may differ subtly from targets. In one series of experiments, LC activity was recorded while monkeys performed a simple signal-detection task in which they were re- quired to respond by releasing a lever imme- diately following a specific visual target (e.g., a small vertical bar of light—target cue, 20% of trials) but to withhold responding for an- other similar cue (e.g., a horizontal bar of light—distractor, 80% of trials). Correct re- sponses were rewarded by the delivery of a small quantity of juice, whereas incorrect re- sponses (target misses and false alarms to the distractor) were punished by a brief time-out. Monkeys performed this task with high ac- curacy, typically greater than 90%. Figure 4 shows a representative recording of LC neu- rons, demonstrating substantial phasic activa- tion shortly following target stimuli but only a weak (if any) response following distractors (Aston-Jones et al. 1994). Systematic exami- nation of the LC phasic response following targets indicated that it is not specific to par- ticular sensory attributes. Also, LC does not respond phasically to distractors even if they
410 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
are infrequent, and (in a forced-choice task) LC responses occur even when targets are presented on every trial. The LC response is also not linked to a specific reward because similar responses are observed for different juice rewards or for water reward in fluid- restricted subjects. Furthermore, in reversal experiments in which the distractor becomes the target and vice versa, LC phasic responses are quickly acquired to the new target and ex- tinguished for the new distractor. This rever- sal in LC response precedes stable behavioral reversal within a single testing session (Aston- Jones et al. 1997). These findings indicate that the LC response is highly plastic and that it is not rigidly linked to specific sensory attributes of a stimulus but rather responds to events in a task-sensitive manner.
The timing of LC phasic responses is also informative and contrasts with traditional concepts of a slowly acting, nonspecific system. The latency of LC phasic activa- tion following targets is surprisingly short (∼100 ms onset) and precedes lever-release responses by about 200 ms. The conduction latency for monkey LC impulses to reach the frontal cortex is ∼60–70 ms (Aston-Jones et al. 1985), making it possible for NE release to occur at about the time that neural activity in motor cortex associated with the behavioral response begins to develop (about 150 ms before the manual response; Mountcastle et al. 1972). Thus, although the conduction velocity of LC impulses is slow, the timing of impulse arrival in cortical targets makes it possible for the LC phasic response and NE release to influence the behavioral response on the same trial. Consistent with this possibility, the latency of LC neuron response and lever release are significantly correlated over trials; shorter LC response is associated with shorter behavioral response to the same cue (Aston-Jones et al. 1994).
LC recordings in monkeys performing two-alternative forced choice (2AFC) tasks strongly suggest that these phasic responses are associated with decision processes. 2AFC tasks have been used in a growing
number of studies examining the neural mechanisms involved in simple forms of decision making (e.g., Gold & Shadlen 2000; Hanes & Schall 1996; Schall & Thompson 1999; Shadlen & Newsome 1996, 2001). In one such experiment with LC recordings, monkeys were rewarded for responding with the left lever for one stimulus and the right for another (Clayton et al. 2004). As in previous experiments, LC phasic re- sponses were observed shortly following task cues and preceding lever responses. More detailed analysis revealed that LC activation was more tightly time-locked to the behav- ioral response than to presentation of the stimulus. This result is shown in Figure 5 with peri-event time histograms that tabulate LC activity with respect to either the sensory stimulus or the behavioral response. Such analyses showed that LC responses precede behavioral responses by about 230 ms regard- less of trial type or response time (RT). Note in particular that, in the stimulus-locked his- tograms, LC responses are greater for correct trials than for error trials. In contrast, in the response-locked histograms, LC activity is comparable for correct and error trials. This can be explained by the additional observation that RT variability was significantly greater for error than for correct trials. Because LC activity is more tightly coupled to the be- havioral response than to stimulus onset, and RT is more variable for error trials, stimulus- locked distributions of LC activity are also more variable (relative to response-locked distributions) for error trials. LC activation did not occur on trials in which the animal made no response despite viewing the cue, and there was no LC response associated with spurious lever responses that occasionally occurred between trials when no stimulus was present. Finally, note that the LC phasic response appears to be closely coupled with, and precedes, the task-related behavioral responses. These observations have been confirmed in a signal-detection task in which trial difficulty was manipulated to produce variable RTs. Once again, LC phasic activity
www.annualreviews.org • LC and Optimization 411
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
412 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
was more tightly linked to the RT than to the sensory stimulus and preceded lever responses by ∼200 ms (Rajkowski et al. 2004).
Similar results were obtained in a recent study that recorded LC neurons in behaving rats (Bouret & Sara 2004). Here, the animal was presented with conditioned odor stimuli that instructed it when to respond to obtain food reward. As found in the monkey, LC neu- rons exhibited phasic responses to the condi- tioned stimulus but not to stimuli (nontargets) that were not associated with food availabil- ity; these responses preceded the behavioral response and were linked more tightly to the behavioral response than to the sensory stim- ulus. Moreover, in reversal conditioning the rat LC responses tracked the significance of the stimuli rather than stimulus identity, and newly acquired LC responses preceded those observed in behavior by several trials. These findings all closely parallel those for the mon- key LC described above (Aston-Jones et al. 1997, Clayton et al. 2004).
The pattern of results described above pre- cludes the possibility that LC phasic activation is driven strictly by stimulus onset, response generation, or reward. As discussed in greater detail below, these results have led us to hy- pothesize that LC phasic activity is driven by the outcome of internal decision processes that may vary in duration from trial to trial (accounting for RT variability) but precede response generation with a regular latency. Along these lines, an important observation
is that LC phasic responses are largest and most consistent when the animal is perform- ing the task well. During epochs of poor per- formance, LC phasic responses are consider- ably diminished or absent. These observations are consistent with our hypothesis that the LC phasic response plays a role in facilitating task- relevant behavioral responses.
As the phasic LC response occurs only for identification of a task-relevant stimulus, and not task-irrelevant stimuli, it can be thought of as an attentional filter that selects for the occurrence (i.e., timing) of task-relevant stim- uli. This filter is temporally specific but, given the broad projections of LC neurons, spa- tially global. In addition, the link to decision outcome, rather than to stimulus presenta- tion, indicates that this LC response primar- ily modulates specific behaviors rather than sensory processing. Therefore, we propose that the LC phasic response provides a tem- poral attentional filter that selectively facili- tates task-relevant behaviors. This conclusion is supported by the results of neural network– modeling studies discussed later.
LC tonic mode involves increased base- line activity and diminished phasic re- sponses of LC neurons. In addition to LC phasic responses, levels of LC tonic (base- line) activity vary significantly in relation to measures of task performance. For example, during performance of a signal-detection task, periods of elevated LC tonic activity were
←−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Figure 5 Phasic activation of monkey LC neurons in a two-alternative forced choice (2AFC) task. Stimulus- and response-locked population PETHs showing LC responses for trials yielding correct and incorrect behavioral responses. (A) Stimulus-locked population PETHs showing LC response to cues (presented at time 0) for trials yielding correct or incorrect behavioral responses. Note that the LC response peaks sooner and is less prolonged on correct compared with incorrect trials in this analysis (17,533 and 1362 trials, respectively). No LC activation was detected on omission trials (orange line, 1128 trials). Vertical dashed lines indicate the mean behavioral RTs. Curves represent the normalized RT distributions for correct and incorrect trials. (B) The difference in the phasic LC response between correct and incorrect trials was not evident in response-locked population PETHs. In addition, no LC activation occurred prior to or following lever releases not associated with stimulus presentation (orange line, 3381 trials). Dashed vertical lines indicate the mean stimulus onset times. From Clayton et al. 2004.
www.annualreviews.org • LC and Optimization 413
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
consistently accompanied by more frequent false-alarm errors (Aston-Jones et al. 1996, Kubiak et al. 1992, Usher et al. 1999). Analy- ses using standard signal-detection measures revealed that, during periods of elevated tonic LC activity, the animal’s ability to discrim- inate targets from distractors (d-prime) and its threshold for responding to stimuli (beta) both decreased (Aston-Jones et al. 1994). RT distributions were also wider. Furthermore, the experimental paradigm required the ani- mal to foveate the center of the computer dis- play prior to stimulus presentation (as an in- dicator of task preparedness). Such foveations were less frequent during periods of ele- vated LC tonic activity, resulting in a sig- nificantly greater number of aborted trials (Aston-Jones et al. 1996, 1998). Collectively, these findings indicate that when baseline LC activity is increased, the animal is less effec- tively engaged in task performance, display- ing increased distractibility with a greater ten- dency to respond to nontarget stimuli (lower response threshold). Such periods are also consistently associated with a diminution or absence of the LC phasic responses seen during periods of best performance. These observations are consistent with the hypoth- esis that LC phasic activity facilitates be- havioral responses engaged by task-related decision processes. However, it begs the ques- tion of whether and how the LC tonic mode is adaptive and what information-processing function it may serve. As described below, we propose that although this mode is disadvan- tageous for performance on a specific task, it may be important for sampling alternative be- haviors and adaptively pursuing other tasks in a changing environment.
The above results indicate an association of LC activity with task performance. However, they do not establish whether alterations in LC activity are causative of, correlated with, or result from other mechanisms responsible for changes in performance. Preliminary evi- dence using microinfusions into the monkey LC supports the view that the LC plays a
causal role in influencing performance. In a recent study, the alpha2 adrenoceptor agonist clonidine was used to decrease tonic LC ac- tivity, and the muscarinic cholinergic agonist pilocarpine was used to stimulate tonic LC discharge. Direct microinfusion of clonidine into the LC of a monkey exhibiting an un- usual degree of distractibility (hyperactivity) and poor performance on the signal-detection task significantly decreased tonic LC activity, increased LC phasic responses to target stim- uli, and improved performance by decreas- ing false alarm and omission errors. In con- trast, during error-free performance in other monkeys, local microinjection of the mus- carinic cholinergic agonist pilocarpine caused tonic activation of LC neurons, decreased phasic responsiveness to task stimuli, and in- terfered with task performance (Ivanova et al. 1997).
Summary. Overall, these results indicate that two modes of LC activity correspond to differ- ent patterns of performance. (a) In the phasic mode, LC cells exhibit phasic activation in re- sponse to the processing of task-relevant stim- uli but display only a moderate level of tonic discharge. This mode of function is consis- tently associated with high levels of task per- formance. (b) In the tonic mode, LC cells fail to respond phasically to task events but exhibit higher levels of ongoing tonic activity. This mode is associated with poor performance on tasks that require focused attention and corre- sponds to apparent increases in distractibility. We should note here that, whereas we have described the phasic and tonic modes as dis- tinct, they likely represent the extremes of a continuum of function. This idea is consistent with findings from the computational models reviewed in the following section, which sug- gest transitions between these modes may be regulated by simple, continuous physiologi- cal variables. Nevertheless, for expository pur- poses we continue to refer to the phasic and tonic modes as distinct in the remainder of this review.
414 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Neural Network Models of LC-NE Function Relate Physiological Mechanisms to Behavioral Effects
The data described above pose important questions about LC activity and its rela- tionship to behavioral performance: Which physiological mechanisms underlie the phasic and tonic modes of LC activity and transitions between them, and how do these mechanisms interact with cortical mechanisms responsible for task performance to produce the patterns of behavior associated with each mode of LC function? Computational modeling has re- cently begun to address these questions. In an initial effort, Usher et al. (1999) constructed a model composed of two components (see Figure 6): (a) a detailed, population-level model of LC; and (b) a more abstract con- nectionist network that was the simplest network capable of simulating performance in the signal-detection task. This model revealed that alterations in electrotonic coupling among LC neurons can produce the two modes of LC activity. The model also revealed how the corresponding alter- ations in gain of cortical units receiving LC inputs can either facilitate task performance (phasic mode) or produce more distractible, less-task-focused responding (tonic mode).
The behavioral network was composed of three layers: an input and a decision layer (each with two units, representing the target and distractor stimuli), and a response layer (with a single unit corresponding to the be- havioral response).2 Connections from units in one layer to the next were excitatory (infor- mation flow), whereas connections between units in the decision layer were inhibitory (competition for representation). A response was recorded when activity of the response unit exceeded a specified threshold.
The LC component of the model was com- posed of 250 integrate-and-fire units, con-
2 Units in this network were assumed to represent popula- tions of recurrently connected cortical neurons contribut- ing to the representation of a given piece of information (e.g., Amit 1989).
Figure 6 Architecture of the neural network model of LC function in the signal-detection task. Arrows represent excitatory links, small circles depict inhibition, and squares (from LC projections) represent modulation of gain (see Figure 3). There is a moderate positive bias on the response unit, which captures the observation that monkeys in this task make many false alarms but very few misses (Aston-Jones et al. 1994). Note the projection from the decision-layer target unit to the LC, which captures the observation that in the well-trained animal LC neurons are selectively activated following target stimuli. From Usher et al. 1999.
nected by mutually inhibitory noradrenergic collaterals as well as weak electrotonic cou- pling, both of which have been observed em- pirically (Aghajanian et al. 1977; Christie et al. 1989; Christie & Jelinek 1993; Egan et al. 1983; Ennis & Aston-Jones 1986; Ishimatsu & Williams 1996; Travagli et al. 1995, 1996). LC units received afferent connections from the decision unit representing the target stimu- lus in the behavioral network (consistent with decision-driven LC activity described above)3
and sent projections back to all units in the be- havioral network (consistent with the known broad efferent projections of LC neurons). All units in the model were subject to noise in their input, producing a baseline level of ac- tivity in the LC and the possibility for spurious responses in the behavioral network.
The effect of NE release in the behav- ioral network was simulated as an increase in the responsivity (i.e., gain of the activation
3 These are assumed to have been learned for this task but in other circumstances could represent “hardwired” con- nections for evolutionarily important signals such as highly salient sensory events.
www.annualreviews.org • LC and Optimization 415
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
function) of these units (Servan-Schreiber et al. 1990). This mimicked the modula- tory influence of NE (discussed above) and augmented the activity of units in the be- havioral network that were already activated (e.g., because of noise), while further sup- pressing the activity of units that were al- ready being inhibited. A systematic examina- tion of these simulated NE effects revealed that their greatest influence on performance was achieved by their impact on the response unit, which produced behavioral changes that closely matched those observed empirically. This point becomes important in our discus- sion below about the role of the LC-NE sys- tem in optimizing task performance.
An important finding from this model was that modest changes in the strength of elec- tronic coupling among LC units reproduced the entire set of neurophysiological and be- havioral results described above concerning modes of LC activity and their relationship to task performance. Within LC, increased coupling facilitated phasic activation of LC neurons in response to activation of the target decision unit in the behavioral network by allowing summation of concurrent responses across electrotonic links. At the same time, coupling reduced spontaneous firing by aver- aging the effects of uncorrelated noise across LC units. Conversely, reduced coupling ac- curately simulated the effects associated with the LC tonic mode by producing a modest rise in baseline LC activity but a diminished phasic response to input from the target decision unit in the behavioral network.
Within the behavioral network, LC phasic responses (associated with high coupling and generated by activation of the target decision unit) produced NE release and correspond- ing increases in gain within the response unit. This occurred within the same time frame that feed-forward activation was being received from the target decision unit, facilitating ac- tivation of the response unit. This, in turn, reproduced the empirical observation of im- proved target detection performance associ- ated with LC phasic responses. In this respect,
the LC phasic response can be thought of as an attentional filter that selects for the occur- rence (i.e., timing) of task-relevant events and facilitates responses to these events. The ex- istence of such a temporal filter, and its asso- ciation with noradrenergic function, is con- sistent with several recent psychophysical and psychopharmacological studies (e.g., Coull & Nobre 1998, Coull et al. 2001). Conversely, during low coupling the attenuated LC phasic response to target detection produced some- what slower and more variable behavioral responses, whereas increased baseline NE re- lease (as a result of increased LC tonic activ- ity) rendered the response unit more suscep- tible to the effects of noise and therefore to the production of spurious (false alarm) re- sponses. Although this degrades performance in the present task, Usher et al. (1999) suggest that the general increase in responsiveness as- sociated with the LC tonic mode facilitates exploration of alternative behaviors, which is adaptive in a changing environment or when current rewards lose their value (as discussed in more detail below).
Recent modeling work has extended these basic results by refining the original popula- tion model of LC using more biophysically detailed, phase oscillator units for each LC neuron (Brown et al. 2004). The individual units of this model have been matched closely with detailed physiological properties empirically observed for LC neurons, such as firing rate, variance in interspike interval, calcium and voltage-dependent potassium currents, electrotonic coupling, and collateral inhibitory connections. One important finding that has emerged from this work is that the phasic and tonic modes of LC activity (described above) can be produced also by changes in baseline excitatory drive to LC cells (in the absence of any change in coupling). Decreases in baseline afferent drive promote the phasic mode of LC activ- ity, whereas increases favor the tonic mode. These observations are consistent with other recent modeling studies (Alvarez et al. 2002) and suggest that changes in baseline activity
416 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
provide another simple mechanism by which LC mode can be regulated. Additional studies are needed to determine if altered baseline drive or electronic coupling, or both, is responsible for the different LC firing modes.
Summary. In the models described above, the outcome of processing at the decision layer drives an LC phasic response, increasing the gain of units throughout the behavioral network. This lowers the threshold for the re- sponse unit, thus facilitating a response to the outcome of the decision process. The timing of the LC phasic response suggests that it can be thought of as a temporal attentional filter, much as cortical attentional mechanisms act as a content filter, facilitating responses to task- relevant events. At the same time, by increas- ing the gain of cortical representations, the LC phasic response may also enhance the ef- fects of attentional selection by content within the neocortex (e.g., Robertson et al. 1998). Together, these effects allow the LC phasic response to facilitate selectively responses to task-relevant stimuli. These mechanisms pro- vide an account of how LC phasic responses may contribute to enhanced task performance associated with the LC phasic mode of func- tion, and how the LC tonic mode may lead to degraded task performance. At the same time, the models have identified specific phys- iological parameters—electrotonic coupling and baseline firing rate—that can drive transi- tions between the LC tonic and phasic modes of function. However, these models leave sev- eral important questions unaddressed: What function does the LC tonic mode serve, and what drives transistions between the phasic and tonic modes? In the sections that follow, we consider both LC phasic and tonic func- tion in the context of recent theories concern- ing the neural mechanisms underlying simple decision processes and mathematical analy- ses concerning the optimization of such pro- cesses. A consideration of LC function within this context suggests how both modes of LC activity may contribute to the overall opti- mization of performance.
AN INTEGRATIVE MODEL OF LC FUNCTION: THE LC PRODUCES ADAPTIVE ADJUSTMENTS IN GAIN THAT SERVE TO OPTIMIZE PERFORMANCE
A Simple Mathematical Model Can Be Used to Describe Decision Processes and Analyze Them for Optimality
To consider how LC may play a role in op- timization of task performance, we must first more precisely define what we mean by opti- mization. This, in turn, requires a more for- mal characterization of the mechanisms that underlie task performance. We can think of these as being composed of a set of deci- sion processes, which may involve perception (“Was that a ball or a strike?”), memory (“Was the count level or full?”), evaluation (“Was the last call fair or unfair?”), and/or action (“Should I swing high or low?”). Cognitive and neuroscientific studies have made consid- erable progress in identifying and character- izing mechanisms associated with the simplest decision processes involved in 2AFC tasks (e.g., Gold & Shadlen 2000; Hanes & Schall 1996; Schall & Thompson 1999; Shadlen & Newsome 1996, 2001). There is a remarkable convergence of views that agree that the dy- namics of both neural activity and behavioral performance observed in such tasks can be de- scribed accurately by a simple mathematical model, often referred to as the drift diffusion model (DDM). This, in turn, provides a use- ful framework for defining and evaluating the optimization of performance.
The DDM describes decision processes in terms of simple accumulators that integrate signals favoring each of the two choices and respond when the difference between these signals exceeds a threshold value. The DDM offers a mathematically precise characteriza- tion of the dynamics and outcome of decision making in such tasks (Laming 1968; Ratcliff 1978, 2004; Stone 1960), on the basis of the
www.annualreviews.org • LC and Optimization 417
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Originally introduced in discrete form as the sequential proba- bility ratio test (SPRT, also known as the random walk model), investigators proved that this is the optimal procedure for making a binary decision under uncertainty (Barnard 1946, Wald 1947). That is, for a specified level of accuracy it is the fastest method to reach a decision; or, conversely, for a speci- fied time in which to make the decision (i.e., deadline) it is the most accurate. Accordingly, this procedure was used by Turing to decipher the Enigma code used by the German navy in World War II. Furthermore, researchers have recently proven that for a given set of task variables (e.g., stimulus strength and intertrial interval), there is a single decision threshold that maximizes reward rate; that is, there is an optimal trade- off between speed and accuracy that maximizes reward rate.4
Recent empirical work indicates that performers can approx- imate this maximum (R. Bogacz, E.T. Brown, J. Moehlis, P. Hu, P. Holmes & J.D. Cohen, manuscript under review).
following three assumptions: (a) the decision- making process is stochastic (that is, it is sub- ject to random fluctuations in accumula- tion of evidence in favor of each alternative); (b) evidence favoring each alternative is accu- mulated over time; and (c) the decision is made when sufficient evidence (exceeding a thresh- old) has accumulated in favor of one alterna- tive over the other. What is remarkable about this model is that, although originally devel- oped to describe behavior (e.g., reaction-time distributions and error rates), it also provides a detailed and accurate account of neuronal responses in such tasks and their relation- ship to performance (Gold & Shadlen 2002, Ratcliff et al. 2003, Schall & Thompson 1999). Furthermore, the DDM provides an
4 In principle, we can imagine utility functions that differ- entially weigh types of outcomes under varying circum- stances (e.g., speed over accuracy, or vice versa; Bohil & Maddox 2003, Mozer et al. 2002, Wald 1947). However, it seems reasonable to assume that under many (if not most) circumstances, a critical objective is to maximize the re- wards accrued by performance. Reward rate can be formal- ized as a function of decision time, interstimulus interval, and error rate and may be used as an objective function in evaluating maximization of utility (R. Bogacz, E.T. Brown, J. Moehlis, P. Hu, P. Holmes & J.D. Cohen under review; Busemeyer & Myung 1992, Gold & Shadlen 2002).
analytically tractable simplification of single- layered neural network models that simu- late performance in such tasks (R. Bogacz, E.T. Brown, J. Moehlis, P. Hu, P. Holmes & J.D. Cohen, manuscript under review; Usher & McClelland 2001).5
The LC Phasic Mode Produces Adaptive Ajustments of Gain that Optimize Performance Within a Task (Temporal Filtering)
The DDM is appealing in the present context because it provides a framework within which we can formally define optimal performance.6
The DDM itself is, in fact, the optimal pro- cess for 2AFC decision making (i.e., it is the most accurate for a given speed of decision
5 Typically, such models involve pairs of mutually inhibiting (i.e., competing) leaky accumulators, the activity of which represents the amount of evidence favoring each alterna- tive in the choice. A response is produced when the activ- ity of one of the units crosses a specified threshold. This is precisely the mechanism used in the decision layer of the LC models described above, and it has been used to simulate a wealth of findings from other 2AFC tasks (e.g., Botvinick et al. 2001; Cohen et al. 1990, 1992; Usher & McClelland 2001) as well as the dynamics of neural activity in response-selective brain areas associated with the perfor- mance of such tasks (e.g., Gold & Shadlen 2002, Shadlen & Newsome 2001). Mathematical analyses of these models suggest that, under certain assumptions, they can be re- duced to the DDM without significant loss of their ability to describe behavioral and neurophysiological data accurately (R. Bogacz, E.T. Brown, J. Moehlis, P. Hu, P. Holmes & J.D. Cohen, manuscript under review). In this respect, the DDM provides a mathematically precise, theoretical bridge from neural mechanisms to behavioral performance. 6 In discussing optimization of performance, we are not as- serting people always behave perfectly optimally. In many instances, people do not behave optimally (e.g., Herrnstein 1961, 1997; Kahneman & Tversky 1984; Loewenstein & Thaler 1989). Nevertheless, the assumption of optimality is useful because it provides theoretical traction: It allows us to define formally and precisely the goal of an adaptive sys- tem, even when this goal may not be fully achieved in prac- tice. This definition can be used to generate testable predic- tions and provides a valuable reference against which actual behavior can be compared and understood, even when it turns out to be suboptimal. This approach has been used effectively in a wide variety of disciplines, including neu- roscience and psychology (Anderson 1990, Barlow 1981, Bialek & Owen 1990, Gallistel et al. 2001, Mozer et al. 2002).
418 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
making, and the fastest for a given level of accuracy; see sidebar). Furthermore, because the DDM provides a good description of sim- ple, single-layered neural networks that im- plement 2AFC decision processing, we can infer that such single-layered networks can approximate optimal performance in 2AFC decision processing. We know, however, that real neural architectures, in fact, involve many layers (e.g., Schall 2003, Reddi 2001), presum- ably because different tasks require decision processes that integrate information of dif- ferent types, at varying levels of analysis, and from a variety of sources. Although all of these decision processes may involve fundamentally similar mechanisms—possibly well described by the DDM—they are implemented by dif- ferent parts of (“layers” within) the full neu- ral architecture. This presents a challenge for optimal performance. On the one hand, as noted above, a single-layered network imple- ments the optimal decision process. On the other hand, the layer implementing this pro- cess for a given task may be several layers away from the response mechanism. It would be inefficient if a decision process in the layer integrating information relevant to the cur- rent task crossed threshold but then had to drive a subsequent series of repeated accu- mulator processes—each of which introduces additional noise and requires addition integra- tion time—before a behavioral response could be elicited. This problem reflects the funda- mental trade-off between the complexity of a multilayered system that can support a wide range of decision processes (and the flexibility of behavior that this affords) and the efficiency of a simpler, single-layered system (that is, the optimality of function that this affords).
The inefficiency of multilayered integra- tion can be ameliorated if, at the time a unit in the task-relevant decision layer crosses threshold, a signal is issued ensuring that this information rapidly and directly influences the behavioral response. The LC phasic sig- nal accomplishes precisely this effect in the models described above. The LC phasic re- sponse is trigged when sufficient activity accu-
mulates in one of the units in the decision layer of the behavioral network. The resulting LC phasic response increases the gain of all units in the behavioral network, which drives units toward binary responding (see Figure 3), in effect eliminating further integration in any subsequent layers. Because this occurs at a time when the relevant decision unit has just crossed threshold and is therefore highly ac- tive, all units “downstream” will assume states that are heavily determined by this partic- ular input. Thus, we can think of the sud- den increase in gain as an adaptive sampling bias (this is the sense in which the LC acts as a temporal filter), favoring the selection of states of the entire system that are most heav- ily influenced by the activity of the units in the decision layer, and thereby allowing that layer efficiently to determine the behavioral response. Thus, the effect of an LC phasic re- sponse driven by the threshold crossing in the task-relevant decision layer can be thought of as collapsing the multilayered network into a single-layered network, thus approximating the optimal decision-making process imple- mented by the task-relevant decision layer. In this way, the LC phasic response resolves a fundamental trade-off between the flexibil- ity of a complex, multilayered system (that can support a wide variety of decision pro- cesses responsive to information from differ- ent sources and different levels of analysis) and the optimality of a single-layered deci- sion mechanism. From this perspective, the decision-driven LC phasic activation provides a mechanism for optimizing task performance by a multilayered system.
In the specific models previously discussed, there are only two layers: the decision layer and the response layer. Nevertheless, even in such a two-layered system, formal analysis re- veals that the adaptive gain-adjustment mech- anism implemented by LC produces reli- able improvements in performance that more closely approximate the optimal performance of a single-layered network (Gilzenrat et al. 2004). These improvements are expected to be considerably greater for more realistic,
www.annualreviews.org • LC and Optimization 419
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
multilayered networks.7 We should note, however, that the function of existing models relies on the detection of discrete threshold- crossing events (e.g., by LC, for generating its phasic response; and by the cortical net- work, for generating an overt behavioral re- sponse). This may reflect intrinsic nonlinear- ities of processing units involved; the specifics of these mechanisms remain to be described in further research.
The LC Tonic Mode Produces Adaptive Adjustments of Gain that Optimize Performance Across Tasks (Exploration Versus Exploitation)
In considering optimal performance thus far, our focus has been on performance within a single task, in which the LC phasic re- sponse produces adaptive adjustments in gain that serve to optimize performance of that task. In contrast, the LC tonic mode pro- duces a persistent increase in gain (i.e., re- sponsivity of widespread LC target neurons) that renders the system more sensitive to task- irrelevant stimuli. With respect to the current task, this is clearly disadvantageous. However, this tonic increase in gain may be adaptive by facilitating a change in behavior if either the current task is no longer remunerative or if the environment has changed and more valuable opportunities for reward or new be- havioral imperatives have appeared. That is,
7 A variant on this model allows the integration process to be distributed over several tightly coupled, interacting lay- ers of the network (rather than a single layer) but assumes that there are two thresholds for the integrators: one for driving the LC phasic response, and a much higher one for driving the behavioral response. The latter insures that spurious activity of task-unrelated units will not drive a be- havioral response. When the threshold for activating the LC is crossed by task-related (decision) units, the LC pha- sic response increases gain throughout the network, driving the most currently activated units toward the behavioral response threshold. This mechanism allows a more dis- tributed implementation of decision processing, while still exploiting the LC phasic response as a “temporal filter.” Formal analyses suggest this distributed case also exhibits improvements in performance with adaptive gain adjust- ment (Gilzenrat et al. 2004).
in addition to pursuing optimal performance within the specific task at hand, organisms face the broader and equally important challenge of deciding whether and for how long it is best to continue performing the current task. From this perspective, optimization involves not only determining how to best perform the current task, but also considering its utility against alternative courses of action and pur- suing these if they are more valuable. This is, of course, a more complex and less well- defined problem, which presents significant challenges to formal analysis. Reinforcement learning models represent one approach to this problem. Such models describe mecha- nisms that seek to exploit opportunities op- timally for reward by sampling a wide range of behaviors and strengthening actions lead- ing to states with the highest value (Montague et al. 2004, Sutton & Barto 1981).
A conundrum faced by reinforcement learning models is how to sample the values of a large number of different states. Early sampling will reveal some states that are more valuable than others. Then the agent must decide whether to spend most of its time engaged in behaviors associated with the most valuable ones that it has already discovered (that is, exploit these known states) or to seek new behaviors by continuing to sample the environment (that is, explore a broader range of states) in search of novel and potentially more valuable opportunities than those already discovered. This conundrum is often referred to as the trade-off between exploitation and exploration (e.g., Kaelbling et al. 1996). How it is handled can have a pro- found effect on reward accrued. If the agent favors exploitation too heavily before the environment has been adequately explored, then it risks missing valuable opportunities (in the terminology of thermodynamics, it may get caught in a local minimum). However, if it favors exploration too heavily, then it may fail to exploit known sources of reward adequately (i.e., it will be “unstable”).
In standard reinforcement learning mod- els, the trade-off between exploitation and
420 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
exploration is handled using a procedure that regulates the amount of noise (randomness) in the agent’s behavior. This is akin to an- nealing in thermodynamic systems. Anneal- ing is a procedure in which a molten metal is slowly cooled so that the molecules can move around thoroughly during recrystalliza- tion and achieve thermodynamic equilibrium producing the most uniform (optimal) lattice structure. This process prevents gaps in the lattice, i.e., local minima in thermodynamic terms. During initial exposure to an environ- ment when exploration is more valuable than exploitation, noise is set at a high level, en- couraging exploration (similar to a free flow of molecules, or melting).8 However, as in- creasingly valuable states are identified, noise is gradually diminished (akin to slow cooling and annealing, restricting flow and solidify- ing the lattice) so that the agent can most ef- fectively exploit the environment by focusing on behaviors tied to the most valuable states. Such annealing procedures help optimize the agent’s ability to extract reward from the en- vironment. However, typically these proce- dures are introduced deus ex machina (that is, using predefined schedules imposed by the modeler). Furthermore, such schedules typi- cally cannot accommodate changes in the en- vironment. For example, what happens when a previously identified source of reward be- comes unavailable or less valuable as the agent becomes sated (e.g., the defining structure of the lattice changes)? Under such condi- tions, the optimal strategy is to resume ex- ploring the environment (melting), sampling different behaviors until new sources of re- ward are discovered. This is exactly the role played by increases in tonic LC activity and
8 An increase in noise promotes the simplest form of explo- ration, which is random search. However, more sophisti- cated agents may, at least under some circumstances, use more structured, model-based forms of search (involving heuristics or explicit algorithms). Even in these cases, how- ever, a transient increase in noise may serve to disengage the current behavioral set, facilitating the discovery and pursuit of a new one.
attenuation of the phasic LC response in the adaptive gain theory: Increased baseline re- lease of NE increases the gain of units in the network indiscriminately, making them more responsive to any stimulus. This uniform in- crease in responsivity is tantamount to in- creasing noise and favoring exploration. The broad efferent network of LC projections is well suited for this role because it applies the tonic gain increase across global targets and circuits, thereby allowing a broad scan of pos- sible new reward sources.
At least two sources of recent evidence sup- port this theory concerning the function of the LC tonic mode. The first is provided by the target reversal experiment described earlier (Aston-Jones et al. 1997). Following reversal of reward contingencies, LC phasic responses to the former target rapidly diminished, while baseline (tonic) LC firing increased. This was maintained until phasic responses appeared for the new target and disappeared for the old one. That is, LC transitioned from a phasic to a tonic mode and then reversed as the new target was acquired. These findings are what would be predicted if LC implemented the an- nealing procedure associated with reinforce- ment learning in the task. Of course, this re- quires that LC has the relevant information to determine when to transition between phasic and tonic modes, an important question that we address shortly.
The second line of evidence comes from studies of human performance, and measure- ments of pupil diameter as an indirect index of LC activity. As shown in Figure 7, pupil diameter correlates remarkably well with LC tonic activity in the monkey (Rajkowski et al. 1993) and shows the same relationship to be- havioral performance as LC tonic activity. The latter finding has recently been corrob- orated in the human (Gilzenrat et al. 2003). Furthermore, numerous studies have shown that task processing is accompanied by rapid and dramatic pupil dilation, consistent with the occurrence of an LC phasic response to task-relevant events (Beatty 1982a,b; Richer
www.annualreviews.org • LC and Optimization 421
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Figure 7 Relationship between tonic pupil diameter and baseline firing rate of an LC neuron in monkey. Pupil diameter measurements were taken by remote eye-tracking camera at each instant in time when the monkey achieved fixation of a visual spot during the signal-detection task (described in text). Note the close direct relationship between the pupil diameter and the rate of LC activity.
& Beatty 1987).9 We recently measured both baseline- and stimulus-related pupil diame- ter in a task involving diminishing utility, designed to test our hypothesis concerning the function of tonic LC activity. Human subjects performed a series of tone discrim- inations of progressively increasing difficulty with rewards for correct performance that in- creased in value with increasing task difficulty. Initially, the increases in reward value out- paced increases in difficulty (and associated increases in errors) so that subjects remained engaged in the task. However, after several trials, the increases in difficulty led to suffi- cient numbers of errors as to reduce reward rate even in the face of the increasing value of correct responses. At the beginning of ev- ery trial, participants were allowed to press a reset button, which would start a new se-
9 Despite the close relationship between pupil diameter and LC activity, the mechanisms underlying this relationship are not yet understood. There are presently no known anatomic pathways that could mediate this relationship di- rectly, which suggests that these effects may reflect parallel downstream influences of a common source mechanism.
ries of discriminations, beginning again with low difficulty and low reward value. Partici- pants behaved optimally on average, choosing to reset when the success (expected utility) of the discriminations began to decline. Early in each trial series there were large phasic pupil dilations for each discrimination. As would be predicted for LC phasic responses, these dilations declined in amplitude, and baseline (tonic) pupil diameter rose as the task became more difficult and expected utility began to decline. Baseline pupil diameter was greatest at the point at which subjects chose to aban- don the current series, consistent with the hy- pothesis that this was mediated by an increase in LC tonic activity.
Although these findings are consistent with the hypothesis that LC tonic activity supports optimal performance by favoring exploration, this remains to be tested using more direct measurements. This dimension of our theory also has yet to be expressed in formal terms. Existing models of LC function provide a mechanism by which this could oc- cur (i.e., tonically increasing gain throughout the network) but have not directly established that such tonic activity optimizes perfor- mance when opportunities exist for sources of reward outside the current task. A recent line of work, however, has begun to make strides in this direction and converges on an interpretation of the function of NE release similar to the one we propose here. A. Yu and P. Dayan (manuscript under review) have used a Bayesian framework to develop a model of how an agent can optimally determine whether a failure of prediction (and therefore performance) reflects variability inherent in the current task (which they term “expected uncertainty”) or an underlying change in the environment (termed “unexpected uncertainty”). They propose that estimates of expected (task-related) uncertainty are mediated by acetylcholine, whereas estimates of unexpected uncertainty—which promote a revision of expectations—are mediated by NE. The latter corresponds closely to our the- ory that LC tonic activity (and corresponding
422 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
increases in tonic NE release) favors ex- ploration. Note, however, that both our hypothesis and that of Yu and Dayan assume the operation of evaluative mechanisms that identify violations of expectation or, more generally, decreases in utility that favor be- havioral changes. Sustained increases in gain mediated by the LC tonic mode, which lead to shifts in behavior, are adaptive in the sense that they are responsive to such evaluative information. Below we review evidence that leads us to propose that projections to LC from the frontal cortex may provide this evaluative information.
Summary. The findings reviewed above sup- port the theory that the LC-NE system helps optimize performance by adaptively adjust- ing gain (responsivity) of target sites in two distinct ways. LC phasic responses produce a transient, system-wide increase in gain driven by task-related decision processes, insuring that the outcome of such processes is ef- ficiently expressed in behavior (recall from above that the timing of phasic LC activation allows it to facilitate task-related behavior). This serves to optimize performance within a given task. In contrast, the LC tonic mode produces a more enduring and less discrimi- native increase in gain. Although this degrades performance within the current task, it fa- cilitates the disengagement of performance from this task and thereby the sampling of others. This action provides a mechanism by which the system can optimize performance in a broader sense—by identifying and pur- suing more remunerative forms of behavior when utility associated with the current task begins to diminish. Note that this may tran- siently accelerate a current reduction in utility by further degrading performance in the cur- rent task. This degradation of performance accounts for the far-right end of the Yerkes- Dodson curve (see Figure 2), where perfor- mance decreases as “arousal” (associated with tonic NE release) increases. According to our theory, this effect reflects the local conse- quences (for the current task) of a mechanism
that is serving to optimize performance on a broader scale. From this perspective, although the right end of the Yerkes-Dodson curve may appear to be maladaptive, in fact it reflects the operation of an important mechanism of longer term adaptation.10
Utility Assessment in Frontal Cortex Regulates LC Mode
Our theory proposes that the LC phasic mode supports optimization of current task perfor- mance as long as task-related utility remains sufficiently high, whereas the LC tonic mode supports optimization on a broader scale, fa- voring exploration when current task-related utility falls below an acceptable value (or that which may be available from other tasks). This begs a critical question: What determines when LC should transition between phasic and tonic modes? That is, how does LC know when current task utility exceeds or has fallen below an acceptable value? If LC is to respond in a truly adaptive fashion, then it must have access to information about rewards and costs. As briefly reviewed below, a growing body of evidence suggests that two frontal structures, the OFC and ACC, play critical roles in eval- uating rewards and costs, respectively. Fur- thermore, recent studies—motivated by our theory of LC—reveal that the most promi- nent descending cortical projections to LC come from these two frontal structures. We review this evidence in the sections that fol- low, concluding with a simple formal theory about how top-down evaluative information from the frontal cortex may be combined to regulate LC function.
The frontal cortex plays an important role in the evaluation of utility. The OFC and ACC have been the focus of increas- ingly intense study, using both direct neuronal
10 Although LC tonic activity may be adaptive, overly per- sistent LC tonic activity (or high arousal) can, in the limit, be maladaptive, as may be the case for clinical syndromes such as anxiety disorders and attention deficit disorder.
www.annualreviews.org • LC and Optimization 423
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
recording techniques in nonhuman primates and neuroimaging methods in humans. The results of these studies consistently indicate a role for these structures in the evaluation of utility.
OFC plays a role in the evaluation of reward. The OFC receives input from all modalities of high-order sensory cortices, in particular ar- eas processing information with strong appet- itive significance, such as taste and olfaction, as well as primary limbic structures such as the ventral striatum and amygdala (Baylis et al. 1995; Carmichael et al. 1994; Carmichael & Price 1995a,b; Ongur & Price 2000; Rolls et al. 1990). Neurons in monkey OFC are ac- tivated by rewarding stimuli in various modal- ities but not by stimulus identification alone nor by response preparation (Roesch & Olson 2004, Rolls 2004). Furthermore, OFC re- sponses vary in magnitude in proportion to the relative reward value of the corre- sponding stimulus (Tremblay & Schultz 1999; Wallis & Miller 2003), and reward-specific re- sponses diminish as the animal becomes sated for that particular reward (Critchley & Rolls 1996, Rolls et al. 1989). OFC neurons in monkey also are sensitive to the anticipation and delivery of reward (Schultz et al. 2000, Hollerman et al. 2000), and recent evidence suggests that OFC responses may be able to integrate the ongoing rate of reward over rel- atively extended periods (Sugrue et al. 2004). Many of these findings have been corrobo- rated in human neuroimaging studies involv- ing a variety of rewards, including money, food, and drugs of abuse (Breiter et al. 1997, 2001; Knutson et al. 2000; O’Doherty et al. 2002; Small et al. 2001; Thut et al. 1997; McClure et al. 2004). Collectively, these find- ings provide strong evidence that OFC plays an important role in the evaluation of reward.
ACC plays a role in the evaluation of cost. Like OFC, ACC receives convergent inputs from a broad range of neocortical and sub- cortical structures, including somatosensory areas and limbic structures such as insular cor-
tex, amygdala, and ventral striatum (Devinsky et al. 1995, Mesulam 1981). ACC is known to be directly responsive to aversive intero- ceptive and somatosensory stimuli, and to pain, in particular (e.g., Peyron et al. 2000). More recently, neurophysiological studies in the monkey as well as human electrophysio- logical and neuroimaging studies have consis- tently demonstrated that ACC is also respon- sive to negatively valenced information of a more abstract nature, such as errors in perfor- mance, negative feedback, monetary loss, and even social exclusion (Eisenberger et al. 2003; Falkenstein et al. 1991; Gehring et al. 1993; Holroyd et al. 2003, 2004a,b; Ito et al. 2003; Kiehl et al. 2000; Miltner et al. 1997; Yeung et al. 2005; Yeung & Sanfey 2004). In addi- tion to explicitly negative information, ACC responds robustly and reliably to task diffi- culty and conflicts in processing (e.g., Barch et al. 1997; Botvinick et al. 1999, 2001, 2004; Carter et al. 1998; Duncan & Owen 2000; Ullsperger & von Crammon 2001). Conflicts occur when simultaneously active processes compete for the expression of incompatible alternatives, a factor that may be directly (and possibly causally) related to task difficulty and has been formalized in neural network mod- els of task performance (Botvinick et al. 2001, Yeung et al. 2004). Thus, converging evi- dence suggests that ACC is responsive to a variety of negatively valenced signals—from pain to internal states that predict degraded performance—all of which may serve as in- dices of performance-related cost.
OFC and ACC send strong convergent projections to LC. Historically, anatomic studies of afferents to LC have focused al- most entirely on subcortical structures; there have been very few published reports of stud- ies examining possible inputs from cortical ar- eas (for a review, see Aston-Jones 2004). This may reflect the emphasis placed by traditional theories on the roles for LC in bottom-up pro- cesses such as sensory encoding and arousal (as discussed in the Introduction section). Impor- tantly, most studies have also been conducted
424 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
in nonprimate species in which cortical struc- tures (and the frontal cortex in particular) are substantially less well developed. Although Jodo and Aston-Jones observed that stim- ulation of rat prefrontal cortex can acti- vate LC neurons ( Jodo & Aston-Jones 1997, Jodo et al. 1998), and others reported in- hibitory responses (Sara & Herve-Minvielle 1995), anatomical studies showed that pre- frontal projections in rat terminate adjacent to, but not within, the LC nucleus (Aston- Jones 2004). Until recently, therefore, little was known about the extent to which pri- mate LC neurons receive direct descending projections from the neocortex. The adap- tive gain theory of LC-NE function outlined above, however, predicts that LC should re- ceive information about task-related utility from high-level structures. Motivated directly by this prediction, a series of anatomic studies was undertaken in monkey to determine the extent to which LC receives top-down corti- cal projections. These studies have revealed a consistent and striking pattern of cortical pro- jections in the primate LC, the preponderance of which come from OFC and ACC.
As illustrated in Figures 8 and 9, focal injections of retrograde tracer into the mon- key LC reveal a large number of labeled neu- rons in OFC and ACC (Aston-Jones et al. 2002; Rajkowski et al. 2000; Zhu et al. 2004; M. Iba, W. Lu, Y. Zhu, J. Rajkowski, R. Morecraft & G. Aston-Jones, manuscript in preparation). These retrograde results have been confirmed by injection of anterograde tracers into OFC or ACC, which yielded prominent fiber and terminal labeling in the monkey LC nucleus and peri-LC dendritic area (Figure 8) (M. Iba, W. Lu, Y. Zhu, J. Rajkowski, R. Morecraft & G. Aston-Jones, manuscript in preparation; Zhu et al. 2004). Importantly, these OFC and ACC projections appear to be the major cortical inputs to LC; relatively few neurons in other cortical ar- eas were retrogradely labeled from LC in- jections. For example, very few neurons were retrogradely labeled in area 46, and antero- grade tracing from area 46 produced label-
ing nearby, but not within, the LC, consistent with a previous report (Arnsten & Goldman 1984). Interestingly, the bulk of OFC inputs to LC appears to originate in the caudolateral OFC, the same area that receives strong direct olfactory and taste primary reinforcer inputs (as noted above). Numerous retrogradely la- beled neurons also extend caudally from the OFC into the anterior insular cortex. ACC neurons that innervate LC are located in both dorsal and ventral ACC subdivisions (includ- ing areas 24, 25, and 32) and densely populate layer 5/6 throughout the rostral ACC. These results indicate that OFC and ACC provide prominent direct input to LC in monkey and that these projections are the major cortical influences on LC neurons. These studies also indicate that OFC and ACC inputs to LC in the monkey are stronger than those in the rat, where prefrontal fibers terminate nearly ex- clusively in the peri-LC dendritic zone and do not appreciably enter the LC nucleus proper (described above).
OFC and ACC may regulate LC function. The evidence reviewed above indicates that OFC and ACC each play an important role in assessing utility and that both of these struc- tures project directly to LC. These findings suggest that these frontal areas could influ- ence LC function on the basis of assessments of utility, consistent with the adaptive gain theory of LC function. There are two ways in which this could occur: OFC and ACC could drive LC phasic activity directly, and they could modulate LC mode of function.
OFC and ACC may drive LC phasic acti- vation. If LC phasic responses are driven by the outcome of decision processes, an impor- tant question is, what cortical regions convey this information to LC? The obvious candi- dates are regions that house the neural ac- cumulators associated with the decision pro- cesses themselves, which have typically been localized to cortical sensorimotor integration areas (e.g., to lateral intraparietal cortex and frontal eye fields for visual tasks requiring an
www.annualreviews.org • LC and Optimization 425
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Figure 8 Projections to the LC from the anterior cingulate cortex (ACC) and the orbitofrontal cortex (OFC) in monkey. (A) Low-power photomicrograph of a frontal section through the ACC showing retrogradely labeled neurons in area 24b/c. Area shown is just ventral to the cingulate sulcus. (B ) High-power photomicrograph showing retrogradely labeled neurons in the ACC (corresponding to rectangle in panel A). Note labeled cells located in deep layer V/superficial layer VI. (C ) Low-power photomicrograph of a frontal section through the OFC showing retrogradely labeled neurons in area 12. Lateral orbital sulcus is at star. (D) High-power photomicrograph showing retrogradely labeled neurons in the OFC (corresponding to rectangle in panel A). Note cells located in deep layer V/superficial layer VI. Neutral red counterstain used for sections in panels A–D. Medial is at right, and dorsal is at top. (E, F ) Low (E )- and high-power (F ) photomicrographs of a frontal section through the LC and peri-LC showing fibers and terminals labeled from an injection of the anterograde tracer biotinylated dextran amine in the ipsilateral OFC (area 12) of an African green monkey. Noradrenergic neurons and processes are stained brown with an antibody against tyrosine hydroxylase. Note close juxtaposition of OFC fibers and terminals with noradrenergic somata and dendrites. Lateral is at right, and dorsal is at top.
426 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Figure 9 Plots of retrogradely labeled neurons in ACC and OFC after injections of CTb into monkey LC. (A) ACC neurons labeled from the monkey LC. Lower sections are high-power views containing plotted cells; upper sections are low-power views to give orientation. (B ) OFC neurons labeled from the monkey LC. Sections at right are high-power views containing plotted cells; sections at left are low-power views to give orientation. For both panels, A22–A27 refer to distances in mm from the interaural line. Plots were composed on atlas sections from Paxinos et al. 2000.
oculomotor response). However, as reviewed above, these areas do not provide direct pro- jections to LC. Alternatively, the outcome of decision processes may be relayed to LC in- directly via OFC, ACC, or both, which re- ceive inputs from a wide array of sensori- motor areas (Baylis et al. 1995, Carmichael
& Price 1995b, Morecraft et al. 1992). Such relays may weight decision-related signals by motivational significance—that is, perceived utility of the current task. This possibility is supported by the pattern of task-specific responses commonly observed within OFC and ACC that are also closely related to
www.annualreviews.org • LC and Optimization 427
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
the motivational significance of the eliciting event. This, in turn, is consistent with the fact that LC phasic responses are limited to goal-related events (e.g., target but not dis- tractor stimuli in a signal-detection task), with an amplitude modulated by the motivational significance of the stimulus (e.g., the reward associated with appropriate performance) (Aston-Jones et al. 1994, Rajkowski et al. 2004). Thus, OFC and ACC may relay the outcome of task-related decision processes, modulated by their assessed utility, driving the LC phasic response. Studies are presently un- derway to test this hypothesis more directly.
OFC and ACC may regulate LC mode. The above considerations address the possibility that OFC and ACC drive LC phasic acti- vation in response to individual events (e.g., decision outcome within the trial of a task). The adaptive gain theory suggests that these frontal structures also influence LC func- tion by driving transitions between phasic and tonic modes to regulate the balance between exploitation and exploration: When evalua- tions in OFC or ACC indicate the current task is providing adequate utility, they drive LC toward the phasic mode (by increasing elec- trotonic coupling, reducing baseline drive, or both), favoring exploitation of that task for associated rewards. However, when utility di- minishes sufficiently over prolonged dura- tions, they drive it toward the tonic mode, fa- voring exploration. This process requires that utility be evaluated over both short and longer time frames.
For example, consider a case in which the current task is associated with high utility (e.g., the animal is thirsty and correct per- formance provides juice). In this situation, it is advantageous to optimize performance on the task and maximally exploit the utility it provides. The adaptive gain theory states that this is promoted by the LC phasic mode, during which event-related phasic LC activ- ity facilitates task-appropriate behavioral re- sponses. Furthermore, if performance should temporarily flag (e.g., owing to a momentary
lapse of attention), the LC phasic mode should be augmented to restore performance.
Compensatory adjustments following lapses in performance have repeatedly been observed (e.g., Botvinick et al. 1999; Gratton et al. 1992; Jones et al. 2002; Laming 1968, 1979; Rabbitt 1966). Furthermore, there is strong evidence that they are mediated by an evaluative function in frontal cortex, consistent with the mounting evidence discussed above that monitoring mechanisms within ACC detect lapses in performance and signal the need to augment top-down control for the current task (Botvnick et al. 2001). However, previous theories have assumed ACC-based monitoring mechanisms act directly on prefrontal systems responsible for top-down control (e.g., Botvinick et al. 2001, Jones et al. 2002). The adaptive gain theory of LC suggests an additional, more general mechanism for improving performance. Signals from ACC to LC (indicating an adverse outcome), possibly complemented by signals from OFC to LC (indicating absence of an expected reward), may augment the LC phasic mode (by further increasing electronic coupling, reducing LC baseline drive, or both). This, in turn, would improve perfor- mance on subsequent trials by enhancing the LC phasic response and thereby augmenting the gain of units responsible for task execu- tion. Increased phasic release of NE may also have direct enhancing effects on task-specific control representations in prefrontal cortex (PFC) (Arnsten et al. 1996, Cohen et al. 2004). This effect could further contribute to the compensatory increase in control following a transient decrease in performance and/or reward. One appeal of this hypothesis is that it provides a general mechanism by which the detection of momentary reductions in utility can augment task control without requiring the monitoring mechanism to have special knowledge about the nature of control required for every possible task. That is, a global signal (LC-mediated NE release) that adaptively adjusts gain throughout the pro- cessing system can interact with task-specific
428 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
control mechanisms (e.g., in PFC) to produce requisite improvements in performance.
The foregoing account addresses circum- stances in which overall task-related util- ity remains high, and momentary lapses in performance (and utility) can be rectified by enhancement of control. However, what happens if there is a persistent decline in util- ity? In such circumstances, augmenting con- trol associated with the current task may no longer be advantageous. For example, if per- formance fails to improve despite compen- satory adjustments, or if task-related utility progressively declines for other reasons (e.g., satiety, depletion of the task-related source of reward, or an increase in the costs associated with its procurement), then the relationship between utility and task investment should re- verse: Further decreases in utility should pro- mote task disengagement rather than attempts to restore performance. That is, they should favor exploration over exploitation. The adap- tive gain theory proposes that this is mediated by a transition to the LC tonic mode. Im- portantly, the determination of when to pro- mote exploration over exploitation requires that evaluative mechanisms take account of both short- and long-term changes in utility. There are many ways of doing so. The fol- lowing equation describes one simple means (shown graphically in Figure 10):
Engagement in current task
= [1 − logistic(short term utility)] ∗ [logistic(long-term utility)],
(Equation 1)
where logistic refers to the sigmoid function 1/(1 + e−utility), and high values of the equation favor the LC phasic mode, whereas low values favor the tonic mode.
We assume that evaluations of utility are computed within both OFC and ACC and then integrated (averaged) over relatively short (e.g., seconds) and longer (e.g., minutes) timescales. How estimations in OFC and ACC are combined and then averaged over different timescales remains a matter for fur-
Figure 10 Plot of the relationship between engagement in the current task and task-related utility integrated over relatively brief (e.g., seconds) and longer (e.g., minutes) timescales given by Equation 1 (see text). The adaptive gain theory of LC-NE function proposes that high values of this equation favor the LC phasic mode, whereas low values favor the tonic mode. Accordingly, low values of long-term utility favor the LC tonic mode (exploration), whereas high values favor the LC phasic mode (exploitation). Note that when long-term utility is low, changes in short-term utility have little impact. However, when long-term utility is high, a decrease in short-term utility augments the LC phasic mode, implementing an adaptive adjustment that serves to restore performance.
ther research. However, allowing that such computations take place, Equation 1 pro- vides a simple means by which OFC and ACC may regulate engagement in the cur- rent task by controlling a single, or small number of, simple physiological parameters in LC. High values of this function favor task engagement (by driving a transition to LC phasic mode through increases in electro- tonic coupling, decreases in baseline drive, or both), discounted by enduring declines in util- ity that favor task disengagement (LC tonic mode). When long-term utility is high, de- creases in short-term utility augment the LC phasic mode. Thus, when overall utility is high, momentary reductions favor improve- ments in task performance. In contrast, these effects are diminished when long-term utility is low: Persistent declines in utility drive LC
www.annualreviews.org • LC and Optimization 429
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
toward the tonic mode, favoring task disen- gagement. Note that this proposed modula- tion of LC mode occurs simultaneously and independently of the phasic excitation of LC proposed also to originate in these two struc- tures in response to the outcome of individual decisions.
Summary. The findings reviewed above indicate that neurons in OFC and ACC show task-selective responses that represent the present value of task-related events. These structures provide substantial direct projections to LC in monkey. As illustrated
Figure 11 Integrated neural system for the adaptive regulation of performance. Components in green represent neural pathways responsible for task execution, including top-down control from prefrontal cortex (PFC). Components in red represent monitoring and evaluative mechanisms in OFC and ACC. These components assess task-related utility based on indicators of performance (including internal conflict and external feedback). This information is used both to drive LC activity (in the phasic mode) and to regulate LC mode of function (phasic versus tonic). These influences on LC, in turn, regulate performance through release of NE throughout the processing system, either phasically in response to task-relevant events (phasic mode) or in a more sustained manner (tonic mode).
in Figure 11, the adaptive gain theory of LC function proposes that these projections pro- vide the information necessary to drive LC phasic responses directly, as well as to drive transitions between its phasic and tonic modes of function. Specifically, the theory proposes that the outcome of decision processes associated with high potential utility are rep- resented in frontal structures (OFC and ACC) that drive LC phasic responses. Furthermore, assessments of utility by these frontal struc- tures are integrated over different timescales and used to regulate LC mode. Brief lapses in performance, in the context of otherwise high utility, augment the LC phasic mode, improv- ing task performance. In contrast, enduring decreases in utility drive transitions to the tonic mode, promoting disengagement from the current task and facilitating exploration of behavioral alternatives. Taken together, these mechanisms constitute a self-regulating sys- tem by which LC—informed by evaluations of utility in frontal structures—can control behavioral strategy through adaptive adjust- ments of gain in its global efferent targets.
GENERAL DISCUSSION
The adaptive gain theory suggests that the LC-NE system plays a more important and specific role in the control of performance than has traditionally been thought. This new theory is an important evolution of an LC theory that proposed roles for this system in vigilance and response initiation (Aston-Jones 1985, 1991b) by further specifying the mecha- nisms that might be involved in executing such functions. The presently proposed framework has a wide range of implications for under- standing the neural mechanisms underlying both normal behavior and its impairment in clinical disorders associated with disturbances of decision making and control. At the same time, the theory raises several questions that remain to be addressed. In the reminder of this review, we consider some of these ques- tions and the broader implications of our theory.
430 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Descending Influences Play an Important Role in LC-NE Function
Many previous analyses of LC function have focused on its role in mediating ascending (“bottom-up”) influences on cortical process- ing. For example, a major focus of research has been on inputs to LC from subcortical mech- anisms involved in the sleep-waking cycle, supporting the view that LC translates these mechanisms into modulatory changes in neo- cortex producing alertness or sleep (Aston- Jones et al. 2005, Berridge & Waterhouse 2003, Jones 1991, Saper et al. 2001). Similarly, theories about the role of LC in sensory pro- cessing have generally assumed that it receives inputs from low-level processing mechanisms, mediating the alerting effects of highly salient events via its widespread ascending projec- tions to the neocortex (Devilbiss & Water- house 2000, 2004; Hurley et al. 2004). Our theory allows a more precise understanding of LC’s role in bottom-up processes such as sleep and sensory processing. For example, de- scending regulation of LC suggests a mechan- sim for volitional control of waking in the face of fatigue, and for increased sensory focus with increased task difficulty. Critcally, how- ever, our theory highlights the importance of descending cortical influences on LC and the refined role that the LC-NE system plays in regulating cortically based mechanisms for decision making and control. There is reason to believe that such influences are more highly developed in the primate brain, commensu- rate with the expansion of frontal structures and their top-down influence on processing and behavior. We have highlighted the roles of orbital and anterior cingulate regions of frontal cortex in particular, both of which send strong projections directly to LC in monkeys (projections that appear to be substantially more developed than in the rat) (Aston-Jones et al. 2002, Rajkowski et al. 2000, Zhu et al. 2004). These areas have been consistently im- plicated in evaluative functions that are com- mensurate with the role we propose for LC in regulating behavioral performance.
LC Activity is Plastic and May Play a Role in Learning
A critical component of the adaptive gain the- ory is that the LC phasic response is driven by the outcome of task-relevant decision pro- cesses. This hypothesis implies that the LC phasic activation must be plastic to adapt to changes in task demands (e.g., the relevance of different decision processes). This impli- cation is consistent with the physiological ev- idence. Most strikingly, reversal experiments demonstrate that LC phasic responses reliably track changes in the target stimulus. How- ever, this plasticity must reflect the response characteristics of the systems that drive LC. This too is consistent with the evidence, given our hypothesis that the LC phasic response is driven by frontal structures, including OFC and ACC. As reviewed above, these struc- tures demonstrate strong task-selective re- sponses. Furthermore, previous studies indi- cate that OFC exhibits marked plasticity in its response to reward-related stimuli during re- versal experiments when the valence of stim- uli is abruptly altered. For example, Rolls has shown that in behaving monkeys OFC re- sponses change quickly to track changes in the motivational relevance of stimuli (Rolls et al. 1996, Thorpe et al. 1983; see also Wallis & Miller 2003). These observations support the idea that both the selectivity and plasticity of the LC phasic response reflect afferent drive by frontal structures. At the same time, LC itself may contribute to plasticity within these structures. A role for the LC-NE system in learning has frequently been suggested in the literature (Amaral & Foss 1975, Anlezark et al. 1973, Archer et al. 1984, Bouret & Sara 2004, Cirelli & Tononi 2004, Harley 1991, Harris & Fitzgerald 1991, Koob et al. 1978, Velley et al. 1991). The adaptive gain theory sug- gests at least one way in which this influence on learning might occur. As discussed above, changes in reward contingency should drive LC into the tonic mode. This shift, in turn, should promote exploration and facilitate the discovery of new reward contingencies
www.annualreviews.org • LC and Optimization 431
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
that in turn provide a new source of drive for the LC phasic response. As discussed earlier, this hypothesis is supported by the observa- tion that reversals in reward contingency (re- quiring new learning) precipitate a shift to the LC tonic mode, followed by shifts back to the phasic mode once the new target has been ac- quired. This hypothesis is also consistent with interactions between the LC-NE and DA sys- tems suggested by the adaptive gain theory, as discussed in the following section. How- ever, a direct test of this hypothesis will re- quire more detailed studies that characterize the relative timing of frontal and LC phasic activity in response to task-relevant stimuli and its evolution over the course of reversal conditioning.11
Interactions Between the LC-NE and DA Systems are Important for Normal and Disordered Cognition
There are many similarities between the LC-NE and DA systems. Both NE and DA are neuromodulatory neurotransmitters that have similar physiological effects on target systems (e.g., modulation of gain; Nicola et al. 2000; Servan-Schreiber et al. 1990; Waterhouse et al. 1980, 1984); both are responsive to motivationally salient events (e.g., reward predictors); and disturbances of both have been implicated in highly overlapping sets of clinical disorders (such as schizophrenia, depression, and attention deficit disorder). Despite these similarities, the relationships between these systems and
11 The form of plasticity discussed here involves learning over many trials. However, behavioral flexibility can also be exhibited in more rapid form from trial to trial. Such flexi- bility is thought to rely on PFC mechanisms responsible for cognitive control (e.g., Duncan 1986, Shallice 1988). Ac- cording to one recent theory, PFC supports such behavioral flexibility by providing top-down modulation of process- ing along task-relevant pathways (Miller & Cohen 2001). In addition to influencing the flow of activity along cortical pathways, such mechanisms could also dynamically modu- late which cortical circuits (e.g., within OFC or ACC) most effectively drive LC activity, providing another mechanism by which LC phasic responses are driven by the outcome of task-relevant processes.
how they interact has remained unclear. In part this ambiguity has been due to the lack of formal theories about the function of either system. Recently, however, Montague et al. (1996) have proposed a sophisticated theory of DA function that suggests it im- plements the learning signal associated with a reinforcement learning mechanism. This theory affords a direct point of contact with the adaptive gain theory of LC-NE function.
As previously discussed, reinforcement learning requires an annealing procedure, favoring exploration during learning in new (or changing) environments and promoting exploitation when reliable sources of reward are discovered. The adaptive gain theory proposes that the LC-NE system serves this function, implementing an annealing mecha- nism that is adaptive to ongoing estimates of current utility. Thus, early in learning, when utility is low, LC remains in the tonic mode, favoring exploration. However, as sources of reward are discovered DA-dependent reinforcement learning strengthens be- haviors that produce these rewards. This strengthening increases current utility, driv- ing LC into the phasic mode, which further stabilizes and exploits the utility associated with DA-reinforced behaviors. This process continues until the current source of reward is either no longer valued or available. As utility declines, LC is driven back into the tonic mode, promoting exploration and learning of new behaviors. In this way, the proposed functions of the LC-NE and DA systems may interact synergistically to implement an auto-annealing reinforcement learning mechanism that is adaptive both to the needs of the organism and changes in the environment. Although these theories of the LC-NE and DA systems are both early in their development, together they poten- tially offer a powerful new account of how these systems interact, which may provide conceptual traction in understanding how disruptions in these systems may impact one another in producing the complex patterns of disturbance observed in clinical disorders.
432 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
How do Specific Effects Arise from a “Nonspecific” System?
The broad efferent anatomy, slow conduc- tion speed, and modulatory postsynaptic ef- fects of LC neurons have traditionally been interpreted as evidence that the LC-NE sys- tem plays a relatively nonspecific role in state setting and arousal (as discussed above). Given these properties, it is natural to wonder how this system could support the more precise form of regulation proposed by the adap- tive gain theory (e.g., involving, in the phasic mode, real-time responses that have within- trial effects on performance). The findings we have reviewed suggest two responses to this concern. First, direct recording studies from LC in animals performing complex con- ditioned tasks have revealed detailed, task- specific patterns of LC response that were not previously observed in nonconditioned sub- jects. This finding indicates that recordings of these neurons during conditioned behav- ior may be critical to observing these prop- erties of the LC-NE system. This point is further underlined by the finding that OFC and ACC send strong projections to LC in monkey that are not apparent in subprimate species. A prominent role for these prefrontal structures in the regulation of LC function indicates that studies of the LC system in pri- mates may be critical to uncovering more fully the precise and subtle roles played in cognitive processing.
Second, the work we have reviewed also highlights the importance of formally explicit theoretical models. With respect to the LC- NE system, such models have demonstrated how a mechanism with low spatial (efferent) specificity and modulatory effects can play an important role in regulating, and even mediat- ing, high-level cortical function. In particular, modeling work has characterized the dynam- ics of the mechanisms involved and has veri- fied that these mechanisms are plausible given the observed temporal properties of LC ac- tivity and NE release. For example, modeling work was essential in showing that LC pha-
sic activity at the appropriate time (typically within 150 ms of the stimulus and immedi- ately postdecision) can increase gain in target areas and have an impact on processing of the behavioral response. Modeling also suggested an important function for the global projec- tions of the LC-NE system: These projections allow the system to facilitate a broad range of possible behaviors favoring exploration in the tonic mode. At the same time, modeling work has shown that these widespread projections do not compromise the more precise func- tion performed by the LC phasic mode (given appropriate timing of the phasic response). Indeed, and perhaps most important, these broad projections allow LC to carry out its function without the need for special knowl- edge of where the changes in gain are needed for a particular task. Finally, modeling work has demonstrated how these effects can arise from the modulation of simple low-level phys- iological variables within LC (electronic cou- pling or baseline afferent drive) that are suffi- cient to regulate transitions between the LC phasic and tonic modes. Thus, modeling work has shown how a distributed system with sub- tle modulatory effects, driven by a small num- ber of simple parameters, can have precise and profound effects on higher-level cogni- tive processes.
Optimization is Critical in a Competitive Context
The adaptive gain theory proposes that the LC-NE system helps to optimize per- formance within and across tasks. Recent modeling analyses indicate that adaptive adjustments of gain in a two-layer network improve reward rate by approximately 10%– 20% (Gilzenrat et al. 2004). Although this is a conservative estimate (analyses of multilay- ered systems may reveal greater benefits), the benefits noted thus far appear to be modest. However, a modest increase in performance in an isolated context may translate into a highly significant effect in a competitive
www.annualreviews.org • LC and Optimization 433
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
environment, where resources are scarce and can be lost to another agent working with a similar motivation and strategy. Thus, increases in response speed or accuracy that have a modest impact on performance in the isolated environment of a laboratory task may make the difference between survival and ex- tinction in a context where food is limited and competition from other agents exists. This hypothesis may provide some insight into the adaptive advantage of a system such as the LC- NE and may give a reason for its evolution in vertebrates. Future modeling studies employ- ing a game-theory approach will be of interest to examine the benefits of adaptive gain ad- justment in a competitive environment.
Relation to Earlier Studies of the LC-NE System
Our review has focused primarily on recent findings from studies of LC in monkeys. Al- though limited in number, such studies pro- vide the most detailed and elaborate data addressing the role of the LC-NE system in behavior. Nevertheless, a theory of LC- NE function should also be consistent with findings using other methods, including le- sion and pharmacological manipulations of the LC-NE system, recording of LC activ- ity and NE release in nonprimate species, and electrophysiological findings related to LC function in humans.
Previous lesion and pharmacological stud- ies implicated the LC in cognitive func- tions. This literature is vast, and page limita- tions permit only a selective review of studies of the role of LC in performance and cogni- tive function (for more extensive reviews see Berridge & Waterhouse 2003, Coull 1994, Robbins & Everitt 1995). Many such stud- ies are compatible with the view that the LC- NE system is important for supporting task- focused behavior. For example, Roberts and colleagues (1976) and others (Oke & Adams 1978) reported that rats with lesions of the ascending LC pathway were more disrupted
by distractors during discriminative learning than were intact animals. Robbins and col- leagues (Carli et al. 1983, Cole & Robbins 1992) found that lesions of ascending LC pro- jections in rats produced deficits in a continu- ous performance task that required sustained monitoring for visual target stimuli that could occur in multiple locations. Lesions impaired target discrimination performance and pro- longed reaction times when distracting stim- uli were presented just before targets or when target presentation was unpredictable. This group also found that lesions of LC projec- tions in rats increased the effectiveness of con- ditioning to contextual stimuli but decreased conditioning to explicit cues (Selden et al. 1990a,b). Although these findings are consis- tent with the idea that the LC-NE system is important in task-focused performance, the mechanisms underlying behavioral changes following these lesions have been difficult to define. Moreover, not all lesion studies of LC produced similar results (e.g., Pisa & Fibiger 1983, Pisa et al. 1988). The conflicting results of some studies may be due at least in part to plasticity following the intervention, known to be substantial for LC (Fritschy & Grzanna 1992, Haring et al. 1986, Levin et al. 1985). In addition, the adaptive gain theory suggests that experimental manipulations should inter- act critically with LC mode of function (phasic vs. tonic), a factor that has not been considered in the design or interpretation of most previ- ous studies. When this is taken into account, previous lesion studies can often be seen in a new light that is compatible with the adaptive gain theory.
For example, a number of plastic responses have been observed following LC lesions, in- cluding increases in NE receptor number or sensitivity (Dooley et al. 1987, Harik et al. 1981, U’Prichard et al. 1980) and in tyrosine hydroxylase (Acheson et al. 1980) and firing rate (Chiodo et al. 1983) in the remaining LC neurons, as well as compensatory changes in other non-LC systems (Carboni & Silvagni 2004, Harik et al. 1981, Martin et al. 1994, Valentini et al. 2004). Such lesion-induced
434 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
plasticity could effectively upregulate LC-NE function tonically. At the same time, LC le- sions would eliminate any temporally spe- cific (e.g., decision-driven) phasic NE release in LC target areas; that is, compensatory changes would not be able to mimic the tem- poral properties of LC neuronal discharge. Thus, the net effect of LC lesions may have been to produce a persistent state similar to the LC tonic mode. This idea would be con- sistent with increased responsiveness to dis- tractors and decreased task-focused perfor- mance, as observed in several lesion studies (discussed above). This postlesion tonic func- tion of LC could also produce decreased con- ditioning with explicit sensory cues (which require more focus on the task at hand, sup- ported by the phasic LC mode) and increased performance and conditioning with contex- tual cues (facilitated by increased responsivity to a broader set of events or exploration as- sociated with the tonic LC mode). Although length constraints prohibit a detailed treat- ment here, similar considerations are applica- ble to prior studies of LC function in attention using pharmacological approaches.
Several pharmacological studies have also investigated the effects of LC-NE function on memory (see Berridge & Waterhouse 2003 for a review). Our focus in this review has been on optimization of performance in sim- ple tasks involving sensory stimuli and mo- tor responses. However, as noted earlier, the adaptive gain theory applies equally to tasks involving other types of processes, such as the retrieval and storage of information in long-term memory. For example, in one study, LC was stimulated by injection of the alpha2 autoreceptor antagonist idazoxan (Sara & Devauges 1989). This agent blocks the local inhibitory effects of NE within the LC, which may have potentiated LC phasic responses. When animals were trained for several weeks on a set of associations, idazoxan administra- tion just prior to a retention test produced fewer errors than in nontreated animals. This result is consistent with the possibility that po- tentiation of LC phasic responses facilitated
the outcome of memory-driven decision pro- cesses, akin to its effects on stimulus-driven decision processes that have been the focus of this review.
Studies by Arnsten and colleagues indicate that manipulations of LC-NE function also have an impact on working-memory function. They found that systemic injections of alpha2 NE autoreceptor agonists, such as clonidine or guanficine, facilitated working-memory performance in aged monkeys (Arnsten et al. 1996). The doses used are thought to preferentially activate alpha2 receptors lo- cated postsynaptically on LC target neurons and are assumed to augment NE effects. It is difficult to interpret these results in the context of the adaptive gain theory with- out a more precise understanding of the dy- namics of these pharmacologic manipulations (e.g., the extent to which they impact phasic versus tonic NE release). Nevertheless, they could be consistent with a relationship of LC- NE function to the Yerkes-Dobson curve (see Figure 2) if it is assumed that aged animals have tonically diminished LC-NE projections and receptor function (consistent with prior data from Burnett et al. 1990, Eriksdotter Jonhagen et al. 1995, Iversen et al. 1983, and Tejani-Butt & Ordway 1992). Thus, injection of NE autoreceptor agonists into aged animals could improve function by increasing stimu- lation of deficient postsynaptic NE receptors and preferentially restoring the phasic mode of LC activity. (The latter is suggested by a previous study showing that stimulation of NE autoreceptors on LC neurons preferen- tially reduces baseline activity while leaving glutamate-driven responses intact, in effect emulating an increased phasic mode; Aston- Jones et al. 1991a). The same manipulation in normal animals, where clonidine may prefer- entially activate intact cortical NE receptors, could maladaptively mimick a tonic LC mode.
Recordings of LC activity or NE release in nonprimate species. Although relatively few studies involving nonprimate species have recorded impulse activity of LC neurons in
www.annualreviews.org • LC and Optimization 435
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
behaving animals, the overall results are con- sistent with the adaptive gain theory. Early studies reported that NE-LC neurons in rats and cats (Aston-Jones & Bloom 1981b, Rasmussen et al. 1986), as in monkeys (Grant et al. 1988), were responsive to salient uncon- ditioned stimuli in many modalities. These LC responses occurred in close relation to orienting behaviors and were much smaller or absent when stimuli produced no overt be- havioral response. These results are consis- tent with other studies in rat, which indicates that LC neurons are activated following novel stimuli (McQuade et al. 1999, Sara et al. 1995, Sara & Herve-Minvielle 1995). This set of findings can be understood in terms of the adaptive gain theory by noting that orienting and novelty responses are rudimentary deci- sions to act in a nonconditioned manner to highly salient events likely to have motiva- tional significance (e.g., a loud cracking sound likely signifies the danger of a falling branch or an approaching predator). Thus, the associ- ated LC activity may represent an evolution- arily hard-wired response that has facilitating effects on behavior analogous to the learned LC phasic responses in the 2AFC tasks stud- ied in the laboratory (described above).
Finally, the adaptive gain theory also pre- dicts that LC activity should increase (with a shift to the tonic mode) as unexpected and prolonged changes in reward contingencies occur. Results of a recent study by Dalley et al. (2001) are consistent with this prediction. Us- ing microdialysis, they found that cortical re- lease of NE (for which LC is the sole source) was increased in a sustained manner when an instrumental task was abruptly changed to noncontingent delivery of reward that was otherwise equal in overall amount. This is consistent with the predicted shift to LC tonic mode and associated increase in sustained NE release.
Human electrophysiological findings and LC function. One of the most robust find- ings from scalp recordings of event-related potentials (ERPs) in humans is the P3. This is
a positive potential with a broad scalp distri- bution that typically occurs ∼300 ms follow- ing task-relevant stimuli. The literature on the P3 is vast and well beyond our ability to re- view here. However, early in the study of this phenomenon investigators proposed that this potential may reflect neuromodulatory func- tion (Desmedt & Debecker 1979) and impli- cated the LC-NE system in particular (Pineda et al. 1989). Elsewhere, we have reviewed the literature on the P3 with regard to this hy- pothesis and the adaptive gain theory of LC- NE function (Niewenhuis et al. 2005). There we suggest that the adaptive gain theory of LC-NE function can explain a wide range of seemingly disparate findings concerning the P3. Both the LC phasic response and the P3 appear shortly following target stimuli in odd- ball paradigms (e.g., the signal-detection task described above), and both depend on the motivational significance of, as well as atten- tion paid to, eliciting stimuli. Furthermore, similar to the LC phasic response, P3 ampli- tude is greater for hits than for false alarms or misses in a signal-detection task; and fac- tors that increase the duration of the deci- sion process (e.g., stimulus degradation, re- duced stimulus intensity) have generally been found to increase P3 latency and RT by a sim- ilar amount, paralleling the consistent rela- tionship between LC phasic response and RT. Finally, the P3 is widely considered to be asso- ciated with completion of stimulus categoriza- tion, consistent with our hypothesis that the LC phasic response is driven by the outcome of task-related decision processes. These con- siderations have led us to hypothesize that the P3 reflects the phasic enhancement of gain in the cerebral cortex induced by LC- mediated release of NE (Niewenhuis et al. 2005). This hypothesis awaits direct testing in nonhuman primates using coordinated LC and P3 recordings in the same subjects. How- ever, if validated, it will provide an impor- tant link between neurophysiological studies in nonhuman species and the study of LC function in humans, including the vast litera- ture that already exists concerning the P3 both
436 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
in normal and disordered cognition (Duncan 2003).
The adaptive gain hypothesis also suggests an explanation for the close relationship of the P3 to the attentional blink. The atten- tional blink refers to the failure to process the second of two sequentially presented tar- gets when they are embedded in a series of rapidly presented stimuli (e.g., every 100 ms; Raymond et al. 1992). Models implement- ing the adaptive gain theory (S. Nieuwenhuis, M.S. Gilzenrat, B.D. Holmes & J.D. Cohen, manuscript under review) have successfully simulated this effect by attributing it to the empirically observed postactivation inhibi- tion of LC neurons (Aghajanian et al. 1977, Andrade & Aghajanian 1984, Williams et al. 1984), which is presumed to render these cells unresponsive to the second target.
Limitations and future directions. Many elements in the adaptive gain theory require testing, further analysis, and additional elab- oration. Nevertheless, even at this early stage of development, the theory makes several spe- cific predictions that may stimulate new re- search. Below we consider some of the areas that call for additional work and that indicate predictions to be tested.
One limitation of the theory is that it is based heavily on animal studies. It will be important, therefore, to develop methods to measure or selectively manipulate LC activ- ity in humans. As discussed in this review, pupilometry and scalp electrophysiology rep- resent two promising avenues. Another is the use of neuroimaging methods, such as functional magnetic resonance imaging and positron emission tomography. Reports have begun to appear about the use of these meth- ods to study brainstem nuclei, including the LC (e.g., Raizda & Poldrack 2003). Although the results of these studies require indepen- dent anatomic verification, methods for doing so are under development.
A key proposal that requires empirical con- firmation is that LC phasic activation reflects decision outcome. One way to test this hy-
pothesis is to analyze LC activity during a de- cision task simultaneously with activity of a brain region that has been strongly linked to decision processes in prior studies. Regions to consider in this regard include the lateral in- teroparietal cortex (LIP) and frontal eye fields (FEF), areas whose neurons meet many cri- teria for neural integrators of information in models of simple decision making (Gold & Shadlen 2001, Hanes & Schall 1996, Mazurek et al. 2003, Schall & Thompson 1999, Shadlen & Newsome 1996). A related component of our theory is that inputs from OFC and ACC drive LC phasic responses to decision out- come. This idea is suggested by the fact that decision-related cortical areas such as LIP and FEF do not strongly innervate the monkey LC (M. Iba, J. Rajkowski & G. Aston-Jones, unpublished observations) and that LC phasic responses are modulated by motivational sig- nificance. This idea can be tested by manip- ulating these prefrontal areas while recording LC neurons during decision tasks. The theory predicts, for example, that inactivation of ipsi- lateral OFC should diminish decision-driven LC phasic responses on positively motivated tasks.
The adaptive gain theory also states that phasic activation of LC facilitates behav- iors (e.g., speeds correct responses) associated with task-related decisions. This idea can be tested by direct phasic electrical stimulation of LC during decision task performance and by combining this with modeling work that predicts the outcome of stimulation at differ- ent times in the task. Conversely, blockade of phasic LC activation (e.g., via local infusions of glutamate antagonists) should decrease the speed of decision-driven behavioral responses and increase the probability that no response will be elicited.
Finally, a central prediction of the adaptive gain theory is that the tonic LC mode facil- itates transitions in behavioral focus so that tasks offering greater reward than the current one can be identified and pursued. Additional recordings of LC activity under conditions in which task-related utility is manipulated (such
www.annualreviews.org • LC and Optimization 437
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
as the diminishing utility task described ear- lier) will be important for testing and more precisely formulating this component of the theory. Moreover, manipulations of LC (e.g., tonic stimulation or inhibition of baseline ex- citatory inputs that drive the tonic mode), as well as manipulations of OFC and ACC that are proposed to regulate transitions between LC phasic and tonic modes, will be important for testing the causal role that these transitions play in behavioral changes.
2AFC: two-alternative forced choice
ACC: anterior cingulate cortex
DDM: drift diffusion model
LC: locus coeruleus
NE: norepinephrine
OFC: orbitofrontal cortex
REM: rapid eye movement
RT: response/ reaction time
PFC: prefrontal cortex
SUMMARY AND CONCLUSIONS
We have reviewed findings indicating that during the awake state the LC-NE system has two distinguishable modes of activity (phasic and tonic) and have described an adaptive gain theory of the LC-NE system, which proposes specific functions for each of these modes. In the phasic mode, a burst of LC activity, driven by the outcome of task-related decision processes, produces a widespread but tempo- rally specific release of NE, increasing gain of cortical processing units and facilitating task-appropriate behavior. In this mode, the event-locked nature of the LC phasic response acts as a temporal attentional filter, facilitat- ing task-relevant processes relative to distract- ing events, thereby augmenting performance of the current task. The LC phasic response serves to optimize the trade-off between the flexibility of a complex, multilayered system and the efficiency of a single-layered decision- making mechanism. The theory further pro- poses that computations regarding decision and utility in OFC and ACC drive these LC phasic responses. We also propose that util- ity computations in the OFC and ACC pro- duce the transitions between phasic and tonic modes in LC. Such transistions occur by the regulation of simple physiological variables within LC, such as electronic coupling, base- line drive, or both. High utility associated with performance of the current task favors the LC phasic mode. This mode is further augmented in response to momentary lapses in perfor- mance to exploit maximally the utility associ-
ated with the current task. However, persis- tent declines in utility drive a transition to the LC tonic mode. In the tonic mode, a lasting increase in baseline NE release augments re- sponsivity of target neurons to a broader class of events, while a concomitant attenuation of the LC phasic response degrades processing of events related to the current task. This in- discriminate release of NE promotes disen- gagement from the current behavioral routine while facilitating the sampling of others that may provide greater utility. These different LC modes serve to optimize the trade-off be- tween exploitation of stable sources of reward and exploration of other, potentially more remunerative, opportunities in a changing environment.
The role proposed here for the LC-NE system indicates that it would interact with many other brain circuits. In addition to its in- teraction with cortical systems, we have con- sidered what this theory implies about the relationship between the functions of the LC- NE and DA systems, which may have par- ticular relevance to psychiatric disorders. We propose that these systems work in synergy, the LC-NE system regulating the balance between exploitation and exploration—a fac- tor that is central to reinforcement learning mechanisms of the sort thought to be im- plemented by the DA system. Improved un- derstanding of these systems has substantial potential for understanding not only normal function, but also disturbances of function in a variety of clinical disorders. Several decades of research have made it clear that disturbances of NE and DA are involved in most of the ma- jor psychiatric illnesses, including schizophre- nia, depression, and anxiety disorders. Some disorders have been associated more closely with NE and others more closely with DA. However, until recently, research has focused primarily on relatively simple hypotheses con- cerning static excesses or deficits of activity in these systems and has given virtually no con- sideration to interactions between them. The simplicity of these hypotheses has reflected a general lack of knowledge about the more
438 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
complex dynamics that characterize the func- tioning of the NE and DA systems individu- ally as well as their interaction. A more sophis- ticated understanding of these dynamics, and their relationship to cognition and behavior, promises to open up new avenues of inquiry.
Realizing this potential, in turn, will afford greater understanding of how disruptions in the LC-NE and DA systems contribute to the complex patterns of behavior associated with psychiatric illness and how appropriate and effective interventions can be designed.
ACKNOWLEDGMENTS
The authors acknowledge the following individuals for the important contributions they have made to the development of the ideas and conduct of the research reviewed in this chapter: Rafal Bogacz, Eric Brown, Edwin Clayton, Mark Gilzenrat, Phil Holmes, Michiyo Iba, Sander Nieuwenhuis, Janusz Rajkowski, David Servan-Schreiber, and Marius Usher. The authors also express their profound appreciation to the National Institute of Mental Health for its enduring support of this work, which has included the following grants: P50 MH62196, R01 MH55309, and R01 MH58480. Work reported here was also supported by the Human Frontiers Science Program and the Air Force Office of Scientific Research. We also thank Mark Gilzenrat, Josh Gold, and Samuel McClure for valuable comments on the manuscript, and Stephanie Aston- Jones and Yan Zhu for assistance with illustrations.
LITERATURE CITED
Abercrombie ED, Keller RJ, Zigmond MJ. 1988. Characterization of hippocampal nore- pinephrine release as measured by microdialysis perfusion: pharmacological and behavioral studies. Neuroscience 27:897–904
Acheson AL, Zigmond MJ, Stricker EM. 1980. Compensatory increase in tyrosine hydrox- ylase activity in rat brain after intraventricular injections of 6-hydroxydopamine. Science 207:537–40
Aghajanian GK, Cedarbaum JM, Wang RY. 1977. Evidence for norepinephrine-mediated collateral inhibition of locus coeruleus neurons. Brain Res. 136:570–77
Alvarez VA, Chow CC, Van Bockstaele EJ, Williams JT. 2002. Frequency-dependent syn- chrony in locus coeruleus: role of electrotonic coupling. Proc. Natl. Acad. Sci. USA 99:4032– 36
Amaral DG, Foss JA. 1975. Locus coeruleus lesions and learning. Science 188:377–78 Amit DJ. 1989. Modeling Brain Function. Cambridge, UK: Cambridge Univ. Press Anderson JR. 1990. The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum Andrade R, Aghajanian GK. 1984. Intrinsic regulation of locus coeruleus neurons: electrophys-
iological evidence indicating a predominant role for autoinhibition. Brain Res. 310:401–6 Anlezark GM, Crow JT, Greenway AP. 1973. Impaired learning and decreased norepinephrine
after bilateral locus coeruleus lesions. Science 181:682–84 Archer T, Jonsson G, Ross SB. 1984. A parametric study of the effects of the noradrenaline
neurotoxin DSP4 on avoidance acquisition and noradrenaline neurones in the CNS of the rat. Br. J. Pharmacol. 82:249–57
Arnsten AF, Goldman RP. 1984. Selective prefrontal cortical projections to the region of the locus coeruleus and raphe nuclei in the rhesus monkey. Brain Res. 306:9–18
www.annualreviews.org • LC and Optimization 439
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Arnsten AF, Steere JC, Hunt RD. 1996. The contribution of alpha 2-noradrenergic mecha- nisms of prefrontal cortical cognitive function. Potential significance for attention-deficit hyperactivity disorder. Arch. Gen. Psychiatry 53:448–55
Aston-Jones G. 1985. Behavioral functions of locus coeruleus derived from cellular attributes. Physiol. Psych. 13:118–26
Aston-Jones G. 2004. Locus coeruleus, A5 and A7 noradrenergic cell groups. In The Rat Nervous System, ed. G Paxinos, pp. 259–94. San Diego: Elsevier Academic
Aston-Jones G, Akaoka H, Charlety P, Chouvet G. 1991a. Serotonin selectively attenuates glutamate-evoked activation of locus coeruleus neurons in vivo. J. Neurosci. 11:760–69
Aston-Jones G, Bloom FE. 1981a. Activity of norepinephrine-containing locus coeruleus neu- rons in behaving rats anticipates fluctuations in the sleep-waking cycle. J. Neurosci. 1:876– 86
Aston-Jones G, Bloom FE. 1981b. Norepinephrine-containing locus coeruleus neurons in behaving rats exhibit pronounced responses to non-noxious environmental stimuli. J. Neurosci. 1:887–900
Aston-Jones G, Chiang C, Alexinsky T. 1991b. Discharge of noradrenergic locus coeruleus neurons in behaving rats and monkeys suggests a role in vigilance. Prog. Brain Res. 88:501– 20
Aston-Jones G, Foote SL, Bloom FE. 1984. Anatomy and physiology of locus coeruleus neu- rons: functional implications. In Norepinephrine: Frontiers of Clinical Neuroscience, ed. M Ziegler, CR Lake, pp. 92–116. Baltimore: Williams and Wilkins. Vol. 2
Aston-Jones G, Foote SL, Segal M. 1985. Impulse conduction properties of noradrenergic locus coeruleus axons projecting to monkey cerebrocortex. Neuroscience 15:765–77
Aston-Jones G, Gonzalez MM, Doran SM. 2005. Role of the locus coeruleus-norepinephrine system in arousal and circadian regulation of the sleep-waking cycle. In Norepinephrine: Neurobiology and Therapeutics for the 21st Century, ed. GA Ordway, M Schwartz, A Frazer. Cambridge UK: Cambridge Univ. Press. In press
Aston-Jones G, Rajkowski J, Cohen J. 1999. Role of locus coeruleus in attention and behavioral flexibility. Biol. Psychiatry 46:1309–20
Aston-Jones G, Rajkowski J, Kubiak P. 1997. Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80:697–715
Aston-Jones G, Rajkowski J, Kubiak P, Alexinsky T. 1994. Locus coeruleus neurons in the mon- key are selectively activated by attended stimuli in a vigilance task. J. Neurosci. 14:4467–80
Aston-Jones G, Rajkowski J, Kubiak P, Ivanova S, Usher M, Cohen J. 1998. Neuromodulation and cognitive performance: recent studies of noradrenergic locus coeruleus neurons in behaving monkeys. In Catecholamines: Bridging Basic Science with Clinical Medicine, ed. D Goldstein, G Eisenhofer, R McCarty, pp. 755–59. New York: Academic
Aston-Jones G, Rajkowski J, Kubiak P, Valentino R, Shipley M. 1996. Role of the locus coeruleus in emotional activation. Prog. Brain Res. 107:379–402
Aston-Jones G, Rajkowski J, Lu W, Zhu Y, Cohen JD, Morecraft RJ. 2002. Prominent pro- jections from the orbital prefrontal cortex to the locus coeruleus in monkey. Soc. Neurosci. Abstr. 28:86–9
Barch DM, Braver TS, Nystrom LE, Forman SD, Noll DC, Cohen JD. 1997. Dissociat- ing working memory from task difficulty in human prefrontal cortex. Neuropsychologia 35:1373–80
Barlow HB. 1981. Critical limiting factors in the design of the eye and visual cortex. Proc. R. Soc. London B 212:1–34
440 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Barnard GA. 1946. Sequential tests in industrial statistics. J. R. Stat. Soc. Suppl. 8:1–26 Baylis LL, Rolls ET, Baylis GC. 1995. Afferent connections of the caudolateral orbitofrontal
cortex taste area of the primate. Neuroscience 64:801–12 Beatty J. 1982a. Phasic not tonic pupillary responses vary with auditory vigilance performance.
Psychophysiology 19:167–72 Beatty J. 1982b. Task-evoked pupillary responses, processing load, and structure of processing
resources. Psychol. Bull. 91:276–92 Beaudet A, Descarries L. 1978. The monoamine innervation of rat cerebral cortex: synaptic
and nonsynaptic axons terminals. Neuroscience 3:851–60 Berridge CW, Waterhouse BD. 2003. The locus coeruleus-noradrenergic system: modulation
of behavioral state and state-dependent cognitive processes. Brain Res. Brain Res. Rev. 42:33–84
Bialek W, Owen WG. 1990. Temporal filtering in retinal bipolar cells: elements of an optimal computation? Biophys. J. 58:1227–33
Bohil CJ, Maddox WT. 2003. On the generality of optimal versus objective classifer feedback effects on decision criterion learning in perceptual categorization. Mem. Cognit. 31(2):181– 98
Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. 2001. Conflict monitoring and cognitive control. Psychol. Rev. 108(3):624–52
Botvinick MM, Cohen JD, Carter CS. 2004. Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn. Sci. 8(12):539–46
Botvinick MM, Nystrom LE, Fissell K, Carter CS, Cohen JD. 1999. Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402:179–81
Bouret S, Sara SJ. 2004. Reward expectation, orientation of attention and locus coeruleus- medial frontal cortex interplay during learning. Eur. J. Neurosci. 20:791–802
Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. 2001. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619–39
Breiter HC, Gollub RL, Weisskoff RM, Kennedy DN, Makris N, et al. 1997. Acute effects of cocaine on human brain activity and emotion. Neuron 19(3):591–611
Brown E, Moehlis J, Holmes P, Clayton E, Rajkowski J, Aston-Jones G. 2004. The influence of spike rate and stimulus duration on noradrenergic neurons. J. Comput. Neurosci. 17:5–21
Brun P, Suaud-Chagny MF, Gonon F, Buda M. 1993. In vivo noradrenaline release evoked in the anteroventral thalamic nucleus by locus coeruleus activation: an electrochemical study. Neuroscience 52:961–72
Burnett DM, Bowyer JF, Masserano JM, Zahniser NR. 1990. Effect of aging on alpha-1 adrenergic stimulation of phosphoinositide hydrolysis in various regions of rat brain. J. Pharmacol. Exp. Ther. 255:1265–70
Busemeyer JR, Myung IJ. 1992. An adaptive approach to human decision making: learning theory, decision theory, and human performance. J. Exp. Psychol. Gen. 121(2):177–94
Carboni E, Silvagni A. 2004. Dopamine reuptake by norepinephrine neurons: exception or rule? Crit. Rev. Neurobiol. 16:121–28
Carli M, Robbins TW, Evenden JL, Everitt BJ. 1983. Effects of lesions to ascending noradren- ergic neurones on performance of a 5-choice serial reaction task in rats; implications for theories of dorsal noradrenergic bundle function based on selective attention and arousal. Behav. Brain Res. 9:361–80
Carmichael ST, Clugnet MC, Price JL. 1994. Central olfactory connections in the macaque monkey. J. Comp. Neurol. 346:403–34
Carmichael ST, Price JL. 1995a. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol. 363:615–41
www.annualreviews.org • LC and Optimization 441
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Carmichael ST, Price JL. 1995b. Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 363:642–64
Carter CS, Braver TS, Barch DM, Botvinick MM, Noll DC, Cohen JD. 1998. Anterior cingu- late cortex, error detection and the on-line monitoring of performance. Science 280:747–49
Chiodo LA, Acheson AL, Zigmond MJ, Stricker EM. 1983. Subtotal destruction of central noradrenergic projections increases the firing rate of locus coeruleus cells. Brain Res. 264:123–26
Christie MJ, Jelinek HF. 1993. Dye-coupling among neurons of the rat locus coeruleus during postnatal development. Neuroscience 56:129–37
Christie MJ, Williams JT, North RA. 1989. Electrical coupling synchronizes subthreshold activity in locus coeruleus neurons in vitro from neonatal rats. J. Neurosci. 9:3584–89
Cirelli C, Tononi G. 2004. Locus coeruleus control of state-dependent gene expression. J. Neurosci. 24:5410–19
Clayton EC, Rajkowski J, Cohen JD, Aston-Jones G. 2004. Phasic activation of monkey locus coeruleus neurons by simple decisions in a forced choice task. J. Neurosci. 24:9914–20
Cohen JD, Aston-Jones G, Gilzenrat MS. 2004. A systems-level perspective on attention and cognitive control: guided activation, adaptive gating, conflict monitoring, and exploitation vs. exploration. In Cognitive Neuroscience of Attention, ed. MI Posner, pp. 71–90. New York: Guilford
Cohen JD, Dunbar K, McClelland JL. 1990. On the control of automatic processes: a parallel distributed processing model of the Stroop effect. Psychol. Rev. 97(3):332–61
Cohen JD, Servan-Schreiber D, McClelland JL. 1992. A parallel distributed processing ap- proach to automaticity. Am. J. Psychol. 105:239–69
Cole BJ, Robbins TW. 1992. Forebrain norepinephrine: role in controlled information pro- cessing in the rat. Neuropsychopharmacology 7:129–41
Coull JT. 1994. Pharmacological manipulations of the alpha 2-noradrenergic system. Effects on cognition. Drugs Aging 5:116–26
Coull JT, Nobre AC. 1998. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci. 18:7426–35
Coull JT, Nobre AC, Frith CD. 2001. The noradrenergic alpha2 agonist clonidine modulates behavioural and neuroanatomical correlates of human attentional orienting and alerting. Cereb. Cortex 11:73–84
Critchley HD, Rolls ET. 1996. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J. Neurophysiol. 75:1673–86
Dahlstrom A, Fuxe K. 1964. Evidence for the existence of monoamine-containing neurons in the central nervous system. I. Demonstration of monoamines in the cell bodies of brain stem neurons. Acta Physiol. Scand. 62:5–55
Dalley JW, McGaughy J, O’Connell MT, Cardinal RN, Levita L, Robbins TW. 2001. Dis- tinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task. J. Neurosci. 21:4908–14
Desmedt JE, Debecker J. 1979. Wave form and neural mechanism of the decision P350 elicited without pre-stimulus CNV or readiness potential in random sequences of near-threshold auditory clicks and finger stimuli. Electroencephalogr. Clin. Neurophysiol. 47:648–70
Devilbiss DM, Waterhouse BD. 2000. Norepinephrine exhibits two distinct profiles of action on sensory cortical neuron responses to excitatory synaptic stimuli. Synapse 37:273–82
Devilbiss DM, Waterhouse BD. 2004. The effects of tonic locus coeruleus output on sensory- evoked responses of ventral posterior medial thalamic and barrel field cortical neurons in the awake rat. J. Neurosci. 24:10773–85
442 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Devinsky O, Morrell MJ, Vogt BA. 1995. Contributions of anterior cingulate cortex to be- haviour. Brain 118:279–306
Dooley DJ, Jones GH, Robbins TW. 1987. Noradrenaline- and time-dependent changes in neocortical alpha 2- and beta 1-adrenoceptor binding in dorsal noradrenergic bundle- lesioned rats. Brain Res. 420:152–56
Duncan CC. 2003. Brain potentials in normal and disordered attention: findings in search of a theory. Presidential address presented at the Annu. Meet. Soc. Psychophysiol. Res., Chicago, IL
Duncan J. 1986. Disorganisation of behaviour after frontal lobe damage. Cogn. Neuropsychol. 3:271–90
Duncan J, Owen AM. 2000. Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends Neurosci. 23:475–83
Egan TM, Henderson G, North RA, Williams JT. 1983. Noradrenaline-mediated synaptic inhibition in rat locus coeruleus neurones. J. Physiol. (London) 345:477–88
Eisenberger NI, Lieberman MD, Williams KD. 2003. Does rejection hurt? An FMRI study of social exclusion. Science 302:290–92
Ennis M, Aston-Jones G. 1986. Evidence for self- and neighbor-mediated postactivation inhi- bition of locus coeruleus neurons. Brain Res. 374:299–305
Eriksdotter Jonhagen M, Hoffer B, Luthman J. 1995. Alterations in alpha-adrenoceptors in aging intraocular hippocampal grafts. Neurobiol. Aging 16:633–38
Falkenstein M, Hohnsbein J, Hoorman J, Blanke L. 1991. Effects of crossmodal divided at- tention on late ERP components: II. Error processing in choice reaction tasks. Electroen- cephalogr. Clin. Neurophysiol. 78:447–55
Foote SL, Aston-Jones G, Bloom FE. 1980. Impulse activity of locus coeruleus neurons in awake rats and monkeys is a function of sensory stimulation and arousal. Proc. Natl. Acad. Sci. USA 77:3033–37
Foote SL, Bloom FE, Aston-Jones G. 1983. Nucleus locus coeruelus: new evidence of anatom- ical and physiological specificity. Physiol. Rev. 63:844–914
Foote SL, Freedman R, Oliver AP. 1975. Effects of putative neurotransmitters on neuronal activity in monkey auditory cortex. Brain Res. 86:229–42
Foote SL, Morrison JH. 1987. Extrathalamic modulation of cortical function. Annu. Rev. Neurosci. 10:67–95
Fritschy JM, Grzanna R. 1992. Restoration of ascending noradrenergic projections by residual locus coeruleus neurons: compensatory response to neurotoxin-induced cell death in the adult rat brain. J. Comp. Neurol. 321:421–41
Gallistel CR, Mark TA, King AP, Latham PE. 2001. The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. J. Exp. Psychol. Anim. Behav. 27:354–72
Gehring WJ, Goss B, Coles MGH, Meyer DE, Donchin E. 1993. A neural system for error detection and compensation. Psychol. Sci. 4:385–90
Gilzenrat MS, Cohen JD, Rajkowski J, Aston-Jones G. 2003. Pupil dynamics predict changes in task engagement mediated by locus coeruleus. Soc. Neurosci. Abstr. No. 515.19
Gilzenrat MS, Brown ET, Aston-Jones G, Cohen JD. 2004. Locus coeruleus, adaptive gain, and the optimization of decision tasks. Soc. Neurosci. Abstr. No. 899.6
Gold JI, Shadlen MN. 2000. Representation of a perceptual decision in developing oculomotor commands. Nature 404:390–94
Gold JI, Shadlen MN. 2001. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5:10–16
www.annualreviews.org • LC and Optimization 443
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Gold JI, Shadlen MN. 2002. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36:299–308
Grant SJ, Aston-Jones G, Redmond DEJ. 1988. Responses of primate locus coeruleus neurons to simple and complex sensory stimuli. Brain Res. Bull. 21:401–10
Gratton G, Coles MGH, Donchin E. 1992. Optimizing the use of information: strategic control of activation and responses. J. Exp. Psychol. Gen. 4:480–506
Hanes DP, Schall JD. 1996. Neural control of voluntary movement initiation. Science 18:427– 30
Harik SI, Duckrow RB, LaManna JC, Rosenthal M, Sharma VK, Banerjee SP. 1981. Cerebral compensation for chronic noradrenergic denervation induced by locus ceruleus lesion: re- covery of receptor binding, isoproterenol-induced adenylate cyclase activity, and oxidative metabolism. J. Neurosci. 1:641–49
Haring JH, Miller GD, Davis JN. 1986. Changes in the noradrenergic innervation of the area dentata after axotomy of coeruleohippocampal projections or unilateral lesion of the locus coeruleus. Brain Res. 368:233–38
Harley C. 1991. Noradrenergic and locus coeruleus modulation of the perforant path-evoked potential in rat dentate gyrus supports a role for the locus coeruleus in attentional and memorial processes. Prog. Brain Res. 88:307–21
Harris GC, Fitzgerald RD. 1991. Locus coeruleus involvement in the learning of classically conditioned bradycardia. J. Neurosci. 11:2314–20
Herrnstein RJ. 1961. Stereotypy and intermittent reinforcement. Science 133:2067–69 Herrnstein RJ. 1997. The Matching Law, ed. H Rachlin, DI Laibson. Cambridge, MA: Harvard
Univ. Press Herve-Minvielle A, Sara SJ. 1995. Rapid habituation of auditory responses of locus coeruleus
cells in anaesthetized and awake rats. Neuroreport 6:1363–68 Hobson JA, McCarley RW, Wyzinski PW. 1975. Sleep cycle oscillation: reciprocal discharge
by two brainstem neuronal groups. Science 189:55–58 Hollerman JR, Tremblay L, Schultz W. 2000. Involvement of basal ganglia and orbitofrontal
cortex in goal-directed behavior. Prog. Brain Res. 126:193–215 Holroyd CB, Larsen JT, Cohen JD. 2004a. Context dependence of the event-related brain
potential associated with reward and punishment. Psychophysiology 41(2):245–53 Holroyd CB, Nieuwenhuis S, Yeung N, Cohen JD. 2003. Errors in reward prediction are
reflected in the event-related brain potential. Neuroreport 14:2481–84 Holroyd CB, Nieuwenhuis S, Yeung N, Nystrom L, Mars R, et al. 2004b. Dorsal anterior cin-
gulate cortex is sensitive to internal and external sources of error information: a functional magnetic resonance imaging study. Nat. Neurosci. 7:497–98
Hurley LM, Devilbiss DM, Waterhouse BD. 2004. A matter of focus: monoaminergic modula- tion of stimulus coding in mammalian sensory networks. Curr. Opin. Neurobiol. 14:488–95
Ishimatsu M, Williams JT. 1996. Synchronous activity in locus coeruleus results from dendritic interactions in pericoerulear regions. J. Neurosci. 16:5196–204
Ito S, Stuphorn V, Brown JW, Schall JD. 2003. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302:120–22
Ivanova S, Rajkowski J, Silakov V, Watanabe T, Aston-Jones G. 1997. Local chemomanipu- lations of locus coeruleus (LC) activity in monkeys alter cortical event-related potentials (ERPs) and task performance. Soc. Neurosci. Abstr. 23:1587
Iversen LL, Rossor MN, Reynolds GP, Hills R, Roth M, et al. 1983. Loss of pigmented dopamine-beta-hydroxylase positive cells from locus coeruleus in senile dementia of Alzheimer’s type. Neurosci. Lett. 39:95–100
444 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Jodo E, Aston-Jones G. 1997. Activation of locus coeruleus by prefrontal cortex is mediated by excitatory amino acid inputs. Brain Res. 768:327–32
Jodo E, Chiang C, Aston-Jones G. 1998. Potent excitatory influence of prefrontal cortex activity on noradrenergic locus coeruleus neurons. Neuroscience 83:63–80
Jones AD, Cho RY, Nystrom LE, Cohen JD, Braver TS. 2002. A computational model of anterior cingulate function in speeded response tasks: effects of frequency, sequence, and conflict. Cogn. Affect. Behav. Neurosci. 2:300–17
Jones BE. 1991. The role of noradrenergic locus coeruleus neurons and neighboring choliner- gic neurons of the pontomesencephalic tegmentum in sleep-wake states. Prog. Brain Res. 88:533–43
Jouvet M. 1969. Biogenic amines and the states of sleep. Science 163:32–41 Kaelbling LP, Littman ML, Moore AW. 1996. Reinforcement learning: a survey. J. Artif. Intell.
Res. 4:237–85 Kahneman D, Tversky A. 1984. Choices, values, and frames. Am. Psychol. 39:341–50 Kiehl KA, Liddle PF, Hopfinger JB. 2000. Error processing and the rostral anterior cingulate:
an event-related fMRI study. Psychophysiology 37:216–23 Koob GF, Kelley AE, Mason ST. 1978. Locus coeruleus lesions: learning and extinction.
Physiol. Behav. 20:709–16 Knutson B, Westdorp A, Kaiser E, Hommer D. 2000. FMRI visualization of brain activity
during a monetary incentive delay task. Neuroimage 12(1):20–27 Kubiak P, Rajkowski J, Aston-Jones G. 1992. Behavioral performance and sensory responsive-
ness of LC neurons in a vigilance task varies with tonic LC discharge rate. Soc. Neurosci. Abstr. 18:538
Laming DRJ. 1968. Information Theory of Choice-Reaction Times. New York: Academic Laming DRJ. 1979. Choice reaction performance following an error. Acta-Psychologica 43:199–
224 Levin BE, Battisti WP, Murray M. 1985. Axonal transport of beta-receptors during the response
to axonal injury and repair in locus coeruleus neurons. Brain Res. 359:215–23 Loewenstein G, Thaler R. 1989. Anomalies: intertemporal choice. J. Econ. Perspect. 3:181–93 Martin P, Ohno M, Southerland SB, Mailman RB, Suzuki K. 1994. Heterotypic sprouting of
serotonergic forebrain fibers in the brindled mottled mutant mouse. Brain Res. Dev. Brain Res. 77:215–25
Mazurek ME, Roitman JD, Ditterich J, Shadlen MN. 2003. A role for neural integrators in perceptual decision making. Cereb. Cortex 13:1257–69
McClure SM, Laibson DI, Loewenstein G, Cohen JD. 2004. Separate neural systems value immediate and delayed monetary rewards. Science 306:503–7
McQuade R, Creton D, Stanford SC. 1999. Effect of novel environmental stimuli on rat be- haviour and central noradrenaline function measured by in vivo microdialysis. Psychophar- macology (Berlin) 145:393–400
Mesulam M-M. 1981. A cortical network for directed attention and unilateral neglect. Ann. Neurol. 10:309–25
Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24:167–202
Miltner WHR, Braun CH, Coles MGH. 1997. Event-related potentials following incorrect feedback in a time-estimation task: evidence for a ‘generic’ neural system for error detec- tion. J. Cogn. Neurosci. 9(6):788–98
Montague PR, Dayan P, Sejnowski TJ. 1996. A framework for mesencephalic dopamine sys- tems based on predictive Hebbian learning. J. Neurosci. 16:1936–47
www.annualreviews.org • LC and Optimization 445
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Montague PR, Hyman SE, Cohen JD. 2004. Computational roles for dopamine in behavioural control. Nature 431(7010):760–67
Moore RY, Bloom FE. 1979. Central catecholamine neuron systems: anatomy and physiology of the norepinephrine and epinephrine systems. Annu. Rev. Neurosci. 2:113–68
Morecraft RJ, Geula C, Mesulam MM. 1992. Cytoarchitecture and neural afferents of or- bitofrontal cortex in the brain of the monkey. J. Comp. Neurol. 323:341–58
Morrison JH, Foote SL, O’Connor D, Bloom FE. 1982. Laminar, tangential and regional organization of the noradrenergic innervation of the monkey cortex: dopamine-beta- hydroxylase immunohistochemistry. Brain Res. Bull. 9:309–19
Mountcastle VB, LaMotte RH, Carli G. 1972. Detection thresholds for stimuli in humans and monkeys: comparison with threshold events in mechanoreceptive afferent nerve fibers innervating the monkey hand. J. Neurophysiol. 35:122–36
Mozer MC, Colagrosso MD, Huber DE. 2002. A rational analysis of cognitive control in a speeded discrimination task. In Advances in Neural Information Processing Systems, ed. T Dietterich, S Becker, Z Ghahramani, pp. 51–57. Cambridge, MA: MIT Press. Vol. XI
Nicola SM, Surmeier J, Malenka RC. 2000. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23:185–215
Niewenhuis S, Aston-Jones G, Cohen JD. 2005. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psych. Bull. In press
O’Doherty JP, Deichmann R, Critchley HD, Dolan RJ. 2002. Neural responses during antic- ipation of a primary taste reward. Neuron 33(5):815–26
Oke AF, Adams RN. 1978. Selective attention dysfunctions in adult rats neonatally treated with 6-hydoxydopamine. Pharmacol. Biochem. Behav. 9:429–32
Olschowka JA, O’ Donohue T, Jacobowitz DM. 1981. The distribution of bovine pancreatic polypeptide-like immunoreactive neurons in rat brain. Peptides 2:309–31
Ongur D, Price JL. 2000. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex 10:206–19
Papadopoulos GC, Parnavelas JG, Buijs R. 1987. Monoaminergic fibers form conventional synapses in the cerebral cortex. Neurosci. Lett. 76:275–79
Papadopoulos GC, Parnavelas JG, Buijs RM. 1989. Light and electron microscopic immunocy- tochemical analysis of the noradrenaline innervation of the rat visual cortex. J. Neurocytol. 18:1–10
Paus T, Koski L, Caramanos Z, Westbury C. 1998. Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: a review of 107 PET activation studies. Neuroreport 9(9):R37–47
Paxinos G, Huang X-F, Toga AW. 2000. The Rhesus Monkey Brain in Stereotaxic Coordinates. San Diego: Academic
Peyron R, Laurent B, Garcia-Larrea L. 2000. Functional imaging of brain responses to pain. A review and meta-analysis. Neurophysiol. Clin.–Clin. Neurophysiol. 30:263–88
Pineda JA, Foote SL, Neville HJ. 1989. Effects of locus coeruleus lesions on auditory, long- latency, event-related potentials in monkey. J. Neurosci. 9:81–93
Pisa M, Fibiger HC. 1983. Evidence against a role of the rat’s dorsal noradrenergic bundle in selective attention and place memory. Brain Res. 272:319–29
Pisa M, Martin-Iverson MT, Fibiger HC. 1988. On the role of the dorsal noradrenergic bundle in learning and habituation to novelty. Pharmacol. Biochem. Behav. 30:835–45
Rabbitt PMA. 1966. Errors and error-correction in choice-response tasks. J. Exp. Psychol. 71:264–72
Raizada RD, Poldrack RA. 2003. Difficult, unpredictable trials coactivate noradrenergic and frontal attentional systems. Soc. Neurosci. Abstr. 401:4
446 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Rajkowski J, Kubiak P, Aston-Jones G. 1993. Correlations between locus coeruleus (LC) neural activity, pupil diameter and behavior in monkey support a role of LC in attention. Soc. Neurosc. Abstr. 19:974
Rajkowski J, Kubiak P, Ivanova S, Aston-Jones G. 1998. State related activity and reactivity of locus coeruleus neurons in behaving monkeys. In Catecholamines: Bridging Basic Science with Clinical Medicine, ed. D Goldstein, G Eisenhofer, R McCarty, pp. 740–44. New York: Academic
Rajkowski J, Lu W, Zhu Y, Cohen J, Aston-Jones G. 2000. Prominent projections from the anterior cingulate cortex to the locus coeruleus in Rhesus monkey. Soc. Neurosci. Abstr. 26:838–15
Rajkowski J, Majczynski H, Clayton E, Aston-Jones G. 2004. Activation of monkey locus coeruleus neurons varies with difficulty and behavioral performance in a target detection task. J. Neurophysiol. 92:361–71
Rasmussen K, Morilak DA, Jacobs BL. 1986. Single unit activity of locus coeruleus neurons in the freely moving cat. I. During naturalistic behaviors and in response to simple and complex stimuli. Brain Res. 371:324–34
Ratcliff R. 1978. A theory of memory retrieval. Psychol. Rev. 85:59–108 Ratcliff R, Cherian A, Segraves MJ. 2003. A comparison of macaque behavior and superior
colliculus neuronal activity to predictions from models of two choice decisions. J. Neuro- physiol. 90:1392–407
Ratcliff R. Smith PL. 2004. A comparison of sequential sampling models for two-choice reac- tion time. Psychol. Rev. 111:333–67
Raymond JE, Shapiro KL, Arnell KM. 1992. Temporary suppression of visual processing in an RSVP task: an attentional blink? J. Exp. Psychol. Hum. Percept. Perform. 18:849–60
Reddi B. 2001. Decision making: two stages of neural judgement. Curr. Biol. 11(15):R603–6 Richer F, Beatty J. 1987. Contrasting effects of response uncertainty on the task-evoked pupil-
lary response and reaction time. Psychophysiology 24:258–62 Robbins TW, Everitt BJ. 1995. Central norepinephrine neurons and behavior. In Psychophar-
macology: The Fourth Generation of Progress, ed. DJ Kupfer, FE Bloom, pp. 363–72. New York: Raven
Roberts DCS, Price MTC, Fibiger HC. 1976. The dorsal tegmental noradrenergic bundle: an analysis of its role in maze learning. J. Comp. Physiol. Psychol. 90:363–72
Robertson IH, Mattingley JB, Rorden C, Driver J. 1998. Phasic alerting of neglect patients overcomes their spatial deficit in visual awareness. Nature 395:169–72
Robinson TE, Berridge KC. 1993. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18:247–91
Roesch MR, Olson CR. 2004. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304:307–10
Rogawski MA, Aghajanian GK. 1982. Activation of lateral geniculate neurons by locus coeruleus or dorsal noradrenergic bundle stimulation: selective blockade by the alpha 1-adrenoceptor antagonist prazosin. Brain Res. 250:31–39
Rolls ET. 2004. The functions of the orbitofrontal cortex. Brain Cogn. 55:11–29 Rolls ET, Critchley HD, Mason R, Wakeman EA. 1996. Orbitofrontal cortex neurons: role in
olfactory and visual association learning. J. Neurophysiol. 75:1970–81 Rolls ET, Sienkiewicz ZJ, Yaxley S. 1989. Hunger modulates the responses to gustatory stimuli
of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. Eur. J. Neurosci. 1:53–60
Rolls ET, Yaxley S, Sienkiewicz ZJ. 1990. Gustatory responses of single neurons in the cau- dolateral orbitofrontal cortex of the macaque monkey. J. Neurophysiol. 64:1055–66
www.annualreviews.org • LC and Optimization 447
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Saper CB, Chou TC, Scammell TE. 2001. The sleep switch: hypothalamic control of sleep and wakefulness. Trends Neurosci. 24:726–31
Sara SJ, Devauges V. 1989. Idazoxan, an alpha-2 antagonist, facilitates memory retrieval in the rat. Behav. Neural Biol. 51:401–11
Sara SJ, Dyon-Laurent C, Herve A. 1995. Novelty seeking behavior in the rat is dependent upon the integrity of the noradrenergic system. Brain Res. Cogn. Brain Res. 2:181–87
Sara SJ, Herve-Minvielle A. 1995. Inhibitory influence of frontal cortex on locus coeruleus neurons. Proc. Natl. Acad. Sci. USA 92:6032–36
Schall J. 2003. Neural correlates of decision processes: neural and mental chronometry. Curr. Opin. Neurobiol. 13(2):182–86
Schall JD, Thompson KG. 1999. Neural selection and control of visually guided eye move- ments. Annu. Rev. Neurosci. 22:241–59
Schultz W, Tremblay L, Hollerman JR. 2000. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex 10:272–84
Segal M, Bloom FE. 1976. The action of norepinephrine in the rat hippocampus. IV. The effects of locus coeruleus stimulation on evoked hippocampal unit activity. Brain Res. 107:513–25
Seguela P, Watkins KC, Geffard M, Descarries L. 1990. Noradrenaline axon terminals in adult rat neocortex: an immunocytochemical analysis in serial thin sections. Neuroscience 35:249–64
Selden NR, Cole BJ, Everitt BJ, Robbins TW. 1990a. Damage to coeruleo-cortical noradren- ergic projections impairs locally cued but enhances spatially cued water maze acquisition. Behav. Brain Res. 39:29–51
Selden NR, Robbins TW, Everitt BJ. 1990b. Enhanced behavioral conditioning to context and impaired behavioral and neuroendocrine responses to conditioned stimuli following coeruleocortical noradrenergic lesions: support for an attentional hypothesis of central noradrenergic function. J. Neurosci. 10:531–39
Servan-Schreiber D, Printz H, Cohen JD. 1990. A network model of catecholamine effects gain signal to noise ratio and behavior. Science 249:892–95
Shadlen MN, Newsome WT. 1996. Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. USA 93:628–33
Shadlen MN, Newsome WT. 2001. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86:1916–36
Shallice T. 1988. From Neuropsychology to Mental Structure. Cambridge, UK: Cambridge Univ. Press
Small DM, Zatorre RJ, Dagher A, Evans AC, Jones-Gotman M. 2001. Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124(Pt. 9):1720–33
Steriade M, McCormick DA, Sejnowski TJ. 1993. Thalamocortical oscillations in the sleeping and aroused brain. Science 262:679–85
Stone M. 1960. Models for choice reaction time. Psychometrika 25:251–60 Sugrue LP, Corrado GC, Newsome WT. 2004. Neural correlates of value in orbitofrontal
cortex of the rhesus monkey. Soc. Neurosci. Abstr. 30:671–78 Sutton RS, Barto AG. 1981. Toward a modern theory of adaptive networks: expectation and
prediction. Psychol. Rev. 88:135–70 Tejani-Butt SM, Ordway GA. 1992. Effect of age on [3H]nisoxetine binding to uptake sites for
norepinephrine in the locus coeruleus of humans. Brain Res. 583:312–15 Thorpe SJ, Rolls ET, Maddison S. 1983. The orbitofrontal cortex: neuronal activity in the
behaving monkey. Exp. Brain Res. 49:93–115
448 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Thut G, Schultz W, Roelcke U, Nienhusmeier M, Missimer J, et al. 1997. Activation of the human brain by monetary reward. Neuroreport 8:1225–28
Travagli RA, Dunwiddie TV, Williams JT. 1995. Opioid inhibition in locus coeruleus. J. Neurophysiol. 74:519–28
Travagli RA, Wessendorf M, Williams JT. 1996. Dendritic arbor of locus coeruleus neurons contributes to opioid inhibition. J. Neurophysiol. 75:2029–35
Tremblay L, Schultz W. 1999. Relative reward preference in primate orbitofrontal cortex. Nature 398:704–8
Ullsperger M, von Cramon DY. 2001. Subprocesses of performance monitoring: a dissociation of error processing and response competition revealed by event-related fMRI and ERPs. NeuroImage 14:1387–401
U’Prichard DC, Reisine TD, Mason ST, Fibiger HC, Yamamura HI. 1980. Modulation of rat brain alpha- and beta-adrenergic receptor populations by lesion of the dorsal noradren- ergic bundle. Brain Res. 187:143–54
Usher M, Cohen JD, Servan-Schreiber D, Rajkowski J, Aston-Jones G. 1999. The role of locus coeruleus in the regulation of cognitive performance. Science 283:549–54
Usher M, McClelland JL. 2001. On the time course of perceptual choice: the leaky competing accumulator model. Psychol. Rev. 108:550–92
Valentini V, Frau R, Di Chiara G. 2004. Noradrenaline transporter blockers raise extracellu- lar dopamine in medial prefrontal but not parietal and occipital cortex: differences with mianserin and clozapine. J. Neurochem. 88:917–27
Velley L, Cardo B, Kempf E, Mormede P, Nassif-Caudarella S, Velly J. 1991. Facilitation of learning consecutive to electrical stimulation of the locus coeruleus: cognitive alteration or stress-reduction? Prog. Brain Res. 88:555–69
Wald A. 1947. Sequential Analysis. New York: Wiley Wallis JD, Miller EK. 2003. Neuronal activity in primate dorsolateral and orbital prefrontal
cortex during performance of a reward preference task. Eur. J. Neurosci. 18(7):2069–81 Waterhouse BD, Moises HC, Woodward DJ. 1980. Noradrenergic modulation of somatosen-
sory cortical neuronal responses to iontophoretically applied putative neurotransmitters. Exp. Neurol. 69:30–49
Waterhouse BD, Moises HC, Woodward DJ. 1998. Phasic activation of the locus coeruleus enhances responses of primary sensory cortical neurons to peripheral receptive field stim- ulation. Brain Res. 790:33–44
Waterhouse BD, Moises HC, Yeh HH, Geller HM, Woodward DJ. 1984. Comparison of norepinephrine- and benzodiazepine-induced augmentation of Purkinje cell responses to gamma-aminobutyric acid (GABA). J. Pharmacol. Exp. Ther. 228:257–67
Waterhouse BD, Woodward DJ. 1980. Interaction of norepinephrine with cerebrocortical activity evoked by stimulation of somatosensory afferent pathways in the rat. Exp. Neurol. 67:11–34
Williams JT, Henderson G, North RA. 1985. Characterization of alpha 2-adrenoceptors which increase potassium conductance in rat locus coeruleus neurones. Neuroscience 14:95–101
Williams JT, North RA, Shefner SA, Nishi S, Egan TM. 1984. Membrane properties of rat locus coeruleus neurones. Neuroscience 13:137–56
Wise RA, Rompre PP. 1989. Brain dopamine and reward. Annu. Rev. Psychol. 40:191–225 Yeung N, Botvinick MM, Cohen JD. 2004. The neural basis of error detection: conflict mon-
itoring and the error-related negativity. Psychol. Rev. 111(4):931–59 Yeung N, Holroyd CB, Cohen JD. 2005. ERP correlates of feedback and reward processing in
the presence and absence of response choice. Cereb. Cortex. In press
www.annualreviews.org • LC and Optimization 449
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
AR245-NE28-16 ARI 19 May 2005 12:10
Yeung NP, Sanfey AG. 2004. Independent coding of reward magnitude and valence in the human brain. J. Neurosci. 24:6258–64
Zhu Y, Iba M, Rajkowski J, Aston-Jones G. 2004. Projection from the orbitofrontal cortex to the locus coeruleus in monkeys revealed by anterograde tracing. Soc. Neurosci. Abstr. 30:211.3
450 Aston-Jones · Cohen
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
Contents ARI 13 May 2005 13:21
Annual Review of Neuroscience
Volume 28, 2005
Contents
Genetics of Brain Structure and Intelligence Arthur W. Toga and Paul M. Thompson � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 1
The Actin Cytoskeleton: Integrating Form and Function at the Synapse Christian Dillon and Yukiko Goda � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �25
Molecular Pathophysiology of Parkinson’s Disease Darren J. Moore, Andrew B. West, Valina L. Dawson, and Ted M. Dawson � � � � � � � � � � � � �57
Large-Scale Genomic Approaches to Brain Development and Circuitry Mary E. Hatten and Nathaniel Heintz � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �89
Autism: A Window Onto the Development of the Social and the Analytic Brain Simon Baron-Cohen and Matthew K. Belmonte � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 109
Axon Retraction and Degeneration in Development and Disease Liqun Luo and Dennis D.M. O’Leary � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 127
Structure and Function of Visual Area MT Richard T. Born and David C. Bradley � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 157
Growth and Survival Signals Controlling Sympathetic Nervous System Development Natalia O. Glebova and David D. Ginty � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 191
Adult Neurogenesis in the Mammalian Central Nervous System Guo-li Ming and Hongjun Song � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 223
Mechanisms of Vertebrate Synaptogenesis Clarissa L. Waites, Ann Marie Craig, and Craig C. Garner � � � � � � � � � � � � � � � � � � � � � � � � � � � � 251
Olfactory Memory Formation in Drosophila: From Molecular to Systems Neuroscience Ronald L. Davis � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 275
The Circuitry of V1 and V2: Integration of Color, Form, and Motion Lawrence C. Sincich and Jonathan C. Horton � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 303
v
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
Contents ARI 13 May 2005 13:21
Molecular Gradients and Development of Retinotopic Maps Todd McLaughlin and Dennis D.M. O’Leary � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 327
Neural Network Dynamics Tim P. Vogels, Kanaka Rajan, and L.F. Abbott � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 357
The Plastic Human Brain Cortex Alvaro Pascual-Leone, Amir Amedi, Felipe Fregni, and Lotfi B. Merabet � � � � � � � � � � � � � � 377
An Integrative Theory of Locus Coeruleus-Norepinephrine Function: Adaptive Gain and Optimal Performance Gary Aston-Jones and Jonathan D. Cohen � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 403
Neuronal Substrates of Complex Behaviors in C. elegans Mario de Bono and Andres Villu Maricq � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 451
Dendritic Computation Michael London and Michael Häusser � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 503
Optical Imaging and Control of Genetically Designated Neurons in Functioning Circuits Gero Miesenböck and Ioannis G. Kevrekidis � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 533
INDEXES
Subject Index � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 565
Cumulative Index of Contributing Authors, Volumes 19–28 � � � � � � � � � � � � � � � � � � � � � � � � � � � 577
Cumulative Index of Chapter Titles, Volumes 19–28 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 582
ERRATA
An online log of corrections to Annual Review of Neuroscience chapters may be found at http://neuro.annualreviews.org/
vi Contents
A nn
u. R
ev . N
eu ro
sc i.
2 00
5. 28
:4 03
-4 50
. D ow
nl oa
de d
fr om
w w
w .a
nn ua
lr ev
ie w
s. or
g by
U ni
ve rs
it y
of C
al if
or ni
a -
S an
D ie
go o
n 03
/3 0/
12 . F
or p
er so
na l
us e
on ly
.
