This is the focus:Behavior Analysis (previously known as behaviorism) SEE articles are attached below

Learning processes underlying avoidance of negative outcomes

MARTA ANDREATTA,a SEBASTIAN MICHELMANN,b PAUL PAULI,a,c AND JOHANNES HEWIGd

aDepartment of Psychology (Biological Psychology, Clinical Psychology, and Psychotherapy), University of W€urzburg, W€urzburg, Germany bSchool of Psychology, University of Birmingham, Birmingham, UK cCenter of Mental Health, Medical Faculty, University of W€urzburg, W€urzburg, Germany dDepartment of Psychology (Differential Psychology, Personality Psychology, and Psychological Diagnostics), University of W€urzburg, W€urzburg, Germany

Abstract

Successful avoidance of a threatening event may negatively reinforce the behavior due to activation of brain structures

involved in reward processing. Here, we further investigated the learning-related properties of avoidance using

feedback-related negativity (FRN). The FRN is modulated by violations of an intended outcome (prediction error, PE),

that is, the bigger the difference between intended and actual outcome, the larger the FRN amplitude is. Twenty-eight

participants underwent an operant conditioning paradigm, in which a behavior (button press) allowed them to avoid a

painful electric shock. During two learning blocks, participants could avoid an electric shock in 80% of the trials by

pressing one button (avoidance button), or by not pressing another button (punishment button). After learning,

participants underwent two test blocks, which were identical to the learning ones except that no shocks were delivered.

Participants pressed the avoidance button more often than the punishment button. Importantly, response frequency

increased throughout the learning blocks but it did not decrease during the test blocks, indicating impaired extinction

and/or habit formation. In line with a PE account, FRN amplitude to negative feedback after correct responses (i.e.,

unexpected punishment) was significantly larger than to positive feedback (i.e., expected omission of punishment),

and it increased throughout the blocks. Highly anxious individuals showed equal FRN amplitudes to negative and

positive feedback, suggesting impaired discrimination. These results confirm the role of negative reinforcement in

motivating behavior and learning, and reveal important differences between high and low anxious individuals in the

processing of prediction errors.

Descriptors: Operant conditioning, FRN, Anxiety trait

Successful avoidance of threats assures organisms’ survival. Nota-

bly, when aversive or threatening situations are successfully

avoided, organisms experience a positive feeling (Delgado, Jou,

LeDoux, & Phelps, 2009; H. Kim, Shimojo, & O’Doherty, 2006),

which may also motivate the organisms to repeat the behavior

(Murty, LaBar, Hamilton, & Adcock, 2011). However, such avoid-

ance does not only entail appropriate and survival-relevant proper-

ties, but this behavior has also been implicated in the maintenance

of anxiety disorders (Bouton, Mineka, & Barlow, 2001; Craske

et al., 2009; Mowrer, 1956). In this study, we want to further inves-

tigate avoidance learning processes with a particular focus on trait

anxiety as a vulnerability factor.

Learning mechanisms underlying avoidance have been exten-

sively described by instrumental conditioning (Skinner, 1932).

During instrumental conditioning, a response is associated to its

consequences (also called response-outcome learning). In other

words, performing the correct response leads to positive conse-

quences such as receiving a reward (positive reinforcer) or avoid-

ing a punishment (negative reinforcer); meanwhile, performing the

wrong response leads to negative consequences such as receiving a

punishment or not obtaining a reward. Thus, successful avoidance

of punishment should elicit rewardlike behavioral (i.e., increased

frequency of the behavior) and neural (i.e., striatal activation)

responses. On a behavioral level, humans usually prefer (i.e., per-

form more often) a response associated with a reward, and compa-

rably prefer a response that allows them to avoid punishment

(Delgado et al., 2009; H. Kim et al., 2006; S. H. Kim, Yoon, Kim,

& Hamann, 2015; Pessiglione et al., 2008). Notably, stimuli associ-

ated with positive or negative reinforcement are salient and

enhance attentional neural sources (S. H. Kim et al., 2015; Mis-

kovic & Keil, 2014). On the neural level, striatum activation was

revealed in experiments where participants received money (H.

Kim et al., 2006; Pessiglione et al., 2008), avoided losing money

(H. Kim et al., 2006; Pessiglione et al., 2008), or successfully

avoided a painful unconditioned stimulus (US; Delgado et al.,

2009). Interestingly, participants reported higher distress to a

threat-predicting stimulus, compared to a stimulus that predicted

similar threat, but they had the possibility to avoid this threat by

pressing a button (Delgado et al., 2009; Miskovic & Keil, 2014).

The work was supported by the Collaborative Research Center - “Fear, Anxiety, Anxiety Disorders” (SFB-TRR 58 project B1) to PP.

Address correspondence to: Dr. Marta Andreatta, Department of Psy- chology (Biological Psychology, Clinical Psychology, and Psychothera- py), University of W€urzburg, Marcusstraße 9-11, D-97070 W€urzburg, Germany. E-mail: [email protected]

578

Psychophysiology, 54 (2017), 578–590. Wiley Periodicals, Inc. Printed in the USA. Copyright VC 2017 Society for Psychophysiological Research DOI: 10.1111/psyp.12822

Based on this evidence, we conclude that successful avoidance

of aversive events elicits appetitive responses, likely due to the

respite of the (expected) aversive event (Lohr, Olatunji, & Saw-

chuk, 2007). Possibly, such successful avoidance may be accompa-

nied by the positive feeling of relief. Animal and human studies

have already shown approach behaviors (i.e., approach, Tanimoto,

Heisenberg, & Gerber, 2004; Yarali et al., 2008) and neural activa-

tion of reward centers (i.e., striatum activation, Andreatta et al.,

2012; Becerra, Navratilova, Porreca, & Borsook, 2013; Seymour

et al., 2005) to stimuli classically associated with such feelings of

relief (e.g., triggered by the ending of a painful event; for recent

reviews, see also Gerber et al., 2014; Navratilova & Porreca, 2014;

Riebe, Pamplona, Kamprath, & Wotjak, 2012). However, the pro-

cesses underlying the appetitive responses associated with respite

(i.e., the absence of an otherwise expected punishment) are rarely

investigated.

It is important to consider that individuals differ in their sensi-

bility to rewards or punishments in that they may be hypersensitive

to reward (e.g., impulsive individuals or gamblers) or to punish-

ment (e.g., anxious individuals, Gray, 1982). Consequently, learn-

ing might be modulated by such individual sensibility. For

instance, individuals with high impulsivity traits prefer to keep per-

forming a response associated with high but uncertain rewards

compared to a response associated with a more secure, but delayed

high reward (Hariri et al., 2006). Furthermore, these responses cor-

related with striatal activation; that is, the more risky the perfor-

mance was, the greater the striatum activation was. Accordingly,

individuals with greater sensibility to reward (according to the

Behavioral Activation System, Carver & White, 1994; Gray, 1990)

responded with greater striatal activity to positive feedback (S. H.

Kim et al., 2015). In contrast, individuals’ anxious state correlated

positively with the strength of the activation of the nucleus accum-

bens (NAcc, which is part of the ventral striatum) following suc-

cessful avoidance (by button pressing) of an aversive consequence

(view of aversive and negative pictures, Levita, Hoskin, & Champi,

2012).

Successful and stable avoidance of aversive or threatening con-

sequences has been implicated in the etiology and maintenance of

anxiety disorders (Bouton et al., 2001; Craske et al., 2009; Mowrer,

1956). According to the two-factor learning theory (Mowrer,

1956), a stimulus (e.g., elevator) may become associated with an

aversive event such as a panic attack. As a consequence, these indi-

viduals will avoid those stimuli, and successful avoidance is

assumed to elicit rewarding and relieving responses. On the other

hand, such avoidance does not allow these individuals to have dif-

ferent and possibly safe experiences with these stimuli; that is,

avoidance behavior prevents extinction of conditioned fear

responses.

Examining this assumption experimentally, two studies exposed

participants after fear conditioning to the conditioned stimulus

(CS1) with or without the possibility of avoidance behavior.

Results revealed that participants who learned that they could avoid

an aversive electric shock (US) by pressing a button when the CS

was present showed less fear extinction to the CS in comparison to

those participants who underwent a classic extinction protocol (i.e.,

CS has been presented several times without US, Lovibond, Mitch-

ell, Minard, Brady, & Menzies, 2009). In the second study, partici-

pants showed quick return of the (classically) conditioned fear

response to the CS once they could not avoid the US any more

(Vervliet & Indekeu, 2015). Similar to the latter study, in a virtual

reality version of a water maze, results showed larger startle

responses in participants who actively delayed entering the pool’s

platform (defined by the author as passive avoidance of the CS1),

where the CS1 (the CS associated with the US) was displayed on

the walls compared to the CS- (the CS never associated with the

US) during late extinction (Cornwell, Overstreet, Krimsky, &

Grillon, 2013). In other words, during the avoidance phase, partici-

pants did not receive the US in association with the CS1 because

of their avoidance response, and as a result no extinction occurred;

their conditioned fear to CS1 was still visible once they were not

allowed to perform the avoidance behavior. Interestingly, individu-

als with high neuroticism scores seemed to show more avoidance

behavior to stimuli that share some physical characteristics with

the CS1 but are actually safe (Lommen, Engelhard, & van den

Hout, 2010). This suggests that anxiety levels modulate the likeli-

hood of avoidance behavior and thus extinction.

We conclude that there is evidence that indicates that personali-

ty traits such as neuroticism or anxiety facilitate avoidance learn-

ing. However, thus far it remains unclear whether anxious

individuals are more sensitive to the positive relieving feeling fol-

lowing a successful avoidance or are more sensitive to the punish-

ment after unsuccessful avoidance. Therefore, we attempted to

disentangle these aspects by using an instrumental conditioning

paradigm during which participants learn to avoid a painful electric

shock by pressing one of two buttons. In order to investigate the

participants’ neural responses to positive relieving feedback after

successful avoidance compared to a negative feedback after unsuc-

cessful avoidance, we took advantage of the N100 and P200

(Olofsson, Nordin, Sequeira, & Polich, 2008; Vogel & Luck,

2000), as well as of the feedback-related negativity (FRN, Holroyd

& Coles, 2002; Miltner, Braun, & Coles, 1997; Walsh & Anderson,

2012).

N100 is an early component of the ERPs characterized by short

latency, while the P200 has a middle latency (Olofsson et al.,

2008). The source of the N100 has been located in the frontal lobe,

and is normally accompanied by an opposing potential located over

parietal-occipital lobe labeled P100 (Heinze et al., 1994; Yamazaki

et al., 2000). N100 and P100 occur within similar time windows,

reflect similar attention-related processes (Heinze et al., 1994), and

present larger amplitude to negative stimuli (Pourtois, Grandjean,

Sander, & Vuilleumier, 2004; Sass et al., 2010; Weymar, Gerdes,

L€ow, Alpers, & Hamm, 2013). Together, the N100 and the P200 have been proposed as sensory gating processes (Lijffijt et al.,

2009). Specifically, presentation of a stimulus triggers attention,

and this process has been related to N100 amplitude. Once the

stimulus is detected, attention may (or may not) be allocated to it,

and this process has been related to P200 amplitude. Furthermore,

impulsivity (Lijffijt et al., 2012) as well as expression style of

angry feelings (measured by the Anger Expression Inventory,

Stewart et al., 2010), positively correlated with N100 amplitude.

Moreover, findings have shown that frontocentral P200 was larger

for wins (indicated by positive feedback), compared to losses (indi-

cated by negative feedback). Importantly, such large P200 ampli-

tude was even more pronounced in the context of two preceding

wins (Osinsky, Mussel, & Hewig, 2012).

The FRN is an ERP initially observed after negative feedback

showing its peak around 230–330 ms after feedback onset at fron-

tocentral midline electrodes (Holroyd & Coles, 2002; Miltner et al.,

1997). Subsequent research indicated that a large amount of vari-

ance in this ERP is explained by rewardlike responses—reward

positivity (RP) in response to monetary wins or positive perfor-

mance feedback (e.g., Hewig et al., 2008; Holroyd, Pakzad-Vaezi,

& Krigolson, 2008). The FRN has been related to the same learning

signal in the dopaminergic system, which reflects the prediction

Avoidance learning 579

error (PE)—a learning signal generated by the discrepancy between

the expected and the actual outcome (Chase, Swainson, Durham,

Benham, & Cools, 2010). Such PE modulation of the FRN is found

in successful avoidance of monetary losses (negative reinforce-

ment) and the achievement of monetary wins (positive reinforce-

ment, e.g., Kreussel et al., 2012). Its source has been located in the

anterior cingulate cortex (Walsh & Anderson, 2012).

The FRN is a reliable learning index (Chase et al., 2010; Walsh

& Anderson, 2012) and also a sensitive index for individual differ-

ences (Gu, Huang, & Luo, 2010; Hewig et al., 2010; Proudfit,

2015). In fact, this ERP was (significantly) more positive in gam-

blers than controls, and this correlated with the level of risky

behaviors carried out by participants. In other words, gamblers are

hypersensitive to rewarding feedback (rather than hyposensitive to

punishment) in a gambling task, and such individual characteristics

are reflected in the FRN (Hewig et al., 2010). In another study that

used a gambling task, low anxious individuals showed significantly

larger FRN to negative feedback compared to highly anxious indi-

viduals. Although this result may appear surprising, the authors

stated that the potentially different expectations of high versus low

anxious individuals might have played a role (Hajcak, Moser, Hol-

royd, & Simons, 2007). Moreover, it is also possible that the task

used is not the most sensitive context for gathering anxiety-related

responses (Holroyd, Larsen, & Cohen, 2004). Therefore, to the best

of our knowledge, there are no studies investigating the role of

FRN in avoidance learning using an anxiety-relevant context such

as a task involving threat.

We hypothesize that respite of an aversive event due to success-

ful avoidance behavior will entail motivating properties, and, con-

sequently, the frequency of the respite-associated response will

increase. Moreover, we expected larger FRN amplitudes to aver-

sive events (negative feedback), especially to those unexpected

after a respite-associated response. Finally, we expect that partici-

pants’ trait anxiety modulates both behavioral as well as electro-

cortical responses. In particular, we expect greater avoidance in

high versus low anxious individuals as well as either larger FRN

amplitude to negative feedback (increased punishment sensitivity) or larger RP to positive feedback (increased negative reinforcement sensitivity).

Method

Participants

Thirty-two volunteers participated in the study and received either

16 eor course credits for their participation. One participant was

excluded from the analysis because of technical problems and three

participants were excluded because they presented too many arti-

facts in the electrocortical signal (see Apparatus and Data Analy-

sis). In the end, we considered 28 participants (18 females, 1 non-

German) with a mean age of 21.68 years (SD 5 3.20, range: 18–30

years). All participants were right-handed according to the standard

handedness inventory (Oldfield, 1971). The study was approved by

the ethics committee of the Department of Psychology of the Uni-

versity of W€urzburg. Before beginning the experiment, all partici- pants read and signed the informed consent form.

There are no available cutoffs for the State-Trait Anxiety Inven-

tory (STAI, Laux, Glanzmann, Schaffner, & Spielberger, 1981), as

it is not a diagnostic questionnaire. Therefore, to explore the anxi-

ety hypothesis, we split the sample into two groups: high anxious

(44.50, SD: 5.40) and low anxious (33.29, SD: 3.20) individuals, based on the median (37.5) of the trait version of the STAI (for

details, see Table 1).

Material and Procedure

Unconditioned stimulus. The aversive stimulus (US) consisted of

a mild painful electric shock delivered over the forearm of the non-

dominant hand by means of two disk electrodes with 9-mm diame-

ter and spacing 30 cm. The electric shock was generated by a

current stimulator (Digitimer DS7A, Digitimer Ltd., Welwyn Gar-

den City, UK, 400 V, maximum of 9.99 mA) consisting of a train

of 50 pulses (each 2 ms long) triggered every 4 ms. Altogether, the

electric pulse stimulation was 200 ms in duration with a frequency

of 250 Hz. The intensity of the shock was individually assessed

with a threshold procedure consisting of two ascending and

descending series of electric shocks in steps of 0.5 mA. Participants

rated each electric stimulus on a visual analog scale (VAS) ranging

from 0 (no pain at all) to 10 (unbearable pain) with 4 as an anchor for the threshold (just noticeable pain). The individual pain thresh- old was then increased by 1 mA in order to avoid habituation. The

mean intensity of the US was 2.44 mA (SD 5 1.03), and partici- pants reported the aversive stimulus as painful (M 5 6.57, SD 5 1.67).

Feedback. Each participant’s response was followed by negative

or positive feedback according to whether they pressed the punish-

ment or the avoidance button, respectively. The negative feedback

consisted of a red flash (1,046 pixels high, 551 pixels wide, 150

dpi), while the positive feedback was green and consisted of a ban

signal over the flash (1,046 pixels high, 979 pixels wide, 150 dpi).

The feedback was presented 1 s after each participant’s response,

18.5 cm high on the screen and lasting 500 ms. Before and after the

experiment, we assessed the valence (positive vs. negative) and the arousal (calm vs. exciting) of the feedback by means of two VASs ranging from 1 to 9.

Task. Participants sat in front of a computer screen (distance

60 cm) and were instructed to press one of two buttons on the key-

board with the index finger of the right (L button) and left (S but- ton) hand (Figure 1). Participants were told that they could avoid

painful electric shocks if they pressed the correct button (without

Table 1. Description of the Sample Separated for High and Low Anxious Participants

Low anxious High anxious

Gender 10 females/4 males 8 females/6 males v 5 0.62, p 5 .430 Age 21.21 years (SD 5 3.02) 22.14 years (SD 5 3.42) t(26) 5 0.76, p 5 .453 US intensity 2.40 mA (SD 5 1.12) 2.48 mA (SD 5 0.96) t(26) 5 0.20, p 5 .840 US ratings 6.50 (SD 5 1.87) 6.64 (SD 5 1.50) t(26) 5 0.22, p 5 .825 Awareness 8 aware/6 unaware 6 aware/8 unaware v 5 0.57, p 5 .450 STAI 33.29 (SD 5 3.20) 44.50 (SD 5 5.40) t(26) 5 6.68, p < .001 BDI 4.92 (SD 5 2.99) 16.29 (SD 5 7.44) t(25) 5 5.28, p < .001

580 M. Andreatta et al.

explicitly indicating which button). Moreover, we told them that if

they did not press any button, they could receive the painful shock.

Hence, if they pressed one button (respite-associated button or

avoidance button), no shock was delivered and the positive feed-

back was presented. If they pressed the other button (pain-associat-

ed button or punishment button), the negative feedback was

presented, and 1 s after its offset, the shock was delivered. This

was chosen in order to prevent confounding effects and artifacts of

the shock. The buttons were counterbalanced among the partici-

pants, and there was no mention of the response/outcome contin-

gency. In order to elicit a more identifiable FRN, the press of the

avoidance button was positively reinforced in 80% of the trials,

while in the remaining 20% participants received the negative feed-

back and the painful electric shock 1 s after feedback offset. In the

same manner, the punishment response was presented in 80% of

the trials, but reinforced in the remaining 20%. Participants were

instructed to press one button when an orange square (17.5 3

17.5 cm) was presented on the black screen. The square lasted

either until participant’s response or maximal 5 s. If participants

did not press any button, the painful electric shock and the negative

feedback were delivered. The intertrial interval (ITI), defined as

the time between feedback offset and square onset, lasted for a ran-

domly jittered interval between 2 s and 4 s. The experiment con-

sisted of four blocks (Block 1, Block 2, Block 3, Block 4) that

consisted of 60 trials each. For 48 of 60 trials (80%), either nega-

tive feedback by the press of the punishment button or positive

feedback by pressing the respite button was presented. While the

remaining 12 trials (20%), either positive feedback by pressing the

punishment button or negative feedback by pressing the respite but-

ton was presented. The four blocks were almost identical except

that, during the last two blocks, no electric shock was delivered.

Although Block 1 to 4 did not differ in regard to the task, the learn-

ing curve during blocks should be different; the learning curve dur-

ing Block 1 should be greater than during Block 2, as during this

second block participants should have been aware that one button

allows avoidance but other does not. Between each block, we asked

participants whether they noticed any association between their

responses and the delivery of the painful electric shock. 1

Moreover,

we asked participants to rate the strength of relief they experienced

when viewing the positive feedback on a VAS from 1 (no relief) to

9 (strong relief).

Questionnaires. The German versions of the STAI (Laux et al.,

1981), the Positive and Negative Affect Schedule (PANAS,

Krohne, Egloff, Kohmann, & Tausch, 1996), and the Beck Depres-

sion Inventory (BDI, Hautzinger, Keller, & K€uhner, 2006) were

collected in order to assess anxiety traits and depression as well as

the current emotional state of the participants. The STAI consists

of 20 items for the state version and 20 items for the trait version.

Each item is rated on a 4-point Likert scale from 1 (almost never)

to 4 (almost always) according to how much it describes the per-

son’s anxiety level in general (trait) or in the actual moment (state).

Trait anxiety scores ranged from 28 to 57 (M 5 38.89, SD 5 7.18),

which is comparable to the published normal range of adults (Laux

et al., 1981). Individual anxiety before (M 5 40.85, SD 5 5.33) and

Figure 1. Method. During one block, participants had to press one of two buttons on a computer keyboard when a geometrical shape was presented

on a computer screen. The shape lasted either until participants’ response or 5 s. If participants did not press a button, a painful electric shock (US)

and, 1 s later, a negative feedback were delivered. If participants pressed the avoidance button (a), they could avoid the painful US in 80% of the tri-

als, and 1 s after button press, a positive feedback was presented for 500 ms. In the remaining 20% of the trials, participants received the US and 1 s

later a negative feedback was presented. If participants pressed the punishment button (b), they received the US followed by the negative feedback in

80% of the trials and in 20% of the trials they avoided the US and saw a positive feedback again for 500 ms. Each block consisted of 60 trials.

1. Seven participants could correctly report by which response they could avoid the pain after the first block. After the second block, seven additional participants could verbally indicate which response allowed them to avoid the painful electric shock. At the end, 14 participants were aware of response-outcome association, whereas 14 remained unaware. Notably, we did not find any relevant modulation of the responses by the contingency awareness.

Avoidance learning 581

after the experiment (M 5 41.37, SD 5 4.82) did not change signif- icantly, F(1,25) 5 0.18, p 5 .677, g2p 5 .007. The PANAS consists of 20 adjectives, and participants indicated to what extent they feel

a particular emotion on a scale ranging from 1 (very slightly) to 5 (extremely). Participants’ negative (before: M 5 12.48, SD 5 2.62; after: M 5 12.30, SD 5 2.73) and positive (before: M 5 26.41, SD 5 4.35; after: M 5 24.78, SD 5 5.83) mood did not change throughout the experiment significantly (negative: F(1,25) 5 0.20, p 5 .659, g2p 5 .008; positive: F(1,25) 5 2.74, p 5 .111, g

2 p 5 .099).

Moreover, no significant effects were revealed involving the factor

anxiety (all ps > .234; see online supporting information Table S1). Both STAI (Laux et al., 1981) and PANAS (Krohne et al., 1996)

were collected at the beginning and at the end of the experiment. The

BDI consists of 21 items for evaluating the depressive state. Each

item describes a depressive thought, and individuals indicate how

often they experience that particular feeling or thought. Although our

sample consisted of healthy individuals, high anxious participants

presented significantly higher depressive feelings (although not clini-

cally relevant) than low anxious individuals (Table 1).

Apparatus and Data Analysis

The electrocortical signal was recorded by means of the 32-channel

ActiCAP (Brain Products GmbH, Munich, Germany) from 28 sites

according to the 10-20 system (Fp1, Fp2, F7, F3, Fz, F4, F8, FC5,

FC1, FC2, FC6, T7, C3, Cz, C4, T8, CP5, CP1, CP2, CP6, P7, P3,

Pz, P4, P8, O1, Oz, O2). The remaining four were used for record-

ing the vertical and the horizontal eye movements (electrooculo-

gram, EOG), two electrodes placed below and above the right eye

and two over the right and left canthi of the eyes. Cz and Fz were

used as reference and ground electrodes, respectively. The elec-

trode impedance was kept below 10 kX, and the signal was contin- uously recorded with a sampling rate of 1000 Hz. Online, a notch

filter (50 Hz) and, for preventing aliasing, a bandwidth filter (low

cutoff 10 s, high cutoff 250 Hz) were applied. The software Brain-

Vision Recorder (Version 1.03.0004, Brain Products GmbH) was

used for recording the EEG signal.

Offline analyses were conducted with the computer software

BrainVision Analyzer Version 2.0 (Brain Products GmbH). Data

were mathematically rereferenced with an average reference across

all electrodes and then pass-band filtered (low cutoff filter: 0.1 Hz;

time constant: 1.59, 24 dB/oct; high cutoff filter: 35 Hz). The EEG

signal was corrected for vertical and horizontal eye movements

(Gratton, Coles, & Donchin, 1983) and segmented for each condi-

tion within each block. Epochs were defined between 200 ms

before and 800 ms after feedback onset. The 100 ms before feed-

back onset were considered for baseline correction. Trials with a

voltage higher than 50 mV were excluded from further analysis. Altogether, 2.26% (SD 5 2.26) of the trials were excluded. There was a significantly larger percentage of rejected trials in Block 1

(4.76%, SD 5 6.63) compared to Block 2 (2.32%, SD 5 3.86; t(27) 5 2.41, p 5 .023), Block 3 (0.95%, SD 5 1.84; t(27) 5 3.09, p 5 .005), and Block 4 (1.01%, SD 5 2.50; t(27) 5 3.60, p 5 .001). However, high (2.92%, SD 5 2.92) and low (1.61%, SD 5 1.57) anxious individuals did not differ, t(26) 5 1.16, p 5 .255. Each con- dition (positive and negative feedback following either respite-

associated or pain-associated response) was then averaged for each

participant and separately for each block (Block 1–4). As partici-

pants were free to choose which button to press, the number of

avoidance trials associated with a negative feedback was quite low

(M 5 11.28, SD 5 1.28; see also supporting information Table S3). However, a minimum of 6–8 trials was enough to elicit a reliable

error-related negativity (ERN, Olvet & Hajcak, 2009), which corre-

sponds to the FRN but follows a wrong response instead of a nega-

tive feedback (Walsh & Anderson, 2012).

Two ERP components, namely, the N100 and the P200, were

analyzed at the Fz and Cz. Such frontocentral sensors were chosen

for analyses based on previous studies investigating individual dif-

ferences (Lijffijt et al., 2009; Stewart et al., 2010). In order to

improve the signal-to-noise ratio (Clayson, Baldwin, & Larson,

2013; Keil et al., 2014), the N100 was defined as the temporal win-

dow between 90 ms and 110 ms after feedback onset; the P200 was

defined as the temporal window between 180 ms and 220 ms after

feedback onset (Stewart et al., 2010). Additionally, the P100 was

analyzed at O1 and O2 within the time window 80–120 ms after

feedback stimulus (results are reported in the supporting informa-

tion). The FRN was defined as the difference between negative and

positive feedback after either avoidance or punishment button press

with the temporal window 180–350 ms following feedback onset

(Osinsky et al., 2012). Based on the topographic maps of this ERP,

we clustered Cz, FC1, and FC2 for FRN analysis.

Statistical Analysis

Analyses were conducted by using repeated measures analyses of

variance (ANOVAs). The frequency and the reaction time of the

responses were analyzed with two separate ANOVAs with the

within-subject factors response (avoidance, punishment) and block

(Block 1–4). For the relief ratings, an ANOVA with the within-

subject factor block (Block 1–4) was calculated. For the electro-

cortical signal, we considered only those trials when participants

pressed the avoidance button, because the number of the punish-

ment button presses was too small for EEG analysis. Separate

ANOVAs for N100 and P200 were calculated considering feed-

back (positive, negative), block (Block 1–4), and electrode (Fz,

Cz) as within-subject factors, while the ANOVA for the FRN

considered the within-subject factors feedback (positive, nega-

tive) and block (Block 1–4). Furthermore, the between-subjects

factor anxiety (low, high anxious individuals) was considered in

all ANOVAs.

As reported above, we controlled for the modulatory role of the

contingency awareness based on classical conditioning studies

(Sehlmeyer et al., 2009) and operant conditioning studies with

rewards (Delgado, Miller, Inati, & Phelps, 2005). No significant

effects were revealed; therefore, we decided not to further consider

participants’ awareness.

As a manipulation check, we calculated two additional

ANOVAs for the valence and arousal ratings of the feedback.

These ANOVAs contained the within-subject factors feedback

(positive, negative) and time (beginning, end) as well as the

between-subjects factor anxiety (high, low anxious).

For post hoc tests, we used t-tests and set the alpha level to .05 for all comparisons. If necessary, we adjusted the p values using the Greenhouse-Geisser correction (GG-e) when the sphericity assumption was violated. The partial eta squared (g2p) values are reported.

Results

Ratings of Feedback Symbols and Behavioral Data

Ratings of feedback symbols. The 2 3 2 3 2 ANOVA indicated

a significant main effect feedback, F(1,26) 5 64.22, p < .001, g2p 5 .712, but no time, F(1,26) 5 0.10, p 5 .756, g

2 p 5 .004, or

582 M. Andreatta et al.

anxiety, F(1,26) 5 0.49, p 5 .492, g2p 5 .018, effects. The interac- tion Feedback 3 Time, F(1,26) 5 4.83, p 5 .037, g2p 5 .157, was significant, but no other interaction effects were found (all

ps > .471). Post hoc t-tests revealed that before the experiment the negative feedback (M 5 3.61, SD 5 0.96) was rated significantly more negative than the positive feedback (M 5 6.32, SD 5 1.34; t(27) 5 7.26, p < .001). Throughout the experiment, feedback became slightly more neutral, and these comparisons (beginning

vs. end) just failed to reach the significance level (negative:

t(27) 5 2.00, p 5 .056; positive: t(27) 5 1.94, p 5 .063). However, the negative feedback (M 5 4.25, SD 5 1.48) was still rated more negatively valenced than the positive feedback (M 5 5.57, SD 5 1.60; t(27) 5 3.11, p 5 .004) at the end of the experiment.

The ANOVA for the arousal ratings of the feedback symbols

before and after the experiment showed a significant main effect

for feedback, F(1,26) 5 102.95, p < .001, g2p 5 .798, indicating that the negative feedback was rated as more arousing than the positive

feedback, but no other significant effects (all ps > .291; for ratings separated for the two groups, see Table S2) were found.

Response frequency. The analysis indicated a significant main

effect response, F(1,26) 5 194.47, p < .001, g2p 5 .882, and an interaction Response 3 Block, F(3,78) 5 4.12, GG-e 5 .593, p 5 .027, g2p 5 .137 (Figure 2a). No significant effects involving the factor anxiety were found (all ps > .083). Post hoc t-tests showed that participants pressed the avoidance button significantly

more often than the punishment button in Block 1, t(27) 5 7.05, p < .001; Block 2, t(27) 5 13.42, p < .001; Block 3, t(27) 5 8.87, p < .001; and Block 4, t(27) 5 13.04, p < .001. Importantly, the strongest learning effect was found between Block 1 and Block 2.

Thus, pressing the avoidance button significantly increased,

t(27) 5 2.63, p 5 .014, from Block 1 to Block 2, while pressing the punishment button significantly decreased, t(27) 5 2.51, p 5 .018. No further changes in the button frequencies were found between

Block 2 and Block 3 (avoidance: t(27) 5 0.75, p 5 .458; punish- ment: t(27) 5 0.76, p 5 .455), and between Block 3 and Block 4

(avoidance: t(27) 5 1.90, p 5 .068; punishment: t(27) 5 1.96, p 5 .061). Furthermore, separated analysis of those trials with no button responses (see Table S3) revealed no significant difference

among blocks, F(3,78) 5 1.77, GG-e 5 .639, p 5.182, g2p 5 .064, or between groups, F(1,26) 5 3.24, p 5 .083, g2p 5 .111, or their interaction, F(3,78) 5 .09, p 5 .966, g2p 5 .003.

Relief ratings. The analysis returned a significant main effect

block, F(3,78) 5 11.13, GG-e 5 0.705, p < .001, g2p 5 .300 (Figure 2b), but no anxiety, F(1,26) 5 .002, p 5 .963, g2p < .001, or Block 3 Anxiety interaction, F(3,78) 5 0.38, GG-e 5 .705, p 5 .698, g2p 5 .014, effects. Polynomial analyses of variance indicated a sig- nificant linear, F(1,26) 5 25.62, p < .001, g2p5 .496, but not qua- dratic, F(1,26) 5 .01, p 5 .906, g2p 5 .001, trend for the factor block, which indicated a linear increase of the relief ratings

throughout the experiment. Thus, participants reported stronger

relief when viewing the positive feedback at the end of the experi-

ment (after Block 4) as compared to the beginning (after Block 1:

t(27) 5 5.03, p < .001).

ERP Data

N100. The ANOVA showed a significant main effect for electrode,

F(1,26) 5 12.12, p 5 .002, g2p 5 .318, but not feedback, F(1,26) 5 3.06, p 5 .092, g2p 5 .105; block, F(3,78) 5 2.23, p 5 .091, g2p 5 .079; or anxiety, F(1,26) 5 0.16, p 5 .696, g

2 p 5 .006. The sig-

nificant main effect indicated that the N100 amplitude was signifi-

cantly larger at Fz compared to Cz.

The interaction Feedback 3 Electrode 3 Anxiety, F(1,26) 5 9.25, p 5 .005, g2p 5 .262, reached the significance level, and we did not find any other interaction effect (all ps > .119). Post hoc analyses separated for low and high anxious individuals indicated

that the low anxious participants, F(1,13) 5 5.52, p 5 .035, g2p 5 .298, but not the high anxious participants, F(1,13) 5 0.10, p 5 .758, g2p 5 .008, showed a significant main effect of feedback (Figure 3). Thus, low anxious individuals, but not high anxious

Figure 2. Behavioral data. Full lines indicate responses of high anxious individuals and dashed lines indicate responses of low anxious individuals. a:

Participants pressed the avoidance button (light gray lines) significantly more often than the punishment button (black lines) throughout the experi-

ment. The strongest learning effect (i.e., higher frequency of avoidance button and less frequency of the punishment button) was observed between

Block 1 and Block 2. b: Participants reported a relieved feeling to the respite-associated feedback, which increased significantly throughout the blocks.

*p < .05; ***p < .001.

Avoidance learning 583

individuals showed larger N100 to the negative feedback compared

to the positive feedback. Moreover, the interaction between

feedback and electrode was significant for high anxious,

F(1,13) 5 5.72, p 5 .033, g2p 5 .305, but not low anxious individu- als, F(1,13) 5 3.64, p 5 .079, g2p 5 .219. Post hoc t-tests for the sig- nificant interaction for high anxious individuals revealed no

significant difference between negative and positive feedback at

Fz, t(13) 5 1.39, p 5 .189, as well as at Cz, t(13) 5 0.76, p 5 .462,

while N100 was significantly larger at Fz compared to Cz for nega-

tive feedback, t(13) 5 3.23, p 5 .007, but not for positive feedback,

t(13) 5 1.95, p 5 .073.

P200. The significant main effects electrode, F(1,26) 5 56.88, p < .001, g2p 5 .686, and block, F(3,78) 5 3.46, GG-e 5 .664, p 5 .039, g2p 5 .117, were significant, as well as the interactions Feedback 3 Electrode, F(1,26) 5 9.31, p 5 .005, g2p 5 .264, and Block 3 Electrode, F(3,78) 5 6.79, p < .001, g2p 5 .207. The inter- action Feedback 3 Block, F(3,78) 5 2.60, p 5 .058, g2p 5 .091, just failed to reach the significance level. Post hoc t-tests for the inter-

action Feedback 3 Electrode showed larger P200 over Cz com-

pared to Fz for both positive, t(27) 5 9.53, p < .001, and negative,

t(27) 5 5.18, p < 0001, feedback, but no significant differences were revealed between the two feedback conditions over Fz,

t(27) 5 0.41, p 5 .682, or Cz, t(27) 5 1.25, p 5 .222. As we were mainly interested in the effects of feedback, we decided not to fol-

low up the interaction Block 3 Electrode.

Interestingly, the interaction Feedback 3 Electrode 3 Block 3

Anxiety was significant, F(3,78) 5 2.90, p 5 .040, g2p 5 .100, while no other effects involving anxiety were significant (all ps > .063). Separated post hoc analyses for the anxiety groups

2 returned a sig-

nificant Feedback 3 Electrode 3 Block interaction for the high

Figure 3. N100 component. Waveforms (upper) and topographic maps (lower) of the N100 amplitude to the positive feedback (light gray) and nega-

tive feedback (black) separately for the low anxious individuals (dashed lines) and high anxious individuals (full lines). The waveforms depict the

main effect stimulus for the two groups separated, that is, the mean voltage over Fz and Cz. The voltage for the topographic maps is presented for 90

ms until 110 ms after feedback onset. The first two heads depict low anxious individuals, while the last two heads depict high anxious individuals.

Only low anxious individuals showed larger N100 to negative as compared to positive feedback. *p < .05.

2. Further effects are listed here: Significant larger P200 over Cz compared to Fz (main effect electrode) for both the low anxiety group, F(1,13) 5 25.56, p < .001, g2p 5 .663, and the high anxiety group, F(1,13) 5 31.37, p < .001, g2p 5 .707. The interaction Feedback 3 Elec- trode was significant for the low anxiety group, F(1,13) 5 16.98, p 5 .001, g2p 5 .566, but not for the high anxiety group, F(1,13) 5 0.48, p 5 .500, g2p 5 .036. The interaction Feedback 3 Block did not reach significance in both groups (low anxiety group: F(1,13) 5 0.99, p 5 .408, g2p 5 .071; high anxiety group: F(1,13) 5 1.71, p 5 .181, g2p 5 .116). Finally, the interaction Electrode 3 Block was significant in the high anxiety group, F(1,13) 5 4.28, p 5 .011, g2p 5 .247, but not in the low anxiety group, F(1,13) 5 2.67, p 5 .061, g2p 5 .170.

584 M. Andreatta et al.

anxious individuals, F(3,39) 5 4.25, p 5 .011, g2p 5 .246, but not for the low anxious individuals, F(3,39) 5 0.30, p 5 .828, g2p 5 .022. We followed up this interaction for the high anxiety group separately for Fz and Cz. The interaction Feedback 3 Block

was marginally significant only for Cz, F(3,39) 5 2.72, p 5 .057, g2p 5 .173 (Figure 4) and not for Fz, F(3,39) 5 1.71, p 5 .181, g2p 5 .116. Finally, explorative t-test comparisons for Cz within the high anxiety group showed no significant P200 amplitude to posi-

tive feedback compared to the negative feedback during Block 1,

t(13) 5 1.82, p 5 .091; Block 2, t(13) 5 0.90, p 5 .383; Block 3, t(13) 5 .06, p 5 .954; and Block 4, t(13) 5 1.12, p 5 .285. Howev- er, P200 amplitude to positive feedback significantly decreased

from Block 2 to Block 3, t(13) 5 3.08, p 5 .009, while P200 ampli- tude to negative feedback did not change significantly throughout

the blocks (all ps > .378).

FRN. Analysis showed significant main effects of feedback,

F(1,26) 5 12.30, p 5 .002, g2p 5 .321, but no block, F(3,78) 5 1.52, GG-e 5 .625, p 5 .228, g2p 5 .055, or anxiety, F(3,78) 5 1.27, p 5 .270, g2p 5 .047. Moreover, the interactions Feedback 3 Block, F(3,78) 5 2.79, p 5 .046, g2p 5 .097, and Feedback 3 Anxiety, F(1,26) 5 4.24, p 5 .050, g2p 5 .140 (Figure 5) were significant, but no other effects involving the factor anxiety (all ps > .215).

Post hoc t-tests for the Feedback 3 Block interaction (Figure S1) revealed significantly larger FRN to negative feedback as com-

pared to positive feedback during Block 1, t(27) 5 5.13, p < .001; Block 2, t(27) 5 2.78, p 5 .010; and Block 3, t(267) 5 2.26, p 5 .032, but not during Block 4, t(27) 5 .97, p 5 0.343. FRN amplitude to negative feedback was comparable throughout all

four blocks (all ps > .232), while FRN amplitude to positive feed- back was significantly larger during the last two blocks as

compared to the first two blocks (Block 1 vs. Block 3: t(27) 5 2.70, p 5 .012; Block 1 vs. Block 4: t(27) 5 3.59, p 5 .001; Block 2 vs. Block 3: t(27) 5 2.89, p 5 .008; Block 2 vs. Block 4: t(27) 5 3.99, p < .001). No differences were found between Block 1 and Block 2, t(27) 5 0.61, p 5 .548, or between Block 3 and Block 4, t(27) 5 1.51, p 5 .142.

Following up the Feedback 3 Anxiety interaction by direct

comparisons between high and low anxious individuals revealed

significantly larger FRN to negative feedback for low compared to

high anxious individuals, t(26) 5 2.12, p 5 .044, while no differ- ences were revealed for the positive feedback between the groups,

t(26) 5 0.15, p 5 .886. Comparisons within anxiety group found, for low anxious individuals, significantly larger FRN to negative

feedback versus positive feedback, t(13) 5 4.39, p 5 .001, but not for high anxious individuals, t(13) 5 0.94, p 5 .366.

Discussion

The main goal of this study was to investigate learning processes

related to successful avoidance of a threat, which has been defined

as respite (Lohr et al., 2007). Respite of threat might elicit a posi-

tively valenced feeling of relief, which entails rewarding properties

and may serve as a reinforcer, motivating organisms to perform a

response (Gerber et al., 2014; Navratilova & Porreca, 2014). More-

over, such relief responses following successful avoidance have

been proposed as a mechanism underlying the etiology and mainte-

nance of anxiety disorders (Craske et al., 2009; Mowrer, 1956).

However, it remains unclear whether it is the respite per se or rather

the (unwanted) punishment that guides the behavior.

In this study, participants learned to press one button on a key-

board in order to avoid a subsequent slightly painful electric shock

Figure 4. P200 component. Waveforms over Cz (upper) and topographic map (lower) of the P200 amplitude (bars with standard errors) to the positive

feedback (light gray) and negative feedback (black) separately for the low anxious individuals (LA, dashed lines) and high anxious individuals (HA,

full lines). High anxious participants showed a significant decrease in P200 amplitude to negative feedback from Block 2 to Block 3 over Cz.

Avoidance learning 585

(respite-associated or avoidance button) in 80% of the trials. By

pressing another button, they received a similar shock (pain-associ-

ated or punishment button) in 80% of the trials. After two learning

blocks, participants underwent two test blocks, which were similar

to the learning blocks except that no shocks were delivered.

We found successful learning, as indicated by a higher frequen-

cy of presses on the avoidance compared to the punishment button.

Notably, learning took place between the first and second block of

the experiment, and then remained stable among the following

blocks. This result is in line with previous studies showing

increased response frequency when its outcome was either a reward

(Chase et al., 2010; Pessiglione et al., 2008) or avoidance of pain

(Delgado et al., 2009; Miskovic & Keil, 2014). Moreover, both par-

ticipants who could correctly indicate the avoidance button (i.e.,

aware participants) and those who could not (i.e., unaware partici-

pants) performed the task equally well and showed no differences

in their electrocortical signals (see below). This goes along with

the idea that instrumental conditioning can occur without conscious

processes (Pessiglione et al., 2008). Hence, avoidance of aversive

consequences seems to entail similar motivational properties such

as a reward leading to a preferred response.

Interestingly, during the test blocks (i.e., Block 3, 4) partici-

pants’ response frequency did not decrease, but remained constant

over time. This is surprising, especially considering that during

these blocks no shock was delivered, which should have led to an

extinction of the response. Possibly, such persistence of the

responses during the last two blocks might be related to the ratio

between punishment and respite of threat (Nevin, Randolph, Hol-

land, & McLean, 2001). Thus, responses were reinforced intermit-

tently (randomly in 80% of the trials), and this should delay

extinction. Conceivably, our participants may have continued

pressing the respite-associated button thinking that this successfully

Figure 5. Feedback-related negativity (FRN). Waveforms (upper) and topographic maps (lower) of the FRN to the positive feedback (light gray lines),

negative feedback (black lines), and the difference between them (blue lines) averaged among Cz, FC1, and FC2, separately for the low anxious indi-

viduals (left, dashed lines) and high anxious individuals (right, full lines). Low anxious participants showed significant larger FRN amplitude to nega-

tive feedback compared to positive feedback, whereas high anxious participants did not discriminate between the feedback and showed comparable

neurocortical signals. FRN to negative feedback was significantly larger in low anxious participants than in high anxious individuals. *p < .05;

**p < .01.

586 M. Andreatta et al.

caused avoidance of punishment. In turn, they missed trying to

press the punishment button and therefore did not notice that now

shocks were no longer delivered. As a consequence, extinction was

prevented (Lovibond et al., 2009; Vervliet & Indekeu, 2015). Fur-

thermore, we observed that this effect was even more pronounced

in high anxious participants. Namely, high anxious participants

pressed the respite-associated button slightly more frequently than

low anxious individuals, and this was especially visible during the

last block of the experiment. 3 .

In relation to the electrocortical signal in trials when partici-

pants pressed the avoidance button, we found significantly larger

N100, P100, and FRN amplitudes to negative feedback compared

to positive feedback following a correct response. We also found

larger P200 amplitudes to positive feedback when compared to

negative feedback. These results are in line with previous studies

(Hajcak et al., 2007; Holroyd & Coles, 2002; Lijffijt et al., 2009;

Miltner et al., 1997; Olofsson et al., 2008; Osinsky et al., 2012;

Pourtois et al., 2004; Sass et al., 2010; Walsh & Anderson, 2012;

Weymar et al., 2013). N100 and P200 are important components

involved in filtering sensory and cognitive inputs. In particular,

N100 is a negative component peaking around 70–160 ms after

stimulus onset over frontocentral sensors (Stewart et al., 2010) and

has been implicated in early attention (Lijffijt et al., 2009; Olofsson

et al., 2008). N100 was larger in response to negative pictures

(Olofsson et al., 2008) and importantly to fear-related pictures (i.e.,

spiders) especially in spider-fearful individuals (Weymar et al.,

2013). Therefore, our findings suggest that unexpected negative

feedback triggered higher attentional sources compared to expected

positive feedback. In parallel, the opponent potential (i.e., the

P100) also presented larger amplitude to the negative feedback sup-

porting previous findings in which P100 resulted in sensible to

aversive stimuli (Pourtois et al., 2004; Sass et al., 2010). If the

N100 triggers attention, the P200 is an index for attention alloca-

tion (Lijffijt et al., 2009). Previously, P200 has been found to be

larger to high arousing positive pictures (Olofsson et al., 2008).

Moreover, the P200 amplitude seems to be context dependent. In

other words, P200 has been found to be more pronounced when a

positive feedback followed another positive feedback, in compari-

son to when a positive feedback followed a negative feedback

(Osinsky et al., 2012). Accordingly, our study found that positive

compared to negative feedback elicited larger P200 amplitudes,

and this effect was evident during the first learning block, but not

during the second learning block or the test/extinction blocks. Pos-

sibly, participants may have allocated their attention to an expected

positive feedback as confirmation of their correct response during

learning of an optimal response strategy. Of note, a similar strong

initial learning effect was observed in the behavioral data. In fact,

in our study, participants quickly learned to avoid pressing the

pain-associated button and to prefer the respite-associated button.

This may imply that participants received a series of positive feed-

back during learning in the first block, and consequently this might

have contributed to the large P200 amplitude related to positive

feedback in the first block. These effects diminish in later blocks,

which is in accordance with reinforcement learning theory; the rel-

evance of the local reinforcement history concerning a response

declines with learning progress and may disappear if a stable

response strategy has been established (Hewig et al., 2008; Holroyd

& Coles, 2002; Holroyd et al., 2008; Miltner et al., 1997).

Of interest, these electrocortical responses (i.e., N100 and

P200) were influenced by the participants’ trait anxiety. Thus, low

anxious individuals discriminated well between negative and posi-

tive feedback after correct responses, as suggested by the larger

N100 amplitude to negative versus positive feedback. Hence, atten-

tion of low anxious participants was triggered by unexpected nega-

tive feedback after a correct respite-associated response. On the

contrary, attentional processes of high anxious individuals were

equally triggered by negative and positive feedback after a correct

avoidance response, as suggested by the equal N100 and P200

amplitudes to the feedback. Considering the general incapability of

high anxious individuals in distinguishing safety from threat

(Craske et al., 2009; Lohr et al., 2007), it is conceivable that they

may also be less able to take advantage of feedback and generalize.

This discrimination deficit seems in contradiction to the results in

spider phobics, who showed higher N100 to phobic-related pictures

(Weymar et al., 2013). We speculate that, in the Weymar et al.

study, the spider phobics’ attention is more specifically and quickly

captured by that particularly feared object. In contrast, participants

in our study are characterized by a more general (perhaps broad)

nonpathological anxiety (based on the STAI, Laux et al., 1981).

Finally, our findings regarding the P200 component are in line with

the N100 results indicating no discrimination between positive and

negative feedback in high anxious individuals. However, high anx-

ious individuals showed a significant decrease in P200 amplitudes

from the learning blocks (i.e., the blocks when slightly painful elec-

tric shocks were delivered) to the test blocks (i.e., the blocks where

no shocks were delivered). We assume that anxious participants

were highly motivated during the blocks with possible electric

shock. Possibly, they kept on trying to optimize behavior, which

includes the calculation of local reinforcement history (Osinsky

et al., 2012). This might explain that P200 amplitudes reflecting

local reinforcement history declined between Block 2 and 3 in

highly anxious individuals. Moreover (see supporting information),

we also found that the more depressive high anxious individuals

were, the more sensitivity (i.e., larger P200 amplitude) to positive

feedback they showed. This is in line with the positive correlation

between anxious state and NAcc activation elicited by avoidance

of negative consequences (Levita et al., 2012). Lastly, it should be

noted that no effect of anxiety traits were revealed for P100 (see

supporting information). Considering that N100 and P100 originate

from different sources (Yamazaki et al., 2000), it is conceivable

that the frontally originated N100 may be more sensitive to interin-

dividual differences than the parietal-occipitally originated P100.

In line with previous studies (Hajcak et al., 2007; Holroyd &

Coles, 2002; Miltner et al., 1997; Osinsky et al., 2012; Pessiglione

et al., 2008; Walsh & Anderson, 2012), FRN amplitudes were larg-

er to negative versus positive feedback. Learning that a button

press predicts likely avoidance of a painful shock may induce

expectation that, after that particular response, a positive feedback

should appear. Hence, the larger FRN to unexpected negative feed-

back in our participants is in line with previous studies saying that

FRN encodes PE (Chase et al., 2010). Also, FRN amplitude to neg-

ative feedback did not change much through the blocks. In contrast,

FRN amplitude to positive feedback significantly increased through

the blocks. Possibly, this increase in FRN amplitude might be relat-

ed to a prediction error signal elicited by the positive feedback.

Specifically throughout the blocks, participants may have noticed

3. During the last block of the experiment, high anxious participants pressed the relief-associated button (M 5 54.5, SD 5 5.8) slightly more frequently and the punishment-associated button slightly less frequently (M 5 5.2, SD 5 5.6) than low anxious individuals (relief-associated but- ton: M 5 48.4, SD 5 10.3, t(26) 5 1.92, p 5 .068; punishment-associated button: M 5 11.4, SD 5 10.3, t(26) 5 1.98, p 5 .062).

Avoidance learning 587

that positive feedback was not 100% certain after the respite but-

ton, which might have implied a PE-dependent FRN.

Lastly and most interestingly, trait anxiety affected our FRN

findings in accordance with the N100 and P200 findings. Thus,

highly anxious individuals did not exhibit differential FRN

responses to positive and negative feedback (discriminative

responses were revealed only during the first block), while low

anxious individuals did. Moreover, highly anxious individuals

responded with a smaller FRN to the unexpected negative feedback

than low anxious individuals. Possibly, low anxious individuals

experienced a strong feeling of relief related to the expected

respite, which elicited a strong prediction error (Chase et al., 2010)

and, consequently, discriminative responses to the positive (respite-

associated) and the negative (pain-associated) feedback. On the

contrary, high anxious individuals might not have felt a strong feel-

ing of relief related to the respite, because they may have been too

worried about a possible shock (Lohr et al., 2007), which was

delivered in an apparently random fashion, in 80% of the trials. As

a consequence, anxious participants showed no discriminative FRN

responses to the feedback. Finally, considering that FRN might be

sensible to individual differences (Hackel, Doll, & Amodio, 2015;

Hajcak et al., 2007; Hewig et al., 2010) and that we used an

anxiety-relevant threat (electric shock), our paradigm might have

been especially suitable to detect anxiety-related individual

differences.

Of interest, electrocortical signals of low anxious individuals

were influenced by their emotional state (see supporting informa-

tion). Thus, the P200 amplitude negatively correlated with the posi-

tive mood, while FRN amplitude positively correlated with

participants’ negative mood. Interestingly, P200, a component sen-

sitive to positive stimuli (Osinsky et al., 2012), was modulated by

participants’ positive mood, whereas FRN, a component sensitive

to negative feedback (Olofsson et al., 2008; Walsh & Anderson,

2012), was modulated by participants’ negative mood.

The current study has two main limitations. First, the number of

trials for the negative feedback after having pressed the avoidance

button is low (10 trials per block), and it is recommended to per-

form this study with a higher number of trials for this condition.

However, we are confident that our results were quite stable as the

large FRN amplitudes indicate (< 2 mV). Moreover, we used active sensors, which have a better signal-to-noise ratio than nonactive

sensors, and this might have contributed to such a stable signal.

The second limitation concerns the sample size. Through the medi-

an split, the anxiety groups consisted of 14 participants each, which

is quite small. Therefore, the results should be taken cautiously and

ought to be replicated with larger sample sizes in future studies.

However, the group differences in processing positive versus nega-

tive feedback are interesting, and they may provide areas of interest

for future studies.

To summarize, our results demonstrated that respite of a nega-

tive consequence is an effective reinforcer for behavior and that the

associated electrocortical signals mirror responses to reward-

associated feedback. Moreover, our results suggest a decreased abil-

ity of high anxious individual to discriminate between negative and

positive feedback. High anxious individuals also seem less able to

adapt their responses once the threat is not present and, therefore,

keep doing what they thought was best to avoid the threat.

References

Andreatta, M., Fendt, M., M€uhlberger, A., Wieser, M. J., Imobersteg, S., Yarali, A., . . . Pauli, P. (2012). Onset and offset of aversive events establish distinct memories requiring fear- and reward network. Learn- ing & Memory, 19, 518–526. doi: 10.1101/lm.026864.112

Becerra, L., Navratilova, E., Porreca, F., & Borsook, D. (2013). Analogous responses in the nucleus accumbens and cingulate cortex to pain onset (aversion) and offset (relief) in rats and humans. Journal of Neurophysi- ology, 110, 1221–1226. doi: 10.1152/jn.00284.2013

Bouton, M. E., Mineka, S., & Barlow, D. H. (2001). A modern learning theory perspective on the etiology of panic disorder. Psychological Review, 108, 4–32. doi: 10.1037//0033-295X.108.1.4

Carver, C. S., & White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punish- ment: The BIS/BAS scales. Journal of Personality and Social Psychol- ogy, 67, 319–333. doi: 10.1037/0022-3514.67.2.319

Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2010). Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23, 936–946. doi: 10.1162/jocn.2010.21456

Clayson, P. E., Baldwin, S. A., & Larson, M. J. (2013). How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study. Psychophysi- ology, 50, 174–186. doi: 10.1111/psyp.12001

Cornwell, B. R., Overstreet, C., Krimsky, M., & Grillon, C. (2013). Passive avoidance is linked to impaired fear extinction in humans. Learning & Memory, 20(3), 164–169. doi: 10.1101/lm.028902.112

Craske, M. G., Rauch, S. L., Ursano, R., Prenoveau, J., Pine, D. S., & Zinbarg, R. E. (2009). What is an anxiety disorder? Depression and Anxiety, 26, 1066–1085. doi: 10.1002/da.20633

Delgado, M. R., Jou, R. L., LeDoux, J. E., & Phelps, E. A. (2009). Avoid- ing negative outcomes: Tracking the mechanisms of avoidance learning in humans during fear conditioning. Frontiers in Behavioral Neurosci- ence, 3, 1–9. doi: 10.3389/neuro.08.033.2009

Delgado, M. R., Miller, M. M., Inati, S., & Phelps, E. A. (2005). An fMRI study of reward-related probability learning. NeuroImage, 24, 862–873. doi: 10.1016/j.neuroimage.2004.10.002

Gerber, B., Yarali, A., Diegelmann, S., Wotjak, C. T., Pauli, P., & Fendt, M. (2014). Pain-relief learning in flies, rats, and man: Basic research and applied perspectives. Learning & Memory, 21, 232–252. doi: 10.1101/lm.032995.113

Gratton, G., Coles, M. G. H., & Donchin, E. (1983). A new method for off- line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology, 55, 468–484. doi: 10.1016/0013-4694(83)90135-9

Gray, J. A. (1982). On mapping anxiety. Behavioral and Brain Sciences, 5, 506–534. doi: 10.1017/S0140525X00013297

Gray, J. A. (1990). Brain systems that mediate both emotion and cognition. Cognition & Emotion, 4, 269–288. doi: 10.1080/02699939008410799

Gu, R., Huang, Y.-X., & Luo, Y.-J. (2010). Anxiety and feedback negativi- ty. Psychophysiology, 47, 961–967. doi: 10.1111/j.1469-8986.2010. 00997.x

Hackel, L. M., Doll, B. B., & Amodio, D. M. (2015). Instrumental learning of traits versus rewards: Dissociable neural correlates and effects on choice. Nature Neuroscience, 18, 1233–1235. doi: 10.1038/nn.4080

Hajcak, G., Moser, J. S., Holroyd, C. B., & Simons, R. F. (2007). It’s worse than you thought: The feedback negativity and violations of reward pre- diction in gambling tasks. Psychophysiology, 44, 905–912. doi: 10.1111/j.1469-8986.2007.00567.x

Hariri, A. R., Brown, S. M., Williamson, D. E., Flory, J. D., de Wit, H., & Manuck, S. B. (2006). Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. Journal of Neu- roscience, 26, 13213–13217. doi: 10.1523/jneurosci.3446-06.2006

Hautzinger, M., Keller, F., & K€uhner, C. (2006). BDI-II. Beck Depressions Inventar Revision Manual. Frankfurt, Germany: Harcourt Test Services.

Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Munte, T. F., . . . Hillyard, S. A. (1994). Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature, 372, 543–546. doi: 10.1038/372543a0

Hewig, J., Kretschmer, N., Trippe, R. H., Hecht, H., Coles, M. G. H., Holroyd, C. B., & Miltner, W. H. R. (2010). Hypersensitivity to reward in problem gamblers. Biological Psychiatry, 67, 781–783. doi: 10.1016/ j.biopsych.2009.11.009

588 M. Andreatta et al.

Hewig, J., Trippe, R. H., Hecht, H., Coles, M. G. H., Holroyd, C. B., & Miltner, W. H. R. (2008). An electrophysiological analysis of coaching in blackjack. Cortex, 44, 1197–1205. doi: 10.1016/j.cortex.2007.07.006

Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709. doi: 10.1037/0033- 295X.109.4.679

Holroyd, C. B., Larsen, J. T., & Cohen, J. D. (2004). Context dependence of the event-related brain potential associated with reward and punish- ment. Psychophysiology, 41, 245–253. doi: 10.1111/j.1469-8986.2004. 00152.x

Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feed- back correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45, 688– 697. doi: 10.1111/j.1469-8986.2008.00668.x

Keil, A., Debener, S., Gratton, G., Jungh€ofer, M., Kappenman, E. S., Luck, S. J., . . . Yee, C. M. (2014). Committee report: Publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology, 51, 1–21. doi: 10.1111/ psyp.12147

Kim, H., Shimojo, S., & O’Doherty, J. (2006). Is avoiding an aversive out- come rewarding? Neural substrates of avoidance learning in the human brain. PLOS Biology, 4, 1453–1461. doi: 10.1371/journal.pbio.0040233

Kim, S. H., Yoon, H., Kim, H., & Hamann, S. (2015). Individual differ- ences in sensitivity to reward and punishment and neural activity during reward and avoidance learning. Social Cognitive and Affective Neuro- science, 10, 1219–1227. doi: 10.1093/scan/nsv007

Kreussel, L., Hewig, J., Kretschmer, N., Hecht, H., Coles, M. G. H., & Miltner, W. H. R. (2012). The influence of the magnitude, probability, and valence of potential wins and losses on the amplitude of the feed- back negativity. Psychophysiology, 49, 207–219. doi: 10.1111/j.1469- 8986.2011.01291.x

Krohne, H. W., Egloff, B., Kohmann, C. W., & Tausch, A. (1996). Unter- suchungen mit einer deutschen Version der “Positive and Negative Affect Schedule” (PANAS). [Development and validation of brief mea- sures of positive and negative affect: The PANAS scales]. Diagnostica, 42, 139–156.

Laux, L., Glanzmann, P., Schaffner, P., & Spielberger, C. D. (1981). Das State-Trait Angstinventar [Manual for the State-Trait Anxiety Invento- ry]. Weinheim, Germany: Beltz Test.

Levita, L., Hoskin, R., & Champi, S. (2012). Avoidance of harm and anxi- ety: A role for the nucleus accumbens. NeuroImage, 62, 189–198. doi: 10.1016/j.neuroimage.2012.04.059

Lijffijt, M., Cox, B., Acas, M. D., Lane, S. D., Moeller, F. G., & Swann, A. C. (2012). Differential relationships of impulsivity or antisocial symp- toms on P50, N100, or P200 auditory sensory gating in controls and antisocial personality disorder. Journal of Psychiatric Research, 46, 743–750. doi: 10.1016/j.jpsychires.2012.03.001

Lijffijt, M., Lane, S. D., Meier, S. L., Boutros, N. N., Burroughs, S., Steinberg, J. L., . . . Swann, A. C. (2009). P50, N100, and P200 sensory gating: Relationships with behavioral inhibition, attention, and working memory. Psychophysiology, 46, 1059–1068. doi: 10.1111/j.1469- 8986.2009.00845.x

Lohr, J. M., Olatunji, B. O., & Sawchuk, C. N. (2007). A functional analy- sis of danger and safety signals in anxiety disorders. Clinical Psycholo- gy Review, 27, 114–126. doi: 10.1016/j.cpr.2006.07.005

Lommen, M. J. J., Engelhard, I. M., & van den Hout, M. A. (2010). Neu- roticism and avoidance of ambiguous stimuli: Better safe than sorry? Personality and Individual Differences, 49, 1001–1006. doi: 10.1016/ j.paid.2010.08.012

Lovibond, P. F., Mitchell, C. J., Minard, E., Brady, A., & Menzies, R. G. (2009). Safety behaviours preserve threat beliefs: Protection from extinction of human fear conditioning by an avoidance response. Behaviour Research and Therapy, 47, 716–720. doi: 10.1016/ j.brat.2009.04.013

Miltner, W. H. R., Braun, C. H., & Coles, M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: Evidence for a “generic” neural system for error detection. Journal of Cognitive Neuroscience, 9(6), 788–798. doi: 10.1162/jocn.1997.9.6.788

Miskovic, V., & Keil, A. (2014). Escape from harm: Linking affective vision and motor responses during active avoidance. Social Cognitive and Affective Neuroscience, 9, 1993–2000. doi: 10.1093/scan/nsu013

Mowrer, O. H. (1956). Two-factor learning theory reconsidered, with spe- cial reference to secondary reinforcement and the concept of habit. Psy- chological Review, 63, 114–128.

Murty, V. P., LaBar, K. S., Hamilton, D. A., & Adcock, R. A. (2011). Is all motivation good for learning? Dissociable influences of approach and avoidance motivation in declarative memory. Learning & Memory, 18(11), 712–717. doi: 10.1101/lm.023549.111

Navratilova, E., & Porreca, F. (2014). Reward and motivation in pain and pain relief. Nature Neuroscience, 17, 1304–1312. doi: 10.1038/nn.3811

Nevin, J. A., Randolph, Holland, S., & McLean, A. P. (2001). Variable- ratio versus variable-interval schedules: Response rate, resistance to change, and preference. Journal of the Experimental Analysis of Behav- ior, 76, 43–74. doi: 10.1901/jeab.2001.76-43

Oldfield, R. C. (1971). The assessment and analysis of handdedness: The Edinburgh Inventory. Neuropsychologia, 9, 97–113. doi: 10.1016/ 0028-3932(71)90067-4

Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective pic- ture processing: An integrative review of ERP findings. Biological Psy- chology, 77, 247–265.

Olvet, D. M., & Hajcak, G. (2009). The stability of error-related brain activity with increasing trials. Psychophysiology, 46, 957–961. doi: 10.1111/j.1469-8986.2009.00848.x

Osinsky, R., Mussel, P., & Hewig, J. (2012). Feedback-related potentials are sensitive to sequential order of decision outcomes in a gambling task. Psy- chophysiology, 49, 1579–1589. doi: 10.1111/j.1469-8986.2012.01473.x

Pessiglione, M., Petrovic, P., Daunizeau, J., Palminteri, S., Dolan, R. J., & Frith, C. D. (2008). Subliminal instrumental conditioning demonstrated in the human brain. Neuron, 59, 561–567. doi: 10.1016/j.neuron.2008.07.005

Pourtois, G., Grandjean, D., Sander, D., & Vuilleumier, P. (2004). Electro- physiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex, 14, 619–633. doi: 10.1093/cercor/bhh023

Proudfit, G. H. (2015). The reward positivity: From basic research on reward to a biomarker for depression. Psychophysiology, 52, 449–459. doi: 10.1111/psyp.12370

Riebe, C. J., Pamplona, F. A., Kamprath, K., & Wotjak, C. T. (2012). Fear relief—Towards a new conceptual framework and what endocannabi- noids gotta do with it. Neuroscience, 204, 159–185. doi: 10.1016/ j.neuroscience.2012.04.015

Sass, S. M., Heller, W., Stewart, J. L., Silton, R. L., Edgar, J. C., Fisher, J. E., & Miller, G. A. (2010). Time course of attentional bias in anxiety: Emotion and gender specificity. Psychophysiology, 47, 247–259. doi: 10.1111/j.1469-8986.2009.00926.x

Sehlmeyer, C., Sch€oning, S., Zwitserlood, P., Pfleiderer, B., Kircher, T., Arolt, V., & Konrad, C. (2009). Human fear conditioning and extinc- tion in neuroimaging: A systematic review. PLOS ONE, 4, e5865. doi: 10.1371/journal.pone.0005865

Seymour, B., O’Doherty, J. P., Koltzenburg, M., Wiech, K., Frackowiak, R., Friston, K., & Dolan, R. (2005). Opponent appetitive-aversive neu- ral processes underlie predictive learning of pain relief. Nature, 8, 1234–1240. doi: 10.1371/journal.pone.0005865

Skinner, B. F. (1932). On the rate of formation of a conditioned reflex. Journal of General Psychology, 7, 274–285.

Stewart, J. L., Silton, R. L., Sass, S. M., Fisher, J. E., Edgar, J. C., Heller, W., & Miller, G. A. (2010). Attentional bias to negative emotion as a function of approach and withdrawal anger styles: An ERP investiga- tion. International Journal of Psychophysiology, 76, 9–18. doi: 10.1016/j.ijpsycho.2010.01.008

Tanimoto, H., Heisenberg, M., & Gerber, B. (2004). Event timing turns punishment to reward. Nature, 430, 983. doi: 10.1038/nature02816

Vervliet, B., & Indekeu, E. (2015). Low-cost avoidance behaviors are resistant to fear extinction in humans. Frontiers in Behavioral Neuro- science, 9. doi: 10.3389/fnbeh.2015.00351

Vogel, E. K., & Luck, S. J. (2000). The visual N1 component as an index of a discrimination process. Psychophysiology, 37, 190–203. doi: 10.1111/1469-8986.3720190

Walsh, M. M., & Anderson, J. R. (2012). Learning from experience: Event-related potential correlates of reward processing, neural adapta- tion, and behavioral choice. Neuroscience & Biobehavioral Reviews, 36, 1870–1884. doi: 10.1016/j.neubiorev.2012.05.008

Weymar, M., Gerdes, A. B. M., L€ow, A., Alpers, G. W., & Hamm, A. O. (2013). Specific fear modulates attentional selectivity during visual search: Electrophysiological insights from the N2pc. Psychophysiology, 50, 139–148. doi: 10.1111/psyp.12008

Yamazaki, T., Kamijo, K.-i., Kenmochi, A., Fukuzumi, S.-i., Kiyuna, T., Takaki, Y., & Kuroiwa, Y. (2000). Multiple equivalent current dipole source localization of visual event-related potentials during oddball par- adigm with motor response. Brain Topography, 12, 159–175. doi: 10.1023/a:1023467806268

Avoidance learning 589

Yarali, A., Krischke, M., Michels, B., Saumweber, T., Mueller, M. J., & Gerber, B. (2008). Genetic distortion of the balance between punish- ment and relief learning in Drosophila. Journal of Neurogenetic, 23, 235–247. doi: 10.1080/01677060802441372

(RECEIVED July 18, 2016; ACCEPTED November 20, 2016)

Supporting Information

Additional supporting information may be found in the online

version of this article:

Appendix S1: P100 analyses.

Appendix S2: Correlation analyses.

Figure 1: FRN component clustered among Cz, FC1, and FC2

for all participants.

Figure 2: P100 component averaged at O1 and O2.

Figure 3: Scatter plots for low and high anxious individuals

separately.

Table 1: Momentary anxiety and positive/negative mood.

Table 2: Ratings for the two kinds of feedback.

Table 3: Frequencies of the feedbacks per block after button

press.

590 M. Andreatta et al.

This document is a scanned copy of a printed document. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material.