question and answer

The events that precede operant behavior and the consequences that follow may be arranged in many different ways. A schedule of reinforcement describes this arrangement. In other words, a schedule of reinforcement is a prescription that states how and when discriminative stimuli and

D

turn on a lamp, which is followed by illumination of the room.

to have little in common. Humans are very complex organisms—they build cities, write books,

cannot do. In addition, pressing a lever for food appears to be very different from switching on a light. Nonetheless, performances controlled by schedules of reinforcement have been found to be remarkably similar across different organisms, behavior, and reinforcers. When the same schedule of reinforcement is in effect, a child who solves math problems for teacher approval may generate a pattern of behavior comparable to a bird pecking a key for water.

and represents the most extensive study of this critical independent variable of behavior science. Today, few studies focus directly on simple, basic schedules of reinforcement. The lawful rela- tions that have emerged from the analysis of reinforcement schedules, however, remain central to the science of behavior—being used in virtually every study reported in the Journal of the Exper- imental Analysis of Behavior. The knowledge that has accumulated about the effects of schedules

Schedules of reinforcement have regular, orderly, and profound effects on the organism’s rate of responding. The importance of schedules of reinforcement cannot be overestimated. No description,

account, or explanation of any operant behavior of any organism is complete unless the schedule of

of schedules is central to the study of behavior. . . . Behavior that has been attributed to the supposed drives, needs, expectations, ruminations, or insights of the organism can often be related much more exactly to regularities produced by schedules of reinforcement.

Modern technology has made it possible to analyze performance on schedules of reinforcement in increasing detail. Nonetheless, early experiments on schedules remain important. The experi- mental analysis of behavior is a progressive science in which observations and experiments build on one another. In this chapter, we present early and later research on schedules of reinforcement. The analysis of schedule performance ranges from a global consideration of cumulative records to a detailed analysis of the time between responses.

- lated and integrated to provide a general account of the behavior of organisms. Often, simple ani- mals in highly controlled settings are studied. The strategy is to build a comprehensive theory of behavior that rests on direct observation and experimentation.

that go substantially beyond the data. Such speculations include reference to the organism’s mem- ory, thought processes, expectations, and undocumented accounts based on presumed physiological states. For example, a behavioral account of schedules of reinforcement provides a detailed descrip- tion of how behavior is altered by contingencies of reinforcement. One such account is based on evidence that a particular schedule sets up differential reinforcement of the time between responses

- grated into larger units of performance according to the molar or macro contingencies of reinforce-

to hypothetical cognitive events or presumed physiological processes. Recall that behavior analysts study the behavior of organisms, including people, for its own

sake. Behavior is not studied to make inferences about hypothetical mental states or real phys- iological processes. Although most behaviorists acknowledge and emphasize the importance of biology and neurophysiological processes, they focus more on the interplay of behavior with the environment during the lifetime of an organism. Of course, direct analysis of neurophysiology of animals provides essential details about how behavior is changed by the operating contingencies of reinforcement and behavioral neuroscientists currently are providing many of these details, as we discuss throughout this textbook.

Contemporary behavior analysis continues to build on previous research. The extension of behavior principles to more complex processes and especially to human behavior is of primary importance. The analysis, however, remains focused on the environmental conditions that control the behavior of organisms. Schedules of reinforcement concern the arrangement of environmental events that regulate behavior. The analysis of schedule effects is currently viewed within a bio- logical context. In this analysis, biological factors play several roles. One way in which biology

that function as reinforcement and discriminative stimuli. Biological variables may also constrain

biological sciences progress, an understanding of biological factors becomes increasingly central to a comprehensive theory of behavior.

con- tingency of reinforcement D r

period, behavior typically settles into a consistent or steady-state performance It may take many experimental sessions before a particular pattern emerges, but once it does, the

performance in his book, The Behavior of Organisms printing of that book, Skinner writes that “the cumulative records . . . purporting to show orderly changes in the behavior of individual organisms, occasioned some surprise and possibly, in some

-

one of these patterns. For example, a hungry rat might be required to press a lever 10 times to get a

food pellet. Following reinforcement, the animal has to make another 10 responses to produce the next bit of food, then 10 more responses. In industry, this requirement is referred to as piece rate and the schedule has characteristic effects on the job performances of the workers. When organisms

break-and-run pattern of behavior often develops. Responses required by the schedule are made rapidly and result in rein- forcement. A pause in responding follows each reinforcement, followed by another quick burst of

over again and occurs even when the ratio size of the schedule is changed.

In everyday life, behavior is often reinforced on an intermittent basis. On an intermittent schedule of reinforcement, an operant is reinforced occasionally rather than each time it is emitted. Every time a child cries, she is not reinforced with attention. Each time a predator hunts, it is not success- ful. When you dial the number for airport information, sometimes you get through, but often the exchange is busy. Buses do not immediately arrive when you go to a bus stop. It is clear that per- sistence is often essential for survival or achievement of success; thus, an account of perseverance on the basis of the maintaining schedule of reinforcement is a major discovery. In concluding his

It is impossible to study behavior either in or outside the laboratory without encountering a schedule of reinforcement: whenever behavior is maintained by a reinforcing stimulus, some schedule is in

schedules operate will it be possible to understand the effects of reinforcing stimuli on behavior.

a seed or insect. These bits of food occur only every now and then, and the distribution of reinforce- ment is the schedule that maintains the animal’s foraging behavior. If you were watching this bird hunt for food, you would probably see the animal’s head bobbing up and down. You might also see the bird pause and look around, change direction, and move to a new spot. This sort of activity is

however, does not explain it. Although evolution and biology certainly play a role in this foraging episode, perhaps as importantly, so does the schedule of food reinforcement.

- aging. In this arrangement, pigeons were able to choose between two food patches by pecking keys

“Progressive-Ratio Schedules” in this chapter; and see discussion of concurrent schedules in Chap-

decreased and more responses were required to produce bits of food—a progressively increasing ratio

would be expected, this change in reinforcement density up and down generated switching back and forth between the two patches. To change patches, however, the bird had to peck a center key—simu-

-

all contributed to the changing patches. This experiment depicts an animal model of foraging—using schedules of reinforcement to simulate natural contingencies operating in the wild.

taken. Experimenters risk misinterpreting results when they ignore possible schedule effects. This is because schedules of reinforcement may interact with a variety of other independent variables,

- -

ous punishment, but otherwise behavior on the schedule remains the same. A possible conclusion is

is not suppressed by contingent aversive stimulation. This conclusion is not completely correct, as further experiments have shown that punishment has

- ent result occurs. On this kind of schedule, when each operant is punished, the pattern of behavior remains the same and the rate of response declines. Obviously, conclusions concerning the effects of punishment on pattern and rate of response cannot be drawn without considering the schedule of reinforcement main- taining the behavior. That is, the effects of punishment depend on the schedule of reinforcement. These

check is the schedule of reinforcement maintaining the behavior labeled as illegal. In summary, schedules of reinforcement produce reliable response patterns, which are con-

sistent across different reinforcers, organisms, and operant responses. In our everyday experience, schedules of reinforcement are so common that we take such effects for granted. We wait for a taxi to arrive, line up at a store to have groceries scanned, or solve 10 math problems for homework. These common episodes of behavior and environment interaction illustrate schedules of reinforce- ment operating in our everyday lives.

Continuous reinforcement, or CRF, is probably the simplest schedule of reinforcement. On this schedule, every operant required by the contingency is reinforced. For example, every time a hungry pigeon pecks a key, food is presented. When every operant is followed by reinforcement, responses are emitted relatively quickly depending upon the time to consume the reinforcer. The organism continues to respond until it is satiated. Sim-

animal is again deprived of reinforcement and exposed to a CRF schedule, the same pattern of responding followed by satiation is repeated. Figure 5.5 is a typical cumulative record of performance on continuous reinforcement. As mentioned in Chapter 4, the typical vending machine delivers products

Conjugate reinforcement is a type of CRF schedule in which properties of reinforcement, including the rate, ampli- tude, and intensity of reinforcement, are tied to particular

energetic, and high-rate operant crying by infants is often

-

by atypically developing children, are automatically reinforced by perceptual and sensory effects

Additional research has used college students responding to clarify pictures on a computer monitor; in this study, students’ responding was sensitive to change in intensity of the visual stimulus, rate of

resistance to extinction is a mea- sure of persistence when reinforcement is discontinued. This perseverance can be measured in several ways. The most obvious way to measure resistance to extinction is to count the number of responses and measure the length of time until operant level is reached. Again, remember from Chapter 4 that

operant level refers to the rate of a response before behavior is reinforced. For example, a laboratory rat could be placed in an operant chamber with no explicit contingency of reinforcement in effect. The number of times the animal presses the lever during a 2-h exploration of the chamber is a measure of operant level, or in this case baseline. Once extinction is in effect, measuring the time taken and number of responses made until operant level is attained is the best gauge of resistance to extinction.

Although continuing extinction until operant level is obtained provides the best measure of behavioral persistence, this method requires considerable time and effort. Thus, arbitrary measures that take less time are usually used. Resistance to extinction may be estimated by counting the

reinforcement could be discontinued and the number of responses made in three daily 1-h sessions counted. Another index of resistance to extinction is based on how fast the rate of response declines during unreinforced sessions. The point at which no response occurs for 5 min may be used to index resistance. The number of responses and time taken to that point are used as indicators of behavioral persistence or resistance to extinction. The important criterion for any method is that it must be quantitatively related to extinction of responding.

In this experiment, birds were trained on CRF and two intermittent schedules that provided reinforce- ment for pecking a key. The number of extinction responses that the animals made during three daily sessions of nonreinforcement was then counted. Basically, Hearst found that the birds made many more extinction responses after training on an intermittent schedule than after continuous reinforcement.

On CRF schedules, the form or topography of response becomes stereotypical. In a classic study,

topography on a CRF schedule. In this study, rats were required to poke their noses anywhere along -

gency, the animals frequently responded at the same position on the slot. Only when the rats were

Further research with pigeons suggests that response variability may be inversely related to the rate of reinforcement. In other words, as more and more responses are reinforced, less and less variation occurs in the

reinforced pigeons for pecking on an intermittent schedule. The birds pecked at a horizontal strip and were occasionally reinforced with food. When some responses were reinforced, most of the birds pecked at the center of the strip—although they were not required to do so. During extinction, the animals made fewer responses to the center and more to other positions on the strip. Eckerman and

- quent study, also with pigeons. They varied the rate of reinforcement and compared response vari- ability under CRF, intermittent reinforcement, and extinction. Responses were stereotypical on CRF and became more variable when the birds were on extinction or on an intermittent schedule.

- ing as reinforcement becomes less frequent or predictable. When a schedule of reinforcement is changed from CRF to intermittent reinforcement, the rate of reinforcement declines and response variability increases. A further change in the rate of reinforcement occurs when extinction is started. In this case, the operant is no longer reinforced and response variation is maximal. The general principle appears to be “When things no longer work, try new ways of behaving.” Or, as the saying

When solving a problem, people usually use a solution that has worked in the past. When the usual solution does not work, most people—especially those with a history of reinforcement for response variability and novelty—try novel approaches to problem solving. Suppose that

inventive when reinforcement is withheld after a period of success. This increase in topographic variability during extinction after a period of reinforcement has been referred to as resurgence

In summary, CRF is the simplest schedule of positive reinforcement. On this schedule, every response produces reinforcement. Continuous reinforcement produces weak resistance to extinction and generates stereotypical response topographies. Resistance to extinction and variation in form of response both increase on extinction and intermittent schedules.

On intermittent schedules of reinforcement, some rather than all responses are reinforced. Ratio schedules are response based—these schedules are set to deliver reinforcement following a pre-

number of responses required for reinforcement. Interval schedules pay off when one response is made after some amount of time has passed. Inter-

number of responses have occurred, or after a con- stant amount of time has passed. On variable sched- ules, response and time requirements vary from one reinforcer to the next. Thus, there are four basic

and variable interval. In this section, we describe

- terns that they produce. We also present an analysis of some of the reasons for the effects produced by these basic schedules.

A

D

SD is sensory stimulation arising from the operant chamber; the response is a lever press and food

presented. After reinforcement, the returning arrow indicates that another 25 responses are required to again produce reinforcement.

should remind you that Mechner notation describes the independent variable, not what the organ- ism does. Indeed, FR 100,000,000 could be easily programmed, but this schedule is essentially an extinction contingency because the animal probably never will complete the response requirement for reinforcement.

In 1957, Ferster and Skinner described the FR schedule and the characteristic effects, patterns, and rates, along with cumulative records of performance on about 15 other schedules of reinforce-

ment. Their observations remain valid after literally thousands of replications: FR sched- ules produce a rapid run of responses, fol- lowed by reinforcement, and then a pause in responding -

ratio is presented in Figure 5.9. The record

very small FR values, as shown by Crossman,

run of responses

known as break and run. During extinction, the break-and-run

pattern shows increasing periods of paus- ing followed by high rates of response. In a cumulative record of a pigeon’s performance

-

reinforcement comes to dominate the record.

often called the postreinforcement pause (PRP), to indicate where it occurs. The pause

in responding after reinforcement does not occur because the organism is consuming the food. -

shown that the moment of reinforcement contributes to the length of the PRP, but is not the only

Detailed investigations of PRP on FR schedules indicate that the upcoming ratio requirement is perhaps more critical. As the ratio requirement increases, longer and longer pauses appear in the cumulative record. At extreme ratios there may be almost no responding. If responding occurs at all, the animal responds at high rates even though the number of responses per reinforcement is very

- pleted response requirements on FR pausing. The number of responses required and the size of the

pause a “post”-reinforcement event accurately locates the pause, but the upcoming requirements exert predominant control over the PRP. Thus, contemporary researchers often refer to the PRP as a preratio pause

Conditioned reinforcers such as money, praise, and successful completion of a task also pro-

of 10 math problems to complete for a homework assignment. A good bet is that you would solve 10 problems, and then take a break before starting on the next set. When constructing a sun deck, one of the authors bundled nails into lots of 50 each. This had an effect on the “nailing behavior” of friends who were helping to build the deck. The response pattern that developed was to put in 50

nailing again. In other words, this simple scheduling of the nails generated a break-and-run pattern typical of FR reinforcement.

These examples of FR pausing suggest that the analysis of FR schedules has relevance for human behavior. We often talk about procrastination and people who put off or postpone doing things. It is likely that some of this delay in responding is similar to the pausing induced by the ratio

low or no productivity. Human procrastination may be modeled by animal performance on ratio schedules; translational research linking human productivity to animal performance on ratio sched-

the economics of work. Researchers in behavioral economics often design experiments using FR

The equal cost assumption holds that each response or unit toward completion of the ratio on an FR schedule is emitted with equal force or effort—implying that the cost of response does not change as the animal completes the ratio. But evidence is mounting that the force of response changes as

Variable-ratio (VR) required for reinforcement changes after each reinforcer is presented. A variable-ratio schedule is

average number of responses

5 times, then 15, 7, 3, and 20 times. Adding these response requirements for a total of 50 and

symbol VR in Figure 5.10 indicates that the number of responses required for any one reinforcer is variable. Other than this change,

In general, ratio schedules produce a high rate of response. When VR and FR schedules are compared, responding is typically faster on VR. One reason for this is that pausing after

when the ratio contingency is changed from

that the PRP does not occur because the animal -

on VR does not pause as many times, or for as long, after reinforcement. When VR schedules are not excessive, PRPs do occur, although these pauses are typically smaller than those

- ure 5.11 portrays a typical pattern of response on a VR schedule of positive reinforcement.

A VR schedule with a low mean ratio can contain some very small ratio requirements. For example, on a VR 10 schedule there cannot be many ratio requirements above 20 responses because, to offset those high ratios and average 10, there will have to be many very low ratios. It is the occasional occurrence of a reinforcer

right after another reinforcer, the short runs to reinforcement, that reduces the likelihood of pausing

have fewer short ratios following one another and typically generate longer PRPs. The change from VR reinforcement to extinction initially shows little or no change in rate of

part of the record shows long pausing and short bursts of responses at a rate similar to the original VR 110 performance. The pauses become longer and longer and eventually all responding stops, as

An additional issue concerning VR schedules is that the number of responses for reinforcement is unpredictable, but it is not random. In fact, the sequence repeats after all the programmed ratios have been completed and, on some VR schedules, short ratios may occur more frequently than with a random sequence. A schedule with a pseudo-random pattern of response to reinforcement values

RR schedules resembles that on a VR schedule, but these probabilistic schedules “lock you in” to high rates of response, as in gambling, by early runs of payoffs and by the pattern of unreinforced

In everyday life, variability and probability are routine. Thus, ratio schedules involving prob-

laboratory. You may have to hit one nail three times to drive it in, and the next nail may take six

- ment. A batter with a .300 average gets 3 hits for 10 times at bat on average, but nothing guarantees a hit for any particular time at bat. The schedule depends on a complex interplay among conditions set by the pitcher and the skill of the batter.

On -

ment. Following reinforcement, another 90-s period goes into effect, and after this time has passed another response will produce reinforcement. It is important to note that responses made before the

-

time FT time. This is also referred to as a response-in- dependent schedule. [Unless otherwise speci-

is required on whatever schedule is in effect.] When organisms are exposed to interval

contingencies, and they have no way of tell- ing time, they typically produce many more responses than the schedule requires. Fixed-in- terval schedules produce a characteristic steady-state pattern of responding. There is a

probe responses, followed by more and more rapid responding to a constant high rate as the interval times out. This pattern of response is called scalloping. Figure 5.13 is an idealized cumulative record of FI performance. Each

into three distinct classes—the PRP, followed by a period of gradually increasing rate, and

Suppose that you have volunteered to be in an operant experiment. You are brought into a small room, and on one wall there is a lever with a cup under it. Other than those objects, the room is empty. You are not allowed to keep your watch while in the room, and you are told, “Do anything you want.” After some time, you press the lever to see what it does. Ten dollars fall into the cup. A good prediction is that you will press the lever again. You are not told this,

- -

ing around or doing anything else, the interval is timing out. You check out the contingency by making

even more time has passed. As the interval continues to time out, the probability of reinforcement increases and your responses are made faster and faster. This pattern of responding is described by the

Following considerable experience with FI 5 min, you may get very good at judging the time period. In this case, you would wait out the interval and then emit a burst of responses. Perhaps you

almost elapsed. This kind of mediating behavior may develop after experience with FI schedules

break-and-run Humans use clocks and watches to keep track of time. Based on this observation, Ferster and Skin-

for pigeons was a light that grew in size as the FI interval ran out. The birds produced FI scallops that were much more uniform than without a clock, showing the control exerted by a timing stimulus.

following reinforcement a high response rate occurred, leading to a pause at the end of the interval. The FI contingencies, however, quickly overrode the stimulus control by the reverse clock, shifting the pat-

to the schedule, behavior eventually conforms to the schedule rather than the controlling stimulus. In everyday life, FI schedules are arranged when people set timetables for trains and buses.

Next time you are at a bus stop, take a look at what people do while they are waiting for the next bus. If a bus has just departed, people stand around and perhaps talk to each other for a while. Then, the operant of “looking for the bus” begins at a low rate of response. As the interval times out, the rate of looking for the bus increases and most passengers are now looking for the arrival of the next bus. The passengers’ behavior approximates the scalloping pattern we have described in this section. Schedules of reinforcement are a pervasive aspect of human behavior, but we seldom recognize the effects of these contingencies.

On a variable-interval VI

changes but the average time is 30 s. The symbol V indicates that the time requirement varies from one reinforcer to the next. The average amount of time the schedule.

Interval contingencies are common in the ordinary world of people and other animals. For example, people

boiling egg, and are put on hold. In everyday life, variable -

ing in line to get to a bank teller may take 5 min one day and half an hour the next time you go to the bank. A wolf pack may run down prey following a long or short hunt. A baby may cry for 5 s, 2 min, or 15 min before a parent picks up the child. A cat waits varying amounts of time in ambush before a bird becomes a meal. Waiting for a bus

given time before leaving. A carpool is an example of a VI contingency with a limited hold. The car arrives

- oratory, this limited-hold contingency—where the reinforcer is available for a set time after a variable interval—when added to a VI schedule increases the rate of responding by reinforcing short interresponse

VI schedule with limited hold are ready for pick-up and rush out of the door when the car arrives.

Figure 5.17 portrays the pattern of response generated on a VI schedule. On this schedule, rate of response is moderate and steady. The pause after reinforcement that occurs on FI does not usually appear in the VI record. Notably, this steady rate of response is maintained during extinction. Ferster

response remains steady and moderate to high, VI performance is often used as a baseline for evaluat- ing other independent variables. Rate of response on VI schedules may increase or decrease as a result of experimental manipulations. For example, tranquilizing drugs such as chlorpromazine decrease the

An ideal baseline would be one in which there is as little interference as possible from other vari- ables. There should be a minimal number of factors tending to oppose any shift in behavior that might result from experimental manipulation. A variable-interval schedule, if skillfully programmed, comes close to meeting this requirement.

In summary, VI contingencies are common in everyday life. These schedules generate a mod- erate steady rate of response, which is resistant to extinction. Because of this characteristic pattern, VI performance is frequently used as a baseline to assess the effects of other variables, especially performance-altering drugs.

The major independent variable in operant conditioning is the program for delivering consequences, called the schedule of reinforcement. Regardless of the species, the shape of the response curve for a

run, and other patterns were observed in a variety of organisms and were highly uniform and regular

of schedule effects has been extended to the phenomenon of biofeedback and the apparent willful control of physiological processes and bodily states.

Biofeedback usually is viewed as conscious, intentional control of bodily functions, such as brainwaves, heart rate, blood pressure, temperature, headaches, and migraines—using instruments that provide information or feedback about the ongoing activity of these systems. An alternative view is that biofeedback involves operant responses of bodily systems regulated by consequences, producing orderly changes related to the schedule of “feedback.”

to the underside of the forearm to measure electrical activity produced by muscles while partic- ipants squeezed an exercise ball. They were instructed to contract their arm “in a certain way” to activate a tone and light; thus, their job was to produce the most tone/light presentations they could. Participants were randomly assigned to groups that differed in the schedule of feedback

sessions are run with the same schedule until some standard of stability is reached. In this applied experiment, however, 15-min sessions were conducted on three consecutive days with a 15-min extinction session added at the end.

Cumulative records were not collected to depict response patterns, presumably because the length and number of sessions did not allow for stable response patterns to develop. Instead,

-

- cle pumping action of the exercise ball.

- cating the operant function of electrical activity in the forearm muscles. Together with studies of bio-

that responses of the somatic nervous system also are under tight operant control of the schedule

responses clearly are warranted, but have been lacking in recent years. In this regard, we recommend the use of steady-state, single-subject designs that vary the interval or ratio schedule value over a wide range to help clarify how schedules of feedback regulate seemingly automatic bodily activity.

On a progressive-ratio (PR) schedule of reinforcement, the ratio requirements for reinforcement

Once the animal emits 5 responses resulting in reinforcement, the next ratio requirement might

give the schedule its name. At some point in the progression of ratios, the animal fails to achieve the requirement. The highest ratio value completed on the PR schedule is designated the breakpoint.

The type of progression on a PR schedule may be arithmetic, as when the difference between two ratio requirements is a constant value such as 10 responses. Another kind of progression is geo-

and geometric PR schedules increased as the ratio requirement progressed and then at some point

arithmetic PR schedules decreased in a linear manner—as the ratio size increased, there was a linear decrease in response rates. Responses rates on geometric PR schedules, however, showed a negative deceleration toward a low and stable response rate—as ratio size increased geometrically, response rates rapidly declined and then leveled off. Thus, the relationship between response rates and ratio requirements of the PR schedule depends on the type of progression—arithmetic or geometric. These relationships can be described by mathematical equations, and this is an ongoing area of

on PR schedules uses the giving-up or breakpoint as a way of measuring or effectiveness, especially of drugs like cocaine. The breakpoint for a drug indicates how much oper- ant behavior the drug will sustain at a given dose. For example, a rat might self-administer morphine on a PR schedule as the dose size is varied and breakpoints are determined for each dose size. It is

assessing the drugs’ relative reinforcement effectiveness. In these tests, it is important to recognize

about which drugs are more “addictive” and how the breakpoint varies with increases in drug dose

Progressive-ratio schedules allow researchers to assess the rein- forcing effects of drugs prescribed to control problem behavior. A drug prescribed to control hyperactivity might also be addictive—an effect recommending against its use. The drug Ritalin®

-

and is chemically related to Dexe- drine® - amine is a drug of abuse, as are other stimulants, such as cocaine. Thus, people who are given methylpheni- date to modify ADHD might develop addictive behavior similar to behav- ior maintained by amphetamine. In one study, human drug-abusing volunteers were used to study the

requirement resulted in oral self-administration of the drug. Additional monetary contingencies were arranged to ensure continued participation in the study. As shown in Figure 5.18, the results indicated that the number of responses to the breakpoint increased at the intermediate dose of meth- ylphenidate and d-amphetamine compared with the placebo control. Thus, at intermediate doses methylphenidate is similar in reinforcement effectiveness to d-amphetamine. One conclusion is that using Ritalin® to treat ADHD may be contraindicated due to its potential for abuse, and interventions based on increasing behavioral momentum may be a preferred strategy, as previously noted in this

on PR schedules; also Bolin, Reynolds, Stoops, & Rush, 2013 provide an assessment of d-amphet-

-

-

a PR3 schedule, requiring an increase or step of 3 responses for each pellet, using a linear pro-

highest ratio completed within a 15-min period

schedule. After establishing the PR3 baselines, both obese-prone and lean-prone mice were administered low and high doses of an anorexic

-

schedule. The results for breakpoints showed

did not reliably differ from the lean-prone mice.

similar for both genotypes. Also, the anorexic drug reduced PR3 breakpoints in a dose–response manner, but this effect did not differ by genotype

One problem with this conclusion is that animals were only given 15 min to complete the ratio requirement, and some animals did not achieve stable baselines on the PR3 schedule, even after

distance traveled or covered in a day. Viewed as behavior, traveling for food is an operant controlled by its consequences—the allocation of food arranged by the location or patch. In the laboratory, a

schedule, increasingly more work or effort is required to obtain the same daily food ration. Typical operant PR experiments are conducted in an open economy where animals receive bits

experimental session to maintain adequate body weight. To model the problem of travel for food in the wild, a closed economy is used where animals that meet the behavioral requirements receive all

especially for food consumption and maintenance of body weight. A novel experiment by a group of biologists from Brazil and England arranged a variation on

the distance required to maintain free-feeding levels was increased above the distance set for the previous 3 days. The NON group was housed and treated identically to the CON group, but food

During a baseline period, all rats were given 3 days of free food and access to running wheels. The animals consumed on average 24 g of food a day for an average consumption rate of 1 g per hour. On average, rats ran 1320 m/day in the wheels during the baseline phase. The next phase involved arranging a PR schedule for the rats in the CON group. To obtain the initial PR value, the 1320 m of baseline wheel running was divided by 24 g of food, yielding 1 g of food for each 55 m. A program-

PR value again remained in effect for 3 days at which point the distance requirement increased again

- tingencies based on Fonseca et al., 2014 and personal communication from Dr. Robert Young, the English

Figure 5.20 shows the -

eled for food for each 3 days of the experiment by the

-

groups. The line joining the grey triangles depicts the increasing distance on the PR schedule for animals to obtain their daily free-feed- ing level of food, six 4-g pellets. For the rats in the NON group, the distance traveled each day is low and constant, about 1300 m on average, as in baseline. These rats maintained daily food intake at about 24 g or

and showed increasing body

scheduled distance. Although distance traveled matched the early PR values and rats received the six 4-g pellets,

wheel running on the PR schedule. Following this initial drop, food consumed partially recovered;

rats’ average distance traveled no longer approximated the PR value—even though rats did complete

than required by the PR value, giving up some of the daily food ration that they could have obtained. One possibility is that the animals were sensitive to energy balance or homeostasis—balancing as best as possible energy expenditure by wheel running with energy intake from food consumption. In fact, body weight initially increased, but then leveled off and decreased as distance traveled fell considerably off the PR requirement and food availability decreased. At PR values between 8000 m

plummeted. The PR schedule and closed economy used in this study generated a severe energy imbalance,

which ultimately would result in eventual death of the animal. Other research addressed in this textbook shows that rats develop activity anorexia when faced with a restricted food supply and free access to running wheels. The animals run more and more, eat less at each meal, and die of self-star-

starve under these conditions—demonstrating how food reinforcement contingencies may induce life-threatening, non-homeostatic behavior.

We have described typical performances generated by different schedules of reinforcement. The patterns of response on these schedules take a relatively long time to develop. Once behavior has stabilized, showing little change from day to day, the organism’s behavior is said to have reached a steady state. The break-and-run pattern that develops on FR schedules is a steady-state performance and is only observed after an animal has considerable exposure to the contingencies. Similarly, the steady-state performance generated on other intermittent schedules takes time to develop. When an organism is initially placed on any schedule of reinforcement, typical behavior patterns are not consistent or regular. This early performance on a schedule is called a transition state. Transition

Consider how you might get an animal to press a lever 100 times for each presentation of food

steady-state performance is established on CRF, you are faced with the problem of how to program the steps from CRF to FR 100. Notice that in this transition there is a large shift or step in the ratio

of reinforcement to bar pressing. This problem has been studied using a progressive-ratio schedule, as we described earlier in this chapter. The ratio of responses following each run to reinforcement is

- tigate the behavioral effects of step size and criteria for stability. If you simply move from CRF to the large FR value, the animal will probably show ratio strain in the sense that it pauses longer and longer after reinforcement. One reason is that the time between successive reinforcements con-

interreinforcement interval and is controlled by it, the animal eventually stops responding. Thus, there is a negative feedback loop between increasing PRP length and the time between reinforcements in the shift from CRF to the large FR schedule.

Transitions from one schedule to another play an important role in human development. Devel- opmental psychologists have described periods of life in which major changes in behavior typically occur. One of the most important life stages in Western society is the transition from childhood to adolescence. Although this phase involves many biological and behavioral processes, one of the most basic changes involves schedules of reinforcement.

When a youngster reaches puberty, parents, teachers, peers, and others require more behav- ior and more skillful performance than they did during childhood. A young child’s reinforcement schedules are usually simple, regular, and immediate. In childhood, food is given when the child says “Mom, I’m hungry” after playing a game of tag, or is scheduled at regular times throughout the

the refrigerator, open packages and cans, sometimes cook, get out plates, eat the food, and clean up. Of course, any part of this sequence may or may not occur depending on the disciplinary practices of the parents. Although most adolescents adapt to this transition state, others may show signs of ratio strain to intermittent reinforcement.

Many other behavioral changes may occur during the transition from childhood to adolescence.

in adolescence:

With adolescence, the picture may change quite drastically and sometimes even suddenly. Now

The adolescent may have to take a job demanding a substantial amount of work for the money, which heretofore he received as a free allowance. Furthermore, he now needs more money than when he was younger to interact with people he deals with. A car or a motorcycle takes the place of the bicycle. Even the price of services such as movies and buses is higher. Money, particularly for boys, frequently becomes a necessary condition for dealing with the opposite sex. The amount of work required in school increases. Instead of simple arithmetic problems, the adolescent may now have to

will require much trial and error.

There are other periods of life in which our culture demands large shifts in schedules of rein-

forced or elected retirement. In terms of schedules, retirement is a large and rapid change in the

work-related consequences. For example, a professor who has enjoyed an academic career is no longer reinforced for research and teaching by the university community. Social consequences for these activities may have included approval by colleagues, academic advancement and income, the

interest of students, and intellectual discussions. Upon retirement, the rate of social reinforcement is reduced or completely eliminated. It is, therefore, not surprising that retirement is an unhappy time of life for many people. Although retirement is commonly viewed as a problem of old age,

As we have seen, the use of drugs is operant behavior maintained in part by the reinforcing effects of the drug. One implication of this analysis is that reinforcement of an incompatible response (i.e., abstinence) can reduce the probability of taking drugs. The effectiveness of an abstinence contingency depends on the magnitude and schedule of reinforcement for nondrug use (e.g., Higgins, Bickel, & Hughes, 1994).

In applied behavior analysis, contingency management involves the systematic use of reinforcement to establish desired behavior and the withholding of reinforcement or punishment of undesired behavior (Higgins & Petry, 1999). An example of contingency management is seen in a study using reinforcement schedules to reduce cigarette smoking. Roll, Higgins, and Badger (1996) assessed the effectiveness of three different schedules of reinforcement for promoting and sustaining drug abstinence. These researchers conducted an experimental analysis of cigarette smoking because cigarettes can function as reinforce- ment, smoking can be reduced by reinforcement of alternative responses, and it is relatively more convenient to study cigarette smoking than illicit drugs. Furthermore, cigarette smok- ers usually relapse within several days following abstinence. This suggests that reinforce- ment factors regulating abstinence exert their effects shortly after the person stops smoking and it is possible to study these factors in a short-duration experiment.

Sixty adults, who smoked between 10 and 50 cigarettes a day, took part in the experi- ment. The smokers were not currently trying to give up cigarettes. Participants were randomly

control group. They were told to begin abstaining from cigarettes on Friday evening so that they could pass a carbon monoxide (CO) test for abstinence on Monday morning. Each per- son in the study went for at least 2 days without smoking before reinforcement for abstinence began. On Monday through Friday, participants agreed to take three daily CO tests. These tests could detect prior smoking.

Twenty participants were randomly assigned to the progressive reinforcement group. The progressive schedule involved increasing the magnitude of reinforcement for remaining drug

Each subsequent consecutive CO sample that indicated abstinence increased the amount of money participants received by $0.50. The third consecutive CO test passed earned a bonus

yielded $3.50, passing the third test yielded $14.00 ($4.00 and bonus of $10.00), and passing the fourth test yielded $4.50. In addition, a substantial response cost was added for failing a CO test. If the person failed the test, the payment for that test was withheld and the value of payment for the next test was reset to $3.00. Three consecutive CO tests indicating abstinence following a reset returned the payment schedule to the value at which the reset occurred (Roll et al., 1996, p. 497), supporting efforts to achieve abstinence.

test. There were no bonus points for consecutive abstinences and no resets. The total amount of -

for reinforcement. The schedule of payment for the control group was the same as the average

people, the payment was given no matter what their carbon monoxide levels were. The control group was, however, asked to try to cut their cigarette consumption, reduce CO levels, and maintain abstinence.

abstinence tests, while the control group only passed about 40% of the tests. The effects of

of participants who passed three consecutive tests for abstinence and then resumed smoking over the 5 days of the experiment. Only 22% of those on the progressive schedule resumed

progressive schedule of reinforcement was superior in terms of preventing the resumption of smoking (after a period of abstinence).

Figure 5.21B shows the percentage of smokers who gave up cigarettes throughout the experiment. Again, a strong effect of schedule of reinforcement is apparent. Around 50% of those on the progressive reinforcement schedule remained abstinent for the 5 days of the

In a subsequent experiment, Roll and Higgins (2000) found that a progressive reinforce- ment schedule with a response–cost contingency increased abstinence from cigarette use

schedule. Overall, these results indicate that a progressive reinforcement schedule, combined

with an escalating response cost, is an effective short-term intervention for abstinence from with an escalating response cost, is an effective short-term intervention for abstinence from smoking. Further research is necessary to see whether a progressive schedule maintains abstinence after the schedule is withdrawn. Long-term follow-up studies of progressive and other schedules are necessary to assess the lasting effects of reinforcement schedules on absti- nence. What is clear, at this point, is that schedules of reinforcement may be an important component of stop-smoking programs (see more on contingency management in Chapter 13).

A schedule of reinforcement describes the arrangement of discriminative stimuli, operants, and con-

understanding of behavior regulation in humans and other animals. The research on schedules and performance patterns is a major component of the science of behavior, a science that progressively builds on previous experiments and theoretical analysis. Schedules of reinforcement generate con- sistent, steady-state performances involving runs of responses and pausing that are characteristic

schedules can serve as an animal model of foraging in the wild, and intermittent reinforcement plays a role in most human behavior, especially social interaction.

To improve the description of schedules as contingencies of reinforcement, we have intro- duced the Mechner system of notation. This notation is useful for programming contingencies in the laboratory or analyzing complex environment–behavior relations. In this chapter, we described

Adult humans have not shown classic scalloping or break-and-run patterns on FI schedules, and the performance differences of humans relate to language or verbal behavior as well as histories of

and higher overall rates of response. Adding a limited hold to a VI schedule increases the response

schedules, the higher the rate of reinforcement the greater the behavioral momentum. The study of behavior during the transition between schedules of reinforcement has not been

well researched, due to the boundary problem of steady-state behavior. Transition states, however, play an important role in human behavior—as in the shift in the reinforcement contingencies from childhood to adolescence or the change in schedules from employment to retirement. Reinforce- ment schedules also have applied importance, and research shows that cigarette smoking can be regulated by a progressive schedule combined with an escalating response–cost contingency. Finally, in the Advanced Section of this chapter, we addressed molecular and molar accounts of response rate and rate differences on schedules of reinforcement. We emphasized the analysis of IRTs for molecular accounts, and the correlation of overall rates of response and reinforcement for molar explanations.

www.thefuntheory.com/

physical activity, and cleaning up litter. See if you can think up new ways to use reinforcement schedules in programming fun to regulate important forms of human behavior in our culture.

www.youtube.com/watch?v=I_ctJqjlrHA This YouTube video discusses basic schedules of rein- forcement, and B. F. Skinner comments on variable-ratio schedules, gambling, and the belief in free will.

www.pigeon.psy.tufts.edu/eam/eam2.html This module is available for purchase and demon- strates basic schedules of reinforcement as employed in a variety of operant and discrimination procedures involving animals and humans.

http://opensiuc.lib.siu.edu/cgi/viewcontent.cgi?article=1255&context=tpr&sei-redir=1- search=“conjugate schedule reinforcement” A review of the impact of Ferster and Skinner’s publication of Schedules of Reinforcement schedules to the operant analysis of choice, behavioral pharmacology, and microeconomics of gambling. Contingency detection and causal reasoning by infants, children, and adults are

www.wadsworth.com/psychology_d/templates/student_resources/0534633609_sniffy2/sniffy/ download.htm If you want to try out shaping and basic schedules with Sniffy the virtual rat, go to this site and use a free download for 2 weeks of fun. After this period, you will have to pay to continue your investigation of operant conditioning and schedules of reinforcement.

2. Infrequent reinforcement generates responding that is persistent. What is this called?

3. Mechner notation describes:

4. Resurgence happens when:

5. Schedules that generate predictable stair-step patterns are:

7. Schedules that combine time and responses are called:

8. The shape of the response pattern generated by an FI is called a:

9. Human performance on FI differs from animal data due to:

10. Behavior is said to be in transition when it is between: