Add to and fix literature review

TOKEN REINFORCEMENT: TRANSLATIONAL RESEARCH AND APPLICATION

TIMOTHY D. HACKENBERG REED COLLEGE

The present paper provides an integrative review of research on token reinforcement systems, organized in relation to basic behavioral functions and economic variables. This type of func- tional taxonomy provides a useful way to organize the literature, bringing order to a wide range of !ndings across species and settings, and revealing gaps in the research and areas especially ripe for analysis and application. Unlike standard translational research, based on a unidirectional model in which the analysis moves from laboratory to the applied realm, work in the area of token systems is best served by a bidirectional interplay between laboratory and applied research, where applied questions inspire research on basic mechanisms. When based on and contributing to an analysis, applied research on token economies can be on the leading edge of theoretical advances, helping set the scienti!c research agenda. Key words: behavioral economics, comparative analysis, review, token reinforcement, token

economy, translational research

Token reinforcement systems are among the oldest and most widely used procedures in applied behavior analysis. Beginning with groundbreaking studies by Ayllon and Azrin (1965) with adult patients in a psychiatric ward and by Phillips (1968) with delinquent youth in a group home, numerous studies have docu- mented the therapeutic and educational bene!ts of token procedures across a wide range of settings and populations (for reviews, see Ayllon & Azrin, 1968; Doll, McLaughlin, & Barretto, 2013; Glynn, 1990; Kazdin, 1982; Kazdin & Bootzin, 1972; O’Leary & Drabman, 1971). Unlike other successful technologies in behavior analysis, however, there has been little substan- tive contact between applied and basic research with token reinforcement over the years.

Despite laboratory research on token systems dating to the 1930s, little is known about how this body of knowledge relates to token rein- forcement procedures in the applied realm. And despite some 50 years of applied work on token economies, surprisingly little is known about the behavioral mechanisms responsible for their effectiveness. Laboratory research on token systems has

centered mainly on basic mechanisms (e.g., acquisition and maintenance of behavior, conditioned reinforcement, temporal organiza- tion, behavioral units), whereas applied research on token systems has centered mainly on prac- tical issues (e.g., generalization and mainte- nance, treatment integrity, staff training). Although token reinforcement systems have generally been quite successful in the applied realm, it is often unclear what variables are responsible for their success; the token systems are rarely based on an understanding of the basic principles involved. As a result, laboratory and applied work on token systems have devel- oped largely in parallel, with little cross- fertilization of ideas and concepts. A major aim of this paper is to bring

together what is known about token

Manuscript preparation was aided by grant R01 DA02617 from the National Institute on Drug Abuse. I am indebted to John Borrero, Jason Bourret, Mary-Kate Carey, and Francesca degli Espinosa for helpful comments on earlier versions of the paper. Address correspondence to: Timothy D. Hackenberg,

Psychology Department, Reed College, 3203 SE Wood- stock Blvd., Portland OR 97202-8199. E-mail: hack@ reed.edu doi: 10.1002/jaba.439

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2018, 51, 393–435 NUMBER 2 (SPRING)

© 2018 Society for the Experimental Analysis of Behavior

393

reinforcement systems from laboratory and applied perspectives. From this, we should gain a better understanding of some of the critical variables, how they operate in both settings and in relation to broader economic contexts. I will suggest promising new lines of research that emerge from a synthesis of laboratory and applied work. Although I will endeavor to point out some of the practical implications of an integrated approach to basic and applied work, the wide-ranging therapeutic and educa- tional bene!ts of token reinforcement proce- dures are beyond the scope of the paper. Rather, my emphasis is on conceptual integra- tion of basic and applied work based on a func- tional analysis of behavior across species and settings. To that end, the review of the litera- ture is selective, emphasizing studies and !nd- ings that illustrate general principles of behavior. The paper is organized into !ve sections. I

begin with an historical overview of token rein- forcement research, tracing the parallel develop- ments in laboratory and applied work that led to the current disjointed state of affairs. I then review the literature on token reinforcement effects in applied settings, comparing and con- trasting what is known about such effects from laboratory research, using a functional taxon- omy similar to that used in a review of labora- tory research on token systems (Hackenberg, 2009). I then consider an economic perspective on token reinforcement systems, building on the emerging synthesis of behavioral economics and behavior analysis. Finally, I consider some practical implications of the !ndings, offering some recommendations for implementing token-based procedures in applied settings in light of the literature reviewed in the earlier sec- tions. I conclude by encouraging an integrated translational approach to research and applica- tion, borrowing concepts and methods from both behavior analysis and behavioral economics.

HISTORICAL BACKGROUND

Token systems have been in existence in one form or another since antiquity (O’Leary, 1978), but it was not until the 1930s that token systems were studied systematically in the laboratory, and not until the 1960s that token systems were implemented in applied set- tings. As the 1960-70s was a period of rapid growth in applied behavior analysis, fueled by laboratory-based research, one might assume that the application of token systems in the applied realm grew out of laboratory research on token reinforcement. In actuality, however, work in the two areas has never been well con- nected. Despite occasional calls for greater inte- gration (Kagel & Winkler, 1972), laboratory and applied work on token systems have devel- oped largely in parallel. Laboratory research on token reinforcement

dates back to a series of groundbreaking studies by Wolfe (1936) and Cowles (1937) with chimpanzees as subjects. Each study consisted of a series of interconnected experiments on a wide range of topics, including discrimination of tokens with and without exchange value, a comparison of tokens and food reinforcers in the acquisition and maintenance of behavior, response persistence under immediate and delayed reinforcement, preference between tokens associated with different reinforcer mag- nitudes and qualities (e.g., food vs. water, play vs. escape), and social behavior engendered by token reinforcement procedures, to name only a few. The primary focus of research was on condi-

tioned reinforcement—the circumstances under which neutral stimuli come to acquire reinfor- cing functions. Many of the experiments were designed to assess whether or to what extent tokens as acquired (conditioned) reinforcers were functionally equivalent to food (uncondi- tioned) reinforcers. At the same time, these authors were clearly aware of some of the dis- tinctive characteristics of token reinforcement

TIMOTHY D. HACKENBERG394

systems, including their durability in the face of different motivational states—what Skinner (1953) would later term generalized reinforce- ment—and their relevance to economic systems more generally. An impressive number of experiments from these studies pre!gured later developments in the !eld of conditioned rein- forcement (Hackenberg, 2009). Despite this promising early research, the

study of token reinforcement lay dormant for two decades, until Kelleher’s work in the mid 1950s. Also using chimpanzees and poker-chip tokens, Kelleher conducted a number of inter- connected experiments (Kelleher, 1956, 1958), which formed part of the emerging !eld of reinforcement schedules (Ferster & Skinner, 1957) and was on the leading edge of research and theory at the time. These schedules permit- ted a broader analysis of conditioned reinforce- ment and of the conditions responsible for establishing and maintaining tokens as acquired reinforcers. This emphasis on basic principles of conditioned reinforcement and temporally organized behavior has persisted in more recent laboratory research, conducted mainly with rats and pigeons as subjects. And although the methods used to study conditioned reinforce- ment have changed over the years, the labora- tory study of token reinforcement has been focused primarily on the elucidation of basic principles. Applied research on token reinforcement has

developed along different lines. Within a decade of Kelleher’s groundbreaking work, token systems had been established in hospitals, schools, and group homes. The main source of inspiration for these studies was not Kelleher’s research with chimpanzees, however, but rather, a pioneering study by Ayllon and Azrin (1965), conducted with patients with chronic psychosis in an institutional setting. The study used a token reinforcement program to modify a variety of vocational and self-care activities of the patients. Tokens were small metal coins that could be exchanged at regularly scheduled

exchange periods three times per day for a vari- ety of functionally de!ned terminal/backup reinforcers (e.g., interaction with staff, preferred dorm rooms, off-ward passes, snacks, cigarettes etc.). On the whole, the program was immensely successful, generating and maintain- ing a wide range of positive behavior in most patients. While the Ayllon and Azrin (1965) study

was analytic, emphasizing experimental control and isolation of effective variables, the clinical setting and the use of socially adaptive responses made its applied signi!cance readily apparent. The Ayllon and Azrin study was fol- lowed by dozens of studies on token systems applied to a wide range of settings and subject populations, including delinquent youth in a group home setting (Phillips, 1968; Phillips, Phillips, Fixsen, & Wolf, 1971), postwar sol- diers in a psychiatric ward (Boren & Colman, 1970), and children in classroom settings (Kaufman & O’Leary, 1972; McLaughlin & Malaby, 1972, 1976). While some of these early studies provided important insights into behavioral functions (described in more detail below), the rapid proliferation of token systems in applied settings led to calls for more research on issues pertaining to practical exigencies of program implementation, staff training, client resistance, generalization, and so on (Kazdin & Bootzin, 1972). A decade later, Kazdin (1982) reviewed progress in these areas and identi!ed several additional areas in need of research, including treatment integrity, administrative/ organization context, and dissemination. All were concerned with large-scale adoption and maintenance of token systems. Applied research on token systems thus came

to be increasingly governed by practical matters of implementation and program evaluation (often in conjunction with other treatment pro- grams) rather than by the illumination of basic behavioral processes. And although one !nds passing references to some of the seminal labo- ratory studies in the major reviews of token

395TOKEN REINFORCEMENT

economies, substantive contact between labora- tory and applied work on token systems has been virtually nonexistent. In addition to drifting from its laboratory

roots, the frequency of applied research on token economies has decreased markedly over time. If one were to conduct a keyword search with the terms token economy and token rein- forcement, one would !nd that the annual rate of published papers rose sharply in the late 1960s, peaked in the mid-1970s, and has since declined. To be sure, citation analyses are likely to underestimate the prevalence of token rein- forcement procedures in applied settings. Along with timeout procedures, token reinforcement procedures are used routinely in applied set- tings (e.g., classroom settings, group homes, institutions) as part of the background behavior management system, but are rarely the subject of research per se. In some ways, token rein- forcement may be a victim of its own successes: The effectiveness of token systems across such a range of settings and subject populations may have created a sense that further research is unnecessary. Whatever the reasons, there has been a rela-

tive dearth of published applied research on token reinforcement in the past few decades. Despite the decreasing publication trends, token economies hold great promise as a trans- lational research environment, permitting rapid advancement of theory and practice. As a start- ing point, I describe the characteristics of com- mon token systems, considering the various potential functions of tokens in applied settings.

FUNCTIONAL TAXONOMY OF TOKEN SYSTEMS

Tokens are stimuli paired with other rein- forcers that can be earned, accumulated, and exchanged for other reinforcers (sometimes called terminal or backup reinforcers) during scheduled exchange opportunities. Tokens are

often manipulable objects (e.g., poker chips, coins, marbles), but can be nonmanipulable as well (e.g., stickers, points, checkmarks on a list). Terminal reinforcers can be unconditioned reinforcers (e.g., food, water) or conditioned reinforcers (e.g., money, privileges, play oppor- tunities). Because clear distinctions between unconditioned and conditioned are often dif!- cult, I will make no further divisions between them, but instead, adopt a procedural de!ni- tion: a terminal reinforcer is the !nal reinforcer in the chain for which the token is exchanged. A token system, or token economy, speci!es the interrelations between token-earning behavior, exchange opportunities, and terminal reinforcers. There are a variety of ways to arrange a

token economy. Contingencies can be arranged for individual or aggregate behavior and for highly speci!c responses or broad classes of behavior; they might involve gain only, loss only, or some combination; and they might be embedded within larger sets of contingencies that govern transition to other levels. These contingencies may themselves be embedded in still-larger sets of social and institutional con- tingencies. Clearly, token systems may exist in many forms, and it is important to catalogue and study those differences. My emphasis here, however, will not be on the detailed analysis of particular token systems, but rather on the more general principles that govern the func- tional operation of all token systems. In the following section, I will outline some

elementary functions of tokens, organized around a behavioral taxonomy similar to that used in a review of laboratory research on token reinforcement (Hackenberg, 2009). This should help identify features common to both settings and the degree to which laboratory and applied research on token systems share func- tional (as opposed to merely super!cial) charac- teristics. This organizational structure will also help reveal gaps in our understanding of token systems and therefore prime topics of further

TIMOTHY D. HACKENBERG396

research. In later sections, I will consider some of the ways in which these basic functions com- bine and interact in a broader token economy, as well as the applied signi!cance of token research, including the types of token econo- mies in which particular variables are most likely to be important.

Reinforcing Functions It is generally assumed that tokens serve as

conditioned reinforcers (i.e., reinforcers whose effectiveness has been established through expe- rience with other events already effective as reinforcers). In early experiments by Wolfe (1936), for example, food-paired tokens were used to establish new behavior. Similarly, Ayl- lon and Azrin (1965) showed that tokens could be used to increase the frequency of work around the hospital ward (e.g., cleaning). Figure 1 shows the number of hours per day on work-related tasks across conditions, aggre- gated across 44 patients, in which tokens were or were not contingent on performance (i.e., tokens were delivered independent of responding). As mentioned above, the tokens were metal coins exchangeable at regular times during the day for a variety of tangibles (e.g., cigarettes) and activities (e.g., off-ward passes). Job performance deteriorated when tokens were not contingent on performance, dropping to less than half of the baseline level in 3 days, and to near-zero levels by the end of the 20-day condition, but performance rebounded quickly when the token reinforce- ment contingency was reinstated. These !ndings were subsequently replicated

and extended by Winkler (1970), also in a psy- chiatric institutional setting. Various patterns of self-care and independent living were reinforced with tokens, the functional control of which was demonstrated through control conditions in which the tokens were delivered independent of behavior. These types of yoked-control pro- cedures in which tokens are presented in a

noncontingent fashion provide an alternative way to arrange an extinction procedure while holding roughly constant the overall reinforce- ment rate. That is, to the extent that the response-independent token rates approximate the response-dependent rates, any differences can be attributed to the token reinforcement contingency per se, rather than to changes in reinforcement rate that would accompany a more traditional extinction procedure, in which reinforcers are discontinued (Thompson & Iwata, 2005). These remain important control procedures, and their use in applied token sys- tems research would augment the surprisingly sparse literature on reinforcing functions of tokens. Pairing with the terminal reinforcer. Tokens

are thought to derive their value as conditioned reinforcers from repeated pairings with the ter- minal reinforcers for which they are exchanged.

Figure 1. Total hours of daily work by 44 patients in a closed psychiatric ward as a function of token reinforce- ment contingencies (from Ayllon & Azrin, 1965). See text for additional details. Reprinted with permission.

397TOKEN REINFORCEMENT

In experiments by Wolfe (1936) and Cowles (1937), for example, poker chip tokens were established as reinforcers by allowing them to be exchanged for food. Metal tokens, on the other hand, were not exchangeable for food, and therefore did not acquire reinforcing func- tions. The chimpanzees came to readily work for and exchange the poker chips, while quickly losing interest in the metal tokens. In one of the few studies with humans on the

topic, Smith (1972) explored the role of token– reinforcer pairings with kindergarten children by comparing the effectiveness of tokens that had and had not been paired with other rein- forcers during exchange periods. Tokens were !rst presented on a !xed-time (FT) schedule (i.e., response-independently) every 10 s. Two token types were presented in strict alternation: One type could be exchanged for a toy trinket each time it was earned (paired token), whereas the other type was not exchangeable (unpaired token). This condition lasted until 50 tokens had been delivered, 25 of each type. The two token types were then made contingent on pressing one of two buttons in a choice task. Pressing one button produced tokens (of one type or the other), whereas pressing of the other did not. The effects of the previous pairing his- tory were studied across two groups of children. For one group, the tokens were those previously paired with toys, whereas for the other group the tokens were those previously unpaired. There was a strong preference for the token- producing button for both groups (i.e., prior history of pairing with toys did not matter), and no difference in the response rates for the paired and unpaired groups. In a subsequent 60-s extinction condition, however, clear differences between paired and unpaired tokens emerged: Nearly three times as many responses occurred on the token alternative than on the no-token alternative in the paired group, whereas no dif- ferences were seen in the unpaired group. The Smith (1972) study is important in

demonstrating the importance of pairing, using

proper comparisons to unpaired tokens. It also underscores the need to study token-reinforced behavior under various conditions, as differ- ences not apparent in the reinforcement condi- tion appeared in extinction. The study also had notable shortcomings, including brief contact with a limited range of contingencies and a between-subject design not well suited to an experimental analysis. Another limitation was a limited range of schedule values with minimal response requirements, which signi!cantly restricts the range of response rates over which an effect might be seen. Motivational operations. Another common

assumption is that token reinforcers (like all conditioned reinforcers) maintain their reinfor- cing functions only insofar as that function is maintained for the terminal reinforcer with which it has been paired. For example, Wolfe (1936) showed that chimpanzees’ choices between working for yellow tokens exchange- able for water and black tokens exchangeable for peanuts depended on the motivational con- text: the chimpanzees favored the yellow/water tokens when water deprived and the black/food tokens when food deprived. Similarly, in a classroom setting with chil-

dren and adolescents with intellectual disabil- ities, Moher, Gould, Hegg, and Mahoney (2008) showed that the reinforcing ef!cacy of speci!c tokens depended on the motivational context. Following a training procedure in which one token type was paired with a high- preference edible reinforcer and a second token type was paired with a low-preference edible reinforcer, response rates were compared under the following four conditions (high-preference edible, high-preference token, low-preference edible, low-preference token) as a function of deprivation and satiation conditions. Response rates were clearly differentiated in the depriva- tion conditions, with higher rates maintained by the high-preference edibles and tokens than the low-preference edibles and tokens. Responding (both for edibles and tokens)

TIMOTHY D. HACKENBERG398

quickly extinguished under satiation conditions with presession free access to the reinforcers, showing that the ef!cacy of the tokens depended on a given deprivation. For four participants in the study, the pair-

ings were established through direct contact with the contingencies; for a !fth, the pairings were established through verbal instructions. The practical import of these latter !ndings is that with verbal participants, tokens can be established as reinforcers via instructions rather than via direct experience with the pairing oper- ation. That generally similar results were obtained across participants in this study sug- gests that verbal instructions may substitute for direct contact with the contingencies. In these conditions, instructions serve as a reasonable proxy for direct contact; to the extent that they hasten acquisition of token-reinforced behavior, instructions may be pragmatically useful. At the same time, such strong substitutability of instructions and contingencies cannot always be assumed. Whereas some research shows an adaptive role of instructions, other !ndings show instructions can impair sensitivity to other contingencies, resulting in markedly suboptimal behavior (Fox & Pietras, 2013; Galizio, 1979; Hackenberg & Joker, 1994). Given how little is known about the role of direct pairings in estab- lishing reinforcing functions of tokens, instruc- tions should be used with caution and in full recognition of their complex and varied effects. Token-directed behavior. Tokens are thought

to be initially neutral stimuli that acquire rein- forcing functions by virtue of their repeated pairings with other reinforcers. Although this may be true of some tokens (e.g., poker chips, checkmarks), tokens can themselves engender behavior directed toward them. In some cases, this behavior takes the form of consummatory responses presumably conditioned by the token–food pairings; for example, rats (Malagodi, 1967) and chimpanzees (Kelleher, 1958) have been observed mouthing marbles and poker chips, respectively.

The origins of other token-directed behavior are less tied to food reinforcers. For example, the self-stimulatory behavior often observed in people with autism can occur in relation to tokens. Charlop-Christy and Haymes (1998) showed that the success of a token reinforce- ment program with children with autism in an academic setting depended, in part, on the degree to which the tokens evoked self- stimulatory behavior. Standard arbitrary tokens (e.g., stars on a sheet of paper) were compared with idiosyncratic tokens, determined individu- ally for each student. These tokens were the so- called “objects of obsession” for each student, stimuli in the presence of which self-stimulatory behavior was allowed to occur. In general, the tokens that occasioned self-stimulatory behavior produced higher levels of correct responding than standard tokens. They also resulted in lower levels of problem (off-task) behavior. Moreover, the use of self-stimulatory opportunities as terminal reinforcers did not lead to a general increase in self-stimulatory behavior. These !ndings were extended more recently

by Carnett et al. (2014) with a 7-year-old child with autism. The basic design and results can be seen in Figure 2. Following a baseline period without a token economy, tokens that engen- dered what the authors termed perseverative interest (PI tokens) were compared against stan- dard arbitrary tokens, such as pennies, in an alternating treatments design in which the two procedures were presented in succession over time. Both token types were exchangeable according to the same schedule and for the same type of terminal reinforcers (food). Both token procedures were effective, but the proce- dure with the embedded PI tokens was more effective than the standard token procedure, increasing on-task behavior and decreasing challenging behavior. The effects of PI tokens then generalized to a classroom setting. Despite the similarities between this type of

token-directed behavior and that seen in other

399TOKEN REINFORCEMENT

animals, there is an important difference. Unlike the animal studies, in which the token- directed behavior derives from pairings with the terminal reinforcer (food), the PI tokens used in the studies with children with autism derive from preexisting reinforcers, established outside the study. These tokens then likely served pri- marily discriminative functions, occasioning self-stimulation, perhaps adding to the reinfor- cing value of that procedure. In other words, the relatively greater ef!cacy of the PI

procedures over the standard procedures may thus derive from the combined effects of the token procedure and the opportunity to engage in self-stimulatory behavior (a high-probability response). Although such self-stimulatory behavior is

normally considered maladaptive, the rationale was based on the Premackian notion that opportunities to engage in high-probability behavior (in this case, self-stimulatory behavior) can reinforce less probable behavior (in this

Figure 2. Percentage of intervals on-task (top panel) and challenging behavior (bottom panel) for one child with autism across token economy conditions and token types: open symbols (standard tokens) and closed symbols (tokens engendering perseverative interest [PI]) (from Carnett et al., 2014). See text for additional details. Reprinted with permission.

TIMOTHY D. HACKENBERG400

case, correct responses); stimuli that occasion such behavior should therefore acquire impor- tant stimulus functions. At !rst glance, such procedures may seem counterproductive in that they explicitly arrange for self-stimulatory behavior to occur—behavior that may in turn interfere with the task being trained. This was not the case in either of the studies described above but is something to carefully consider in designing a token economy.

Antecedent Functions Much of the evidence reviewed above is con-

sistent with a conditioned reinforcement func- tion, based on repeated pairing of tokens and terminal reinforcers during exchange periods. In typical token economies, however, tokens also occasion other behavior (namely, token produc- tion and exchange behavior), and thus likely also serve important discriminative as well as reinfor- cing functions. For example, in the Smith (1972) study, described above, the poker chips were not only paired with trinkets during exchange periods, but also provided occasions upon which exchange responses (deposits) were reinforced. In robust token economies, these two functions (reinforcing and discriminative) are complemen- tary and mutually supportive: Stronger condi- tioned reinforcers tend to go hand in hand with stronger discriminative control. And for most practical purposes, there may be little need to separate the multiple functions of tokens. But for analytic purposes, because reinforcing and dis- criminative functions of tokens are perfectly con- founded in typical procedures, little is currently known about the separate contributions of the two functions in token economies. Future research should explore the effects of

different types of pairing operations and schedules—variables known to exert substantial impact on conditioned reinforcers generally. For example, applied research has shown that response contingency plays an important role in establishing conditioned reinforcing

functions of stimuli, such that stimuli !rst established as discriminative become effective conditioned reinforcers (Isaksen & Holth, 2009; Taylor-Santa, Sidener, Carr, & Reeve, 2014). When compared with stimulus-pairing methods in which neutral stimuli are temporally paired with reinforcers, such discriminative pro- cedures tend to be superior (Holth, Vandbakk, Finstad, Gronnerud, & Sorenson, 2009). To date, however, the !ndings have been limited to the new-response method, in which the effects of the different procedures (discriminative vs. stimulus pairing) are tested to determine whether they are capable of supporting a new response in extinction (i.e., the absence of fur- ther pairings with reinforcement). Such methods have two main limitations. First, because the stimulus-pairing and discriminative operant procedures are compared in extinction, while responding is in decline, the effects are typically brief and transient. Second, such methods, in which the conditioned reinforcers are presented in the signaled absence of uncon- ditioned reinforcement, are quite unlike the conditions in which they will be used in treat- ment. For both of these reasons, comparisons of the two procedures for establishing conditioned reinforcers would be greatly enhanced if con- ducted under steady-state conditions, such as with extended chained and concurrent chained schedules, in which added stimuli continue to be paired with terminal reinforcers. Such proce- dures generate more robust responding, and have proven useful in the analysis of conditioned reinforcement more broadly (Fantino, 1977; Gollub, 1977; Shahan, 2010; Williams, 1994). Directing these types of procedures to token- reinforced behavior would provide important insights into the relative contributions made by stimulus-pairing, discriminative, and condi- tioned reinforcing functions.

Generalized Reinforcing Functions Tokens are frequently exchangeable for more

than one terminal reinforcer (Ayllon & Azrin,

401TOKEN REINFORCEMENT

1965; McLaughlin & Malaby, 1972; Phillips, 1968). Such tokens are said to be generalized conditioned reinforcers, in the sense of not being tied to a particular deprivation. This con- ceptualization of generalized reinforcement originates with Skinner (1953). According to Skinner, and to subsequent interpretations (including virtually every behavioral textbook that covers the topic), generalized reinforcers are assumed to retain reinforcing functions across a wider range of conditions than nonge- neralized, speci!c reinforcers (those associated with a single terminal reinforcer). This is thought to be due to the wider range of termi- nal reinforcers and motivational conditions to which generalized reinforcers are related—any one or more of which is likely to prevail at the time the response is made. Generalized reinforcers, ranging from mone-

tary to social reinforcers, abound in everyday human contexts, including those in which much of applied behavior analysis occurs. To begin with, monetary reinforcers (the prototyp- ical generalized reinforcer) have been shown to play a powerful role in contingency manage- ment and other incentive-based intervention programs (Higgins, Heil, & Lussier, 2004). Skinner also implicated generalized reinforce- ment in social behavior. Signs of attention and approval are closely related to a wide range of other reinforcers, and thereby acquire general- ized functions. Such social reinforcers play a crucial role in functional assessment and treat- ment programs as well. Yet despite the practical and theoretical importance of the topic, little is currently known about the provenance of gen- eralized reinforcement—how generalized rein- forcers are established and maintained, and how they compare to nongeneralized, speci!c reinforcers. In applied research, Sran and Borrero (2010)

found that three typically developing children preferred generalized tokens (paired with multi- ple edible reinforcers) over speci!c tokens (paired with a single edible reinforcer). The

children worked on a tracing task in which every !ve completed units produced a token (a !xed ratio [FR] 5 token production sched- ule; see below). At the end of the 3-min ses- sions, any earned tokens could be exchanged for preferred edibles (a !xed-interval [FI] 3-min exchange production schedule; see below). Response rates were slightly but consistently higher in sessions in which the generalized tokens were earned. Moreover, a modest prefer- ence for the generalized over the speci!c tokens was observed, suggesting that the generalized tokens were of somewhat higher ef!cacy than the speci!c tokens. These !ndings are broadly consistent with

laboratory research with pigeons in token econ- omies (DeFulio, Yankelevitz, Bullock, & Hack- enberg, 2014). Under various experimental arrangements, pigeons could produce either speci!c tokens or generalized tokens. In one experiment, food components (in which green tokens could be earned and exchanged for food) alternated with water components (in which red tokens could be earned and exchanged for water) in a multiple schedule for- mat (i.e., two schedule components presented in alternation, each correlated with distinctive stimuli). In both components, white (general- ized) tokens could be earned and exchanged for either food or water. The cost of producing the speci!c or generalized tokens was always the same, yet the pigeons produced disproportion- ately more generalized than speci!c tokens. Together with the results of a more recent study with the same subjects and apparatus (Andrade & Hackenberg, 2017), pigeons pre- ferred generalized to speci!c tokens in 32 of 40 conditions in which both options were available at equal costs. Preference for generalized over more speci!c

reinforcers may be due to the expanded range of motivational conditions over which the gen- eralized reinforcers are effective. One set of conditions in the Moher et al. (2008) study is relevant to this issue. The design and main

TIMOTHY D. HACKENBERG402

!ndings for one participant are portrayed in Figure 3. The !rst three conditions were similar to an earlier set of conditions in the study, described brie"y above, in which the reinfor- cing ef!cacy of the token was shown to depend on deprivation from the edible with which it had been paired. Following the return to depri- vation (third phase), the token was paired with a second highly preferred edible and subse- quently tested under satiation conditions, in which presession free access to one or the other reinforcer (but not both) was provided. If the token had acquired generalized functions, responding should be less sensitive to satiation conditions because deprivation remained in effect for one of the terminal reinforcers. Con- sistent with this prediction, response rates remained high across satiation conditions. Thus, only two terminal reinforcers were suf!- cient to produce generalized functions of the tokens. These effects were replicated with the two other participants, although for one, three terminal reinforcers were needed to produce generalized functions (see also Becraft &

Rolider, 2015, for replication and extension of the basic !ndings). In sum, the studies reviewed in this

section suggest a few promising directions for research on generalized functions of tokens, and many important questions remain. To begin with, we know very little about the his- tory of experiences needed to establish tokens as generalized reinforcers. It is common in token economies to include a store in which tokens can be exchanged for an array of backup reinforcers (preferred items and activities). It is assumed—but rarely demonstrated—that such experiences are suf!cient to establish the tokens as generalized reinforcers. And although one !nds occasional recom-

mendations to use a healthy variety of terminal reinforcers (Kazdin, 1982), little research has been directed to this topic per se—what might be called the generalizability of the reinforcer. It may be most useful to view reinforcers on a continuum of generalizability, ranging from highly speci!c (dependent on a single backup reinforcer and its deprivation) to highly

Figure 3. Response rates for one participant across different pairing and motivational conditions in Experiment 3 of Moher et al. (2008). See text for additional details. Reprinted with permission.

403TOKEN REINFORCEMENT

generalized (associated with multiple backup reinforcers and less dependent on speci!c depri- vation conditions, such as money). Laboratory methods with nonhumans are well suited to exploring parts of this continuum, but the number of terminal reinforcers is usually quite limited (typically food, water, or drug). Applied token research, on the other hand, can greatly expand the number and range of terminal rein- forcers, sharpening the analysis and permitting greater quanti!cation. When expanded to include a wider range of terminal reinforcers, procedures such as those reviewed here on pref- erence and schedule assessments (Becraft & Rolider, 2015; Moher et al., 2008; Sran & Borrero, 2010) are well suited to an analysis of this continuum of generalizability.

Schedules of Token Reinforcement Beginning with Kelleher’s work in the

1950s, early token reinforcement research over- lapped quite substantially with work on extended chained and second-order schedules. Boren and Colman (1970), for example, used insights gained from laboratory research on chained and higher-order schedules of rein- forcement to conceptualize and interpret the results of token-based experiments conducted on a psychiatric ward with soldiers. One experi- ment was concerned with assessing the effec- tiveness of a token program on attendance at a morning meeting when organizational details of the token system were discussed. Although tokens were available for attendance, many sol- diers skipped the meetings. It was only when attendance at the meeting (initial link of the chain) was required to access the exchange period (the terminal link of the chain) that attendance increased. Conceptualized as extended chained or

second-order schedules, a token reinforcement schedule can be analyzed with respect to three principal components: (a) the token-production schedule, by which tokens are earned, (b) the

exchange-production schedule, by which exchange periods are produced, and (c) the token-exchange schedule, by which tokens are exchanged for ter- minal reinforcers. Research related to these func- tional components of a token economy will be reviewed below, along with recommendations for further research. Token-production schedules. The token-

production schedule speci!es the contingencies for earning tokens. There are two main lines of evidence of token-production schedule effects in laboratory research with nonhumans. In one, Kelleher (1958) and Malagodi (1967) showed that token-production schedules produce local patterns with chimpanzees and rats, respec- tively, that resemble the patterns typically seen with simple schedules. That is to say, FR schedules of token production generate typical “break-run” patterns observed under simple FR schedules of reinforcement, characterized by a two-state pattern of pausing shortly after rein- forcement with an abrupt transition to a higher rate of responding until reinforcement. The second line of evidence comes from the

functional relationship between response rates and FR schedule value. In simple FR schedules, this function is bitonic—it !rst increases and then decreases (Mazur, 1983). Bullock and Hackenberg (2006) showed that a similar bitonic function characterized the relationship between response rates and FR token- production schedule value with pigeons. Taken together, these patterns of !ndings show that tokens resemble food reinforcers in functional control over behavior, suggesting that the tokens indeed serve important conditioned reinforcing functions. There is little in the way of a direct analogue

in applied research, though a few studies have examined schedule-related effects in token pro- duction schedules, taking as a starting point what is known about schedule-related differ- ences from laboratory research. For example, De Luca and Holborn (1990) compared the relative ef!cacy of FR and FI token-production

TIMOTHY D. HACKENBERG404

schedules on exercise and weight loss in sixth- grade boys. The target responses were wheel turns of a stationary bike in four boys, two of whom were obese and two of whom were non- obese, according to standard criteria. Following a baseline phase during which wheel turns had no programmed consequences, wheel turns produced tokens (points signaled by light and bell) across three successive phases. In the !rst, tokens were produced according to an FI 30-s schedule, in which the !rst wheel turn after 30 s produced a token. In a second phase, tokens were produced according to an FR schedule, in which a !xed number of responses was required to produce a token (with the response requirement matched to the responses in the preceding FI condition). This was fol- lowed by a reversal to the FI 30-s schedule. Tokens earned during the 30-min sessions could be later exchanged for functionally deter- mined terminal reinforcers (priced according to each participant’s value ratings). Response rates were consistently higher for the nonobese than for the obese boys, but a clear reinforcement effect was seen for all four participants. To begin with, response rates for all participants increased in the transition from baseline to FI schedule. Rates increased further in the subse- quent FR condition, consistent with the higher response rates under FR than comparable FI nontoken reinforcement schedules (Ferster & Skinner, 1957). Responding declined in the return to the FI condition, providing further evidence of schedule control. De Luca and Holborn (1992) replicated and

extended these !ndings, using variable ratio (VR) rather than FR schedules of token pro- duction, in which a variable number of responses was required to produce a token. The token procedures were used to establish and maintain exercise in 11-year-old obese and nonobese boys over a 12-week period. The rationale for using VR schedules was derived from basic schedule research showing higher response rates under VR than comparable FR

schedules (Mazur, 1983). Following a baseline period in the absence of tokens with little activ- ity, wheel turns increased immediately upon introduction of the VR token-reinforcement contingency, and then increased across condi- tions in which the mean VR requirements increased by 15%. A subsequent 3-day return to baseline produced rapid declines in respond- ing, showing functional control by the contin- gency. This was followed by a return to the highest VR in the study, ranging from 110 to 130 wheel turns per token. This generated response rates in excess of 100 per min in all six boys. These !ndings are important in dem- onstrating clear functional control by a high- rate contingency over a clinically important activity. Also taking a lead from basic research on

schedule-related differences in response rates, Repp and Deitz (1975) compared FR 60 and VR 60 schedules of token production with two boys, aged 10-12. Button presses produced a marble token immediately exchangeable for a penny. In Phase 1 (20 sessions), the FR and VR token-production schedules were arranged in a multiple schedule, with FR and VR com- ponents strictly alternating after each token delivery and exchange period. Response rates were approximately equal in both components. In Phase 2, the FR and VR token-production schedules were arranged concurrently in a Findley-type changeover procedure (Findley, 1958). In this procedure, only one of the two schedules was active; a response on a second (changeover) button changed the schedule from one to the other. Both boys made a majority of changeover responses from the FR to the VR, indicating a strong preference for the VR schedule. This preference for VR over FR (owing perhaps to the disproportionate in"u- ence of the occasional shorter reinforcer delays in VR distributions) parallels results with other schedules and species (Mazur, 2004). The par- allel between these effects obtained with tokens and with other reinforcers suggests that the

405TOKEN REINFORCEMENT

tokens were serving as conditioned reinforcers. That concurrent schedules can reveal schedule- related differences not observed with the sched- ule in isolation is also consistent with prior research with pigeons and food reinforcers (Neuringer, 1967). Together with the results of De Luca and

Holborn (1990, 1992), the Repp and Deitz (1975) results show clear functional control over behavior by token-production schedules. These studies are conceptually signi!cant in deriving their rationale from an understanding of basic schedule effects, using what is known about rate differences between FI, FR, and VR schedules to develop maximally effective token schedules. One limitation of these studies, however, is that because the tokens could be exchanged as soon as they were earned, the delays to token delivery were confounded with delays to the exchange period (when the tokens could be exchanged for other reinforcers). And it is this variable—the delay to the exchange period—that has emerged as a powerful vari- able in token reinforcement research, both in laboratory and applied settings. Exchange-production schedules. In addition to

token-production schedules, several studies have shown that rates and patterns of token- reinforced behavior are systematically related to exchange-production schedules. In laboratory studies, for example, when token-production schedules are held constant, pausing increases and response rate decreases with increases in FR exchange-production ratio (Bullock & Hackenberg, 2006, 2015; Foster, Hackenberg, & Vaidya, 2001), similar to that seen under simple FR schedules (Felton & Lyon, 1966; Ferster & Skinner, 1957; Powell, 1968). Similar exchange-schedule effects have been

reported in applied studies. For example, an early study by Staats, Staats, Schutz, and Wolf (1962) showed that tokens were effective in generating and maintaining behavior in a read- ing program with children, even with exchange-production ratios as high as

24 (i.e., tokens could be exchanged after 24 had been earned). Staats et al. also reported evidence of break-run patterns similar to those obtained with FR exchange-production sched- ules with chimpanzees (Kelleher, 1956), rats (Webbe & Malagodi, 1978), and pigeons (Bullock & Hackenberg, 2006; Foster et al., 2001). The similar schedule-related effects seen across settings and species suggest that token- reinforced behavior in the laboratory and in applied settings may be governed by similar principles. In one of the experiments in the Phillips

et al. (1971) study with delinquent youth in a group home, an attempt was made to encour- age savings by making tokens contingent on deposits into a bank account. Figure 4a (left column) shows cumulative deposits, averaged across !ve subjects, in baseline (when deposits were recorded but did not produce any addi- tional consequences) and contingent points (when deposits produced points). Deposits occurred infrequently in baseline but increased substantially when tokens (points) were contin- gent on deposits. And although deposits could occur at any time in the week, the frequency of deposits increased with proximity to the exchange period, scheduled each Friday. Most of the deposits occurred in close proximity to exchange, showing that token-reinforced behav- ior was under temporal control of exchange opportunities. Figure 4b shows data from an analogous

experiment with rats (Waddell, Leander, Webbe, & Malagodi, 1972), in which lever presses produced marble tokens according to an FR 20 schedule and exchange periods according to FI schedules. The FI schedules varied para- metrically across conditions (1.5, 4.5, and 9 min), with a representative cumulative record from each shown in Figure 4b. The rate of token-reinforced behavior tended to increase across the exchange-production interval, in proximity to the exchange periods. That simi- larities in the temporal patterning of token

TIMOTHY D. HACKENBERG406

production occurred in the Waddell et al. (1972) and Phillips et al. (1971) studies, despite differences in responses (lever pressing, money deposits), tokens (marbles, points), spe- cies (rats, humans), and settings (laboratory, group home), suggest that exchange-schedule effects have wide generality. Further evidence of exchange-schedule

effects in token economies comes from studies in which exchange-production schedules are directly manipulated. For example, in spite of generally similar temporal distribution of token production across the different FI curves in Figure 4b, increasing the FI value tended to decrease the overall response rates, mainly through increasing pausing early in the interval. This type of inverse relationship between

response rates and FI exchange-schedule dura- tion has also been seen with FR exchange schedules (Bullock & Hackenberg, 2006; Foster et al., 2001). Applied research in which exchange-schedule variables were manipulated in this fashion would make a signi!cant contri- bution to basic understanding of exchange- schedule effects, expanding the generality of this functional relationship across species and settings. It would also hold important practical signi!cance, as a method for identifying opti- mal schedule values to use in applied token economies. Another way to manipulate exchange-

schedule variables is to compare behavior under !xed and variable schedules. In general, response rates tend to be higher under variable

Figure 4. (a, left panel): cumulative deposits in savings account across days of week in a residential token economy with delinquent youth. Different panels indicate conditions in which savings were reinforced (Points) or not (Baseline); arrows indicate exchange periods (from Phillips et al., 1971). (b, right panel): cumulative token-production lever presses per minute across time in a laboratory token economy with rats. The different panels include records from three exchange-production schedules: 1.5, 4.5, and 9 min FI schedules; circles indicate exchange periods (from Waddell et al., 1972). Both !gures reprinted with permission.

407TOKEN REINFORCEMENT

than under otherwise comparable !xed sched- ules. This has been seen in laboratory research with both simple !xed and variable schedules (Mazur, 1983), as well as with !xed and vari- able exchange-production schedules in token reinforcement procedures (Foster et al., 2001; Webbe & Malagodi, 1978). It has also been shown in a classroom token economy with sixth graders (McLaughlin & Malaby, 1972). Points could be earned for assignment comple- tion and lost for inappropriate behavior. Points could be exchanged for functionally de!ned terminal reinforcers (mainly classroom privi- leges) scheduled after either a !xed or variable number of days. Assignment completions in both token conditions were superior to control conditions in which the privileges were not contingent on assignment completion, showing that the mere presence of the token economy was not suf!cient to maintain academic gains; the tokens had to be contingent on the target behavior. Although the variable schedule (tech- nically, a variable-time, or VT) seemed to pro- duce even stronger effects than the !xed-time (FT) schedule, this was evident in only a subset of students with inconsistent academic perfor- mance; the others were already performing at high levels under the FT exchange schedule. The effects of the VT schedule were thus obscured by a ceiling effect. Because the VT schedule had a slightly lower

mean requirement than the FT schedule (4.5 vs. 5.0 days), however, the effects of the vari- able scheduling could not be completely disen- tangled from the relatively shorter exchange delay for the VT exchange-production sched- ule. To remedy this, the authors conducted a follow-up study that equated the FT and VT exchange schedules (McLaughlin & Malaby, 1976). Each was equal to 5 days, but the VT exchange was scheduled according to a variable distribution with an equal likelihood of 3, 5, 7, and 9 days between exchange periods. Using a combined reversal and multiple-baseline design across responses (math, language, and

spelling), both FT and VT exchange schedules generated high rates of assignment completion. Similar to the earlier study, the VT schedule reduced variability, producing near perfect levels of completion (though the effects were again obscured by ceiling effects). Following the second VT 5 was a higher-valued VT (mean = 13.2 days, range = 8-21 days) with a minimum delay exceeding the mean of the other conditions. Assignment completion increased from 88%-100% during FT exchange to nearly 100% during VT exchange (including the VT 13.2). Tarbox, Ghezzi, and Wilson (2006) also

demonstrated the importance of the exchange- production schedule in the context of a thor- ough within-subject experimental analysis of token-reinforcement effects with a 5-year-old boy with autism. The response of interest, attending to the therapist, was examined sys- tematically as a function of token reinforce- ment: Each response produced a sticker later exchangeable for the terminal reinforcer— periods away from the session and access to preferred items. Attending occurred on nearly 100% of opportunities under token reinforce- ment conditions, and at low levels under base- line conditions when the token reinforcement schedule was inoperative. Responding was sus- tained across substantial increases in the exchange-production ratio, from an initial FR 10 up to FR 50 (i.e., 50 tokens required to produce the exchange period). Token-exchange schedules. The Tarbox

et al. (2006) study went on to demonstrate the importance of two additional variables: pairing and token-exchange schedule. Results from these conditions are summarized in Figure 5, which shows the percentage of trials with attending behavior across a series of reversals comparing tokens paired or not with the termi- nal reinforcer (designated TR, for Token Rein- forcement and NB for No Backup, in the !gure). In both conditions, the exchange- production schedule was !xed at FR 20. In the

TIMOTHY D. HACKENBERG408

NB conditions, social praise followed the exchange but no terminal reinforcer occurred. Removing the terminal reinforcer led to marked reductions in attending, but attending quickly recovered when a token reinforcement schedule was reinstated, demonstrating func- tional control by this variable. Then, with the exchange-production sched-

ule held constant at FR 20, exchange opportu- nities were delayed according to a !xed-time (FT) schedule, which increased in 5-s incre- ments across blocks of sessions, up to delays of 190 s. In these FT token-exchange schedules, tokens could be produced but could not be exchanged until after the delay. Attending decreased but was not eliminated as exchange opportunities were delayed; even at FT dura- tions of 190 s, attending occurred at approxi- mately 40% of the FT 0-s delay levels. Attending recovered to high levels when the

token reinforcement contingencies were reinstated. A more extensive parametric analysis of

delayed reinforcement effects was conducted by Leon, Borrero, and DeLeon (2016) in a token reinforcement context with three individuals with intellectual disabilities. Three conditions were compared: (a) delayed access to preferred food (no token, delayed food); (b) delayed access to tokens exchangeable for preferred food at the end of the session (delayed token, delayed exchange/food); and (c) immediate access to tokens exchangeable for preferred food at the end of each trial (immediate token, delayed exchange/food). The delays were sys- tematically increased across successive sessions until responding dropped to low levels. For the two participants who completed all three condi- tions, responding weakened most quickly under delayed token delivery, reaching low levels with

Figure 5. Percentage of trials with a target (attending) response for one child with autism across token reinforcement conditions: baseline (BL), token reinforcement (TR), schedule thinning (ST), no terminal/backup reinforcer (NB), and delayed exchange (DR) (from Tarbox et al., 2006). See text for additional details. Reprinted with permission.

409TOKEN REINFORCEMENT

6-s delays, and remained relatively strong under the other two conditions until longer delays were reached. Delay tolerance (delay values at which responding dropped to low levels) was slightly higher under delayed food than under delayed exchange for one participant, whereas for the other, delay tolerance was slightly higher for delayed exchange than for delayed food. The !ndings from the delayed exchange con-

dition are consistent with what is known about effects of delayed exchanges under laboratory conditions. In one early experiment with chim- panzees by Wolfe (1936), response mainte- nance under delayed exchange conditions was compared across four conditions, in which a response produced (a) delayed food (no token, delayed food); (b) immediate access to a poker- chip token exchangeable for food after a delay (immediate token, delayed exchange/food); (c) immediate access to a brass token that was not exchangeable for food (immediate token/ unpaired); and (d) immediate access to a poker- chip token and deposit but a delay to food. Within each condition, the delays increased progressively with each earned reinforcer, a progressive-interval (PI) schedule. Only condi- tion (b) sustained responding, reaching break- points (or last completed ratios) of 20 and 60 min for two subjects and 60+ for the other two (that continued to respond at the maxi- mum delay studied). The other three condi- tions generated weak and inconsistent responding, with breakpoints rarely in excess of 3 min. It is worth noting that the conditions that

generated the strongest behavior (immediate tokens, delayed exchange) were also those in place in the Tarbox et al. (2006) study and in some conditions in the Leon et al. (2016) study, in which the student produced tokens but had to tolerate increasingly longer delays to the exchange period. These procedures could readily be adapted to include conditions like those used by Wolfe (1936) to systematically identify the optimal arrangement of token-

production and exchange contingencies. The procedures might also be expanded to include generalized tokens rather than the speci!c tokens used by Leon et al. (2016). Identifying ways to enhance exchange delay tolerance has obvious applied signi!cance; it has theoretical implications as well, bearing on fundamental issues such as the necessity of token–food pair- ings in maintaining token-reinforced behavior. Antecedent functions revisited. In the research

reviewed in these prior sections, various ante- cedent functions are dif!cult to disentangle. As part of an extended chained schedule, a token is both paired with a terminal reinforcer and occasions responses that occur in its presence. As such, tokens will normally have multiple functions, the relative in"uence of which is likely to vary across conditions. A laboratory study by Bullock and Hacken-

berg (2015) attempted to disentangle these multiple stimulus functions of tokens. The tokens were lights arranged in a horizontal row above the response keys and the subjects were pigeons, well experienced with token econo- mies. Two-component multiple schedules were used to compare responding in token schedules to responding in otherwise identical (tandem) schedules but without tokens present. In the token components, tokens were earned accord- ing to a FR 50 token-production schedule and exchanged according to a FR x exchange- production schedule (in which x varied across conditions: 2, 4, and 8). In the tandem compo- nents, an equivalent number of responses were required to produce food reinforcers, but in the absence of tokens. So, for example, in the com- parison with the FR 4 exchange-production schedule, 200 responses were required to pro- duce four food reinforcers in the tandem com- ponent. These tandem components served as a control for the added stimuli in token proce- dures. If the tokens are serving primarily rein- forcing functions, one might expect higher responding in the token components than in the tandem components, owing to the presence

TIMOTHY D. HACKENBERG410

of four additional conditioned reinforcers. Overall response rates were in fact consistently lower in token than in tandem components, a !nding that is consistent with prior results comparing behavior in chained and tandem schedules of food reinforcement (Jwaideh, 1973). Such differences strongly suggest dis- criminative effects, namely, weak responding in the presence of stimuli early in the chain (i.e., temporally distant from the terminal reinforcer). It is worth noting that the rate differences

between token and tandem were not merely a product of weak responding in the initial links in the absence of tokens, but rather, to lower rates across the entire link. Figure 6 shows run

rates (responses with preratio pauses subtracted out) under token and tandem components across links within the chain at all three exchange-production ratios. The functions are graded in proximity to the exchange period, re"ecting temporal control, but are displaced downward for the token component, re"ecting discriminative control by the tokens. A later experiment showed that similar such patterns were evident even in the signaled absence of a contingency between responding and token presentation, albeit at somewhat lower levels. Thus, tokens can potentially serve one of three functions—reinforcing, discriminative, or eliciting—depending on the particular arrange- ment of contingencies.

Figure 6. Token-production response rates across token-production segments for three pigeons in token (!lled sym- bols) and tandem (open symbols) schedules (from Bullock & Hackenberg, 2015). The different columns represent dif- ferent exchange-production schedules: FR2, FR4, and FR8. See text for additional details. Reprinted with permission.

411TOKEN REINFORCEMENT

Although no comparable studies from the applied realm exist, including tandem control conditions alongside token reinforcement pro- cedures can be bene!cial in disentangling con- trol by conditioned reinforcement from control by discriminative stimuli. Such multiple control is not limited to token reinforcement, but is true of conditioned reinforcement more gener- ally, both in the laboratory (Shahan, 2010; Williams, 1994) and in applied settings, as dis- cussed earlier (Holth et al., 2009). Token rein- forcement procedures that separate these different stimulus functions may thereby aid in a more general understanding of conditioned reinforcement and discriminative control.

Aversive Functions If token gains serve reinforcing functions, to

what extent do token losses serve aversive func- tions? The two main ways to assess the aversive functions of a stimulus event are via punish- ment (response-contingent loss) and negative reinforcement (response-contingent reduction in losses). In a seminal early laboratory investi- gation of token-loss negative reinforcement, Weiner (1963) studied adult human subjects on avoidance and escape procedures under lab- oratory conditions. The aversive events were point-loss periods (PLPs), or periods of signaled token loss. Responding was established and maintained across three components in a multi- ple schedule: avoidance, in which responses postponed the upcoming PLP; escape, in which responses terminated the PLP; and escape/avoid- ance, in which both termination and postpone- ment were possible. Responding was established and well maintained across all three components, and adjusted to the local contin- gencies: just after PLP onset in escape, and prior to PLP onset in avoidance. On the whole, the response patterns are impressively consis- tent with those seen with other species and other aversive stimuli, suggesting that token losses are functionally equivalent with other

aversive stimuli (Azrin & Holz, 1966; Pazuli- nec, Meyerrose, & Sajwaj, 1983). Additional evidence for the aversive func-

tions of token loss comes from the aforemen- tioned study by Phillips et al. (1971). In one experiment, the target was promptness for din- ner; tokens (points tallied on index cards) were earned for appropriate behavior and lost for inappropriate behavior. Tokens could be exchanged for privileges (e.g., hobbies and games, allowance). A punishment contingency was instituted, such that 100 tokens were lost for each minute late. The contingency was effective in reducing lateness from about 10 min to 0-1 min in an ABAB design. Threats of losing points had an initial but tran- sient effect; only when lateness cost tokens was it reduced. A second experiment involved negative rein-

forcement contingencies: If 80% of the avail- able points were earned, points were given for the day; if less than 80% of the points were earned, points were subtracted. Corrective feed- back was also given. This contingency was effective in producing and sustaining a high level of behavior relative to a second condition with no points. Reinstating the point contin- gency brought behavior back to high levels. The contingency was then faded, !rst by removing corrective feedback, then by decreas- ing the proportion of days on which the con- tingency was in effect, while holding reinforcement magnitude constant. High levels of behavior were maintained across the fading and 6-month follow-up conditions. Comparing token-loss with other aversive con-

trol procedures. An important question sur- rounding the aversive functions of token loss is how token loss compares to other aversive stim- uli. In an early systematic analysis of this ques- tion, Burchard and Barrera (1972) compared punishment via token loss to comparable time- out (TO) procedures on antisocial behavior in adolescents diagnosed with intellectual disabil- ities in an institutional setting. The magnitude

TIMOTHY D. HACKENBERG412

of the punisher was manipulated across groups of participants: smaller magnitude (!ve tokens or 5-min TO) versus larger magnitude (30 tokens or 30-min TO). The results can be seen in Figure 7, which shows the overall fre- quency with which each type of consequence was delivered, averaged across participants in each group. (Although the y-axis label reads “total timeouts,” a timeout referred to either a response-cost !ne or to the more traditional de!nition of removal from the setting.) Both types of punishment produced magnitude- related reductions in the rate of each conse- quence, re"ecting the rate of responding. Greatest suppression was seen under the larger magnitude conditions, consistent with punish- ment magnitude effects seen in laboratory

research with humans (Weiner, 1962). It is also broadly consistent with shock-punishment effects seen in a wide range of nonhuman spe- cies (Azrin, 1960). The consistency in the func- tional relationship between response suppression and punishment magnitude strongly shows that token losses are effective punishing events. Along with TO procedures, token loss proce-

dures are negative punishment procedures, in that the aversive functions arise from the removal of positive reinforcers. Thus, unlike positive punishers (e.g., shock, social disap- proval), token-loss contingencies depend on the reinforcing value of token gain. That is, because losses, by de!nition, reduce the overall rate of token reinforcement, the response- suppressive effects of punishment are dif!cult to tease apart from the response-weakening effects of (sometimes marked) reductions in reinforcement rate or quality. In the Burchard and Barrera (1972) study, for example, the point-loss (and TO) contingencies also reduced overall reinforcement, making it dif!cult to iso- late controlling variables. Disentangling these different sources of control requires control procedures in which token losses are separable from concomitant changes in overall reinforce- ment density. No applied research on token loss has been

directed to isolating punishment effects from underlying changes in reinforcement rate, but some laboratory research may provide a starting point. In a laboratory-based token economy with pigeons, Raiff, Bullock, and Hackenberg (2008) assessed token-loss punishment across two components of a multiple schedule. A stan- dard token reinforcement schedule operated in both components in baseline no-loss conditions. Under loss conditions, a token-loss punishment contingency operated conjointly with the token-gain schedule in one component, and only the gain contingency operated in the other component. That is, in loss conditions, responses both produced and subtracted

Figure 7. Frequency of timeouts (TO) across time and procedure type: token loss (circles) and timeout (tri- angles); small magnitude (!ve tokens, 5-min TO) and large magnitude (30 tokens, 30-min TO) punishers are represented by open and closed symbols, respectively (from Burchard and Barrera, 1972). Reprinted with permission.

413TOKEN REINFORCEMENT

tokens, whereas in no-loss conditions, responses only produced tokens. Responding was substan- tially reduced in the loss component, relative to the no-loss component, suggesting a punishing function. The conditions also resulted in sub- stantial reductions in the overall number of tokens exchanged for food, however. To tease apart the in"uence of these variables, a subse- quent control condition provided noncontin- gent food, yoked to the obtained rate in the no- loss component. Responding was somewhat lower in the yoked-food than in the baseline no- loss conditions, re"ecting some effects of reduced reinforcement rate, but was even lower in the loss condition, re"ecting the additional suppres- sive effects of punishment. Thus, even with pos- itive reinforcement variables held constant, token loss appears to function as an effective punisher (see also Pietras & Hackenberg, 2005, for analogous !ndings using different yoked- control methods). Similar effects have been reported with TO as an aversive event: respond- ing can be maintained by TO postponement even in the absence of increases (and sometimes in spite of decreases) in overall food reinforce- ment rate (Pietras & Hackenberg, 2000). Together with the !ndings discussed earlier,

these !ndings support a view of tokens as con- ditioned reinforcers and token losses as condi- tioned aversive stimuli. These laboratory !ndings also point to some critical control con- ditions needed to separate important behavioral functions. Such procedures could easily be adapted to applied research. Indeed, presenting tokens irrespective of behavior (such as the yoked-control methods of Raiff et al., 2008) is a variant of a commonly used procedure in applied research—noncontingent reinforcement (NCR; Phillips, Iannaccone, Rooker, & Hago- pian, 2017). Incorporating NCR-like proce- dures as control procedures into token-loss punishment programs will help reveal key behavioral functions of tokens. Comparing token-loss with token-gain proce-

dures. Another important question surrounding

token-loss procedures is how they compare to token-gain procedures. Although many token economies involve combinations of loss and gain, a few studies have directly compared the relative ef!cacy of loss and gain procedures. In an early study along these lines, Kaufman and O’Leary (1972) examined token reinforcement and punishment effects in a classroom token economy. Tokens could be earned for appro- priate behavior or lost for disruptive behavior. One group of students could earn up to 10 points per class period, whereas a second group could avoid losing up to 10 points. Both sets of contingencies (loss and gain) produced clear and systematic reductions in disruptive behavior; one was not appreciably superior to the other. Similarly, Iwata and Bailey (1974) compared

token reinforcement and punishment effects on academic performance in a classroom token economy. Unlike the Kaufman and O’Leary (1972) study, in which gain and loss conditions were varied across groups, the procedures here alternated on a within-subject basis, with order counterbalanced across groups of students. Thus, students in each group received both loss and gain conditions, but in a different sequence, along with interspersed baseline con- ditions with neither contingency in effect. Sim- ilar to Kaufman and O’Leary, both procedures were about equally effective in reducing disrup- tive behavior. More recently, Donaldson, DeLeon, Fisher,

and Kahng (2014) compared token-loss and token-gain contingencies on disruptive behavior in a classroom setting. Loss and gain conditions were varied in a multielement design, with the overall earning potential in each condition held constant at 10 tokens. A time-sampling method was used whereby students either received a token (check mark) for appropriate behavior, or had one removed for inappropriate behavior, under gain and loss conditions, respectively. Tokens were later exchangeable for tangible and edible reinforcers. Similar to prior research,

TIMOTHY D. HACKENBERG414

both procedures reduced disruptive behavior to comparably low levels, and no adverse collateral behavior was reported. On average, participants earned slightly more tokens in, and displayed a modest preference for, the token-loss than the token-gain conditions. The preferences for loss over gain procedures parallels previous !ndings (Hanley, Piazza, Fisher, & Maglieri, 2005), raising at least some doubts about negative side effects of the token-loss procedures. Collateral effects of token loss. One consider-

ation surrounding the use of aversive control procedures, more generally, is the incidence of collateral effects (e.g., emotional behavior, escape, social disruption) observed with nonhu- man animals and noxious stimuli such as elec- tric shock (Azrin & Holz, 1966; Hutchinson, 1977). How prevalent are such effects with token loss, and how do they compare with other aversive stimuli? One common collateral effect of some aver-

sive stimuli is an increase in physiological arousal (Brady, Kelly, & Plumlee, 1969). Does loss produce similar emotional arousal? Although there are few studies along these lines, an early experiment by Schmauck (1970) compared autonomic arousal just after shock, token loss, and social disapproval. Autonomic responses were elicited by shock but not by the other two aversive stimuli; all were judged equal in subjective reports of anxiety. Thus, despite the equivalent subjective effects, the physiological response pro!le differed, with token loss producing no change in physiologi- cal responding. Although further study is clearly needed, the limited available evidence suggests that token losses do not necessarily give rise to the negative emotional responses seen with electric shock. Thus, emotional responding does not seem to be an inevitable or perhaps even likely accompaniment of token-loss procedures. Another commonly reported side effect of

aversive events is escape from the situation in which the aversive events are occurring

(Azrin & Holz, 1966). The study by Boren and Coleman (1970), mentioned earlier, reported negative side effects associated with a token-loss procedure with soldiers in a psychiat- ric ward. When tokens were contingent upon attendance at a morning meeting, most but not all soldiers attended. When penalties, in the form of token losses, were introduced for absences while holding the reinforcement con- stant, however, attendance decreased markedly—the opposite of a punishment effect. In addition to poor attendance, the authors reported clear changes in other aspects of behav- ior. The subjects acted more rebelliously (e.g., encouraging others to miss the meeting) and generally more aggressively with each other and with staff. When the point-loss punishment procedure was discontinued in a subsequent condition, attendance increased to previous levels and the punishment-induced aggressive behavior quickly extinguished. In this case, at least, the indirect effects of the token-loss con- tingency were large and generally detrimental in that they opposed the target behavior. Aversive control has also been known to pro-

duce social disruption. An illustrative case is provided by a study conducted in a residential laboratory setting (Emurian, Emurian, & Brady, 1985). Participant triads (three-person groups) lived together for up to 3 weeks, com- pleting work units for points later exchangeable for money. Points contributed to a group account, with earnings divided evenly among the three participants at the end of the study. Positive and negative reinforcement contingen- cies were compared across blocks of days on a within-triad basis. In positive reinforcement conditions, each completed work unit added a token to the group account; in negative rein- forcement conditions, satisfying a !xed require- ment (yoked to earnings from the prior positive reinforcement condition) prevented reductions in accumulated earnings. Four triads were studied with repeated rever-

sals across blocks of daily sessions to

415TOKEN REINFORCEMENT

demonstrate functional control. Despite roughly similar levels of overall productivity under the two sets of contingencies, major dif- ferences in collateral behavior were observed. In positive reinforcement conditions, the partici- pants worked together well and were socially amicable, earning and exchanging a high rate of tokens for other reinforcers. In negative rein- forcement conditions, on the other hand, there were marked increases in negative verbal responses about the experiment. Figure 8 shows mean daily ratings on a 4-point scale (ranging from 1, not bothered by the experiment, to 4, extremely bothered by the experiment) on negative reinforcement (avoidance) and positive reinforcement (appetitive) days. Ratings were consistently more negative under negative rein- forcement conditions. Similar patterns were evi- dent in analogous scales rating the experimenters and other participants. In one group (G-4, in Figure 8), these changes in ver- bal behavior coincided with decrements in pro- ductivity. One participant stopped working altogether, and another reduced productivity under negative reinforcement conditions, both

in response to a third group member whose productivity rate was insuf!cient to meet the performance criterion. The authors interpreted such negative collateral behavior as a side effect of aversive control, analogous to those seen with nonhuman animals and noxious stimuli such as electric shock. It is worth noting, however, that collateral

behavior is not always produced by token-loss procedures. Neither Kaufman and O’Leary (1972) or Iwata and Bailey (1974) found evi- dence of negative collateral behavior; indeed, both studies reported positive indirect effects of the token-loss procedures: increases in academic output (not speci!cally targeted by the token- loss contingency). Clearly, the effects of point- loss procedures, including the incidence of col- lateral byproducts of such procedures, are a complex function of many variables. In some cases, token-loss procedures produce effects comparable to token-gain procedures and are free of unwanted collateral effects. In other cases, token-loss procedures produce similar direct effects but negative collateral effects. In still other cases, token-loss procedures produce weaker direct effects along with negative collat- eral effects. The variability in !ndings may be due in

part to different subsets of contingencies oper- ating within the token economy. One differ- ence between the studies was in how the tokens were presented. In the studies reporting mainly positive effects of point-loss contingen- cies, the tokens were presented noncontingently at the beginning of a class period; they were then subject to loss for speci!ed responses. In the studies reporting weaker and negative effects, on the other hand, the tokens removed according to the token-loss procedure had pre- viously been earned. Why should this make a difference? Loss aversion. One possibility is that this dif-

ference in how tokens are presented (whether they are given freely or have to be earned) in the token economy may alter the value of the

Figure 8. Mean daily verbal ratings on a 4-point scale (ranging from 1, not bothered by the experiment, to 4, extremely bothered by the experiment) on negative reinforcement (avoidance) and positive reinforcement (appetitive) days for four social groups of three subjects apiece (from Emurian et al., 1985). Reprinted with permission.

TIMOTHY D. HACKENBERG416

tokens. A token earned, as opposed to one freely given, may be more highly valued. A study by Miller, DeLeon, Toole, Lieving, and Allman (2016), for example, found that partici- pants who earned their own tokens were more risk averse (less likely to gamble those tokens) than participants given noncontingent (free) tokens. These !ndings, along with those from a similar study (Brandt, Sztykiel, & Pietras, 2013), suggest that the contingent delivery of tokens may enhance their value relative to those freely delivered. Perhaps at least some of the negative collateral effects reported by Boren and Colman (1970) and Emurian et al. (1985), when point- loss conditions were instituted, re"ected a similar kind of value adjustment, in which tokens in a loss context are weighted more heavily than the same tokens in a gain context. This tendency to weigh losses more than

gains may be related to the literature on the so- called endowment effect—the tendency to value more a commodity in one’s possession (to which one is endowed) to the same com- modity at a price. In a prototypical experiment (e.g., Kahneman, Knetsch, & Thaler, 1990), participants engage in a market exchange for two items of equivalent value: candy bars and coffee mugs. Despite being randomly assigned, the people tend to place greater value on items in their possession, charging more to sell the item than they were willing to pay for the same item. The endowment effect is normally con- ceptualized as re"ecting loss aversion—the ten- dency to value losses more heavily than gains. Loss aversion has been empirically demon- strated across a wide range of studies in various decision-making domains, and is considered theoretical bedrock in some prominent approaches to behavioral economics, such as Prospect Theory (Kahneman & Tversky, 1979). Yet the studies to date are based almost entirely on verbal-hypothetical scenarios with adult humans (but see Levitt, List, Neckermann, & Sadoff, 2012, for an interesting exception), which raises questions about their applicability

both to laboratory-based and everyday situa- tions with repeated choices and real outcomes. Token systems permit a more general

approach to assessing the relative value of gains and losses, appropriate to either verbal or non- verbal tasks. That roughly comparable results are produced in gain and loss contingencies (Donaldson et al., 2014; Iwata & Bailey, 1974; Kaufman & O’Leary, 1972) suggests losses and gains are equally weighted—a result broadly inconsistent with loss aversion. On the other hand, the modest preferences for token loss over token gain reported by Donaldson et al. (2014) may be viewed as consistent with loss aversion. Similarly, in a laboratory-based test of loss aversion, Rasmussen and Newland (2008) found evidence supporting loss aversion. Adult humans made repeated choices between two schedules of reinforcement (token gain), one of which sometimes also included punish- ment (token loss). Punishment conditions alter- nated with no-punishment conditions across blocks of sessions. If losses loom larger than otherwise equivalent gains, then strong bias for the unpunished alternative would be predicted. This is indeed what happened. Such dispropor- tionate effects of token loss are consistent with a loss aversion account. An earlier laboratory study by Critch!eld,

Paletz, MacAleese, and Newland (2003), how- ever, did not !nd evidence of loss aversion. Like Rasmussen and Newland (2008), they gave humans repeated choices between sched- ules of token gain and loss. The conditions were arranged in such a way as to evaluate the predictive accuracy of a punishment model that views losses and gains as symmetrical but oppo- site in their effects. Across a broad parametric range of conditions, preferences were generally well aligned with the predictions of the model, supporting a version of the symmetrical law of effect dating back to at least Thorndike (1911), and counter to loss aversion. Although the few studies designed to assess

the symmetry between losses and gains have

417TOKEN REINFORCEMENT

yielded mixed results, token economies are especially well suited to functionally scaling losses and gains along a common dimension, as losses and gains can be de!ned in conceptually analogous terms, as common currency units of loss or gain. And with currency units more like those of monetary currencies, token procedures may provide a bridge between laboratory-based choice procedures used in behavior analysis and those used in other behavioral economic approaches. Such token-based procedures with humans would also connect to token-based procedures with other animals, facilitating cross-species comparisons more generally (see Hackenberg, 2009).

APPLIED BEHAVIORAL ECONOMICS

In a series of papers published between 1970 and 1980, Kagel, Winkler, and colleagues encouraged a view of token systems as experi- mental economies (Battalio et al., 1974; Kagel, 1972; Winkler, 1973, 1980). Kagel and Winkler (1972), who were credited with having coined the term behavioral economics (Madden, 2000), argued that token systems were ripe for a behavioral economic analysis. Tokens are earned for completion of some task and exchanged for access to commodities. In eco- nomic terms, token production is analogous to a wage, exchange opportunities to procurement costs, exchange rate to the price of a good, accumulation to savings, and terminal rein- forcers to expenditures. Using this type of analysis, several early stud-

ies, conducted mainly in institutional settings, found broad support for economic demand the- ory—a family of models and concepts con- cerned with how consumption of a commodity changes with price and the availability of other commodities (Winkler, 1971, 1972). Overall, the results were consistent with those obtained in other token economic research (Battalio et al., 1974; Schroeder & Barrera, 1976). They are also in general agreement with econometric

data on consumption patterns in the national economy, supporting the general view of token systems as analogues of real economic systems. Despite the promise of this work, however,

an economic approach to token systems did not gain a strong foothold in applied behavior analysis at the time. And while behavioral eco- nomic methods and concepts have been used productively in applied behavior analysis research over the past two decades (Bickel & Vuchinich, 2000; DeLeon, Iwata, Loh, & Worsdell, 1997; Francesco, Madden, & Bor- rero, 2009; Madden, 2000; Roane, Call, & Fal- comata, 2005; Tustin, 1994), little of this work has focused on token economies per se. Combining an economic approach with token systems has great translational appeal, poised to yield rapid advances in the analysis and application of token reinforcement effects. In this section, I review some !ndings relevant to an economic account, and consider some promising areas of research and application sug- gested by an applied behavioral economics.

Different Price Measures In a token economy, wages correspond to

the token earnings—the number of tokens per unit of work (labor). A straightforward way to manipulate wages in a token economy is via changes in reinforcement magnitude. This has emerged as a powerful variable in contingency management programs, in which vouchers (tokens) are earned for drug abstinence. Pro- grams with higher wages (monetary incentives) tend to produce better treatment outcomes (Dallery, Silverman, Chutuape, Bigelow, & Stitzer, 2001; Levitt et al., 2012). Similar wage effects have been reported in token economies in institutional settings. Fisher (1979), for example, altered the wage rate in a token econ- omy with 16 adults in an institutional mental health ward. The target response, brushing teeth, was compared across baseline (no token) and two different magnitudes of token

TIMOTHY D. HACKENBERG418

reinforcement (one vs. !ve tokens) in an A-B- A-C-A within-subject design. Relative to base- line (no-token) conditions, brushing teeth increased under both token reinforcement con- ditions, and was sensitive to wage rates, increas- ing to a greater extent in the high-wage (5 token) than in the low-wage (1 token) conditions. In addition to labor, costs can be de!ned in

terms of the work to produce an exchange opportunity (procurement costs) and the number of tokens to exchange for a terminal reinforcer (prices). These distinctions are critical in under- standing reinforcement effects in token systems, as there is clear evidence from schedule research (summarized above) that token-reinforced behavior is jointly determined by all three types of costs. When economic analyses are applied to behavior, costs are generally labor costs—the number of responses to produce a reinforcer— but with a token system, it is possible to de!ne costs in terms of common currency units (tokens)—the price (number of tokens) one is willing to pay for it. This more closely resem- bles real-world economic systems, in which money is used to purchase different reinforcers, and one can infer relative reinforcer value from expenditures. Of perhaps greater importance is that token systems provide the basis for a com- mon scale for comparing qualitatively different reinforcers in a quantitatively precise way—a major aim of applied behavioral economics.

Quantifying Reinforcer Value Among the main strengths of an economic

approach to behavior are tools for identifying and quantifying different reinforcers. The prin- cipal method for quantifying reinforcer value from an economic standpoint is the demand analysis: consumption as a function of price. Demand for commodities highly sensitive to price changes (i.e., greater than proportional changes in price) are said to be elastic, whereas those less sensitive to price changes (i.e., less

than proportional changes) are inelastic. Rein- forcers are typically characterized by mixed elasticity—they are less elastic at low prices and more elastic at higher prices. For example, demand for a highly valued good such as coffee might be inelastic at relatively low prices, but elastic if the price rises enough. The resulting response output (work) functions are bitonic— increasing over the inelastic portion of the curve before eventually declining at higher prices. Considering the mathematical characteristics

of demand functions for different reinforcers can yield important information about optimal schedule values for generating peak response output. For example, in early investigation along these lines, Tustin (1994) analyzed the reinforcing value of visual, auditory, and social reinforcers with young adults with moderate to severe intellectual disabilities. Social reinforcers consisted of brief bouts of teacher attention and sensory reinforcers consisted of brief periods of computer-generated auditory, visual, or combined auditory and visual stimuli. For one participant, demand functions were obtained separately for social and sensory rein- forcers. Consumption (reinforcers earned) and work output (response rate) was similar for both reinforcer types at the lowest price (FR 1). As the price (FR size) was increased, both con- sumption and work output declined for both social and sensory reinforcers, but the slope of the function was shallower for sensory than for social reinforcers. In economic terms, demand for sensory reinforcers was less elastic than for social reinforcers, suggesting higher relative reinforcer ef!cacy. Such differences in rein- forcer ef!cacy have important practical implica- tions, as typical reinforcer assessments use low cost (typically FR 1) procedures and could thus mask potentially signi!cant differences in rein- forcer value that emerge at higher prices. For example, in this case, probing only the FR 1 price would provide a misleading picture of the relative reinforcing ef!cacy of the two

419TOKEN REINFORCEMENT

reinforcers. Roane and colleagues (Roane, 2008; Roane, Lerman, & Vorndran, 2001) made a similar point regarding the use of pro- gressive ratio (PR) schedules as assessment tools. Similar demand methods have been applied

to token reinforcement contingencies. Tan and Hackenberg (2015), for example, generated demand functions for speci!c and generalized reinforcers in a token economy with pigeons. The pigeons could earn speci!c tokens (exchangeable for either food or water) or gen- eralized tokens (exchangeable for both food and water). Production of all three token types (food-speci!c, water-speci!c, and generalized) declined systematically with price, consistent with a demand analysis. Demand curves for all three token types were well described by Hursh and Silberberg’s (2008) essential value model, which has proven successful in a range of exper- iments with a variety of species and reinforcers (Barrett & Bevins, 2012; Bentzley, Fender, & Aston-Jones, 2013; Cassidy & Dallery, 2012; Christensen, Silberberg, Hursh, Huntsberry, & Riley, 2008). In addition to the similarities, there were also

some differences between the token types. Pro- duction was less elastic for food-speci!c tokens than the other two token types, but more gen- eralized than speci!c tokens were produced at the lowest price. Thus, as in the Tustin (1994) study, demand elasticity was not predictable on the basis of production at the lowest price; these two aspects of reinforcer ef!cacy (con- sumption at low price and sensitivity to changes in price) must therefore be considered separately. In subsequent phases of the study, Tan and

Hackenberg (2015) included additional assess- ments of token reinforcer ef!cacy—PR break- point and preference—to assess the correlation between the different measures. Elasticity from the demand analysis was highly correlated with preference in pairwise choices between the dif- ferent token types. That is, the less elastic the

demand for the token, the more it was pre- ferred, and vice versa. The correlation between preference and PR breakpoints was weaker than that for demand elasticity, however. These results, along with other !ndings with rats (Madden, Smethells, Ewan, & Hursh, 2007), suggest limits on a unitary conception of rein- forcer ef!cacy, based on convergence between the measures. This lack of perfect correspon- dence between different measures of reinforcer value has important implications for applied research, as it is commonly assumed that a rein- forcer assessment conducted under one sched- ule arrangement will generalize to others. These results, however, urge a more cautious, empiri- cal approach to determining when, and under what conditions, reinforcer ef!cacy in one situ- ation predicts ef!cacy in a different situation. Demand analyses can also be applied to con-

currently available reinforcers. In the !nal phase of the Tan and Hackenberg (2015) study, pigeons were given three sets of pairwise choices between the three token types: food versus water, food versus generalized, and water versus generalized. When food tokens were pitted against water tokens, pigeons produced and exchanged approximately 80% of the tokens for food and 20% for water. In the con- ditions with the generalized tokens, preferences depended on whether the alternative was food or water tokens. When the alternative was food tokens, pigeons produced about 80% food tokens and about 20% generalized tokens— which were subsequently exchanged for water. When the alternative was water tokens, pigeons produced about 20% water tokens and about 80% generalized tokens—which were subse- quently exchanged for food. In other words, the generalized tokens were spent on (function- ally substituted for) water in one context and on food in another, enabling roughly equiva- lent levels of food and water consumption across the different choice contexts. This type of functional substitution is key to

understanding the higher relative reinforcing

TIMOTHY D. HACKENBERG420

ef!cacy of generalized token reinforcers. Gener- alized reinforcers can be exchanged for multiple terminal reinforcers and, in this sense, serve as functional substitutes for those reinforcers. The economic concept of substitution is simply a name given to this type of relationship. More than a mere synonym, however, substitution— when linked to a demand analysis—puts a sharper quantitative point on the relationship, permitting one to determine not only whether one reinforcer substitutes for another but also the degree to which it does so. In the Tan and Hackenberg (2015) study, generalized tokens substituted to a higher degree for food (nearly full substitutes) than for water (partial substi- tutes). The type of quanti!cation permitted by an economic analysis is a valuable tool in map- ping the continuum of generalizability dis- cussed above, and for reinforcer assessments in applied work.

Collateral Effects of Token Reinforcement An economic perspective calls attention to the

broader context in which speci!c reinforcement contingencies are embedded, and suggests the need to measure more global changes in the distri- bution of behavior produced by contingency changes. In the Winkler (1970) study, described above, noise and aggression were monitored but not speci!cally targeted for reinforcement. Reduc- tions in these two patterns of problem behavior accompanied the changes in appropriate behavior produced by the token reinforcement procedure. In other words, noise and aggression were part of a functional class of responses affected by the token reinforcement contingency—what are sometimes called collateral effects of reinforce- ment contingencies. Such response covariation is a critical but often overlooked dimension of behavior-change procedures (see Parrish, Cataldo, Kolko, Neef, & Egel, 1986), sometimes as impor- tant as the main effects of contingencies. Token economies are also ideal contexts in

which to explore collateral effects of

contingencies, including potential adverse effects of reinforcement procedures. Fisher (1979), for example, designed an experiment to assess overjusti!cation effects, de!ned as reduced response output following periods of extrinsic reinforcement (Greene & Lepper, 1974). According to standard interpretations of over- justi!caton effects, the low responding stems from reductions in intrinsic motivation pro- duced by extrinsic reinforcement, reducing behavior to below baseline levels. Another way to think about overjusti!cation effects is in terms of behavioral contrast: a greater-than- expected reduction in responding in transition from a rich to a lean reinforcement context (Reynolds, 1961; Williams, 1983). However they are best conceptualized, such effects, if valid, would pose major challenges to token reinforcement procedures, as they usually entail explicit tangible reinforcement for socially desirable behavior. In a study discussed brie"y above, Fisher

(1979) compared brushing teeth across baseline (no token) and two magnitudes of reinforce- ment (one and !ve tokens). In the comparisons most critical to the overjusti!cation hypothesis (no-token conditions before and after token conditions), levels of brushing teeth in no- token conditions following token reinforcement conditions did not differ from baseline levels. This !nding runs counter to the overjusti!ca- tion hypothesis, which predicts that brushing teeth would drop below baseline levels, owing to the reduced interest in the activity. That nei- ther this study nor others (Feingold & Maho- ney, 1975; McGinnis, Friman, & Carlyon, 1999; Vasta & Stirpe, 1979) have found evi- dence for detrimental effects of token reinforce- ment suggests that token economies may insulate behavior from such effects. Unlike the typical laboratory-based study, in which expo- sure to conditions is extremely brief, the partic- ipants in Fisher’s study were long-time members of the token economy, with variables systematically manipulated over time.

421TOKEN REINFORCEMENT

Moreover, tokens were contingent on job or task performance—variables shown to attenuate the so-called adverse effects of contingent rein- forcement (Eisenberger & Cameron, 1996). These results thus join with the results reported for reinforcement procedures more generally (Eisenberger & Cameron, 1996; Levy et al., 2016), in showing no detrimental effects of reinforcement contingencies.

Economic Context A token economy is a system of interlocking

contingencies, and to properly understand one part of it (e.g., responding on a speci!c proce- dure), it is useful to know more about the broader context surrounding it. One part of this context is the price and availability of other reinforcing events and activities, shown in a previous section to be powerful determinants of demand and preference. In this section, I will touch on other aspects of the economic con- text. In considering the impact of these back- ground economic variables, Winkler and Burkhard (1990) distinguished between inter- nal interdependencies and external interdepen- dencies; the former are factors external to a speci!c procedure but internal to the token economy (e.g., income), whereas the latter are factors external to the token economy (e.g., the broader national economy). Internal interdependencies. An example of an

internal interdependency is income—the num- ber of tokens available for spending. This is a factor that is external to a speci!c contingency but internal to the token economy as a whole. For example, Fisher et al. (1978) showed income to be a powerful determinant of token earning and spending in token economies. For low-income earners, the higher wages produced increases in income, and did not adversely affect other token-earning behavior; they enjoyed a higher standard of living, and worked at about the same rate. For high-income earners, on the other hand, the higher wages

led to reductions in other (lower-wage) token- earning behavior; they maintained a roughly similar standard of living, and worked less on nontarget tasks. The therapeutic implications are clear. The

success of any speci!c intervention (e.g., self- help skills) will depend not only on the speci!c procedure (e.g., token reinforcement), but also on how output and consumption on that job relate to broader patterns of income and savings of the individual within the token economy. If the wage increases for brushing teeth result in less behavior directed toward other activities of applied signi!cance (e.g., academic work), then tradeoffs will need to be made between differ- ent responses, as they relate to the overall goals of the program. The utility of an economic analysis is its explicit focus on the broader changes surrounding speci!c reinforcement contingencies, bringing the tradeoffs into focus. Winkler (1972) discussed how income com-

bines with savings in determining the marginal rate of consumption—the ratio of changes in spending to changes in income. In both token and natural economies, the marginal rate of consumption is linear but with slopes typically less than one, indicating expenditures do not completely keep pace with income. Winkler showed how the marginal rate of consumption is affected both by income and savings (tokens accumulated for later exchange). This has clear applied signi!cance. Given that tokens are usu- ally earned for socially desirable behavior, ways to increase token earning (marginal consump- tion) translates directly into more effective clin- ical practices. Winkler showed how the ef!cacy of token procedures varied with savings (tokens in the bank). The procedures were highly effec- tive for individuals with little savings; with greater levels of savings, however, behavior was less well maintained by the procedures. To account for these differential effects,

Winkler (1971) introduced the concept of criti- cal range of savings (CRS)—a savings threshold below which response output and consumption

TIMOTHY D. HACKENBERG422

will increase and above which will decrease. Manipulations that decrease savings should therefore increase consumption, and this is indeed what happens (Winkler, 1971). When the price of tea was doubled, more was spent on tea. This in turn reduced savings, and pro- duced increments in token-earning behavior. Savings may also be decreased in other ways (e.g., improving the quality of the backup rein- forcers). By pointing to variables that alter CRS, and thereby enhance token-earning behavior, this approach has important applied implications (see below). External interdependencies. In addition to the

internal interdependencies described above, there are external interdependencies—structural factors external to the token economy that may interact with the behavior within the economy. One example of an external factor is the broader economy in which the token system is situated. Winkler (1980), for example, com- pared two different token economies in terms of their interaction with economic factors out- side the setting. In one setting, an institution with long-term patients, the economy was closed, in the sense that tokens could only be exchanged within the setting. Here the econ- omy thrived, with high levels of token earning and spending, little savings, and high levels of marginal consumption. In a second setting, a 98-day experimental program with volunteer marijuana smokers paid to work on a variety of tasks, the token economy was open, in that it interacted with broader economic circum- stances. Tokens could be exchanged for goods (including marijuana) within the program, or could be saved and exchanged for money at the end of the program. This essentially opened the economy to broader in"uences, namely, the money available at the end of the program. Indeed, an increasing portion of earnings went into savings, with corresponding changes in consumption and income. The two settings correspond to different

points on a continuum ranging from relatively

open (the experimental setting) to relatively closed economies (the ward setting). Closed economies are increasingly common in basic behavioral economic research, including much of the research on token reinforcement reviewed above. With a few notable exceptions (Roane et al., 2005), closed economies are somewhat less common in applied behavioral economics. The study by McLaughlin and Malaby (1972), discussed earlier, provides a useful illustrative example of a relatively closed token economy. In this study, sixth-grade stu- dents earned tokens for assignment completion and other appropriate behavior, and lost tokens for inappropriate behavior, with exchange opportunities scheduled every 5-6 days over the course of an entire academic year. With only negligible investment by the teacher, the pro- gram was highly successful in establishing and maintaining high levels of academic perfor- mance over sustained time periods. A distinc- tive feature of the token system was the use of functionally de!ned activities (e.g., caring for the class pet) that occurred only within the classroom setting. This helped maintain a rela- tively closed and self-contained economy in which tokens were earned and spent only within the economy. A practical bene!t of using only reinforcers intrinsic to the setting was that no additional !nancial expenditures were needed. The token economy simply restructured the activities in such a way that preferred objects and activities were made more usefully contingent on academic behavior. The applied implications are substantial.

First, the study utilized functionally de!ned reinforcers. Second, by limiting reinforcers to those available within the classroom setting, the study essentially closed the economy, eliminat- ing interactions with extraclassroom reinforcers. Both of these factors enhance the value of the reinforcers and the ef!cacy of the token econ- omy as a whole. In holding constant factors external to the classroom economy, such class- room token systems provide a behaviorally rich

423TOKEN REINFORCEMENT

and analytically tractable environment in which to explore basic mechanisms of token-economic behavior: token earning, saving, and spending in a controlled and relatively closed economy. These types of settings provide applied labora- tories for studying basic processes.

TRANSLATIONAL IMPLICATIONS

The focus of the paper to this point has been on reviewing what is known about token rein- forcement systems from laboratory and applied perspectives. Although some applied implica- tions have been mentioned along the way, the primary focus has been on identifying gaps in the literature and promising areas for future research. As an overall translational program of research, consisting of (a) basic research, (b) applied research, and (c) practice, the emphasis thus far has been on (a) and (b)—the science of token reinforcement. In this section, I emphasize part (c), the practical implementa- tion of token reinforcement procedures in the light of the science.

Acquisition Research reviewed above shows how much

we still have to learn about the speci!c condi- tions responsible for establishing the various functions of tokens. Nevertheless, a few speci!c suggestions can be made. As with other condi- tioned reinforcers, tokens should occur in a rich reinforcement context, closely coupled with other already-effective reinforcers. Simply pair- ing the tokens with terminal reinforcers may be suf!cient in some cases to establish conditioned reinforcement value, but there are reasons to favor response-contingent pairings of tokens and terminal reinforcers, in which an exchange response is required to access terminal rein- forcers. First, response-contingent pairings (i.e., training a discriminative operant) are more effective than simply pairing the tokens with the terminal reinforcers. Second, there are prac- tical bene!ts as well: Training an exchange

response and bringing it under stimulus control of a token is the initial step in a more general token reinforcement program. It is advisable in the early stages to use FR 1 schedules, when- ever possible, adding links via backward chain- ing until the entire unit of token production and exchange is established.

Maintenance Once token-reinforced behavior has been

established, an important applied concern is with the maintenance of behavior—as exchange periods are often temporally removed from the periods during which tokens are earned. Indeed, increasing this delay between token delivery and token exchange is frequently a goal of token reinforcement programs. Examining ways to sustain behavior in the face of such changes thus holds important practical value. What variables should be taken into account in the maintenance of such behavior? When viewed as an extended chained sched-

ule, token reinforcement procedures consist of a sequence of interconnected schedules. For example, suppose one were to arrange a token program with the following three component schedules: (a) FR 5 token production (!ve aca- demic problems completed produce a star), (b) FR 10 exchange production schedule (10 stars produce an exchange period), and (c) FR 1 exchange schedule (handing over the full token board to a teacher produces 30-s access to a preferred toy train). Suppose one wished to maintain responding while thinning the schedule; which schedule (token production or exchange production) should be changed? Research on schedule effects suggests it will be most effective to raise exchange-production requirements and keep token-production requirements low, rather than the reverse. This was indeed the strategy employed by Tarbox et al. (2006), and perhaps the reason they saw relatively small decrements in responding in the face of large increases in FR exchange price.

TIMOTHY D. HACKENBERG424

This ability to sustain responding in the face of substantial increases in the FR value may be due, in part, to a unique feature of token rein- forcement schedules: The number of tokens produced is correlated with the magnitude of the terminal reinforcer. Each increment in the FR exchange-production price means that more tokens will be available to exchange for the ter- minal reinforcer. Hence, increases in the exchange-production price produce correspond- ing increases in terminal reinforcer magnitude. In the above example, assuming each token could be cashed in for 30-s access to the train, increasing from FR 10 to FR 20 would also increase the toy access time from 300 to 600 s. As a result, despite a two-fold increase in FR, the overall unit price (responses per terminal reinforcer duration) remains constant. A two- fold increase in the token-production FR, on the other hand, would double the unit price, because each increment in the FR size does not produce corresponding increases in terminal reinforcer magnitude. Thus, incrementally rais- ing the exchange-production schedule is likely to sustain higher levels of responding than comparable increases in the token-production schedule. Not all applied token programs are arranged

in such a way that terminal reinforcer amount is correlated with number of tokens earned, however. In some, tokens signal proximity to the exchange but not amount of reinforcement in exchange. This is the case, for example, when the terminal reinforcer magnitude is con- stant, irrespective of the numbers of tokens pre- sent at exchange. In the above example, suppose the terminal reinforcer is 30-s toy access whether the exchange-production sched- ule is FR 10 or FR 20. Unlike standard token schedules with correlated reinforcer magnitude, this type of token schedule is more akin to a typical extended chained schedule, with a single terminal reinforcer. Absent the correlation between token num-

ber and terminal reinforcer amount, the

methodological advantages of manipulating the exchange-production schedule diminish somewhat. Even so, however, there are addi- tional reasons to recommend changing the exchange-production over token-production schedule. For one thing, exchange-production manipulations involve relatively more tokens than otherwise comparable token-production manipulations. That is, target responses con- tinue to produce tokens when the exchange- production ratio is increased, but not when the token-production ratio is increased. As conditioned reinforcers, the added tokens alone should facilitate responding, even with comparable unit prices and reinforcer magni- tude. Thus, if an applied goal is to sustain response output as the schedule is thinned, it will be more ef!cient to increase the work requirements for exchange production (e.g., number of tokens needed to access the exchange) than for the token production schedule (e.g., number of responses needed to produce a token).

Discriminative Control One exception to this general rule is if the

exchange-production FR is too large, and the tokens in the early links exert strong discrimi- native effects. For example, research reviewed earlier showed that weak early-link responding is due to the discriminative functions of tokens temporally removed from exchange periods and terminal reinforcers. As additional tokens are earned, exchanges and terminal reinforcers grow nearer. As a result, the number of earned tokens is correlated with delays to the exchange and terminal reinforcers. In the above example, with the FR 10 exchange-production schedule, relatively large numbers of tokens (e.g., 8, 9, 10) signal relatively short exchange delays, whereas relatively small numbers of tokens (e.g., 0, 1, 2) signal relatively long exchange delays. These discriminative functions effec- tively weaken early-link responding.

425TOKEN REINFORCEMENT

One method to strengthen such early-link behavior is to reverse the usual stimulus-control relations, such that late-link stimuli become early-link stimuli, and vice versa (e.g., begin the token-production link in the presence of 10 tokens). If the tokens are serving discrimina- tive functions, this manipulation might work in bringing weak early-link responding under the control of stimuli that normally accompany stronger later-link behavior. Such effects are likely to be temporary, however, as new stimu- lus functions based on temporal proximity to terminal reinforcers would eventually be established. Another method for increasing weak behav-

ior in the early links of token reinforcement schedules is to use variable schedules, which have been shown to produce relatively higher response rates than otherwise comparable !xed schedules (as reviewed above). In terms of the above example, using a VR 10 exchange- production schedule (e.g., where exchange periods might occur after 2, 6, 10, 14, or 18 tokens) should sustain higher response out- put than an FR 10 schedule, owing to the stronger early-link behavior in the VR schedule. Thus, putting this together with the above, when arranging token reinforcement schedules for maintenance, one should, whenever possi- ble, (a) manipulate exchange-production rather than token-production schedules, and (b) use variable rather than !xed schedules. By point- ing to variables that strengthen responding when it is at its weakest (in periods remote from exchange), these recommendations sug- gest ways to increase rates of token-reinforced behavior—an important applied goal of most token programs.

Aversive Control Just as tokens can be used to increase behav-

ior, token losses can be used to decrease behav- ior. How might such research be translated into practical recommendations? Returning to

the above example, suppose that, in addition to the token-gain schedule (FR 5 token produc- tion, FR 10 exchange production) arranged for academic behavior, a token-loss contingency is arranged for challenging behavior. What type of token-loss procedure should be used? In light of the punishment magnitude effects

described above, it might seem appropriate to recommend losses as large as possible. Such considerations must also take into account token-gain contingencies, however. With ratio- based token-gain schedules, as in our example, removing a star from the token board is essen- tially a return to an earlier link in the chain. This results in longer delays to the exchange and toy access, and thus may weaken behavior on its own. For this reason, ratio-based sched- ules of token gain may not be the best choice in sustaining behavior, especially given that such token-earning behavior is usually desirable (in this case, academic work). Suppose instead that a time-based schedule (e.g., FI, VI) is used, in which an exchange is produced following the !rst token after a period of time. These sched- ules are less sensitive to punishment effects, and may therefore provide a better way to sus- tain the target academic responding via token gain while punishing challenging behavior via token loss. On the other hand, all else equal, ratio schedules generally produce higher rates than interval schedules. The particular arrange- ment of token-gain and token-loss contingen- cies may depend on tradeoffs between these different goals, including the relative impor- tance of the academic and challenging behavior. It may also depend on practical considerations (e.g., time-based schedules may be more dif!- cult to implement than response-based sched- ules). However these tradeoffs are resolved, the need for balancing losses against gains is espe- cially crucial in negative punishment proce- dures (including both token loss and timeout), in which the reinforcing value of token gains and the aversiveness of token losses are interdependent.

TIMOTHY D. HACKENBERG426

Another variable contributing to the aver- siveness of the loss is the manner in which the tokens are presented. Research reviewed above suggests that earned tokens may be valued more than otherwise equivalent freely delivered tokens. And just as access to earned tokens may be more reinforcing, removal of earned tokens may be more aversive. In our example, it might be more effective to punish the challenging behavior by removing earned stars from the token board than by removing freely provided stars. A !nal consideration in the decision about

whether to use token-loss procedures is their potential for negative collateral effects of aver- sive control (e.g., escape, social disruption), seen with other types of aversive stimuli. Although some such effects have occasionally been reported, their overall incidence is low. The majority of studies have reported no nega- tive side effects, some have reported positive side effects (e.g., increasing academic behavior), whereas others have reported preference for loss over gain procedures. On the whole, token-loss procedures are

both effective and safe; when designed and implemented in light of what is known about punishment effects more generally, token-loss procedures can be valuable behavior-change tools. When such procedures are used, consider a few general points. First, with respect to the loss schedule, losses should be immediate, fre- quent, and of moderate to high magnitude. In our example, all stars might be removed from the board (return to the initial link) each time the challenging behavior occurs. Second, with respect to the token-gain schedules, whether time-based or response-based schedules are used depends on how the token-reinforced behavior (academic work, in our example) interacts with the token-loss schedule. If the loss schedule is weakening token-earning behavior due to the delays to the exchange and terminal reinforcer (toy access) brought on by the loss schedule, then time-based schedules should be used.

Otherwise, ratio-based schedules would be favored, as they tend to produce higher response rates than comparable time-based schedules. And third, with respect to both schedules, both gains and losses should be response-dependent. This will enhance both the reinforcing value of the tokens and the aversiveness of token loss.

Economic Variables Several of the issues raised in the applied

behavioral economics section have important translational implications. To begin with, the demand analysis provides a valuable supple- ment to existing reinforcer assessment methods, suggesting quantitatively precise ways to assess relative reinforcer ef!cacy in ways most useful to treatment. It is common, for example, in applied studies to identify a reinforcer based on the results of a preference assessment under FR 1 schedules. Work described above, however, shows how relative reinforcer ef!cacy can and often does change with schedule variables (e.g., FR price). If limited to FR 1 schedules, these differences in reinforcer ef!cacy would not be apparent. The typical FR 1-type of assessment may be suf!cient if the intervention schedules are similar (i.e., FR 1 or small FR requirements). If, on the other hand, one wishes to use leaner and more intermittent schedules for behavior maintenance, it may be prudent to conduct a demand analysis, asses- sing preference under a wider range of schedule values that map better onto maintenance con- ditions. A demand analysis expands the range of schedule values used in assessment, provid- ing a more comprehensive and nuanced picture of reinforcer ef!cacy, and for these reasons may be more predictive of treatment success. Should one choose to conduct a demand

analysis, what type of procedure should be used? The typical demand analysis would entail measuring response output and terminal rein- forcers earned as the (token or exchange-

427TOKEN REINFORCEMENT

production) price was increased incrementally across sessions (e.g., FR 1, FR 5, FR 10, FR 20). An alternative method, gaining currency in recent years, is a PR schedule, in which the price changes within a session, usually after each reinforcer (Roane, 2008). The same span of ratios can be programmed, but on a com- pressed time scale. This has clear practical advantages over typical methods, not the least of which is the time needed to generate the demand function (the main practical disadvan- tage of the standard method). Unfortunately, research has shown that reinforcing ef!cacy, as measured by demand and PR methods, are not always highly correlated, and thus may be mea- suring different aspects of reinforcer value. More speci!cally, price sensitivity from a demand analysis better predicts preference between reinforcers than does PR breakpoint (Madden et al., 2007; Tan & Hackenberg, 2015). The choice of measure will depend, then, on the treatment goals. If the goal is to predict preferences between different rein- forcers, then the demand analysis may be the best choice; if, on the other hand, the goal is to predict broad differences in response output in a single-response situation, then the quicker PR method may suf!ce. A major insight from economic analyses is

that the value of one reinforcer depends not only on its own price, but also on the price and availability of other reinforcers. This is espe- cially pertinent to applied settings, in which a myriad of reinforcers are generally available, and needs to be considered in designing treat- ment programs. Returning to our prior exam- ple, suppose that the token-based academic program is embedded in a larger token econ- omy, in which tokens are produced and lost for adaptive and challenging behavior, respectively. The success of the program will depend, in part, on the degree to which the academic behavior and these other behavior patterns pro- duce functionally similar reinforcers; or in eco- nomic terms, the degree to which they serve as

substitutes, one for the other. Most token pro- grams have this type of functional substitution built into them, in that the reinforcer for all responses is the same: functionally equivalent generalized tokens exchangeable for a wide array of reinforcers. Considering the broader context surrounding

speci!c token programs can call attention to larger classes of variables to manipulate in applied settings. One such variable is income— the number of tokens available to spend. Income is a joint function of the price and availability of tokens for adaptive behavior (e.g., academic work) and the number of tokens already accumulated in the bank (i.e., savings, in economic terms). Suppose that in our example, stars can be saved across exchange periods, with unspent stars remaining in a token bank account. Because the stars in the bank and the stars earnable on the task are functionally interchangeable, a large savings account is essentially a supply of substitutable reinforcers, which may then weaken token- earning behavior. A robust token economy is one in which savings are low or nonexistent; this promotes the high rates of token earning and spending critical to the overall success of the program. One way to encourage spending is to

improve the quality of the terminal reinforcers (e.g., including a wider range of preferred rein- forcers). Another is to arrange mandatory exchange periods, either after each token and exchange-production cycle (as commonly done) or periodically (e.g., at the end of each session or at the end of each day), thereby emptying the token bank account and maintaining a more robust economy. Either way, methods that shift the income toward the present task and away from the savings account will bene!t the economy as a whole. A !nal variable to consider in designing

effective token programs is the overall eco- nomic context; more speci!cally, the availabil- ity of substitutable reinforcers outside the

TIMOTHY D. HACKENBERG428

token-earning context. Suppose that in our example, toy trains (the preferred terminal rein- forcer) were abundantly available before and after the token-based academic sessions (in economic terms, an open economy). This would likely weaken academic behavior, as this would be only one of many ways to access toy trains. Instead, restricting access to trains out- side the academic sessions (in economic terms, a closed economy), would likely enhance the value of the tokens and the academic respond- ing on which they depend. In general, the greater the restriction placed on access to termi- nal reinforcers in the setting as a whole (i.e., the more closed the economy), the more effective the token economy.

SUMMARY AND CONCLUSIONS

For a variety of reasons, laboratory and applied research on token reinforcement have drifted apart over the years. This split between the laboratory and applied work on token sys- tems has had unfortunate consequences for the !eld. Although token reinforcement systems have generally been quite successful in the applied realm, it is often unclear what variables are responsible for their effectiveness; they are rarely based on an understanding of the basic principles involved. We have known for over !ve decades that token reinforcement proce- dures work; we need to know more about why they work. We need a better analysis of the critical variables operating in token economies. A key premise of the paper is that a better

understanding of the basic principles operating in token systems will enhance application. To take just one example from a prior section, understanding that token-reinforced behavior is better maintained by variable than by !xed schedules (e.g., Ferster & Skinner, 1957) leads to more effective ways of scheduling exchange periods in classroom token economies (e.g., McLaughlin & Malaby, 1972). Knowl- edge of basic reinforcement schedules, derived

largely from laboratory research with nonhu- man animals, translates into more effective applications with students in a classroom environment. This is but one of numerous examples pro-

vided in the present paper where applied research serves translational research in this way—mainly as a vehicle for extrapolation of laboratory-based principles to an applied prob- lem. This is the prototypical model of transla- tional research, in which principles uncovered in the laboratory move outward to application. In the realm of token reinforcement, however, applied research can play a more active role in translational work; it can drive the analysis for- ward, opening new research vistas. For example, as discussed earlier, exploring the continuum of generalizability requires a healthy range of termi- nal reinforcers. These conditions are common- place in applied token environments (e.g., in the menu of terminal reinforcers for which tokens can be exchanged), and therefore provide an ideal context for an experimental analysis of a critical function of token reinforcers—their degree of generalizability—an analysis that is best suited to an applied setting. I have suggested the utility of a theoretical

framework that combines basic behavior- analytic and behavioral-economic principles. This illustrates another aspect of translational research—the intertranslation of concepts and methods from related disciplines. Although the direct translation between certain concepts (e.g., FR size in behavior analysis and price in behavioral economics) may lead some to ques- tion the utility of consulting two different the- oretical approaches, my aim here was to show ways in which the disciplines complement one another by asking similar questions but emphasizing different experimental and explanatory variables. For instance, behavior- analytic methods have traditionally focused on qualitatively similar reinforcers, whereas behavioral-economic methods have focused on qualitatively different reinforcers. In situations

429TOKEN REINFORCEMENT

with a variety of reinforcers, as in the case of generalized token reinforcers, well-established behavioral-economic methods (e.g., substitu- tion effects) may therefore serve as a valuable supplement to traditional behavior-analytic methods. Combining these behavioral economic

methods with applied token research—what I call applied behavioral economics—would simul- taneously advance the science and the applica- tion of token reinforcement. In the case of generalized reinforcement, elasticity measures would advance the science by providing detailed quantitative information on interac- tions between qualitatively different reinforcers; such information would be extremely valuable in designing more effective applied contingen- cies. And because the analyses would take place in the applied context itself, the results would be maximally generalizable. In this and other domains, there are impor-

tant questions applied research is actually better suited than laboratory research to address. Indeed, in this domain, clear distinctions between laboratory and applied research are dif- !cult to make, as the analysis takes place in the very context in which it is applied. This type of bidirectional interplay between analysis and application is one dimension of use-inspired research (Critch!eld, 2011; after Stokes, 1997), in which applied problems provide the theoreti- cal context for basic research. A major aim of this paper has been to stimu-

late such use-inspired research, in which token economies serve as applied laboratories for addressing fundamental questions, and theoreti- cal questions about basic principles go hand in hand with practical questions about how best to implement them. Figuring out how things work in the course of solving important social problems is a de!ning feature of applied behav- ior analysis (Baer, Wolf, & Risley, 1968), and when combined with the tools of behavioral economics, offers fresh new insights into the science and application of token reinforcement.

REFERENCES Andrade, L. F., & Hackenberg, T. D. (2017). Substitu-

tion effects in a generalized token economy with pigeons. Journal of the Experimental Analysis of Behavior, 107, 123-135. https://doi.org/10.1002/ jeab.231

Ayllon, T., & Azrin, N. H. (1965). The measurement and reinforcement of behavior of psychotics. Journal of the Experimental Analysis of Behavior, 8, 357-383. https://doi.org/10.1901/jeab.1965.8-357

Ayllon, T., & Azrin, N. H. (1968). The token economy: A motivational system for therapy and rehabilitation. New York, NY: Appleton-Century-Crofts.

Azrin, N. H. (1960). Effects of punishment intensity under variable-interval reinforcement. Journal of the Experimental Analysis of Behavior, 3, 123-142. https://doi.org/10.1901/jeab.1960.3-123

Azrin, N. H., & Holz, W. C. (1966). Punishment. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp. 380-437). New York, NY: Appleton-Century-Crofts.

Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Jour- nal of Applied Behavior Analysis, 1, 91-97. https://doi. org/10.1901/jaba.1968.1-91

Barrett, S. T., & Bevins, R. A. (2012). A quantitative analysis of the reward-enhancing effects of nicotine using reinforcer demand. Behavioural Pharmacology, 23, 781-789. https://doi.org/10.1097/fbp. 0b013e32835a38d9

Battalio, R. C., Kagel, J. H., Winkler, R. C., Fisher, E. G., Jr., Basman, R. L., & Krasner, L. (1974). An experimental investigation of consumer behavior in a controlled environment. Journal of Con- sumer Research, 1, 52-60. https://doi.org/10. 1086/208591

Becraft, J. L., & Rolider, N. U. (2015). Reinforcer varia- tion in a token economy. Behavioral Interventions, 30, 157-165. https://doi.org/10.1002/bin.1401

Bentzley, B. S., Fender, K. M., & Aston-Jones, G. (2013). The behavioral economics of drug self- administration: A review and new analytical approach for within-session procedures. Psychopharmacology, 226, 113-125. https://doi.org/10.1007/s00213-012- 2899-2

Bickel, W. K., & Vuchinich, R. (2000). Reframing health behavior change with behavioral economics. Mahwah, NJ: Lawrence Erlbaum & Associates.

Boren, J. J., & Colman, A. D. (1970). Some experiments on reinforcement principles within a psychiatric ward for delinquent soldiers. Journal of Applied Behavior Analysis, 3, 29-37. https://doi.org/10.1901/jaba. 1970.3-29

Brady, J. V., Kelly, D., & Plumlee, L. (1969). Autonomic and behavioral responses of the rhesus monkey to behavioral conditioning. Annals of the New York

TIMOTHY D. HACKENBERG430

Academy of Sciences, 15, 959-975. https://doi.org/10. 1111/j.1749-6632.1969.tb12991.x

Brandt, A. E., Sztykiel, H., & Pietras, C. J. (2013). Labo- ratory simulated gambling: Risk varies across participant-stake procedure. The Journal of General Psychology, 140, 130-143. https://doi.org/10. 1080/00221309.2013.776509

Bullock, C. E., & Hackenberg, T. D. (2006). Second- order schedules of token reinforcement with pigeons: Implications for unit price. Journal of the Experimen- tal Analysis of Behavior, 85, 95-106. https://doi. org/10.1901/jeab.2006.116-04

Bullock, C. E., and Hackenberg, T. D. (2015). The sev- eral roles of stimuli in token reinforcement. Journal of the Experimental Analysis of Behavior, 103, 269-287. https://doi.org/10.1037/e625362007-001

Burchard, J. D., & Barrera, F. (1972). An analysis of timeout and response cost in a programmed environ- ment. Journal of Applied Behavior Analysis, 5, 271- 282. https://doi.org/10.1901/jaba.1972.5-271

Carnett, A., Raulston, T., Lang, R., Tostanoski, A., Lee, A., Sigafoos, J., & Machalicek, W. (2014). Effects of a perseverative interest-based token econ- omy on challenging and on-task behavior in a child with autism. Journal of Behavioral Education, 23, 368-377. https://doi.org/10.1007/s10864-014- 9195-7

Cassidy, R. N., & Dallery, J. (2012). Effects of economy type and nicotine on the essential value of food in rats. Journal of the Experimental Analysis of Behavior, 97, 183-202. https://doi.org/10.1901/jeab.2012. 97-183

Charlop-Christy, M.H., & Haymes, L. K. (1998). Using objects of obsession as token reinforcers for children with autism. Journal of Autism and Developmental Disorders, 28, 189-198. https://doi.org/10.1023/ a:1026061220171

Christensen, C. J., Silberberg, A., Hursh, S. R., Huntsberry, M. E., & Riley, A. L. (2008). Essential value of cocaine and food in rats: tests of the expo- nential model of demand. Psychopharmacology, 198, 221-229. https://doi.org/10.1007/s00213-008- 1120-0

Cowles, J. T. (1937). Food-tokens as incentives for learn- ing by chimpanzees. Comparative Psychological Monographs, 12, 1-96. https://doi.org/10. 1037/14268-000

Critch!eld, T. S. (2011). Translational contributions of the experimental analysis of behavior. The Behavior Analyst, 34, 3-17. https://doi.org/10.1007/ bf03392227

Critch!eld, T. S., Paletz, E. M., MacAleese, K. R., & Newland, M. C. (2003). Punishment in human choice: Direct or competitive suppression? Journal of the Experimental Analysis of Behavior, 80, 1-27. https://doi.org/10.1901/jeab.2003.80-1

Dallery, J., Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (2001). Voucher-

based reinforcement of opiate plus cocaine abstinence in treatment-resistant methadone patients: effects of reinforcer magnitude. Experimental and Clinical Psychopharmacology, 9, 317-325. https://doi.org/10. 1037//1064-1297.9.3.317

DeFulio, A., Yankelevitz, R. L., Bullock, C. E., & Hackenberg, T. D. (2014). Generalized conditioned reinforcement with pigeons in a token economy. Journal of the Experimental Analysis of Behavior, 102, 26-46. https://doi.org/10.1002/jeab.94

DeLeon, I. G., Iwata, B. A., Loh, H. L., & Worsdell, A. S. (1997). Emergence of reinforcer pref- erence as a function of schedule requirements and stimulus similarity. Journal of Applied Behavior Analysis, 30, 439-449. https://doi.org/10.1901/jaba. 1997.30-439

De Luca, R. V., & Holborn, S. W. (1990). Effects of !xed-interval and !xed-ratio schedules of token rein- forcement on exercise in obese and non-obese boys. Psychological Record, 40, 67-82.

De Luca, R. V., & Holborn, S. W. (1992). Effects of a variable-ratio reinforcement schedule with changing criteria on exercise in obese and non-obese boys. Journal of Applied Behavior Analysis, 25, 671-679. https://doi.org/10.1901/jaba.1992.25-671

Doll, C., McLaughlin, T. F., & Barretto, A. (2013). The token economy: A recent review and evaluation. International Journal of Basic and Applied Science, 2, 131-149.

Donaldson, J. M., DeLeon, I. G., Fisher, A. B., & Kahng, S. W. (2014). Effects of and preference for conditions of token earn versus token loss. Journal of Applied Behavior Analysis, 47, 1-12. https://doi. org/10.1002/jaba.135

Eisenberger, R., & Cameron, J. (1996). Detrimental effects of reward: Reality or myth? American Psychologist, 51, 1153-1166. https://doi.org/10. 1037//0003-066x.51.11.1153

Emurian, H. H., Emurian, C. S., & Brady, J. V. (1985). Positive and negative reinforcement effects on behav- ior in a three-person microsociety. Journal of the Experimental Analysis of Behavior, 44, 157-174. https://doi.org/10.1901/jeab.1985.44-157

Fantino, E. (1977). Conditioned reinforcement: Choice and information. In W. K. Honig & J. E. R. Staddon (Eds), Handbook of operant behavior (pp. 313-339). Englewood Cliffs, NJ: Prentice Hall.

Feingold, B. D., & Mahoney, M. J. (1975). Reinforce- ment effects on intrinsic interest: Undermining the overjusti!cation hypothesis. Behavior Therapy, 6, 367-377. https://doi.org/10.1016/s0005-7894(75) 80111-0

Felton, M., & Lyon, D. O. (1966). The post- reinforcement pause. Journal of the Experimental Analysis of Behavior, 9, 131-134. https://doi.org/10. 1901/jeab.1966.9-131

Ferster, C. B., & Skinner, B. F. (1957). Schedules of rein- forcement. New York, NY: Appleton-Century-Crofts.

431TOKEN REINFORCEMENT

Findley, J. D. (1958). Preference and switching under concurrent scheduling. Journal of the Experimental Analysis of Behavior, 1, 123-144. https://doi.org/10. 1901/jeab.1958.1-123

Fisher, E. B., Jr. (1979). Overjusti!cation effects in token economies. Journal of Applied Behavior Analysis, 12, 407-415. https://doi.org/10.1901/jaba.1979.12-407

Fisher, E. B., Jr., Winkler, R. C., Krasner, L., Kagel, J., Battalio, R. C., & Basmann, R. L. (1978). Economic perspectives in behavior therapy: Complex interde- pendencies in token economies. Behavior Therapy, 9, 391-403. https://doi.org/10.1016/s0005-7894(78) 80083-5

Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token reinforcement with pigeons: Effects of !xed- and variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 76, 159-178. https://doi.org/10.1901/jeab. 2001.76-159

Fox, A. E., & Pietras, C. J. (2013). The effects of response-cost punishment on instructional control in a choice task. Journal of the Experimental Analysis of Behavior, 99, 346-361. https://doi.org/10.1002/ jeab.20

Francesco, M. T., Madden, G. J., & Borrero, J. (2009). Behavioral economics: Principles, procedures, and utility for applied behavior analysis. The Behavior Analyst Today, 10, 277-293. https://doi.org/10.1037/ h0100671

Galizio. M. (1979). Contingency-shaped and rule- governed behavior: Instructional control of human loss-avoidance. Journal of the Experimental Analysis of Behavior, 31, 53-70. https://doi.org/10.1901/jeab. 1979.31-53

Glynn, S. M. (1990). Token economy approaches for psy- chiatric patients: progress and pitfalls over 25 years. Behavior Modi!cation, 14, 383-407. https://doi. org/10.1177/01454455900144002

Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds), Handbook of operant behavior (pp. 288-312). Engle- wood Cliffs, NJ: Prentice Hall.

Greene, D., & Lepper, M. R. (1974). Effects of extrinsic rewards on children’s subsequent intrinsic interest. Child Development, 45, 1141-1145. https://doi. org/10.2307/1128110

Hackenberg, T. D. (2009). Token reinforcement: A review and analysis. Journal of the Experimental Analy- sis of Behavior, 91, 257-286. https://doi.org/10.1901/ jeab.2009.91-257

Hackenberg, T. D., & Joker, V. R. (1994). Instructional versus schedule control of humans’ choices in situa- tions of diminishing returns. Journal of the Experi- mental Analysis of Behavior, 62, 367-383. https://doi. org/10.1901/jeab.1994.62-367

Hanley, G. P., Piazza, C. C., Fisher, W. W., & Maglieri, K. A. (2005). On the effectiveness of and preference for punishment and extinction

components of function-based interventions. Journal of Applied Behavior Analysis, 38, 51-65. https://doi. org/10.1901/jaba.2005.6-04

Higgins, S. T., Heil, S. H., & Lussier, J. P. (2004). Clini- cal implications of reinforcement as a determinant of substance abuse disorders. Annual Review of Psychology, 55, 431-461. https://doi.org/10.1146/ annurev.psych.55.090902.142033

Holth, P., Vandbakk, M., Finstad, J., Gronnerud, E. M., & Sorenson, J. M. A. (2009). An operant analysis of joint attention and the establishment of conditioned social reinforcers. European Journal of Behavior Analysis, 10, 143-158. https://doi.org/10.1080/15021149.2009. 11434315

Hursh, S. R., & Silberberg, A. (2008). Economic demand and essential value. Psychological Review, 115, 186-198.

Hutchinson, R. R. (1977). By-products of aversive con- trol. In W. K. Honig & J. E. R. Staddon (Eds), Handbook of operant behavior (pp. 415-431). Engle- wood Cliffs, NJ: Prentice Hall.

Isaksen, J., & Holth, P. (2009). An operant approach to teaching joint attention skills to children with autism. Behavioral Interventions, 24, 215-236. https://doi. org/10.1002/bin.292

Iwata, B. A., & Bailey, J. S. (1974). Reward versus cost token systems: An analysis of the effects on student and teacher. Journal of Applied Behavior Analysis, 7, 567-576. https://doi.org/10.1901/jaba.1974.7-567

Jwaideh, A. R. (1973). Responding under chained and tandem !xed-ratio schedules. Journal of the Experi- mental Analysis of Behavior, 19, 259-267. https://doi. org/10.1901/jeab.1973.19-259

Kagel, J. H. (1972). Token economies and experimental economics. Journal of Political Economy, 80, 779-785. https://doi.org/10.1086/259926

Kagel, J. H., & Winkler, R. C. (1972). Behavioral eco- nomics: Areas of cooperative research between eco- nomics and applied behavior analysis. Journal of Applied Behavior Analysis, 5, 335-342. https://doi. org/10.1901/jaba.1972.5-335

Kahneman, D., Knetsch, J. L, & Thaler, R. H. (1990). Experimental studies of the endowment effect and the Coase theorem. Journal of Political Economy, 98, 1325-1348. https://doi.org/10.1086/261737

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decisions under risk. Econometrica, 47, 263-291. https://doi.org/10.2307/1914185

Kaufman, K. F., & O’Leary, K. D. (1972). Reward, cost, and self-evaluation procedures for disruptive adoles- cents in a psychiatric hospital school. Journal of Applied Behavior Analysis, 5, 293-309. https://doi. org/10.1901/jaba.1972.5-293

Kazdin, A. E. (1982). The token economy: A decade later. Journal of Applied Behavior Analysis, 15, 431- 445. https://doi.org/10.1901/jaba.1982.15-431

Kazdin, A. E., & Bootzin, R. R. (1972). The token econ- omy: An evaluative review. Journal of Applied

TIMOTHY D. HACKENBERG432

Behavior Analysis, 5, 343-372. https://doi.org/10. 1901/jaba.1972.5-343

Kelleher, R. T. (1956). Intermittent conditioned rein- forcement in chimpanzees. Science, 124, 679-680. https://doi.org/10.1126/science.124.3224.679

Kelleher, R. T. (1958). Fixed-ratio schedules of condi- tioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281-289. https://doi.org/10.1901/jeab.1958.1-281

Leon, Y., Borrero, J. C., & DeLeon, I. G. (2016). Para- metric analysis of delayed primary and conditioned reinforcers. Journal of Applied Behavior Analysis, 49, 1-17. https://doi.org/10.1002/jaba.311

Levitt, S. D., List, J. A., Neckermann, S., & Sadoff, S. (2012). The behavioralist goes to school: Leveraging behavioral economics to improve educational perfor- mance. ZEW Discussion Papers, No. 12-038, http:// nbn-resolving.de/urn:nbn:de:bsz:180-madoc-323358

Levy, A., DeLeon, I. G., Martinez, C. K., Fernandez, N., Gage, N. A., Sigurdsson, S. Ó., & Frank- Crawford, M. A. (2016). A quantitative review of overjusti!cation effects in persons with intellectual and developmental disabilities. Journal of Applied Behavior Analysis, 50, 206-221. https://doi.org/10. 1002/jaba.359

Madden, G. J. (2000). A behavioral economics primer. In W. K. Bickel & R. Vuchinich (Eds), Reframing health behavior change with behavioral economics (pp. 3-26). Mahwah, NJ: Lawrence Erlbaum & Associates.

Madden, G. J., Smethells, J. R., Ewan, E. E., & Hursh, S. R. (2007). Tests of behavioral-economic assessments of relative reinforcer ef!cacy II: Eco- nomic complements. Journal of the Experimental Analysis of Behavior, 88, 355-367. https://doi.org/10. 1901/jeab.2007.88-355

Malagodi, E. F. (1967). Acquisition of the token-reward habit in the rat. Psychological Reports, 20, 1335-1342. https://doi.org/10.2466/pr0.1967.20.3c.1335

Mazur, J. E. (1983). Steady-state performance on !xed-, mixed-, and random-ratio schedules. Journal of the Experimental Analysis of Behavior, 39, 293-307. https://doi.org/10.1901/jeab.1983.39-293

Mazur, J. E. (2004). Risky choice: Selecting between cer- tain and uncertain outcomes. Behavior Analyst Today, 5, 190-203. https://doi.org/10.1037/h0100031

McGinnis, J. C., Friman, P. C., & Carlyon, W. D. (1999). The effect of token rewards on “intrinsic” motivation for doing math. Journal of Applied Behav- ior Analysis, 32, 375-379. https://doi.org/10.1901/ jaba.1999.32-375

McLaughlin, T. F., & Malaby, J. E. (1972). Intrinsic reinforcers in a classroom token economy. Journal of Applied Behavior Analysis, 5, 263-270. https://doi. org/10.1901/jaba.1972.5-263

McLaughlin, T. F., & Malaby, J. E. (1976). An analysis of assignment completion and accuracy across time and under !xed, variable, and extended token exchange periods in a classroom token economy.

Contemporary Educational Psychology, 1, 346-355. https://doi.org/10.1016/0361-476x(76)90054-0

Miller, J. R., DeLeon, I. G., Toole, L. M., Lieving, G. A., & Allman, M. J. (2016). Contin- gency enhances sensitivity to loss in a gambling task with diminishing returns. The Psychological Record, 66, 301-308. https://doi.org/10.1007/s40732-016- 0172-5

Moher, C. A., Gould, D. D., Hegg, E., & Mahoney, A. M. (2008). Non-generalized and gener- alized conditioned reinforcers: Establishment and val- idation. Behavioral Interventions, 23, 13-38. https:// doi.org/10.1002/bin.253

Neuringer, A. J. (1967). Effects of reinforcement magni- tude on choice and rate of responding. Journal of the Experimental Analysis of Behavior, 10, 417-424. https://doi.org/10.1901/jeab.1967.10-417

O’Leary, K. D. (1978). The operant and social psychol- ogy of token systems. In A. C. Catania & T. A. Brigham (Eds.), Handbook of applied behavior analysis: Social and instructional processes. New York, NY: Irvington.

O’Leary, K. D., & Drabman, R. (1971). Token reinforce- ment programs in the classroom: A review. Psychologi- cal Bulletin, 75, 379-398. https://doi.org/10.1037/ h0031311

Parrish, J. M., Cataldo, M. F., Kolko, D. J., Neef, N. A., & Egel, A. L. (1986). Experimental analysis of covariation among compliant and inappro- priate behaviors. Journal of Applied Behavior Analysis, 19, 241-254. https://doi.org/10.1901/jaba.1986. 19-241

Pazulinec R., Meyerrose. M., & Sajwaj T. (1983). Pun- ishment via response cost. In S. Axelrod (Ed.), The effects of punishment on human behavior (pp. 71-86). New York, NY: Academic Press.

Phillips, C. L., Iannaccone, J. A., Rooker, G. W., & Hagopian, L. P. (2017). Noncontingent reinforce- ment for the treatment of severe problem behavior: An analysis of 27 consecutive applications. Journal of Applied Behavior Analysis, 50, 357-376. https://doi. org/10.1002/jaba.376

Phillips, E. L. (1968). Achievement Place: Token rein- forcement procedures in a home-style rehabilitation setting for pre-delinquent boys. Journal of Applied Behavior Analysis, 1, 213-223. https://doi.org/10. 1901/jaba.1968.1-213

Phillips, E. L., Phillips, E. A., Fixsen, D. L., & Wolf, M. W. (1971). Modi!cation of the behaviors of pre-delinquent boys within a token economy. Jour- nal of Applied Behavior Analysis, 4, 45-59. https://doi. org/10.1901/jaba.1971.4-45

Pietras, C. J., & Hackenberg, T. D. (2000). Timeout postponement without increased reinforcement fre- quency. Journal of the Experimental Analysis of Behavior, 74, 147-164. https://doi.org/10.1901/jeab. 2000.74-147

433TOKEN REINFORCEMENT

Pietras, C. J., & Hackenberg, T. D. (2005). Response- cost punishment via token-loss with pigeons. Beha- vioural Processes, 69, 343-356. https://doi.org/10. 1016/j.beproc.2005.02.026

Powell, R. W. (1968). The effects of small sequential changes in !xed-ratio size upon the post- reinforcement pause. Journal of the Experimental Analysis of Behavior, 11, 589-593. https://doi.org/10. 1901/jeab.1968.11-589

Raiff, B. R., Bullock, C. E., & Hackenberg, T. D. (2008). Response-cost punishment with pigeons: Fur- ther evidence of response suppression via token loss. Learning & Behavior, 36, 29-41. https://doi.org/10. 3758/lb.36.1.29

Rasmussen, E. B., & Newland, M. C. (2008). Asymmetry of reinforcement and punishment in human choice. Journal of the Experimental Analysis of Behavior, 89, 157-167. https://doi.org/10.1901/jeab.2008.89-157

Repp, A. C., & Deitz, S. M. (1975). A comparison of !xed-ratio and variable-ratio token-production sched- ules with human subjects. The Psychological Record, 25, 131-137. https://doi.org/10.1007/bf03394296

Reynolds, G. S. (1961). Behavioral contrast. Journal of the Experimental Analysis of Behavior, 4, 57-71. https:// doi.org/10.1901/jeab.1961.4-57

Roane, H. S. (2008). On the applied use of progressive- ratio schedules of reinforcement. Journal of Applied Behavior Analysis, 41, 155-161. https://doi.org/10. 1901/jaba.2008.41-155

Roane, H. S., Call, N. A., & Falcomata, T. S. (2005). A preliminary analysis of adaptive responding under open and closed economies. Journal of Applied Behav- ior Analysis, 38, 335-348. https://doi.org/10.1901/ jaba.2005.85-04

Roane, H. S., Lerman, D. C., & Vorndran, C. M. (2001). Assessing reinforcers under progressive sched- ule requirements. Journal of Applied Behavior Analysis, 34, 145-167. https://doi.org/10.1901/jaba.2001. 34-145

Schmauck, F. J. (1970). Punishment, arousal, and avoid- ance learning in sociopaths. Journal of Abnormal Psychology, 76, 325-335. https://doi.org/10.1037/ h0030398

Schroeder, S. R., & Barrera, F. J. (1976). Effects of price manipulations on consumer behavior in a sheltered workshop token economy. American Journal of Men- tal De!ciency, 8, 172-180.

Shahan, T. A. (2010). Conditioned reinforcement and response strength. Journal of the Experimental Analysis of Behavior, 93, 269-289. https://doi.org/10.1901/ jeab.2010.93-269

Skinner, B. F. (1953). Science and human behavior. New York: Macmillan.

Smith, T. D. (1972). Development of tokens as secondary reinforcers. Journal of Experimental Child Psychology, 14, 133-138. https://doi.org/10.1016/0022-0965(72) 90038-0

Sran, S.K, & Borrero, J.C. (2010). Assessing the value of choice in a token system. Journal of Applied Behavior Analysis, 43, 553-557. https://doi.org/10.1901/jaba. 2010.43-553

Staats, A. W., Staats, C. K., Schutz, R. E., & Wolf, M. (1962). The conditioning of textual responses using “extrinsic” reinforcers. Journal of the Experimental Analysis of Behavior, 5, 33-40. https://doi.org/10. 1901/jeab.1962.5-33

Stokes, D. E. (1997). Pasteur’s quadrant: Basic science and technological innovation. Washington, DC: Brookings Institution Press.

Tan, L., & Hackenberg, T. D. (2015). Pigeons’ demand and preference for speci!c and generalized condi- tioned reinforcers in a token economy. Journal of the Experimental Analysis of Behavior, 104, 296-314. https://doi.org/10.1002/jeab.181

Tarbox, R. S. F., Ghezzi, P. M., & Wilson, G. (2006). The effects of token reinforcement on attending in a young child with autism. Behavioral Interventions, 21, 155-164. https://doi.org/10.1002/bin.213

Taylor-Santa, C., Sidener, T. M., Carr, J. E., & Reeve, K. F. (2014). A discrimination training proce- dure to establish conditioned reinforcers for children with autism. Behavioral Interventions, 29, 157-176. https://doi.org/10.1002/bin.1384

Thompson, R. H., & Iwata, B. A. (2005). A review of reinforcement control procedures. Journal of Applied Behavior Analysis, 38, 257-278. http://doi.org/10. 1901/jaba.2005.176-03

Thorndike, E. L. (1911). Animal intelligence. New York, NY: Macmillan.

Tustin, R. D. (1994). Preference for reinforcers under varying schedule arrangements: A behavioral eco- nomic analysis. Journal of Applied Behavior Analysis, 27, 597-606. https://doi.org/10.1901/jaba.1994. 27-597

Vasta, R., & Stirpe, L. A. (1979). Reinforcement effects on three measures of children’s interest in math. Behavior Modi!cation, 3, 223-244. https://doi.org/10. 1177/014544557932006

Waddell, T. R., Leander, J. D., Webbe, F. M., & Malagodi, E. F. (1972). Schedule interactions in second-order !xed-interval (!xed-ratio) schedules of token reinforcement. Learning and Motivation, 3, 91- 100. https://doi.org/10.1016/0023-9690(72)90050-1

Webbe, F. M., & Malagodi, E. F. (1978). Second-order schedules of token reinforcement: Comparisons of per- formance under !xed-ratio and variable-ratio exchange schedules. Journal of the Experimental Analysis of Behavior, 30, 219-224. https://doi.org/10.1901/jeab.1978.30-219

Weiner, H. (1962). Some effects of response cost upon human operant behavior. Journal of the Experimental Analysis of Behavior, 5, 201-208. https://doi.org/10. 1901/jeab.1962.5-201

Weiner, H. (1963). Some effects of response cost and aversive control of human operant behavior. Journal

TIMOTHY D. HACKENBERG434

of the Experimental Analysis of Behavior, 6, 415-421. https://doi.org/10.1901/jeab.1963.6-415

Williams, B. A. (1983). Another look at behavioral con- trast in multiple schedules. Journal of the Experimental Analysis of Behavior, 39, 345-384. https://doi.org/10. 1901/jeab.1983.39-345

Williams, B. A. (1994). Conditioned reinforcement: Neglected or outmoded explanatory construct? Psy- chonomic Bulletin & Review, 1, 457-475. https://doi. org/10.3758/bf03210950

Winkler, R. C. (1970). Management of chronic psychotic patients by a token reinforcement system. Journal of Applied Behavior Analysis, 3, 47-55. https://doi. org/10.1901/jaba.1970.3-47

Winkler, R. C. (1971). The relevance of economic theory and technology to token reinforcement systems. Behavior Research and Therapy, 9, 81-88. https://doi. org/10.1016/0005-7967(71)90064-7

Winkler, R. C. (1972). A theory of equilibrium in token economies. Journal of Abnormal Psychology, 2, 169- 173. https://doi.org/10.1037/h0032722

Winkler, R. C. (1973). An experimental analysis of eco- nomic balance, savings, and wages in a token econ- omy. Behavior Therapy, 4, 22-40. https://doi.org/10. 1016/s0005-7894(73)80072-3

Winkler, R. C. (1980). Behavioral economics, token economies, and applied behavior analysis. In J. E. R. Staddon (Ed.). Limits to action (pp. 269- 297). New York, NY: Academic Press. https://doi. org/10.1016/b978-0-12-662650-6.50015-5

Winkler, R. C., & Burkhard, B. (1990). A systems approach to behavior modi!cation through behavioral economics. In L. Green and J. H. Kagel (Eds.), Advances in behavioral economics, Vol. 2 (pp. 288- 315). Norwood, NJ: Ablex.

Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative Psychological Monographs, 12, 1-72.

Received June 6, 2016 Final acceptance October 2, 2017 Action Editor, Iser DeLeon

435TOKEN REINFORCEMENT