Literature review on behavior analysis

Vol.:(0123456789)1 3

Journal of Contemporary Psychotherapy (2018) 48:145–154 https://doi.org/10.1007/s10879-017-9376-5

O R I G I N A L PA P E R

Token Economies: Using Basic Experimental Research to Guide Practical Applications

Jeffrey F. Hine1 · Scott P. Ardoin2 · Nathan A. Call3

Published online: 12 December 2017 © Springer Science+Business Media, LLC, part of Springer Nature 2017

Abstract This paper highlights the applicability of patterns seen within basic experimental research in relation to contemporary appli- cation of token economies. Token economies are one of the most widely used interventions to promote behavior change, and this procedure has evolved to be effective across many settings, behaviors, and individuals. Due to this widespread use, casual implementation of the token economy might result in inconsistencies in responding and therefore an overall skepti- cism in the procedure itself. We present multiple barriers that encumber practical application of token economies, including insufficient conditioning and pairing of tokens, determining quality of backup reinforcers, unforeseen effects of motivating operations, teaching the token exchange, effects of higher-order reinforcement schedules, ratio strain, and use of response cost procedures. To assist practitioners in implementing more effective treatments, for each barrier we revisit the often overlooked basic research involving features of conditioned reinforcement and reinforcement schedules. It is important to translate the often complex implications of basic research so that practitioners can use this information to improve their own practice as well as their confidence in disseminating use of this evidence-based treatment. To further guide practitioners in using this knowledge in everyday settings, we also provide recommendations specific to each barrier as well as relevant applied research and practical examples.

Keywords Token economy · Conditioned reinforcement · Applied behavior analysis

Introduction

Since first proposed by Ayllon and Azrin (1968) and subse- quently refined by Kazdin (1977), the use of token econo- mies has become one of the most venerable and widespread applied interventions for producing behavior change (Kazdin 1982; Matson and Boisjoli 2009). Given this widespread use and effectiveness across settings, many practitioners

(i.e., psychologists, teachers, and applied behavior analysts) may have a general understanding of the procedures behind establishing a token economy. Although there may be some differences in the specifics of establishing a token economy (e.g., Drabman and Tucker 1974; Miltenberger 2008), there seems to be general consensus that establishing an effec- tive token economy should at least include: (1) identifying and operationally defining appropriate target behaviors; (2) selecting appropriate tokens (e.g., durable, engaging, indi- vidualized); (3) identifying backup reinforcers (e.g., primary reinforcers, other conditioned reinforcers); (4) determining values of tokens and exchange rates for backup reinforc- ers; (5) determining methods of exchange; (6) determin- ing how individuals can earn or lose tokens; (7) accurately monitoring the program’s effects on the target behaviors; and (8) adjusting the program to meet the long-term goals and addressing barriers to success. If practitioners implement these steps in a consistent and systematic manner, positive behavior changes are likely to occur. In general, practitioners can use the above framework as a “base” behavior manage- ment system with which they can enact multiple options.

* Jeffrey F. Hine [email protected]

1 Vanderbilt University Medical Center Department of Pediatrics, Vanderbilt Kennedy Center/Treatment and Research Institute for Autism Spectrum Disorders (TRIAD), 1211 21st Ave S, #110, Nashville, TN 37212, USA

2 University of Georgia Department of Educational Psychology, Center for Autism and Behavioral Education Research, Athens, GA, USA

3 Emory University School of Medicine, Marcus Autism Center, Atlanta, GA, USA

146 Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

However, during implementation of a token economy, practitioners may encounter complex barriers to behavior change and may struggle to enact solutions targeting those barriers (Bailey et al. 2011; Kazdin 1982). Revisiting foun- dational basic research on the underlying mechanisms of token economies can assist practitioners in overcoming such difficulties when they are encountered in practice. Wide- spread failure to implement token economies without an understanding of these mechanisms likely has an impact on the progress of the individuals with whom such procedures are adopted and the reputation of this strategy.

Much basic research exists demonstrating the effects of the underlying mechanisms of token economies (Foster et al. 2001; Hackenberg 2009). Ideally, practitioners would consult this literature when faced with a practical dilemma. A variable that often interferes with this venture; however, includes the considerable effort required for practitioners to assimilate and apply information gained from reading basic experimental research. Practitioners might view this research as inapplicable to everyday practice; yet, general patterns of performance found with animals and humans in the laboratory consistently emerge in applied research (Mace and Critchfield 2010). We will highlight the applicability of these patterns that may assist practitioners in identify- ing potential barriers to individual success and implement- ing a more fundamentally sound and thus effective token economy.

Conditioning Tokens as Effective Reinforcers

A conditioned reinforcer is defined as an initially neutral event or stimulus acquiring value through its relation to pri- mary reinforcers and subsequently can serve as an effec- tive independent reinforcer (Skinner 1974; Williams 1994). Comprehensive research programs such as Fantino (1977), Kelleher (1966), and Williams (1994) collectively demon- strate that several species’ response rates increase if respond- ing produces conditioned reinforcers. Perhaps the most widely cited laboratory investigations of the effects of tokens as conditioned reinforcers are the classic primate studies of Wolfe (1936) and Cowles (1937). Contingent presentation of tokens maintained responding across multiple experiments even when subjects were not allowed to exchange the tokens until the end of an experimental session. Malagodi (1967a, b, c) added to this research by demonstrating that rats acquired new responses through use of token reinforcement alone and that token-specific response rates were similar to those seen under primary reinforcement. Given the substantial body of research demonstrating that findings from the basic ani- mal research can be generalized to applied use of the same behavioral mechanisms, it would seem that the principles that govern the effectiveness of token economies in animal

studies have value when troubleshooting an ineffective token economy. Thus, when token economies are not as effective as projected, practitioners need first to investigate a number of general factors relating to the effectiveness of the token.

Barrier: Insufficient Quality of Backup Reinforcers

One barrier to effective use of a token economy is when the token has not been established as an effective condi- tioned reinforcer. This problem may become evident when the individual ceases to readily exchange tokens for previ- ously accessed backup reinforcers. If this occurs, decreased responding will likely ensue and the individual may discard tokens instead of exchanging them. An initial issue that practitioners need to investigate is the quality of the backup stimuli.

Basic Experimental Research

Some manipulable dimensions of reinforcement found to increase the likelihood of responding within token econo- mies include reinforcer rate, magnitude, and quality of rein- forcement (Mace and Roberts 1993). Quality of reinforce- ment is often described as involving reinforcer potency or efficacy, and can be quantified in terms of an individual’s preferences. One way to measure preference is to consider stimuli that are reliably selected as highly preferred in stimu- lus preference assessments (Neef et al. 1994). An individu- al’s preference for the to-be-paired stimuli will undoubtedly influence a token’s effectiveness as a conditioned reinforcer. In the classic Wolfe (1936) studies, researchers demonstrated primates’ proclivity to select tokens that had been paired with food (as opposed to nothing) and tokens that had been paired with two pieces of food (rather than one). Additional animal studies demonstrate that with all other dimensions of reinforcement held constant (e.g., amount and rate) subjects will bias responding toward reinforcers of higher quality. Thus, if manipulating the quality of primary reinforcement is an effective method of biasing responding, doing so will likely impact the reinforcing effectiveness of tokens paired with the primary (or backup) reinforcers.

In Applied Settings

Tokens will not become effective reinforcers if they are paired with stimuli that are not of sufficient quality or that have not been established as effective reinforcers themselves. A large body of applied literature has identified numer- ous strategies for selecting stimuli most likely to function as effective backup reinforcers including single stimulus, free operant, paired stimulus, and multiple stimulus with- out replacement formats (DeLeon and Iwata 1996; Roane et  al. 1998). Given that preferences may fluctuate over

147Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

time, practitioners might consider having an assortment of backup stimuli and institute periodic assessment of prefer- ences. Systematically rotating preferred items can main- tain the reinforcing properties of the backup stimuli. For example, DeLeon et al. (2000) demonstrated that providing access to only a single set of toys limited the effectiveness of their intervention due to satiation effects. Instead, when providing access to a rotating set of toys as reinforcement for competing responses, automatically maintained self- injurious behavior was reduced. Thus, tokens can maintain their reinforcing properties despite fluctuating preferences if they can be exchanged for a variety of high quality reinforc- ers and can become much more flexible as a reinforcer in treatment programs. Additionally, if tokens can effectively be exchanged for many different backup reinforcers, the con- venience and social validity of the program increases by not requiring practitioners to keep a wide range of reinforcers constantly and immediately available.

Barrier: Insufficient or Inconsistent Pairing

After ensuring quality backup reinforcers, another factor that might impede consistent responding is the association of the token with the backup reinforcer. These associations arise through the original token-backup pairing and how often this pairing occurs.

Basic Experimental Research

Foundational basic research by Wolfe (1936) and Cowles (1937) demonstrated the importance of pairing tokens with primary reinforcers by teaching primates to respond differ- entially to tokens with exchange value as opposed to those without. A token will likely not have a reinforcing influ- ence over an organism’s behavior if the token is not paired with the backup stimuli a sufficient number of times or close enough in time. Williams and Dunn (1991) provided some evidence for the necessity of token-backup pairing through a series of experiments examining conditioned reinforcement in pigeons. Overall, the effectiveness of conditioned rein- forcers depended on the frequency with which the stimulus was paired with the primary reinforcer as well as how often the stimulus was followed by reinforcement. Kelleher and Gollub (1962) also noted the significance of the number of pairings between the eventual conditioned stimuli (tokens) and primary reinforcers.

Research investigating respondent conditioning further supports that, with repeated pairing, the token should retain the reinforcing properties of the backup reinforcer even without the individual engaging in any behaviors outside of accepting and consuming the reinforcer (Williams 1994). Shahan (2010) equates this circumstance to the principles of respondent conditioning that result in stimuli acquiring

the capacity to act as conditioned stimuli when paired with unconditioned stimuli. In this case, neutral stimuli (tokens) acquire the capacity to function as reinforcers when paired with primary reinforcers. Classic basic research demon- strating application of conditioned reinforcers to shape new responses (Malagodi 1967a, b, c; Kelleher and Gollub 1962) provides further supporting evidence for the importance of the foundational relationship between tokens and backups.

In Applied Settings

If the token does not seem to function as a conditioned rein- forcer, practitioners will most likely need to pair the two stimuli more frequently, more consistently, or temporally closer. Before ever requiring the individual to engage in a behavior to gain access to the token, practitioners would benefit from repeatedly and contiguously pairing the token with a backup reinforcer. For instance, a practitioner could fill up 9 spaces on a 10-space token board and then noncon- tingently deliver the 10th token while immediately allowing access to a preferred item. This process would be repeated until the individual accepts both the token and backup rein- forcer a majority of the time. In an applied study, Moher, Gould, Hegg, and Mahoney (2008) successfully established tokens as conditioned reinforcers by pairing tokens with backup reinforcers in two stages. The first stage involved the experimenter delivering a backup reinforcer within 0.5 s of delivering a token noncontingently. In the second stage, the participant was encouraged to physically exchange the token for the backup reinforcer. When evaluated in a pref- erence assessment, tokens contingently paired with highly preferred edible items became preferred stimuli themselves.

Barrier: Overcoming Problematic Effects of Motivating Operations

Even after ensuring a strong pairing between tokens and backup reinforcers, inconsistent responding may still occur. One potential cause is the variable effectiveness of backup stimuli from moment-to-moment. In an effort to prevent inconsistent effectiveness of backup reinforcers and thus responding, practitioners can investigate the effects of moti- vating operations. By definition, motivating operations can alter the reinforcing effectiveness of tokens either by increas- ing (establishing) or decreasing (abolishing) the effective- ness of a given consequence (Laraway et al. 2003; Vollmer and Iwata 1991). It might be the case that motivating opera- tions are affecting responding in unforeseen ways.

Basic Experimental Research

Wolfe (1936) first demonstrated this fact by exposing pri- mates to various states of food deprivation. Specifically,

148 Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

subjects were given choices between tokens (some exchange- able for food and some exchangeable for water) while under alternating deprivation conditions. All subjects preferred the tokens corresponding to the current deprivation conditions; however, the preference was not exclusive due to relatively modest deprivation states. Therefore, two additional sub- jects were given the same choices between tokens under longer deprivation conditions and the researchers allowed access to the alternate reinforcer prior to each session. The subjects under the more stringent conditions preferred the deprivation-specific reinforcer to a higher degree. Thus, rate of token-exchange was consistent with the state of depri- vation specific to each backup reinforcer and lessening the motivation for one reinforcer strengthened the motivation for the other.

Another motivating operation factor studied within basic research and applicable to practical application involves the degree to which the reinforcer is available outside of the experimental session.Hursh (1984) described closed econo- mies as those in which reinforcers are only available through an organism’s interaction with the experimental environ- ment, and open economies as those in which consumption of the reinforcer is not completely dependent on within-session performance. For example, two classic studies (Felton and Lyon 1966; Catania and Reynolds 1968) performed experi- ments in which pigeons were given supplemental (noncon- tingent) feedings outside of the experimental session (open economy). Relative rates of responding were markedly less under these conditions than in conditions where subjects could only access reinforcers through responding accord- ing to in-session schedules of reinforcement (Collier et al. 1972).

In Applied Settings

Motivating operations can be seen as an advantage to prac- titioners who can regulate the amount of access to a single backup reinforcer. This can be achieved by ensuring the indi- vidual does not have access to the preferred backup item outside of the token economy. For instance, Roane, Call, and Falcomata (2005) demonstrated more responding during closed economies in which participants were only able to obtain reinforcement through interaction with progressive- ratio schedules of reinforcement during session. This was in contrast to open-economy sessions during which partici- pants demonstrated decreased responding while obtaining both within-session reinforcers and supplemental access to reinforcers outside of session.

Depending on the nature of the primary backup rein- forcer, applied research has shown that the effectiveness of tokens decreases during periods in which participants are satiated on backup reinforcers; however, rotation and choice across multiple backup reinforcers may guard against

these effects (Moher et al. 2008; Sran and Borrero 2010). Additionally, inconsistent responding due to the effects of motivating operations can also be neutralized by creation of generalized conditioned reinforcers. A token becomes a gen- eralized conditioned reinforcer when it can be exchanged for a variety of backup reinforcers and is less sensitive to moti- vating operations (Ferster and Culbertson 1982). Increasing the number of backup reinforcers with which the token is paired should also result in the maintenance of responding even when individuals are satiated on the most preferred backup reinforcer (Moher et al. 2008). Thus, efforts can be made to decrease the potential negative effects of abolish- ing operations by having a menu of options from which an individual can select when exchanging tokens for backup reinforcers.

Barrier: Difficulty Shaping the Exchange Response

Given that most practitioners themselves have had a long history of operating within a token economy (e.g., receiv- ing and cashing paychecks), there may be some inclination to assume an individual can exchange tokens for backup reinforcers spontaneously. For ease and efficiency of token- backup exchange, there may be some benefit in removing the exchange response altogether. That is, by exchang- ing the tokens for someone who is struggling to learn the exchange response, the reinforcing properties of the token might stay intact with practitioner-mediated token-backup pairing. However, explicit teaching of token exchange may be necessary and beneficial if the practitioner intends on the individual eventually determining components such as the magnitude and rate of reinforcement.

Basic Experimental Research

Laboratory studies often rely on multiple stages to shape the exchange response. After magazine training, in which ani- mals are taught to approach the food receptacle and consume primary reinforcers, a shaping procedure is used to rein- force approximations of lever pressing. Experimenters then focus on teaching the animal the token deposit response. For instance, Malagodi (1967a) distributed 80 marbles on the floor of an operant chamber and reinforced successive approximations of rats depositing the marbles into a recep- tacle. Stimulus control over the response was established by reinforcing deposit responses on a continuous reinforcement schedule in the presence of a discriminative stimulus (recep- tacle light and clicker). In the foundational Wolfe (1936) and Cowles (1937) studies, exchange opportunities were freely available for primates and depositing of the token was ini- tially modeled by the experimenter. Each token deposited by the subject was reinforced immediately with food.

149Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

In Applied Settings

Even when tokens have acquired the properties of effec- tive conditioned reinforcers, not all individuals will imme- diately have mastery over the response chain necessary to physically exchange the token and consume the backup reinforcer. Numerous empirically validated methods for teaching response chains are possible; including graduated guidance, errorless learning, constant-time delay, and video modeling. Initially, the act of exchanging the token should be the primary task in which the individual must engage in order to gain access to the reinforcer. It could also be the case that the response effort of the act of exchanging is too great (e.g., walking to a different area, locating the practi- tioner, making choices between backup reinforcers, engag- ing in a communicative response, etc.); thus, individuals sometimes save tokens to increase the amount of reinforc- ment they receive per exchange (Yankelevitz et al. 2008). For instance, an establishing operation for saving tokens might be in effect during low-effort tasks if the exchange response is too demanding; at least until a sufficient number of tokens have been accumulated to overcome the effort of the exchange response. Thus, practitioners must consider the overall effort of the exchange itself, as it may influence the effectiveness of token program.

Acknowledging and Investigating First‑ and Second‑Order Schedules of Reinforcement

Reinforcement schedules do not operate in isolation; instead, one schedule (a first-order schedule) can be a unit of behav- ior upon which another schedule operates (higher- or sec- ond-order schedules). In other words, completion of the first- order schedule (e.g., fixed-ratio [FR]-5) is a behavioral unit that is reinforced according to a second schedule (e.g., varia- ble-interval [VI]-25). An oversimplified view of responding within token economies would include practitioners viewing responding as vulnerable to only the “local” contingencies available through first-order reinforcement schedules. If this were the case, behavior patterns under token economies would only mimic those seen under programs using pri- mary reinforcement, which most often is not the case. Fixed- ratio schedules, for instance, produce post-reinforcement pauses—also referred to as “pre-run” pauses—in which responding briefly ceases following reinforcement deliv- ery. This momentary lag in responding is often followed by an increase in response rate until the organism meets the requirement for reinforcement. Conversely, a variable- ratio (VR) schedule produces relatively higher and steadier rates of responding (Ferster and Skinner 1957). Patterns of behavior within an extended token economy, however,

should instead be considered as unitary responses influenced and reinforced according to two other higher-order sched- ules. Kelleher (1958, 1966) described token economies as involving three interconnected schedules of reinforcement and behavior that is responsive to a token economy will be jointly determined by both the first- and second-order reinforcement schedules. The three schedules include: (1) the token-production schedule: the first-order schedule of reinforcement under which the behavior targeted for change will result in tokens (e.g., FR5: the individual must emit 5 responses to receive one token); (2) the exchange-production schedule: the schedule that determines when the opportunity to exchange tokens for backup reinforcers is available (e.g., fixed-time [FT]-5: the individual can exchange tokens for backups every 5 min); and (3) the token-exchange schedule: the rate of exchange or “cost” for backup reinforcers (e.g., FR5: the individual must exchange 5 tokens for a certain backup reinforcer).

Practitioners might view ongoing patterns of behavior as being reinforced by the local contingencies available through the immediate token-production schedule; however, responding is not under the sole control of any of these three schedules at any given time. Given research supporting the separate and combined effects of these schedules (Hacken- berg 2009), practitioners are advised to take notice of the three specific schedules of reinforcement and need to inves- tigate both the “local” contingencies of the token-production schedule as well as the often superseding contingencies of the second-order schedules.

Barrier: Appropriately Adjusting Local Contingencies of the Token‑Production Schedule

Beginning with the token-production schedule, practitioners may struggle in deciding whether to provide tokens after a fixed or variable number of responses (FR or VR); after a fixed or variable duration of engaging in the targeted behav- ior (FRD or VRD); or after the first response following a fixed or variable amount of time (FI or VI). Most token economies implemented in applied settings run on fixed schedules. There are obvious logistical benefits of using fixed token-production schedules (i.e., ease of implementa- tion, predictability) and FR schedules can often result in high, steady responding and are especially beneficial when teaching new behaviors. One important property of fixed schedules, however, is that they can introduce discrimina- ble periods during which reinforcement does not occur, and responding can appear erratic such as scalloped respond- ing under an FI schedule in which responding is slow in the beginning of an interval and increases just before token delivery. Furthermore, FI schedules can become problematic when practitioners attempt to reinforce longer durations of

150 Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

continuous appropriate behavior such as increasing time on task.

Basic Experimental Research

As stated previously, the token-production schedule exerts some control over response patterns in token economies in a manner resembling those obtained under schedules of primary reinforcement (Kelleher 1956; Malagodi 1967b). For example, Kelleher (1958), taught primates to press a lever that produced poker chips according to an FR30 schedule where every 30 responses produced one poker chip. Exchange periods were scheduled following an FR50 exchange-production schedule. Corresponding to basic experimental research with simple FR schedules using pri- mary reinforcement, responding occurred at a high-steady rate, with short pauses prior to each ratio run. The research- ers then increased the token-production schedule to an FR125 while keeping the exchange-production schedule constant. Again, emulating effects seen with FR schedules of primary reinforcement, overall response rates decreased and post-reinforcement pausing increased. In addition to this study, basic research also shows that by switching the token-reinforcement schedule to variable schedules, one can expect high and steady responding under VR token-produc- tion schedules, and slow and steady responding under VI schedules (Ferster and Skinner 1957).

In Applied Settings

During acquisition phases, practitioners have the option to begin with an FR1 token production schedule so that every occurrence of the new behavior produces reinforcement. As the individual gains experience with the token economy, practitioners can systematically adjust the production sched- ule to include intermittent schedules using procedures such as thinning the schedule to an FR5. Alternatively, moving to a VR token-production schedule will result in maintenance of the replacement behavior over longer periods of time and may guard against post-reinforcement pauses and satiation of backup reinforcers. Basic research suggests that under FI schedules practitioners will observe long periods of inactiv- ity with slight increases in responding towards the interval’s end (scalloped responding; Kelleher 1956). To modify this pattern, practitioners can change to either a VI token-produc- tion schedule or a response duration schedule. Under a VI schedule, responding should be steadier and more moderate due to the unpredictability of the interval’s end (Malagodi 1967c). With an FRD or a VRD schedule, practitioners have the option to only reinforce behaviors that are of a specific duration. An example of effectively using an FRD token- production schedule would involve an individual receiving a token after 3 min of continuous appropriate conversation,

and not receiving a token if the duration of the conversation was less than 3 min.

Barrier: Unforeseen Effects of the Exchange‑Production Schedule

Whereas the token-production schedule refers to the num- ber of target responses that must be emitted by individu- als to receive a token, the exchange-production schedule refers to how often an individual is given the opportunity to exchange tokens for the backup reinforcers. Practitioners may tend to focus too narrowly on the more local effects of the token-production schedule and, as a result, encounter decreased responding even with a dense token-production schedule. One consideration that needs to be emphasized in this instance is the influence of the exchange-production schedule.

Basic Experimental Research

Basic research has suggested that the exchange-production schedule may in fact have greater control over responding patterns than the local contingencies operating within the token-production schedule (Webbe and Malagodi 1978). Foster et al. (2001) highlighted the relatively greater influ- ence of the exchange-production schedule over token-pro- duction schedules by comparing one condition with a VR- token-production schedule and a FR-exchange-production schedule with another condition involving an FR-token- production schedule and a VR-exchange-production sched- ule (FR-token/VR-exchange vs. VR-token/FR-exchange). Because pause durations were longer in the VR token-pro- duction schedule (a schedule known to produce relatively pause-free, constant rates of responding) relative to the FR token-production schedule (a schedule known to produce break-run patterns), the researchers concluded that overall rates of behavior were primarily organized by the exchange- production schedule requirements. Bullock and Hackenberg (2006) extended these results by showing more pronounced effects of the exchange-production schedule when the token- production ratios were higher and when more responses per token were required. In trials that included lower token-pro- duction schedules (e.g., FR2), response rates varied much less with the exchange-production schedule. That is, the schedules that allowed more frequent access to tokens mim- icked those of primary reinforcement schedules; whereas, when more responses were required per token the frequency of exchange periods had more influence over responding.

In Applied Settings

A practical example of the token-production schedule acting as a unitary response, and thereby producing reinforcement

151Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

according to the exchange-production schedule, includes a student earning tokens for completing math problems. For every two problems completed, he will receive a token (FR2-token-production schedule). Given a relatively fre- quent exchange period (FR8-exchange-production sched- ule), one could expect responding to adhere to patterning seen under simple-schedules with primary reinforcement: the student would complete problems rapidly and accumu- late more tokens within a given time period. The frequency of exchange periods becomes more influential once teach- ers thin the token-production schedule. For instance, if the teacher now requires the student to complete 15 problems for one token (FR15-token-production schedule), these clus- ters of behaviors are now more vulnerable to the effects of the higher-order exchange-production schedule. The teacher could now allow the student to exchange all the tokens for a backup reinforcer either (a) after completing a fixed number of problems (e.g., FR45-exchange-production schedule), or (b) after an average number of problems (e.g., VR45). Under the FR45 exchange-production schedule, the student is likely to engage in post-reinforcement pausing with a quick transi- tion to rapid responding until reinforcement is received (i.e., break-run patterning). Under the VR45 exchange-production schedule, however, the student could be expected to com- plete problems more quickly while pausing for shorter peri- ods. Thus, even though the token-production schedule is the same in either scenario (FR15), it is the exchange-production schedule that disproportionately controls the overall rate of responding.

Barrier: Overcoming Ratio Strain

Within token economies, ratio strain occurs with abrupt increases in ratio requirements, resulting in decreases in behavior similar to those seen during extinction (Ferster and Skinner 1957). Ratio strain can unexpectedly occur through the interaction of the token- and exchange-produc- tion schedule. Sifting through why ratio strain is occurring and which schedule is influencing responding the most can be a complex task. Long pauses in ratio performance could occur when response requirements within the token-produc- tion schedule are too high, or when too much time elapses between exchange opportunities.

Basic Experimental Research

Specific to token-production, organisms whose schedules of reinforcement are “thinned” are required to engage in an increased amount of responding before reinforce- ment. Decreased responding is often a function of increas- ing ratio requirements too quickly; however, ratio strain can also occur through the interaction of the token- and exchange-production schedules whereby organisms

might earn tokens at an appropriate rate, yet would show decreased responding if the requirement for exchange- production was too stringent (i.e., exchange opportunities are not often enough). For instance, Bullock and Hacken- berg(2006) demonstrated ratio strain in pigeons by show- ing an inverse relation between responding and the token- production ratio. High response requirements decreased responding due mainly to long pauses and low response rates in early segments. Researchers also demonstrated, however, an inverse relation between response rate and exchange-production ratios when token-production ratios were kept constant.

In Applied Settings

Ratio strain can occur within applied environments when practitioners delay opportunities to exchange tokens until the end of the day (FT- or VT-exchange-production sched- ules) or do not restore the exchange-production schedules to the ratio that previously maintained adequate rates of responding. For example, if an individual receives tokens on an FR5-token-production schedule, and can exchange tokens after accumulating an average of 5 tokens (VR5- exchange-production schedule), all other things accounted for, the practitioner can expect relatively rapid responding with short post-reinforcement pauses. If, however, the practi- tioner abruptly requires the individual to either (a) engage in 50 responses for 1 token (FR-50-token-production schedule), (b) exchange tokens only after accumulating an average of 100 tokens (VR100-exchange-production schedule), or (c) both, one would expect decreased responding due to ratio strain, and potentially complete extinction of responding before reinforcement at the new schedule can occur. Ratio strain can be avoided by increasing response requirements gradually, temporarily reducing ratio requirements, or by increasing backup magnitude or quality (Roane et al. 2007).

Barrier: Adjusting Prices of Backup Reinforcers

The token-exchange schedule, refers to how many tokens are required for a specific backup reinforcer, or the token- specific “price” of the backup reinforcer. It is not always the case that once an exchange opportunity is earned, the indi- vidual can exchange only one token for one unit of a backup reinforcer. What is more likely to occur in applied settings is the option to “purchase” a variety of backup reinforcers that vary in price. Token-exchange schedules often may be chosen arbitrarily or might even be based on the actual retail price of the item. However, to ensure predictable respond- ing, practitioners should consider a number of different fac- tors when determining this schedule.

152 Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

Basic Experimental Research

Basic research does not provide extensive information about the specific influence of token-exchange schedules other than that token-exchange influence is similar to exchange- production influence. Malagodi et al. (1975) demonstrated decreased responding in rats’ lever pressing given increased token-exchange schedules: the more demanding the token- exchange schedule, the longer post-reinforcement pausing. Basic research on “unit price,” or the ratio of responses to every unit of reinforcer, includes a more sophisticated description of price that details the characteristics of cost- benefit tradeoffs (e.g., Delmendo et  al. 2009; Foster and Hackenberg 2004). That is, price does not necessarily refer just to the actual token-exchange schedule of the backup reinforcer; rather, the interaction of the token-exchange schedule with the costs of producing tokens (the token- production schedule) and the costs of producing exchange opportunities (the exchange-production schedule; Bullock and Hackenberg 2006). Basic research on unit price demon- strates that decreases in response rates are associated with increases in unit price (more stringent token-production, exchange-production, and token-exchange requirements).

In Applied Settings

Practitioners can expect decreased overall responding if the labeled price of the backup reinforcer is relatively too high (i.e., too demanding of a token-exchange schedule). In applied settings; however, the actual price of the backup reinforcer is better determined by the combined effects of how often responses result in tokens, how often the indi- vidual can exchange tokens, and the number of tokens required to exchange for specific reinforcers (Bullock and Hackenberg 2006; Malagodi et  al. 1975). In this sense, Hackenberg (2009) likened the token-production schedule to a worker’s wage, the exchange-production schedule to the effort required to purchase the item (e.g., driving to the store, getting cash out of the bank), and the token-exchange schedule to the number listed on the price tag. The basic premise of unit price can be applied to practical contexts by using reinforcer assessments to identify a hierarchy of backup reinforcers. Delmendo et al. (2009) suggested that this information could be used to differentially program con- tingencies for task completion based on the subjective effort associated with the task. Within token economies, those tasks that require more effort can be associated with more preferred reinforcers. Conversely, reinforcers that are less preferred may be more suitable for maintaining less effort- ful responses. An example of this situation could involve an individual being able to use tokens to purchase high quality rewards that are not available at other times after a period of effortful tasks (e.g., difficult homework). Less preferred

reinforcers, therefore, would be available for the periods that involve less effortful tasks (e.g., sitting at the table while eating).

Barrier: Response Cost and Reducing Inappropriate Behavior

Common barriers not only to token economies, but also to applied practice as a whole, may include the individual engaging in problem behavior. Practitioners most likely are well versed in the use of differential reinforcement proce- dures to reinforce an appropriate behavior in place of an inappropriate one, but they may struggle to enact appropri- ate measures to respond to inappropriate behavior using the token economy. Modifying the token-production schedule to include response cost procedures could be an effective addition to target problem behavior if reinforcement alone is ineffective in reducing problem behavior.

Basic Experimental Research

Response cost is conceptualized as a punishment procedure in which reinforcers are removed contingent upon some response. For example, Pietras and Hackenberg (2005) used LED lights as tokens to reinforce pecking in pigeons. Use of these lights allowed researchers to easily remove tokens by turning the light off. Key pecking was maintained on two separate schedules and when FR schedules of response- cost were introduced in one of the schedules, response rates under only that schedule decreased. It is interesting to note that in this experiment, response rates for the response-cost schedule were not completely suppressed; rather, only dur- ing extinction did response rates decrease to near-zero levels. In a basic experiment with humans (Weiner 1962), responses produced brief stimuli (lights) signaling availability of rein- forcers/points according to either VI or FI schedules. By subtracting one point from a counter during response cost conditions, response rates were suppressed and did not recover with continued exposure to the response-cost con- tingency. Thus, basic experimental research has consistently demonstrated decreased response rates of target behaviors using contingent removal of conditioned reinforcers.

In Applied Settings

A response cost procedure can be effective within any type of token-production schedule as long as the tokens are act- ing as conditioned reinforcers. Much of the applied token economy research implementing response cost involves the individual being given a number of tokens at the begin- ning of an interval and losing tokens for each inappro- priate response (e.g., Conyers et al. 2004; McGoey and DuPaul 2000). If the individual has enough tokens at the

153Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

end of the set interval, he or she can exchange them for a backup reinforcer. Conyers et al. (2004) found that both a response-cost and differential reinforcement of other behavior (DRO) procedure were effective in reducing problem behavior when implemented in isolation; how- ever, they recommended implementing them together to increase treatment acceptability. Thus, a response cost pro- cedure will be more valuable if implemented in conjunc- tion with a token-production differential reinforcement schedule.

A practical limitation of using response cost can involve an instance in which an individual loses all tokens before an exchange period and has a long wait before an oppor- tunity to earn them back. This instance might produce a segment of time in which contingencies for appropri- ate behavior are vague, perhaps creating an establishing operation for problem behavior. Some other practical lim- itations are encompassed by the potential negative side effects of punishment procedures in general. Specifically, punishment procedures such as response cost can produce negative side effects that include collateral increases in punishment-elicited aggression, escape behaviors, and emotional reactions (Lerman and Vorndran 2002). Lastly, using response cost in isolation might be disadvantageous considering exchange opportunities depend on respond- ing (FR or VR exchange-production schedules). That is, if response-cost conditions result in low response rates, infrequent pairings of tokens with backup reinforcers may reduce the reinforcing value of the tokens.

Conclusion

As robust as the literature surrounding token economies is, practitioners may not make regular contact with the basic research that has defined many of the practices in use today. However, when faced with practical problems or barriers to success, it is likely beneficial for practitioners to revisit this literature and examine these underlying principles. This is especially true if the practitioner responsible for training others in the implementation of the token economy has a less sophisticated understanding of how and why certain procedures function as they do. It is true that typical and even substandard implementation of token economies can have positive effects on behavior; however, many practition- ers are called upon to consult for complex cases. Complex cases likely require a deeper understanding of the underlying mechanisms actuating the seemingly everyday practices that we use. Thus, our objective was to make this research more accessible and to translate this research for applied settings; hopefully assisting practitioners in implementing a more fundamentally sound and thus effective treatment.

Compliance with Ethical Standards

Conflict of interest All authors declare no conflicts of interest. This article does not contain any studies with human participants or animals performed by any of the authors.

References

Ayllon, T., & Azrin, N. H. (1968). The token economy: A moti- vational system for therapy and rehabilitation. New York: Appleton-Century-Crofts.

Bailey, J. R., Gross, A. M., & Cotton, C. R. (2011). Challenges asso- ciated with establishing a token economy in a residential care facility. Clinical Case Studies, 10(4), 278–290.

Bullock, C. E., & Hackenberg, T. D. (2006). Second-order schedules of token reinforcement with pigeons: Implications for unit price. Journal of the Experimental Analysis of Behavior, 85, 95–106.

Catania, A. C., & Reynolds, G. A. (1968). Quantitative analysis of the responding maintained by interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 11, 327–383.

Collier, G. H., Hirsch, E., & Hamlin, P. H. (1972). The ecological determinants of reinforcement in the rat. Physiology and Behav- ior, 9, 705–716.

Conyers, C., Miltenberger, R., Maki, A., Barenz, R., Jurgens, M., Sailer, A., & Kopp, B. (2004). A comparison of response cost and differential reinforcement of other behavior to reduce disruptive behavior in a preschool classroom. Journal of Applied Behavior Analysis, 37, 411–415.

Cowles, J. T. (1937). Food-tokens as incentives for learning by chim- panzees. Comparative Psychological Monographs, 12, 1–96.

DeLeon, I. G., Anders, B. M., Rodriguez-Catter, V., & Neidert, P. L. (2000). The effects of noncontingent access to single- versus mul- tiple-stimulus sets on self-injurious behavior. Journal of Applied Behavior Analysis, 33(4), 623–626.

DeLeon, I. G., & Iwata, B. A. (1996). Evaluation of multiple-stimulus presentation format for assessing reinforcer preference. Journal of Applied Behavior Analysis, 29, 519–532.

Delmendo, X., Borrero, J. C., Beauchamp, K. L., & Francisco, M. T. (2009). Consumption and response output as a function of unit price: Manipulation of cost and benefit components. Journal of Applied Behavior Analysis, 42, 609–625.

Drabman, R. S., & Tucker, R. D. (1974). Why token economies fail. Journal of School Psychology, 12(3), 178–188.

Fantino, E. (1977). Conditioned reinforcement: Choice and informa- tion. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 313–339). Englewood Cliffs: Prentice-Hall.

Felton, M., & Lyon, D. (1966). The post-reinforcement pause. Journal of the Experimental Analysis of Behavior, 9, 131–134.

Ferster, C. B., & Culbertson, S. A. (1982). Behavior Principles (3rd edn.). Englewood Cliffs: Prentice Hall.

Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton Century-Crofts.

Foster, T. A., & Hackenberg, T. D. (2004). Unit price and choice in a token-reinforcement context. Journal of the Experimental Analy- sis of Behavior, 81(1), 5–25.

Foster, T. A., Hackenberg, T. D., & Vaidya, M. (2001). Second-order schedules of token reinforcement with pigeons: Effects of fixed- and variable-rate exchange schedules. Journal of Experimental Analysis of Behavior, 76, 159–178.

Hackenberg, T. D. (2009). Token reinforcement: A review and analysis. Journal of Experimental Analysis of Behavior, 91, 257–286.

Hursh, S. R. (1984). Behavioral economics. Journal of the Experimen- tal Analysis of Behavior, 42, 435–452.

154 Journal of Contemporary Psychotherapy (2018) 48:145–154

1 3

Kazdin, A. E. (1977). The token economy: A review and evaluation. New York: Plenum.

Kazdin, A. E. (1982). The token economy: A decade later. Journal of Applied Behavior Analysis, 15, 431–445.

Kelleher, R. T. (1956). Intermittent conditioned reinforcement in chim- panzees. Science, 124, 679–680.

Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforce- ment with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281–289.

Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475–485.

Kelleher, R. T., & Gollub, L. R. (1962). A review of positive condi- tioned reinforcement. Journal of the Experimental Analysis of behavior, 5(4), 543–597.

Laraway, S., Snycerski, S., Michael, J., & Poling, A. (2003). Motivating operations and terms to describe them: Some further refinements. Journal of Applied Behavior Analysis, 36(3), 407–414.

Lerman, D. C., & Vorndran, C. M. (2002). On the status of knowledge for using punishment: Implications for treating behavior disorders. Journal of Applied Behavior Analysis, 35(4), 431–464.

Mace, F. C., & Critchfield, T. S. (2010). Translational research in behavior analysis: Historical traditions and imperative for the future. Journal of the Experimental Analysis of Behavior, 93(3), 293–312.

Mace, F. C., & Roberts, M. L. (1993). Factors affecting selection of behavioral interventions. In J. Reichle & D. P. Wacker (Eds.), Communicative alternatives to challenging behavior (pp. 113– 134). Baltimore: Brookes.

Malagodi, E. F. (1967a). Acquisition of the token-reward habit in the rat. Psychological Reports, 20, 1335–1342.

Malagodi, E. F. (1967b). Fixed-ratio schedules of token reinforcement. Psychonomic Science, 8, 469–470.

Malagodi, E. F. (1967c). Variable-interval schedules of token reinforce- ment. Psychonomic Science, 8, 471–472.

Malagodi, E. F., Webbe, F. M., & Waddell, T. R. (1975). Second-order schedules of token reinforcement: Effects of varying the sched- ule of food presentation. Journal of the Experimental Analysis of Behavior, 24, 173–181.

Matson, J. L., & Boisjoli, J. A. (2009). The token economy for children with intellectual disability and/or autism: A review. Research in Developmental Disabilities, 30(2), 240–248.

McGoey, K. E., & DuPaul, G. J. (2000). Token reinforcement and response cost procedures: Reducing the disruptive behavior of preschool children with attention-deficit/hyperactivity disorder. School Psychology Quarterly, 15(3), 330–343.

Miltenberger, R. (2008). Behavior modification. Belmont: Wadsworth Publishing.

Moher, C. A., Gould, D. D., Hegg, E., & Mahoney, A. M. (2008). Non-generalized and generalized conditioned reinforcers: Estab- lishment and validation. Behavioral Interventions, 23(1), 13–38.

Neef, N. A., Shade, D., & Miller, M. S. (1994). Assessing influen- tial dimensions of reinforcers on choice in students with serious emotional disturbance. Journal of Applied Behavior Analysis, 27, 575–583.

Pietras, C. J., & Hackenberg, T. D. (2005). Response-cost punish- ment via token loss with pigeons. Behavioral Processes, 69(3), 343–356.

Roane, H. S., Call, N. A., & Falcomata, T. S. (2005). A preliminary analysis of adaptive responding under open and closed economies. Journal of Applied Behavior Analysis, 38(3), 335–348.

Roane, H. S., Falcomata, T. S., & Fisher, W. W. (2007). Applying the behavioral economics principle of unit price to DRO schedule thinning. Journal of Applied Behavior Analysis, 40(3), 529–534.

Roane, H. S., Vollmer, T. R., Ringdahl, J. E., & Marcus, B. A. (1998). Evaluation of a brief stimulus preference assessment. Journal of Applied Behavior Analysis, 31(4), 605–620.

Shahan, T. A. (2010). Conditioned reinforcement and response strength. Journal of the Experimental Analysis of Behavior, 93, 269–289.

Skinner, B. F. (1974). About behaviorism. New York: Knopf. Sran, S. K., & Borrero, J. C. (2010). Assessing the value of choice

in a token system. Journal of Applied Behavior Analysis, 43(3), 553–557.

Vollmer, T. R., & Iwata, B. A. (1991). Establishing operations and rein- forcement effects. Journal of Applied Behavior Analysis, 24(2), 279–291.

Webbe, F. W., & Malagodi, E. F. (1978). Second-order schedules of token reinforcement: Comparisons on performance under fixed- ratio and variable-ratio exchange schedules. Journal of Experi- mental Analysis of Behavior, 30, 219–224.

Weiner, H. (1962). Some effects of response cost upon human oper- ant behavior. Journal of the Experimental Analysis of Behavior, 5(2), 201–208.

Williams, B. A. (1994). Conditioned reinforcement: Experimental and theoretical issues. The Behavior Analyst, 17, 261–285.

Williams, B. A., & Dunn, R. (1991). Preference for conditioned rein- forcement. Journal of the Experimental Analysis of Behavior, 55, 37–46.

Wolfe, J. B. (1936). Effectiveness of token-rewards for chimpanzees. Comparative Psychology Monographs, 12, 1–72.

Yankelevitz, R. L., Bullock, C. E., & Hackenberg, T. D. (2008). Rein- forcer accumulation in a token reinforcement context. Journal of the Experimental Analysis of Behavior, 90, 283–299.