REINFORCER SURVEYS summary

335

JOURNAL OF APPLIED BEHAVIOR ANALYSIS 2000, 33, 335–338 NUMBER 3 (FALL 2000)

FURTHER EVALUATION OF THE ACCURACY OF REINFORCER SURVEYS: A SYSTEMATIC REPLICATION

JOHN NORTHUP LOUISIANA STATE UNIVERSITY

The present report evaluates the accuracy of a reinforcer survey by comparing the survey results to the results of subsequent reinforcer assessments for 20 children using a con- current-operants arrangement to assess relative reinforcer preference. Total accuracy for the survey was determined to be approximately 57%. The results provide a systematic replication of Northup et al. (1996) with a much larger sample of children. A need for the development of more accurate and comprehensive reinforcer assessment methods for verbal children is discussed.

DESCRIPTORS: reinforcer assessment, functional assessment, surveys, attention def- icit hyperactivity disorder

Self-reports of potential reinforcers con- tinue to be widely used to develop treat- ments for the most common and pervasive childhood behavior problems. The use of re- inforcer surveys to identify potential rein- forcers has a long history in child behavior therapy. The practice appears to be well es- tablished, and continued use may be further supported by the ease and apparent efficien- cy of administration. Nevertheless, poor cor- respondence between verbal self-reports and subsequent behavior has been long noted and often demonstrated (e.g., Bernstein & Michael, 1990; Risley & Hart, 1968). Chil- dren’s ability to accurately name events that will reinforce future behavior may be partic- ularly questionable. Northup, George, Jones, Broussard, and Vollmer (1996) demonstrat- ed that correspondence between the results of a reinforcer survey and a subsequent eval- uation of reinforcement effects was no better than chance for 4 children with a diagnosis of attention deficit hyperactivity disorder (ADHD). However, the evaluation of only 4 subjects for a comparison of this type is a major limitation of the Northup et al. study.

Correspondence and requests for reprints should be sent to John Northup, Department of Psychology, 236 Audubon Hall, Louisiana State University, Baton Rouge, Louisiana 70803.

The Northup et al. (1996) study has ap- parently not been replicated, and no other similar evaluations of the accuracy and treat- ment utility of reinforcer surveys could be found. The present report provides a system- atic replication of the procedures of Northup et al. by (a) conducting a comparison of the results of a reinforcer survey and subsequent reinforcer assessments for a much larger sample of 20 children, and (b) presenting potential reinforcers (token coupons) simul- taneously in a more efficient concurrent-op- erants arrangement rather than singly as in Northup et al. (1996). Presenting the token coupons simultaneously is both more effi- cient and provides a more direct assessment of relative reinforcement effects.

METHOD

Participants

The files of all children who attended a summer program for children with a diag- nosis of ADHD during the past 5 years were reviewed. Children were included in this evaluation on the basis of the following cri- teria: (a) The results of a reinforcer survey as described below were available; (b) the re- sults of a reinforcer assessment that included at least three baseline sessions, three assess-

336 JOHN NORTHUP

ment sessions, and three return to baseline sessions were also available; and (c) the re- sults had not been published elsewhere.

General Procedure

The survey and reinforcer assessment pro- cedures were similar to those of Northup et al. (1996) with the following exceptions: (a) Token coupons were presented simulta- neously in a concurrent-operants arrange- ment, (b) a separate category for peer atten- tion was included, and (c) easy math prob- lems, rather than a coding task, were used during the reinforcer assessment. Token cou- pons were used to represent a wide range of potential reinforcers that are common to this population but that are difficult or impos- sible to tangibly or immediately provide dur- ing an assessment (e.g., to get out of a school activity).

Reinforcer survey. The reinforcer survey in- cluded a total of 42 items, with seven items representing each of the following six cate- gories; edible items, peer attention, activities, tangible items, teacher attention, and escape. The survey was administered verbally with the instruction that ‘‘I’m going to name some things that kids sometimes get in school. I want to know how much you like each of these things. After I name each thing you tell me if you like it ‘not at all,’ ‘a little,’ or ‘a lot.’ For example . . .’’ Rankings of each item were given the value of 1, 2, and 3, for not at all, a little, and a lot, respectively, and a percentage score was calculated for each category by dividing the summed score of the item rankings by the total possible score. Categories with a score of 75% or greater were considered to be high preference. The complete survey is available from the author.

Reinforcer assessment. During baseline, the child was seated across from an examiner, given a worksheet containing easy math problems (i.e., problems previously complet- ed with greater than 90% accuracy), and giv- en the instruction that ‘‘you can do as much

as you want, as little as you want, or none at all.’’ The session ended if the child stated that he or she did not want to do any, said that he or she was done, or worked for 5 min. During the reinforcer assessment, to- ken coupons representing each of the six re- inforcer categories and the associated stimuli on the survey (and a control coupon) were placed directly above the worksheet. The stimuli associated with each coupon were re- viewed and sampled by each child prior to the assessment. The child was given the same instructions as during baseline but was also told that if he or she completed a cri- terion number of problems he or she could stop and take a coupon of his or her choice and that he or she could earn as many cou- pons of any kind as he or she wanted. The criterion number of problems required to choose a coupon was determined individu- ally based on the average number of prob- lems completed during baseline and was marked on the worksheet prior to the ses- sion. Problems did not have to be completed correctly; however, the correct number of digits for each answer had to be provided (accuracy was typically 100%, in that prob- lems were predetermined to be easy). Ses- sions ended as in baseline except that no time limit was imposed and the child was allowed to work (and choose coupons) until he or she stated that he or she was done. A return to baseline was conducted to dem- onstrate any reinforcement effects associated with the token coupons. The chosen cou- pons could be exchanged for any of the as- sociated stimuli at any time upon request, as in Northup et al. (1996).

RESULTS AND DISCUSSION

The overall accuracy of the survey was de- termined as follows. For the survey, each cat- egory of stimuli (i.e., edible items, peer at- tention, etc.) was first determined to be ei- ther high (.75%) or low (,75%) prefer-

337REINFORCER SURVEYS

Table 1 Accuracy of the Reinforcer Survey

True positives

True negatives

False positives

False negatives

Total possibilities

Total accuracy

41 (34%)

28 (23%)

35 (29%)

16 (13%)

120 57%

ence for each participant. For the reinforcer assessment, the total number of problems as- sociated with each coupon was first deter- mined for each session (i.e., all problems completed prior to each coupon selection were attributed to the coupon selected). Subsequently, all token coupons associated with clear reinforcement effects were iden- tified for each participant. The criteria for clear reinforcement effects were the same as in Northup et al. (1996) and were as fol- lows: (a) The number of completed math problems associated with the coupon was greater than baseline, and (b) the number completed was greater than those for the control coupon for each session. The accu- racy of the reinforcer survey then was cal- culated by determining the number of (a) true positives (categories identified as high preference on the survey that functioned as reinforcers), (b) false positives (categories that were identified as high preference that did not function as reinforcers), (c) true neg- atives (categories that were identified as low preference on the survey that did not func- tion as reinforcers), and (d) false negatives (categories that were identified as low pref- erence on the survey that did function as reinforcers). Total accuracy was calculated as the sum of true positives and true negatives divided by the total of all possibilities.

Table 1 shows the overall accuracy of the reinforcer survey compared to the results of the reinforcer assessments. The total accu- racy of 57% was nearly identical to the 55% accuracy reported by Northup et al. (1996). The percentage of true and false positives and negatives was also quite similar and nev-

er varied by more than 7%. Overall, the re- sults of this comparison replicate those of Northup et al. (1996) with a much larger sample of children. The results further sug- gest that reinforcer surveys may add little to an assessment beyond chance. The relatively high number of false positives and low num- ber of false negatives again suggest that sur- veys may more accurately identify stimuli that are not reinforcers than those that are. False positive results would typically be ex- pected to be of much greater concern than false negative ones in applied settings. That is, selecting stimuli for use in a behavioral treatment that do not in fact function as re- inforcers (a false positive) will presumably result in treatment failure, whereas a failure to identify a potential reinforcer (false neg- ative) may be of little or no concern (as long as at least one true reinforcer is identified). Because the results were very similar to those of Northup et al. (1996), they further sug- gest that presenting the token coupons si- multaneously, rather than singly, had little or no effect on overall results.

A limitation of the procedures is that the potential effect of a single highly preferred item on the survey could have been masked by its inclusion in a category with other low- preference items (a false negative for only that item). However, an inspection of indi- vidual survey responses did not indicate this outcome, and a comparison of overall cate- gories was of primary interest in this study. Similarly, it is possible that the absolute val- ue of any one coupon may be obscured by the simultaneous presentation of all cou- pons. However, reinforcer assessments are

338 JOHN NORTHUP

most likely to be conducted in applied set- tings when there is a particular interest in identifying the most potent or valued rein- forcers for use in a behavioral treatment. It may also be considered a design limitation of this study that survey categories received a score independent of any comparison to any other category.

Some stimuli, such as access to widely popular edible items and toys, may typically function as reinforcers for so many children that no formal assessment is needed. How- ever, the current challenge for behavior an- alysts is to accurately identify naturally oc- curring contingencies that maintain problem behaviors such as the relative value of teach- er, parent, and peer attention, or escape from aversive events. What is needed is the further development of practical and efficient as- sessment methods for verbal children that can more accurately identify such contingen- cies. For example, the further development of verbal choice procedures and the evalua- tion of other procedures (e.g., instructions) or conditions that may mediate reinforcer

assessment accuracy for verbal children may be fruitful directions for future research. The use of more experimentally based reinforcer assessments, such as those described in this report, may offer both a valid criterion mea- sure for future research with verbal children and an interim alternative method of assess- ment if a more rigorous reinforcer assess- ment is considered necessary or desirable.

REFERENCES

Bernstein, D. J., & Michael, R. L. (1990). The utility of verbal and behavioral assessment of value. Jour- nal of the Experimental Analysis of Behavior, 54, 173–184.

Northup, J., George, T., Jones, K., Broussard, & Voll- mer, T. (1996). A comparison of reinforcer as- sessment methods: The utility of verbal and pic- torial choice procedures. Journal of Applied Behav- ior Analysis, 29, 201–212.

Risley, T. R., & Hart, B. (1968). Developmental cor- respondence between the nonverbal and verbal be- havior of preschool children. Journal of Applied Behavior Analysis, 1, 267–281.

Received April 20, 1999 Final acceptance May 14, 2000 Action Editor, Cathleen Piazza