Discussion Five

Vol.:(0123456789)

Journal of Behavioral Education (2018) 27:435–460 https://doi.org/10.1007/s10864-018-9304-0

1 3

ORIGINAL PAPER

Exploring Issues of Generalization and Maintenance in Training Instructional Aides in a Public School Setting

Solandy Forte1,2 · Michael F. Dorsey1 · Mary Jane Weiss1 · Mark J. Palmieri2 · Michael D. Powers2

Published online: 3 July 2018 © Springer Science+Business Media, LLC, part of Springer Nature 2018

Abstract In today’s climate, with the ever-increasing demand for competent behavior ana- lytic services, it is necessary for behavior analysts to work across a variety of set- tings, including home, school, and community, overseeing instructional aide staf as the frontline providers of applied behavior analysis (ABA) services. It is com- mon practice for ABA practitioners to provide the supervision and training to par- ents, caregivers, and direct-care staf, who may have limited knowledge and experi- ence within the feld of ABA. In 2013, the Behavior Analysis Certifcation Board (BACB®) initiated an efort to establish standards for the training of instructional aides and direct-care staf in the delivery of instructional and treatment services based on the principles of ABA. This new standard and credential is known as a Registered Behavior Technician™ (RBT®). The purpose of this study was to evalu- ate the training process outlined by the BACB, focusing specifcally on the general- izability and long-term maintenance of newly acquired skills taught through a com- petency-based approach, to novel opportunities for implementation. Training was conducted through two methods: in vivo training with clients versus role-play with peers. Both these types of staf training approaches are commonly used instructional practices in the feld of ABA.

Keywords Staf training · Competency-based training · Registered Behavior Technician · Pyramidal training · Behavioral skills training · Developmental disabilities · Role-play · In vivo

• Solandy Forte [email protected]

1 Endicott College, 376 Hale Street, Beverly, MA 01915, USA 2 Center for Children with Special Needs, 2300 Main Street, Glastonbury, CT 06033, USA

436 Journal of Behavioral Education (2018) 27:435–460

1 3

Introduction

With the increase in the prevalence of autism spectrum disorder (ASD) over the years from 1 to 2 per 10,000 in 1980 to 1 in 59 today (Centers for Disease Con- trol and Prevention 2018), and the research supporting the efectiveness of early intervention and the use of ABA interventions (e.g., Myers and Johnson 2007; Howard et  al. 2005; Eikeseth et  al. 2007; Lovaas 1987), there has also been an increase in the demand for competent and qualifed behavior analysts (Hughes and Shook 2007). For the past 20  years, the feld of applied behavior analysis (ABA) has focused on the increase in the demand of ABA services and the lack of professionals trained in this area of expertise (Hughes and Shook 2007; Moore and Shook 2001). As noted by Dorsey et al. (2009),

The feld of applied behavior analysis (ABA) has experienced extraordinary growth in the number of practitioners as well as those seeking services. This change appears to be related to the explosion in the number of children diagnosed with pervasive developmental disorder/autism and the recogni- tion of the success of behavior analytic treatments (p. 53)

According to ChildStats.gov (the Federal Interagency Forum on Child and Family Statistic), there are currently 73.5 million children in the USA between the ages of 0 and 17 years old. Based on these statistics and the current preva- lence rates for Autism Spectrum Disorders (ASD), there are 1,079,411 children who meet the Diagnostic and Statistical Manual V standards for a diagnosis of ASD. According to the Behavior Analysis Certifcation Board (BACB®), there are 26,763 Board Certifed Behavior Analysts (BCBA®)/Board Certifed Assis- tant Behavior Analysts (BCaBA®) in the USA. Even with this density of qualifed professionals, and even if every Behavior Analyst in the USA worked exclusively with children on the autism spectrum, the ratio of clinicians to children with ASD would be 1:43. As a result, there is a growing demand for non-certifed individu- als such as instructional aides or direct-care staf, to receive the level of train- ing necessary for implementing behavioral interventions under the supervision of a behavior analyst. Recently the Behavior Analyst Certifcation Board (BACB) introduced its own credentialing of behavior technicians referred to as Registered Behavior Technicians (RBT®). In 2014, the BACB established a set of require- ments that individuals need to meet in order to be credentialed. These criteria have since been revised to include an examination requirement.

The requirements for the RBT credential are as follows: (a) be at least 18 years of age, (b) complete and pass a criminal background check, (c) have obtained a minimum of a high school diploma, (d) complete a 40-h, combined didactic and experiential training program that is aligned with the RBT® task list, (e) pass a competency checklist (demonstrate the ability to implement behavioral interven- tions and tasks in a role-play or in vivo format) as evaluated by the trainer, (f) pass the RBT® examination (requirement introduced in January 2015), and (g) pay all of the fees associated with the RBT® credentialing process. Once an individual is credentialed as an RBT, they must receive ongoing supervision for a minimum of

437

1 3

Journal of Behavioral Education (2018) 27:435–460

5% of the hours spent per month implementing the principles of ABA in the work setting. In addition, they must apply for re-credentialing annually and adhere to the maintenance requirements which include meeting satisfactory completion of 12 out of 20 task list items on the RBT competency assessment, maintain ongo- ing supervision, and adhere to ethic requirements (BACB 2014).

Behavior analysts are faced with the challenge of executing a variety of job responsibilities to ensure that treatment derived from the principles of ABA is implemented with integrity and that these interventions are making a positive impact on the life of the consumer. It is often necessary for the implementers of behavioral technology (BCBA® or RBT®) to work across a variety of settings, including home, school, and community. Within these settings, it is common practice for behavior analysts to provide training to caregivers as well as direct-care staf, who may have limited knowledge and experience within the feld of ABA (Lerman 2009), but who are often charged with the implementation of behavioral technology with minimal supervision. As of April 3, 2017, the BACB reported that there were as many as 31,055 individuals credentialed as a RBT®. Once credentialed, the RBT® is not required to participate in continuing education activities to help to ensure that they are trained in contemporary behavior analytic interventions but are required to be supervised for a minimum of 5% of the hours they provide behavior analytic ser- vices per month (e.g., 1.5 h. for every 30 h. of direct service provided), one of which must be while delivering such services. Under such circumstances, an RBT® may be assigned to work alone in a 1:1 staf–child ratio, with a child diagnosed on the autism spectrum for 30-to-40-h per week in the child’s home, receiving a minimum of 1.5–2  h of supervision per week. Some have suggested that these provisions might not meet the needs for continued professional oversight and skill development (Leaf et al. 2016).

Specifc concerns have been raised about the RBT® training requirements (Leaf et  al. 2016), questioning whether the requirements were adequate enough to train staf to provide high-quality ABA services to a vulnerable population of individu- als served by professionals within the feld. Leaf et al. (2016) raised the following concerns: (a) the number of training hours required to become an RBT® are not suf- fcient enough to be “extensive”; (b) the content of the RBT® task list is incomplete, and the items are not operationally defned; (c) the assessment of knowledge and on-the-foor competencies are developed by the individual trainer, and there are no recommended measures for objectively determining if a trainee has, in fact, passed these competencies; (d) the majority of the on-the-foor competencies can be admin- istered using a role-play format; and (e) there has not been any evaluation to deter- mine the impact this credential will have on the cost of ABA treatment, the quality and integrity of services provided, and the impact on consumers receiving ABA ser- vices by an RBT® who may not have been trained sufciently. Leaf et al. stress the importance of developing standards for credentialing based on empirical data.

In response to this critique, Carr et al. (2017) noted that the BACB® adopts stand- ards for credentialing based on the recommendations from a group of subject matter experts (SMEs), and that the standards are reviewed and evaluated every 5 years. While these authors welcome researchers within the feld to evaluate the standards for RBT® credentialing, they admit that operationalizing and measuring the critical

438 Journal of Behavioral Education (2018) 27:435–460

1 3

variables will be a difcult task. They also note that such research would need to be conducted across a variety of settings, as there most likely will be diferences that will impact the results. It seems that Carr et al. call for immediate research, which may contribute to the future revision of credentialing standards, to be conducted in this area and note that the research will take time to conduct and disseminate none- theless, they indicate that there is an immediate need for credentialing of nonprofes- sionals. Whether the decision to move forward before operationalizing and measur- ing this critical variable is a prudent one or not is beyond the scope of this study, but the points made by Leaf et al. (2016) and Carr et al. are certainly noteworthy.

In support of both the concerns raised by Leaf et al. (2016) and the suggestions made by Carr et al. (2017), the purpose of the present study was to conduct an initial assessment of a training package aligned with the RBT® task list utilizing a compe- tency-based training approach, specifcally evaluating the generalization and main- tenance of targeted skills using this training approach with paraprofessional public school staf. The study investigated in  vivo versus role-play training methods for providing competency-based training, including: the rate of skill acquisition of train- ees and the generalizability and maintenance of the trainees’ newly acquired skills.

There is a wealth of literature supporting the efectiveness of the competency- based training model utilized in this study for training staf in the implementation of behavioral techniques using its various teaching components (providing explicit directions, modeling, and feedback) as well as on the efects of staf training on client outcomes, consumer satisfaction, and future directions for the feld of ABA with respect to the study of competency-based training (Parsons and Reid 1995; Kratochwill and Bergan 1978; O’Reilly et al. 1992; Weinkauf et al. 2011; Lepage et al. 2004). Competency-based training packages typically include a didactic train- ing component that provides the trainee with information on a given topic related to intervention through a series of lectures. There are benefts and limitations for both the didactic and competency-based training models.

Competency-based training programs have been described for decades, primar- ily in the training of teachers and counselors (Bergan et  al. 1980) and have been expanded to encompass the training of direct-care staf working with individuals with developmental disabilities (Parsons and Reid 1995; Parsons et al. 2012). While there is a wealth of literature on competency-based training models, there is limited research evaluating the outcomes of competency-based training on the generalizabil- ity and long-term maintenance of skills acquired by direct-care staf.

Method

Setting

The present investigation was conducted in two classrooms located within a local public elementary school. The frst was an integrated preschool classroom and the second a special education classroom serving kindergarten to ffth-grade students. All training activities, including lectures, role-plays, and in vivo competency-based training, occurred within the same public school setting. The didactic training was

439

1 3

Journal of Behavioral Education (2018) 27:435–460

delivered in the library media center, which was equipped with a laptop computer, projector, and screen for displaying PowerPoint presentations. The tables and chairs were arranged in a classroom style to allow participants to be able to view the mate- rial on the screen easily and take notes.

The role-play and in vivo training occurred within the same preschool integrated classroom and across special education classrooms in an elementary school setting (i.e., kindergarten to ffth grade). The preschool and special education classrooms were located on the frst foor of an elementary school. The preschool classrooms were arranged with designated center-based play areas, a carpet for circle time, tables and chairs, and a variety of materials and visual supports for individual, small, and large group instruction.

The special education classrooms included an instructional area for each student. Each instructional area was equipped with a desk or table and chair (appropriate for the student’s height), a storage shelf with materials organized into bins and folders, visual supports (schedules, choice boards, calendars, checklists, etc.), toys, and other tangible items used for reinforcement. Each student’s instructional area contained material that was individualized for his or her instruction (token economy system, lesson plans, data collection sheets, slant boards, timers, etc.).

Participants

The participants in this research study were instructional aides assigned to provide one-to-one individualized instruction based on the principles of ABA to children with a variety of developmental disabilities (including but not limited to ASD, Down syndrome, cerebral palsy, intellectual disability, as well as other related neu- rodevelopmental disorders). The participants were all full-time employees of the public school district. The educational training and experience of the participants varied relative to their knowledge of the principles of ABA and the implementation of behavioral strategies, and in their years working within educational settings. All instructional aides were at least 18 years of age and had a minimum of a high school diploma or national equivalent. Each instructional aide was assigned to implement skill acquisition and behavior reduction plans designed by a BCBA® to at least one student within the ABA program. All participants voluntarily consented to partici- pating in the research study. Their consent was documented via a formal consent form approved by a university Institutional Review Board (Title 45 Code of Federal Regulations Part 46), in which they were informed that they could withdraw their participation at any time and for any reason, with no penalty.

Prior to beginning the study, a questionnaire was distributed to each participant to gather demographic information as well as information related to past profes- sional experience within educational settings, prior training in ABA, and experience with individuals with special needs. The pool of participants was comprised of nine females, ranging in age from 26 and 65 years, all of whom completed the 40-h com- bined didactic and experiential training package aligned with the BACB® RBT® cre- dential task list. Five of the participants had high school diplomas, two had associ- ate’s degrees, and two had bachelor’s degrees.

440 Journal of Behavioral Education (2018) 27:435–460

1 3

Seven of the participants reported having prior training in the implementation of ABA strategies (ranging from 3 to 10  years of prior experience), and all par- ticipants reported having training in a lecture-series format. Specifcally, fve of the participants reported receiving prior training in the implementation of discrete trial instruction and error correction procedures, and six participants reported receiving prior training in the delivery of reinforcement.

For this study, fve participants were randomly assigned to receive their experi- ential training through a role-play format, and four participants through an in vivo format. Participants were assigned to their respective groups for experiential training using a random number generator (random.org, 2016).

Design

A multiple baseline design (Baer et al. 1968) across participants was used to evalu- ate the efcacy of the role-play versus in vivo training on skill acquisition, generali- zation, and maintenance. A minimum of three skills was targeted for acquisition per participant (all participants received training across the same targeted skills). The targets were chosen based on recommendations from the Association for Behavior Analysis International—Autism Special Interest Group Consumer Guidelines.

The nine participants were randomly assigned across two groups (role-play vs. in vivo groups). Data for each skill were compared across training phases and across participants, thereby addressing the need for equivalence of skills. Data collection occurred during the following phases: baseline, post-didactic training probe, role- play or in vivo training, generalization, and maintenance. More specifcally, the data used to assess rate of acquisition for each skill were collected during the role-play and in vivo training phases of the study. The data used to assess the generalizabil- ity and maintenance of each skill were collected during the post-training phase of the study, after the participants had completed the didactic training for the specifc skill, and after the role-play or in vivo training phase had been completed by each participant.

Dependent Variable(s)

In order to assess the impact of the competency training method (role-playing vs. in  vivo competency training) competency scores were calculated (percentage of treatment integrity) from data collected for each participant using a task analysis for each skill. Again, there was a minimum of three targeted skills per participant. Collecting baseline data assisted the researcher in determining if the participant had already met profciency with regard to the targeted skill before the start of either direct training model.

The targeted skills selected for this study were based on recommendations from the Association for Behavior Analysis International—Autism Special Inter- est Group Consumer Guidelines, which identify the importance of training specifc behavioral techniques to implementers. These skills included but were not limited to the implementation of discrete trial instruction, error correction, prompting, and

441

1 3

Journal of Behavioral Education (2018) 27:435–460

reinforcement (ABAI—Autism SIG, 2007). The skills targeted for acquisition in this study were also identifed on the RBT® task list developed by the Behavior Analyst Certifcation Board (BACB 2014).

The three skills targeted for training included the implementation of a discrete trial, an error correction, and the delivery of reinforcement. Each skill was objec- tively operationally defned using a task analysis outlining the steps for successfully completing each procedure, and these defnitions (steps) are described in Table 1.

Independent Variable(s)

Role‑Play Training

Participants randomly assigned to this group were presented with role-play scenarios for the opportunity to demonstrate each targeted skill. Components of the role-play training included providing: (1) the participant with explicit instructions on how to perform each skill, (2) a model, (3) feedback on performance, and (4) opportunities to practice each skill until they met criterion for mastery. A BCBA®, who was asked to play the role of a client, was instructed to look in the opposite direction of the par- ticipant (in order to provide the participant with an opportunity to gain the student’s

Table 1 The table below outlines each of the steps necessary for completing each of the targeted skills for acquisition (i.e., dependent variables)

Skill Steps of task analysis

Targeted skills for acquisition Implementation of discrete trial Clears area of extraneous materials

Places instructional material in front of student (in student’s view) Places materials in a feld size of 3 (when asked) Gains student’s attention (e.g., waits for eye contact or for student to

look at materials) Delivers correct SD

Provides clear and concise initial instruction Waits for student’s response (no more than 3 s) Delivers a light physical prompt within 3 s of observing incorrect

response Implementation of error correction Refrains from providing any reinforcement or comments about error

Provides light physical prompt to hand or elbow within 3 s of observing incorrect response

Rearranges materials on desk or table to prepare for representation of trial

Represents the original discrete trial (last trial presented where student made error)

Delivers reinforcement within 3 s of observing correct response Delivery of reinforcement Delivers verbal praise within 3 s of observing correct response

Uses an enthusiastic (not neutral) tone of voice when delivering verbal praise

Delivers token within 3 s of observing correct response Delivers tangible item for completion of token economy system (i.e.,

flled token board)

442 Journal of Behavioral Education (2018) 27:435–460

1 3

attention) and respond incorrectly to the instruction given by the participant. Each training opportunity (i.e., trial) was novel to the participant. In other words, the par- ticipant was given a new scenario each time they were given the opportunity to prac- tice the skill. The length of the training session(s) varied depending on how many trials it took each individual participant to meet criterion for mastery (see results).

In Vivo Training

Participants randomly assigned to this group were presented with a real-life scenario for the opportunity to demonstrate each targeted skill with a client. Components of the role-play training included providing: (1) the participant with explicit written and verbal instructions on how to perform each skill, (2) a model, (3) feedback on performance, and (4) opportunities to practice each skill until they met criterion for mastery. The participants were each asked to demonstrate each skill with a client after being given a model and explicit instructions. The clients were asked to per- form a skill that was novel (in order to provide the participant with an opportunity to deliver a prescribed prompt). Similar to the role-play training procedure, each train- ing opportunity (i.e., trial) was novel to the participant. The length of the training session(s) varied depending on how many trials it took each individual participant to meet criterion for mastery (see results).

Procedures

Baseline

Participants were asked to perform each of the skills targeted for acquisition in a role-play scenario with a BCBA. Observations were conducted on each participant to obtain competency scores (percentage of treatment integrity) across a minimum of three sessions for each targeted skill. These data served as a baseline; therefore, no feedback was provided to the participants.

Post‑baseline

A 40-h, didactic training series was provided to each participant in a group format, comprised or all nine participants. The training series was developed such that the content was aligned with the BACB RBT® task list. For each of the targeted skills, a written topic-specifc lesson (within a didactic training format) was provided to each participant. The didactic training was delivered by a BCBA® in a lecture format using PowerPoint slides. The lessons were specifc to the skills targeted for training, and they included but were not limited to commonly used prompts, reinforcement, error correction, discrete trial instruction, incidental teaching, teaching and data col- lection protocols, task analysis, shaping behavior, and strategies for managing chal- lenging behavior.

Pre- and post-tests were administered to each participant to assess knowledge competencies before and after training. Each test included fve multiple-choice

443

1 3

Journal of Behavioral Education (2018) 27:435–460

questions, directly related to the material outlined in the didactic training series. The purpose of this pretest was to identify any prior knowledge the participant had regarding the principles of ABA.

Training Phase

Competency-based training was provided to two training groups (role-play and in vivo) in an individual format for each participant. In the frst group, each partici- pant was trained individually through role-play to perform a targeted skill. The role- play training phase was then followed by a generalization and maintenance phase, where probe data were collected to assess generalization of the skill after the role- play training session. In the second group, each participant was trained individually with clients in vivo to perform a targeted skill. The in vivo training phase was then also followed by a generalization and maintenance phase exactly as outlined above. The participants did not move to the generalization and maintenance phases under either training paradigm until they had met criteria for mastery (a score of 100% across three consecutive trials) for each targeted skill.

Prior to the role-play or in vivo training, a BCBA® reviewed any protocols (e.g., skill acquisition or behavior reduction) with the participant specifc to the role-play or client’s instructional or behavioral protocol as this was a standard of treatment. The participant was then given the opportunity to ask questions.

Role‑Plays

The role-play participants moved into the role-play individual training phase after they had passed a topic-specifc quiz following the initial didactic training, which was delivered after the baseline phase.

After two trainers had conducted a demonstration of the targeted skills within a role-play situation, the participant was then presented with a scenario to perform the targeted skills (implementation of a discrete trial, error correction, or reinforce- ment). A trained BCBA® followed a script with explicit instructions on how to per- form during a role-play, and the participant was asked to demonstrate the targeted skill during the role-play scenario. Upon completing each role-play, feedback was provided, and the participant was asked to perform the skill a minimum of three times, or until the participant had met profciency (a score of 100% across three con- secutive trials).

Feedback (positive and/or corrective) was delivered by reviewing each step of the procedure with specifc detail. Each step of the task was reviewed, and the par- ticipants were told what steps they had performed correctly. They were also pro- vided with corrective feedback for incorrect responses. To ensure that each partici- pant received the same level and quality of feedback, each skill had a task analysis outlining the steps necessary for completing the skill successfully. The steps out- lined in the task analysis provided an operational defnition for each of the targeted skills. Each role-play session was completed within 30 min or less. If the participant did not meet the criteria for mastery after 10 training trials, the training phase was

444 Journal of Behavioral Education (2018) 27:435–460

1 3

deferred for retraining in order to prevent training fatigue. Retraining procedures were identical to the training procedures for each respective training group. This did not occur for any of the participants in the in vivo group.

In Vivo

The in vivo participants moved into the in vivo individual training phase after they had demonstrated knowledge competencies in the initial didactic training, which was delivered following the baseline phase.

A trained BCBA® followed a script with explicit instructions indicating what should be said to the participant and modeled the implementation of the skill with a client. After watching a trainer implement, the targeted skill with a child and then implementing the procedure with the trainer, the participant was asked to demon- strate the skill independently. The participants did not have any prior working rela- tionship with the children in the in vivo training phase. This training procedure was done for each targeted skill (implementation of a discrete trial, error correction, and reinforcement). Feedback was provided upon completing each skill.

Participants were asked to perform each skill a minimum of three times, or until the participant met profciency (a score of 100% across three consecutive trials). As described previously, each step of the task was reviewed, and the participants were told what steps they had performed correctly. They were also provided with cor- rective feedback for incorrect responses; each step of the task was reviewed with participant as a form of providing feedback. To ensure that each participant received the same level and quality of feedback, each skill had a task analysis outlining the steps necessary for completing the skill successfully. The steps outlined in the task analysis provided an operational defnition for each of the targeted skills. Each in vivo training session was completed within 30 min or less. If the participant did not meet criteria for mastery after 10 training trials, the training phase was deferred for retraining in order to prevent training fatigue. Retraining procedures were identi- cal to the training procedures for each respective training group. This did not occur for any of the participants in the in vivo group.

Generalization Phase

One of the objectives of this study was to examine the generalizability of training efects of staf training models (role-play vs. in  vivo). The generalization phase included the collection of data to evaluate the efcacy of the training methods in demonstrating generalized skills when applied to working with clients in novel opportunities. These novel opportunities are situations that present the participant with a chance to demonstrate the newly acquired skill in an untrained situation that is new or unfamiliar to the participant. Further, it is a situation that warrants the implementation of the newly acquired skill. For instance, the participant would be asked to implement a discrete trial with a novel student.

To assess the impact of the competency-based training method on generalizabil- ity, competency scores were calculated from data collected for each targeted skill.

445

1 3

Journal of Behavioral Education (2018) 27:435–460

The generalization phase was conducted within 5 days of completing the role-play and the in  vivo training phase for each skill across all participants. No reinforce- ment or prompting was provided during this generalization phase. The generaliza- tion phase included one generalization probe for each targeted skill per participant.

Maintenance Phase

A fnal goal of the study was to evaluate the maintenance of skills over time. To accomplish this, observations were conducted to obtain competency scores at 30 and 60 days post-training for each participant (one data point was collected at 30 and 60 days). To account for diferences of each client’s learning profle, the par- ticipant was asked to demonstrate the newly acquired skill with the same learner from the training phase. In other words, the participant was observed performing the skills with the same learner during the training and maintenance phase. The full- time BCBA® normally assigned to the program was instructed to continue providing consultative services, which included the training of all school-based instructional aides, including the participants in this study, in the implementation of instructional and behavioral supports. Thus, the participants in this study were routinely provided with corrective feedback and in  vivo supplemental training, consistent with pre- established program supervision procedures, during the maintenance phase of the study.

Treatment Integrity

Treatment integrity was evaluated to ensure consistent implementation of training methods for each participant. Before beginning this study, the primary researcher reviewed the didactic and competency-based training materials with each BCBA®. Reviewing the material included discussing the points on each slide, answering questions related to training material, conducting the role-plays with the researcher, reviewing child-specifc instructional or behavioral protocols, and reviewing data collection systems. Some training components required demonstration (data collec- tion and role-play scenarios).

In addition, for each training session (didactic and competency-based train- ing session) the trainer would record whether the participant was present or absent from the training session. In order for the participant to be present, he or she had to have attended the entire training, which meant viewing all slides as delivered by the trainee and completing all training activities.

Interobserver Reliability Measures

Reliability observations were conducted by two BCBA’s for 49% of data collection sessions for each participant, across all conditions. The Interobserver Reliability score was obtained by dividing the total number of agreements for each observa- tion by the total number of agreements plus disagreements and multiplying that number by 100. The observed sessions were comprised of the following phases:

446 Journal of Behavioral Education (2018) 27:435–460

1 3

baseline, post-didactic training, role-play and in  vivo training, generalization, and maintenance. The results were calculated using an interval by interval agreement method (dividing the total agreement by the sum of agreement and disagreement; Hawkins and Dotson 1975). This method was used since it is more sensitive than total reliability in that it separates observational time into intervals and assesses the agreement and disagreement for every interval between two observers. The interval agreement results were 88.5% across all subjects and sessions.

Evaluation of Social Validity

In order to provide a measure of social validity, a pencil and paper anonymous sur- vey was administered to all participants in the study, 5 days after the completion of the competency-based training phase, to assess their satisfaction with the training provided. The survey allowed participants to rate various components of the train- ing package delivered by the research team, using a Likert scale rating system. The participants were presented with a series of statements and questions (see Table 2) related to the training delivered. They were asked to rate their level of agreement or disagreement and the level of learning difculty for each skill taught in the training using a symmetric scale. The symmetric systems included the following two scales: (a) strongly agree, agree, neither agree or disagree, disagree, and strongly disagree and (b) very difcult, somewhat difcult, somewhat easy, and very easy. In addition, the participants were provided with one open-ended question for providing feedback with regard to improving the training.

Results

The efects of the role-play and in vivo training provided to instructional aides to teach the three targeted skills are shown in Figs. 1, 2, 3, 4, 5 and 6. Figure 1 displays the results of role-play training for the implementation of a discrete trial across fve diferent participants. Baseline scores ranged from 0 to 63% for correct responding, and the average score across all participants was 40%. Post-didactic training scores ranged from 37 to 50% for correct responding, and the average score across all par- ticipants was 45%. During the role-play training phase, data indicate that trials to criterion (mastery) ranged from six to 13, with participants requiring an average of 10 trials to meet mastery of this skill (implementation of a discrete trial). Generali- zation scores ranged from 50 to 75% for correct responding, and the average score across all participants was 62%. Maintenance scores ranged from 38 to 100% for correct responding, and the average score across all participants was 65%.

Figure 2 displays the results of role-play training for the implementation of an error correction across fve diferent participants. Baseline scores ranged from 0 to 60% for correct responding, and the average score across all participants was 27%. Post-didactic training scores ranged from 0 to 60% for correct responding, and the average score across all participants was 25%. During the role-play training phase, data indicate that trials to criterion (mastery) ranged from fve to 12, with

447

1 3

Ta bl

e 2

Th e

ta bl

e be

lo w

o ut

lin es

th e

st at

em en

ts , i

ns tru

ct io

ns , a

nd q

ue sti

on s p

re se

nt ed

in th

e su

rv ey

; t he

sy m

m et

ric sc

al es

u se

d; a

nd p

ar tic

ip an

t r es

ul ts

St at

em en

t/i ns

tru ct

io n/

qu es

tio n

Sy m

m et

ric L

ik er

t s ca

le (s

) R

at in

g re

su lts

Sa tis

fa ct

io n

su rv

ey re

su lts

 T he

st af

tr ai

ni ng

p ro

vi de

d by

th e

ag en

cy w

as im

po rta

nt

St ro

ng ly

a gr

ee

St ro

ng ly

a gr

ee (2

) A

gr ee

A

gr ee

(2 )

N ei

th er

a gr

ee o

r d is

ag re

e D

is ag

re e

St ro

ng ly

d is

ag re

e  T

he st

af tr

ai ni

ng p

ro ce

ss w

as e

fe ct

iv e

in im

pr ov

in g

m y

A BA

im pl

em en

ta tio

n of

sk ill

s St

ro ng

ly a

gr ee

A

gr ee

(3 )

A gr

ee

N ei

th er

a gr

ee o

r d is

ag re

e (1

) N

ei th

er a

gr ee

o r d

is ag

re e

D is

ag re

e St

ro ng

ly d

is ag

re e

 I fe

lt co

m fo

rta bl

e w

ith th

e st

af tr

ai ni

ng p

ro ce

ss

St ro

ng ly

a gr

ee

A gr

ee (1

) A

gr ee

N

ei th

er a

gr ee

o r d

is ag

re e

(1 )

N ei

th er

a gr

ee o

r d is

ag re

e D

is ag

re e

(2 )

D is

ag re

e St

ro ng

ly d

is ag

re e

 I fe

el th

at th

e sk

ill s t

au gh

t d ur

in g

th e

st af

tr ai

ni ng

w er

e ap

pl ic

ab le

to th

e da

ily w

or k

th at

I do

w ith

m y

St ro

ng ly

a gr

ee

A gr

ee (3

) stu

de nt

s A

gr ee

D

is ag

re e

(1 )

N ei

th er

a gr

ee o

r d is

ag re

e D

is ag

re e

St ro

ng ly

d is

ag re

e  T

he fe

ed ba

ck I

re ce

iv ed

fr om

th e

tra in

er (s

) d ur

in g

th e

tra in

in g

pr oc

es s h

el pe

d to

im pr

ov e

m y

pe rfo

r- St

ro ng

ly a

gr ee

A

gr ee

(2 )

m an

ce

A gr

ee

N ei

th er

a gr

ee o

r d is

ag re

e (1

) N

ei th

er a

gr ee

o r d

is ag

re e

D is

ag re

e (1

) D

is ag

re e

St ro

ng ly

d is

ag re

e

Journal of Behavioral Education (2018) 27:435–460

448 Journal of Behavioral Education (2018) 27:435–460

1 3

Ta bl

e 2

(c on

tin ue

d)

St at

em en

t/i ns

tru ct

io n/

qu es

tio n

Sy m

m et

ric L

ik er

t s ca

le (s

) R

at in

g re

su lts

 I fe

el th

at th

e am

ou nt

o f t

im e

al lo

ca te

d fo

r t he

st af

tr ai

ni ng

w as

a de

qu at

e St

ro ng

ly a

gr ee

A

gr ee

(1 )

A gr

ee

D is

ag re

e (3

) N

ei th

er a

gr ee

o r d

is ag

re e

D is

ag re

e St

ro ng

ly d

is ag

re e

 P le

as e

ra te

th e

le ve

l o f d

if cu

lty w

he n

le ar

ni ng

to im

pl em

en t a

d is

cr et

e tri

al

Ve ry

d if

cu lt

So m

ew ha

t d if

cu lt

(2 )

So m

ew ha

t d if

cu lt

So m

ew ha

t e as

y (2

) So

m ew

ha t e

as y

Ve ry

e as

y  P

le as

e ra

te th

e le

ve l o

f d if

cu lty

w he

n le

ar ni

ng to

im pl

em en

t a n

er ro

r c or

re ct

io n

Ve ry

d if

cu lt

So m

ew ha

t d if

cu lt

(1 )

So m

ew ha

t d if

cu lt

So m

ew ha

t e as

y (3

) So

m ew

ha t e

as y

Ve ry

e as

y  P

le as

e ra

te th

e le

ve l o

f d if

cu lty

w he

n le

ar ni

ng to

d el

iv er

re in

fo rc

em en

t Ve

ry d

if cu

lt So

m ew

ha t e

as y

(4 )

So m

ew ha

t d if

cu lt

So m

ew ha

t e as

y Ve

ry e

as y

 A re

th er

e an

y re

co m

m en

da tio

ns y

ou h

av e

fo r i

m pr

ov in

g th

e st

af tr

ai ni

ng p

ro ce

ss ?

O pe

n- en

de d

“S pe

nd m

or e

tim e

tra in

in g

w ith

st ud

en ts

o n

th e

fo or

”

449

1 3

Journal of Behavioral Education (2018) 27:435–460

Post Didac�c Implementa�on of Discrete Trial Gen-Test MaintenanceBaseline Training Probe Role Play Training

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

P5

P1

P4

P2

P3

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

Trials

Fig. 1 Graph represents data for the role-play training group on the implementation of a discrete trial

participants requiring an average of eight trials to meet mastery of this skill (imple- mentation of an error correction). Generalization scores ranged from 20 to 80% for correct responding, and the average score across all participants was 56%. Mainte- nance scores ranged from 0 to 100% for correct responding, and the average score across all participants was 52%.

Figure 3 displays the results of role-play training for the delivery of reinforcement across fve diferent participants. Baseline scores ranged from 25 to 75% for cor- rect responding, and the average score across all participants was 50%. Post-didactic training scores ranged from 0 to 50% for correct responding, and the average score

450 Journal of Behavioral Education (2018) 27:435–460

1 3

Post Didac�c Implementa�on of Error Correc�on Baseline Training Probe Role Play Training Gen-Test Maintenance

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

P2

0%

20%

40%

60%

80%

100%

P3

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

P5

P1

P4

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

Trials

Fig. 2 Graph represents data for the role-play training group on the implementation of an error correction

across all participants was 50%. During the role-play training phase, data indicate that trials to criterion (mastery) ranged from fve to nine, with participants requiring an average of seven trials to meet mastery of this skill (delivery of reinforcement). Generalization scores ranged from 0 to 100% for correct responding, and the aver- age score across all participants was 70%. Maintenance scores ranged from 50 to 100% for correct responding, and the average score across all participants was 80%.

Figure 4 displays the results of in vivo training for the implementation of a dis- crete trial across fve diferent participants. Baseline scores ranged from 17 to 88%

451

1 3

Journal of Behavioral Education (2018) 27:435–460

Post Didac˜c Baseline Training Probe

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

Delivery of Reinforcement Gen-Test MaintenanceRole Play Training

P2

P5

P1

P3

P4

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Trials

Fig. 3 Graph represents data for the role-play training group on the delivery of reinforcement

for correct responding, and the average score across all participants was 49%. Post- didactic training scores ranged from 25 to 50% for correct responding, and the aver- age score across all participants was 44%. During the in vivo training phase, data indicate that trials to criterion (mastery) ranged from fve to 15, with participants requiring an average of eight trials to meet mastery of this skill (implementation of a discrete trial). Generalization scores ranged from 50 to 87% for correct responding, and the average score across all participants was 66%. Maintenance scores ranged from 50 to 87% for correct responding, and the average score across all participants was 71%.

452 Journal of Behavioral Education (2018) 27:435–460

1 3

Post-Didac�c Implementa�on of Discrete Trial

0%

20%

40%

60%

80%

100%

0%

20%

40%

60%

80%

100%

P7

0%

20%

40%

60%

80%

100%

P8

0%

20%

40%

60%

80%

100%

P6

Baseline Training Probe In-vivo Training Gen-Test Maintenance

P9

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Trials

Fig. 4 Graph represents data for the in vivo training group on the implementation of a discrete trial

Figure 5 displays the results of in vivo training for the implementation of an error correction across fve diferent participants. Baseline scores ranged from 0 to 40% for correct responding, and the average score across all participants was 18%. Post- didactic training scores ranged from 0 to 40% for correct responding, and the aver- age score across all participants was 25%. During the in vivo training phase, data indicate that trials to criterion (mastery) ranged from fve to 13, with participants requiring an average of seven trials to meet mastery of this skill (implementation of an error correction). Generalization scores ranged from 0 to 100% for correct responding, and the average score across all participants was 40%. Maintenance scores ranged from 0 to 80% for correct responding, and the average score across all participants was 48%.

Figure 6 displays the results of in vivo training for the delivery of reinforcement across fve diferent participants. Baseline scores ranged from 50 to 75% for correct

453

1 3

Journal of Behavioral Education (2018) 27:435–460

Post-Didac˜c Implementa˜on of Error Correc˜on Baseline Training Probe In-vivo Training Gen-Test Maintenance

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

P7

P8

P6

P9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Trials

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

Fig. 5 Graph represents data for the in vivo training group on the implementation of an error correction

responding, and the average score across all participants was 58%. Post-didactic training scores ranged from 0 to 75% for correct responding, and the average score across all participants was 50%. During the in vivo training phase, data indicate that trials to criterion (mastery) ranged from fve to 12, with participants requiring an average of eight trials to meet mastery of this skill (delivery of reinforcement). Gen- eralization scores ranged from 0 to 100% for correct responding, and the average score across all participants was 44%. Finally, maintenance scores ranged from 0 to 100% for correct responding, and the average score across all participants was 75%.

It was noted that some participants were consistently making the same errors across consecutive baseline trials, and this may have helped to stabilize the data dur- ing the baseline phase. It should also be noted that scores increased from baseline to the post-didactic training probe phase for a number of participants (see Figs. 1, 2, 3, 4, 5, 6), and this can be attributed to the didactic training that was delivered relative

454 Journal of Behavioral Education (2018) 27:435–460

1 3

Post-Didac˜c Delivery of Reinforcement Baseline Training Probe In-vivo Training Gen-Test Maintenance

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

P7

P8

P6

P9

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Trials

Pe rc

en ta

ge o

f C or

re ct

R es

po nd

in g

Fig. 6 Graph represents data for the in vivo training group on the delivery of reinforcement

to each targeted skill. However, it was determined that these participants needed to participate in the competency training, as they did not meet the criterion for mastery. It should also be noted that scores decreased from baseline to the post-didactic train- ing probe phase for a number of participants (see Figs. 1, 2, 3, 4, 5, 6), and this may be due to participants working hard (after completing their didactic training) to cor- rect their errors during the probe trial and, therefore, not completing the sequence of responses necessary to adequately implement the targeted skill. It is emphasized that the participants, even those reporting some prior experience with ABA training, were not able to reliably implement the targeted skills after completing the 40-h, didactic training.

455

1 3

Journal of Behavioral Education (2018) 27:435–460

The satisfaction survey was completed by 4 out of the 9 participants so the results were interpreted with caution. The results of the survey are outlined in Table  2. Overall, the participants reported the training provided as part of this study was important and applicable to the work that they do within the public school setting. With the exception of one participant, they all agreed that the training was efective in improving their implementation of ABA skills. Most of the participants indicated that the amount of time allocated for the training was not sufcient. With respect to the level of difculty of each skill, the participants rated the skills somewhat difcult to somewhat easy (Table 3).

Discussion

The results of this study indicate that the training provided across both training groups was efective in teaching the skills to the participants; however, the data also show that the acquired skills were not generalized or maintained by a majority of the participants post-training. It was noted that all the participants benefted from formal training with respect to the targeted skills for this study.

The training phase outcomes for all the participants showed an increasing trend for each skill. After a series of training trials, each participant met mastery crite- rion for each skill. It is noted that the rate of acquisition was variable across par- ticipants and skills in each training group. These results cannot be explained by a participant’s prior experience, as the data indicate that the number of years of ABA training (i.e., through professional development, seminar or workshop, or on the foor), as reported by the participants, did not necessarily have a notable impact on their rate of acquisition. For example, participant four in the role-play training group reported to have had 5 years of prior ABA training, but required an equal or greater amount of training trials than the other group participants (who, respectively, reported 0–2 years of prior experience) in order to meet criterion for mastery for each targeted skill. Similarly, in the in  vivo training group, with the exception of participant nine who reported no years of prior ABA training and required the most amount of training trials compared to others in the group, participants six and eight, who reported having received 10 years each of ABA training, required an equal or slightly greater amount of training trials than participant seven, who reported having had 3 years of prior ABA training experience.

Table 3 The table below represents the range of scores and averages for each group across all phases of the study

Phase

Baseline

Role-play group

Range Average (%)

0–63 40

In vivo group

Range Average (%)

17–88 49 Post-didactic 37–50 45 25–50 44 Training 6–13 10 5–15 8 Generalization 50–75 62 50–87 66 Maintenance 39–100 65 50–87 71

456 Journal of Behavioral Education (2018) 27:435–460

1 3

Further, the data did not indicate a diference in the rate of acquisition across skills for both training groups. In other words, the data did not show that any one skill was acquired at a faster rate than another, but, again, rate of acquisition did vary across participants. For example, participant four in the role-play training group acquired the frst skill (implementation of a discrete trial) after 12 trials, the second skill (implementation of an error correction) after 12 trials, and the third skill (delivery of reinforcement) after nine trials. Similarly, participant nine in the in vivo training group acquired the frst skill in 15 trials, the second skill in 13 tri- als, and the third skill in 12 trials. Again, a variation in the rate of acquisition across skills was not noted for participants in either training group.

Generalization probe data indicated that the participants scored below mastery criterion during this phase, with most of the participants returning to baseline lev- els across skills. From the role-play group, participants two, three, and fve general- ized the third target skill (delivery of reinforcement), and from the in vivo group, participant six generalized the second (implementation of an error correction) and the third target skill (specify). With respect to the few participants who generalized the targeted skills mentioned above, this could be attributed to a diference in the number of steps required to complete each skill. For example, the task analysis for the implementation of a discrete trial requires the implementer to complete a series of eight steps; the implementation of an error correction requires fve steps; and the delivery of reinforcement requires four steps. Further, for those participants who demonstrated generalizability, it is unknown if their prior experience in ABA train- ing had an impact on these outcomes.

Maintenance probe data after approximately 30 days show that the majority of the participants across both training groups either maintained their generalization scores, which were below mastery criterion, or scored lower. There were three par- ticipants (fve, six, and nine) who met mastery criterion for one targeted skill each during the 30-day maintenance probe. The maintenance probe data after approxi- mately 60 days showed improvement in scores across participants, skills, and train- ing groups; however, the participants did not all meet mastery criterion for all the skills. Notably, there was one participant from the role-play group who met mas- tery criterion for all three targeted skills during the 60-day maintenance probe. Fur- ther, for those participants who reliably implemented the skills post-training it is unknown whether their prior ABA training had an impact on these outcomes. As noted earlier, the participants in this study were provided with feedback and in vivo supplemental training regularly with pre-established program supervision proce- dures, during the maintenance phase of the study.

Limitations and Future Directions

There were some limitations to this study. First, the researcher had limited access to information regarding each participant’s prior history with respect to ABA training, and the information that was obtained was reported by the participants in a written survey. As such, the researcher did not have a way to confrm independently if a participant’s report was accurate with regard to prior training in ABA. While this

457

1 3

Journal of Behavioral Education (2018) 27:435–460

researcher recognizes the difculty in obtaining information about an individual’s history that can be quantifed and that accurately represents an individual’s expe- rience, it is still necessary for this information to be considered, as it may impact training outcomes such as the rate of acquisition, generalization, and maintenance of skills. More specifcally it would be useful to be able to obtain access to records related to participants’ prior ABA training, such as the number of training hours, format or delivery of content, training topics, and applied activities. It might also be useful to replicate the procedures with naïve participants who do not have the con- found of prior training.

A second limitation of the study was that the targeted skills varied in the numbers of steps that each skill required for completion. Future studies should evaluate skills whose levels of difculty to perform are more equivalent to each other. Administer- ing a survey to experts in the feld of ABA may assist the researcher in identify- ing such skills with similar degrees of difculty. Weinkauf et al. (2011) provide an excellent discussion on the importance of utilizing content experts in this area.

A third limitation was the lack of information related to the amount (e.g., num- ber of hours) and quality of feedback that the participants received post-training. There was variability in the amount of supervision provided to trainees, and this could have accounted for their diferential maintenance. To be clear, these individ- uals were trained in alignment with guidelines for RBT certifcation, but did not become credentialed. In addition, the guidelines for post-training supervision were not implemented. Future research could examine how diferential ongoing levels of supervision are associated with diferential maintenance. In any case, the extension of these fndings to the post-training efectiveness of RBT standards is limited by these diferences from those standards.

The feedback and support provided by the BCBA consultant resembled the training provided to the in vivo training group; therefore, the feedback was univer- sal across all the participants, but still may have had an impact on the maintenance scores for the participants who demonstrated improvement or maintenance of skills after the generalization phase. It is unclear if the consultation support provided to all the participants had an impact on the outcomes. However, it is important to note that it would have been unethical to postpone any consultation activities with regard to staf training, as it most likely would impact the consumer. A consideration for future studies would be to incorporate booster sessions that outline specifc guide- lines for delivering structured feedback to participants (Parsons et al. 2012), so that the researcher can account for quality and richness of the maintenance schedule.

Finally, this study did not evaluate the impact of staf training on the consumer. It is important that future research consider evaluating the impact of staf training, particularly those receiving training aligned with the BACB © RBT® standards, on the clients they serve. There have been a number of studies conducted that examine the efects of staf training on client outcomes (e.g., Sheridan 1992; Huskens et al. 2012), particularly when working with individuals who exhibit severe challenging behaviors (Shore et al. 1995; Lepage et al. 2004) and a lack of communication skills (Schlosser et  al. 2006; Seiverling et  al. 2010). However, research evaluating the efects of staf training with respect to the RBT® task list is in its infancy.

458 Journal of Behavioral Education (2018) 27:435–460

1 3

There is a wealth of research supporting the efectiveness of behavioral inter- ventions when treating individuals with developmental disabilities, particularly those diagnosed with autism spectrum disorder (Myers and Johnson 2007; How- ard et  al. 2005; Eikeseth et  al. 2007; Lovaas 1987). However, the efectiveness of these interventions relies on high-quality staf training and, further, the fdel- ity of implementation (Bibby et  al. 2001). Shook et  al. (2002) make this point succinctly,

Perhaps the two most important variables in ensuring that people with autism learn the skills needed to increase their overall quality of life and to actively participate in society are (a) the instructional approach that is used and (b) the competence of the individuals providing the instruction (p. 27)

Many agree that these components are critical to any intervention expected to have an impact on the future outcomes of the individuals receiving behavioral inter- ventions (Weinkauf et al. 2011); therefore, it is necessary that any paraprofessional staf training program includes the most efective teaching strategies in order to ensure treatment fdelity. Competency-based training programs are designed to pro- mote learning in an efective and efcient manner, but they also must promote the generalization and maintenance of acquired skills.

Given the rise in the need of ABA services to treat a variety of learning and behavioral needs, particularly for those diagnosed with ASD, the feld of behav- ior analysis must ensure that the quality of services is high, and this can only be addressed by providing adequate training of staf as well as promoting the gener- alization and maintenance of behavioral skills acquired by trainees. Credentialing professionals within behavior analysis is one way of helping to diferentiate between those who are qualifed and those who are not (Shook and Neisworth 2005); how- ever, this may not lead to the ultimate solution, as it does not ensure that the amount of training an individual receives is sufcient and adequate for identifying that per- son as a qualifed implementer of ABA services.

The feld of nursing provides an excellent model for the training of paraprofes- sional human services staf. Within their literature, research supports the use of didactic training combined with clinical/practicum hour standards required for nurs- ing assistants to ensure that they have acquired the necessary skills to promote the delivery of high-quality care (Riggs and Rantz 2001). The US federal government mandates require that nursing assistants participate in a minimum of 75 h of class- room training before they can work with a patient within a clinical setting (Federal Register 42 Part 483, 1991). In addition, the individual must complete in vivo clini- cal training hours, with the number of required hours established by state nursing boards. This standard for nursing assistants may provide a useful rubric for training direct-care implementers. Future research needs to evaluate the impact that the ini- tial and post-training (specifcally training activities and the number of experiential hours) has on the quality of implementation of ABA services by direct-care staf. Findings from this research will help credentialing organizations as well as con- sumer protection and public health entities in order for the feld of ABA to further defne training and evaluation standards to ensure that trainees are able to implement behavioral technology with high quality.

459

1 3

Journal of Behavioral Education (2018) 27:435–460

Innovations in competency-based training have included the use of new instruc- tional technologies in delivering instruction to trainees (e.g., video modeling). These are promising and have expanded the tools available for training staf to implement ABA. However, more research is needed to understand the most efective and ef- cient ways to achieve skill acquisition, maintenance, and generalization of the funda- mental skill sets for direct-care workers. As noted earlier, it is a challenge for organi- zations and trainers to allocate or guarantee sufcient time for training, but trainees who do not receive adequate training will not master the skills targeted and will not be efective change agents for their clients.

Behavior analysts are responsible for ensuring that behavior analytic treatment is implemented with integrity and at efective levels. Therefore, it is the responsibil- ity of our feld to conduct further research in the area of staf training, as well as to evaluate whether generalizability and maintenance of staf skills acquired through a competency-based training package is achieved. This training model is supported by empirical evidence, and continued research in this area may potentially inform standards for direct-care staf training and address the problems with generaliza- tion and maintenance of skills acquired through the delivery of a competency-based training package.

Compliance with Ethical Standards

Confict of interest The authors declare that they have no confict of interest.

Ethical Standard All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Hel- sinki Declaration and its later amendments or comparable ethical standards. The study was reviewed and approved by an approved University Institutional Review Board (Title 45 Code of Federal Regulations Part 46).

Informed Consent Informed consent was obtained from all individual participants included in the study.

References

Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Current dimensions of applied behavior analysis. Jour- nal of Applied Behavior Analysis, 1, 91–97.

Behavior Analyst Certifcation Board (2014). Registered Behavior Techician. Retrieved from https:// www.bacb.com/rbt/.

Bergan, J. R., Kratochwill, T. R., & Luiten, J. (1980). Competency-based training in behavioral consulta- tion. The Journal of School Psychology, 18(2), 91–97.

Bibby, P., Eikeseth, S., Martin, N. T., Mudford, O. C., & Reeves, D. (2001). Progress and outcomes for children with autism receiving parent-managed intensive interventions. Research in Developmental Disabilities, 22, 425–447.

Carr, J. E., Nosik, M. R., & DeLeon, I. G. (2017). The Registered Behavior Technician™ creden- tial: A response to Leaf et  al. (2017). Behavior Analysis Practice. https://doi.org/10.1007/s4061 7-017-0172-1.

Dorsey, M. F., Weinberg, M., Zane, T., & Guidi, M. M. (2009). A case for licensure of applied behavior analysis. Behavior Analysis in Practice, 2(1), 53–58.

Eikeseth, S., Smith, T., Jahr, E., & Eldevik, S. (2007). Outcome for children with autism who began intensive behavioral treatment between ages 4 and 7: A comparison controlled study. Behavior Modifcation, 31, 264–278.

460 Journal of Behavioral Education (2018) 27:435–460

1 3

Hawkins, R. P., & Dotson, V. A. (1975). Reliability scores that delude: An Alice in wonderland trip through the misleading characteristics of interobserver agreement scores in interval recording. In E. Ramp & G. Semb (Eds.), Behavior Analysis: Areas of Research and Application (pp. 359–376). Englewood Clifs, NJ: Prentice-Hall.

Howard, J. S., Sparkman, C. R., Cohen, H. G., Green, G., & Stanislaw, H. (2005). A comparison of inten- sive behavior analytic and eclectic treatments for young children with autism. Research in Develop- mental Disabilities, 26, 359–383.

Hughes, J. C., & Shook, G. L. (2007). Training and certifcation of behaviour analysts in Europe: Past, present, and future challenges. European Journal of Behavior Analysis, 8, 239–249.

Huskens, B., Reijers, H., & Didden, R. (2012). Staf training efective in increasing learning opportunities for school-aged children with autism spectrum disorders. Developmental Neurorehabilitation, 15(6), 435–447.

Kratochwill, T. R., & Bergan, J. R. (1978). Training school psychologists: Some perspectives on a com- petency-based behavioral consultation model. Professional Psychology, 9, 71–82.

Leaf, J. B., Leaf, R., McEachin, J., Taubman, M., Smith, T., Harris, S. L., et al. (2016). Concerns about the registered behavior technician™ in relation to efective autism intervention. Behavior Analysis in Practice. https://doi.org/10.1007/s40617-016-0145-9.

Lepage, K., Kratochwill, T. R., & Elliott, S. N. (2004). Competency-based behavior consultation training: An evaluation of consultant outcomes, treatment efects, and consumer satisfaction. School Psychol- ogy Quarterly, 19(1), 1–28.

Lerman, D. C. (2009). An introduction to the volume 2, number 2 of behavior analysis in practice (BAP). Behavior Analyst in Practice, 2(2), 2–3.

Lovaas, I. O. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3–9.

Moore, J., & Shook, G. L. (2001). Certifcation, accreditation, and quality control in behavior analysis. The Behavior Analyst, 24(1), 45–55.

Myers, S. M., & Johnson, C. P. (2007). Management of children with autism spectrum disorders. Pediat- rics, 120, 1162–1182.

O’Reilly, M. F., Renzaglia, A., Hutchins, M., Koterba-Buss, L., Clayton, M., Halle, J. W., et al. (1992). Teaching systematic instruction competencies to special education student teachers: An applied behavioral supervision model. The Association for Persons with Severe Handicaps, 17(2), 104–111.

Parsons, M. B., & Reid, D. H. (1995). Training residential supervisors to provide feedback for main- taining staf teaching skills with people who have severe disabilities. Journal of Applied Behavior Analysis, 28, 317–322.

Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staf training: A guide for practi- tioners. Behavior Analyst in Practice, 5(2), 2–11.

Riggs, C. J., & Rantz, M. J. (2001). A model of staf support to improve retention in long-term care. Nursing Administration Quarterly, 25(2), 43–54.

Schlosser, R. W., Walker, E., & Sigafoos, J. (2006). Increasing opportunities for requesting in children with developmental disabilities residing in group homes through pyramidal training. Education and Training in Developmental Disabilities, 41(3), 244–252.

Seiverling, L., Pantelides, M., Ruiz, H. H., & Sturmey, P. (2010). The efect of behavioral skills training with general-case training on staf chaining of child vocalizations within natural language paradigm. Behavioral Interventions, 25, 53–75.

Sheridan, S. M. (1992). Consultant and client outcomes of competency-based behavioral consultation training. School Psychology Quarterly, 7(4), 245–270.

Shook, J., Ala’i-Rosales, S., & Glenn, S. (2002). Training and certifying behavior analysts. Behavior Modifcation, 26, 253–258.

Shook, G. L., & Neisworth, J. T. (2005). Ensuring appropriate qualifcations for applied behavior analyst professionals: The behavior analyst certifcation board. Exceptionality, 13(1), 3–10.

Shore, B., Iwata, B., Vollmer, T. R., Lerman, D. C., & Zarcone, J. R. (1995). Pyramidal staf training in the extension of treatment for severe behavior disorders. Journal of Applied Behavior Analysis, 28(3), 323–332.

Weinkauf, S. M., Zeug, N. M., Anderson, C. T., & Ala’i-Rosales, S. (2011). Evaluating the efective- ness of a comprehensive staf training package for behavioral interventions for children with autism. Research in Autism Spectrum Disorders, 5, 864–871.

Journal of Behavioral Education is a copyright of Springer, 2018. All Rights Reserved.