ABA506 - Assignment 2

Page 1

ABA 506 Lesson 6

Implementation, Management, and Supervision of Behavior Analysis Services

This lesson provides an overview of BACB® Task List content area Personnel Supervision and

Management. Topics include the purpose of supervision, who is qualified to provide BCBA

supervision, potential outcomes of ineffective supervision, characteristics of effective

supervisors, effective supervision, important features of supervision and successful supervisors,

evidence-based supervision, indicators of poor supervision, supervision of direct service staff,

schedule and preparing for supervision, evaluation systems, the recommended process for

supervision evaluation, ongoing professional development, performance feedback, supervisor

goals, effective feedback, behavioural skills training, ongoing quality management, approaches

to supervision, supervision objectives, supervision contracts, and the evaluation of supervision

process.

Learner Outcomes Upon completing of this lesson, the student will be able to:

• Describe ten main reasons for effective, competency-based approaches to the supervision of others who are providing behavior analysis services

• Describe the potential outcomes of ineffective supervision • Describe or demonstrate the ten components of effective, evidence-based supervision • Describe and demonstrate ten components of behavioral-skills training of the

supervisee • Apply behavioral-skills training across relevant skill areas including, but not limited to,

those outlined in the BACB® Task List • Describe and demonstrate three components of performance feedback • Describe five ways of providing feedback • Describe three methods to evaluate supervisory effectiveness • Describe five methods for ongoing professional development as a supervisor • Describe five methods for the ongoing professional development of the supervisee

What is the Purpose of Supervision?

The purpose of supervision is to facilitate the delivery of high-quality services that result in client improvement. To provide effective supervision, learning the critical skills and elements of supervision from the start is essential. This is the main reason for this course and why it is so important to have training as part of the coursework as required by the BACB®.

During supervision, the supervisor has the responsibility of developing behavior- analytic, professional, and ethical repertoires in specific areas of behavior-analytic practice, as specified by the BACB®'s current Task List and found in the Ethics Code for Behavior Analysts at www.bacb.com.

Page 2

The supervisor also guides behavioral case conceptualization, problem-solving, decision making, and assistance seeking of the supervisee and works to improve the current repertoires of the supervisee. Other aspects of supervision include maintaining beneficial repertoires of the supervisee and modeling effective supervision practices for the supervisee.

Who is Qualified to Provide Supervision for BACB Candidates?

In this section, we will review who is qualified to provide supervision for the required experience hours for certification with the Behavior Analyst Certification Board (BACB). According to the most recent requirements by the BACB, supervisors must be one of the following:

• A BCBA/BCBA-D in active status. Supervisors who hold BCBA/BCBA-D certification must also meet an ongoing supervision CEU requirement. • A licensed or registered psychologist certified by the American Board of Professional Psychology in Behavioral and Cognitive Psychology who was tested in Applied Behavior Analysis; or • An authorized Verified Course Sequence (VCS) Instructor who is providing supervision in a Practicum or Intensive Practicum (Verified Experience Instructor).

In addition, the supervisor may not be related to, subordinate to, or employed by, the supervisee during the experience period. Note that employment does not include compensation received by the supervisor from the supervisee for purposes of providing the supervision services. It is also recommended to frequently review the BCBA Handbook for relevant and applicable standards pertaining to providing supervision of behavior analysis services.

Potential Outcomes of Ineffective Supervision

It is useful to identify the potential dangers and outcomes of ineffective supervision and how this affects the quality of services. Some of these dangers include:

• Low-quality services to clients that result in a lack of client progress or harm • Poor-performing supervisees with insufficient practice repertoires • Poor-performing supervisees unable to problem solve or who cannot make decisions in

novel or unfamiliar situations

Also worth considering are supervisees who do not become effective supervisors in their own practices and supervisees who engage in unethical behavior, which are both considerable concerns for clients and the future of the field.

Page 3

Characteristics of Effective Supervisors

Effective supervisors should model for the supervisee ethical and effective practice based on evidence from behavior analytic literature. Also, they should use up-front and clear directions. This includes the next steps for expanding knowledge, relevant experiential and practical experiences, and strategies that support positive impacts on current cases or the work of the supervisee. The effective supervisor also encourages the supervisee's growth within her/his scope of current competency and is reflective while maintaining clear and effective boundaries in the supervision sessions.

Carrying Out Effective Supervision

Supervisors' goals should include ensuring competent and comprehensive development of skills that instill a clear understanding of ethical practices. These skills should include:

• Respecting client rights and interests • Maintaining professional boundaries • Practicing only within the scope of one's competencies (as specified by the BACB)

For those they supervise, supervisors should create a model and process for skill development. This would involve establishing expectations and boundaries of the supervision relationship. These elements should be included clearly in a contract for supervision that both the supervisor and supervisee agree to and sign, which is also an expectation by the BACB for those entering supervision arrangements. The supervision relationship may include such components as:

• Completion of the required BACB supervision contract • Frequency, scheduling, and method of communication and contact • Ways of communicating the supervision relationship • Goals for supervision for the supervisor and supervisee, with expectations for each

party

The supervisor should help to create a vision for how to achieve the supervisee's goals by:

• Gaining foundational knowledge, with strategies for this (examples include experience, observation, reading the literature, and attending lectures/conferences/workshops)

• Accessing practical experience

In addition, supervisors need to provide information to the supervisee regarding essential experiences that must be incorporated throughout the course of supervision and formulate a sequential plan for achieving both required and desired goals throughout the course of supervision. This would include elements such as:

• Timelines • Practical experiences

Page 4

• Resources available and those to be acquired

It is a good idea to review with the supervisee the BACB® Guidelines and points of emphasis. This includes the purpose of supervision as previously reviewed in this lesson.

Important Features of Supervision

The supervisor should be able to describe and/or demonstrate the components of effective, evidence-based supervision. This includes such factors as:

• Establishing performance expectations for both the supervisor and supervisee • A description of how supervision will proceed according to the BACB experience

standards (which may include the frequency of individual supervision, whether it is individual or group supervision, in-person or online, and other parameters)

• A description of potentially appropriate activities as described in the BACB experience standards

• A description of determining the supervisee's competency in understanding and applying interventions and methods from the behavior analytic literature and current best practices

• Development and review of the contractual agreement of how the supervision will be provided, responsibilities of each party, and other specifics, including any fees and payments, scheduling, responsibilities of each party, and other factors, as mentioned previously.

• Assessment of initial skills of the supervisee and how the ongoing assessment will occur

Successful Supervisors

Supervisors who function successfully understand the experiential and knowledge- based starting points of the supervisee and are fluent in understanding the supervisee's needs, goals, and motivations. The supervisor should have a process for training the supervisee on specific skills (which will be discussed at a later point in this course), observing the supervisee executing these skills, and providing feedback on performance and how to shape skill improvements or refinements that may be needed. The supervisor must also review both direct client interactions, as well as written or permanent product outcomes, such as:

• Functional assessment data • Functional assessment reports • Intervention data • Behavior intervention/support plans • Other reports as warranted by the circumstances and needs of the supervisee and the

situation in which he/she is working

Page 5

Components of Evidence-Based Supervision

The supervisor should be able to describe or demonstrate the components of effective, evidence-based supervision, which include:

• Behavioral skills training of the supervisee • Scheduled observations that may include in-vivo, video, web-based, and/or computer-

assisted observations at various points during the training process (before, during, and after)

• Review of written materials developed by the supervisee, such as behavior plans, functional assessment reports and data, graphs of client progress, and progress notes, if appropriate to the situation.

Supervisors should also be able to describe or demonstrate the following components of effective supervision:

• Scheduling of supervision meetings • Performance evaluations • Evaluation of the supervision process • Evaluation of the supervisor by the supervisee

Challenges and Indicators of Poor Supervision

As the supervisor of Behavior Analysis services, you need to be able to recognize and correct problems with supervision. Some undesired effects of poor supervision can include:

• Low-quality services to clients • A lack of progress by the client • Harm to the person being served and possibly to others

Supervisees may perform poorly in the applied setting and not have sufficient practice opportunities and thus possess insufficient and/or ineffective repertoires to provide the necessary behavior analytic services. In addition, the supervisee may be unable to problem solve and make sound decisions in novel or unfamiliar situations.

Another potential outcome is that these supervisees will themselves have ineffective repertoires as supervisors in their own practice after becoming certified (given these individuals are able to meet all requirements and pass the exam for certification). Thus, an important component of the training program would be to provide supervision as early as possible in order to prevent poor service, ineffective practice, and/or unethical behavior by behavior analysts who become certified. Supervised experience of BCBA candidates is a vital aspect of training to become effective, ethical practitioners, making it also critical that supervisors are trained to provide competency-based supervision to their supervisees.

Page 6

Supervision of Direct Service Staff

Supervision by a BCBA will also include how the supervisee implements the behavior analysis programs and services that were devised by the BCBA with the input of the team, family members and/or legal guardians, and others involved. The tasks involved are similar to those stated previously in this course but need to be approached from a different perspective. This involves training in key behavioral skills for the direct service staff (also referred to as paraprofessional staff and may have different titles in different programs, schools, and organizations that provide behavior analysis services), as well as an understanding of behavior analytic concepts, approaches, and procedures. It may also include training on how to look for the functions of behavior and methods to assess function, such as:

• The use of interview instruments

• The use of direct observation methods, such as ABC charting or Pattern Analysis

These practices need to be conducted under close supervision of a qualified professional, such as the supervising BCBA. Ensuring that consistent and accurate data collection is taking place, along with implementing the plan as written, are critical aspects of supervision and training of the direct staff. Using methods such as Inter-Observer Agreement and Fidelity checks can help ensure data collection is accurate and taking place as directed by the supervisor and as outlined in the behavior plan, as well as assuring the consistency of the implementation of the plan as written.

Scheduling Supervision Sessions and Content of Meetings

The BCBA supervisor should set a schedule to arrange regular supervision meetings with the direct service staff for purposes of reviewing progress, reviewing data, and to discuss any potential difficulties or barriers that may be occurring in the setting or with the client. The supervisor also needs to set aside time to conduct training sessions on a regular basis.

It is suggested that training sessions be arranged monthly and can be best accomplished in a group situation with all direct service staff being supervised and perhaps others that need training in the organization. These training sessions may be considered supervision sessions, but separate supervision meetings with individual staff, as well as group sessions for case reviews, are highly recommended as a best practice, in addition to training on skills, methods, new research, and so on.

It is also a sound practice for BCBAs to provide competency-based pre-service training for all newly hired staff before they begin providing services to clients. Most organizations are required by various state or school contracts to ensure such pre-service training, as well as regular ongoing training and annual re-training requirements. So, it is important to check and implement regulatory requirements that affect what training the supervisor may need to provide, as well as the number of hours and frequency of training and supervision meetings with the staff.

Page 7

Preparing for Supervision The following are skills that supervisors need to master, which include:

• Supporting ongoing professional development • Behavioral Skills training • Delivering feedback • Evaluating the process

To support professional development, the supervisor must remain current with BACB requirements and keep contact information up to date with the BACB. Also, the supervisor needs to read the relevant literature in their specialization or training and needs to ensure that the supervisee understands the supervisor's specialty areas and areas of competence. The supervisor needs to ensure that during supervision, the supervisee is providing services that are within the scope of practice and training of the supervisor, as well as the supervisee, to the extent the supervisee has had relevant training in coursework and experience. At a later point, we will review aspects of the types of areas to focus on within behavioral skills training.

Another area of competence for the supervisor is providing feedback to the supervisee. This needs to be carried out in a constructive, supportive manner and should be designed to improve the skills and competencies of the supervisee. Therefore, part of the purpose of feedback when the skills are not yet at the desired or optimal level is to provide opportunities for practice and correction. In this regard, the supervisor needs to establish performance goals and objectives for the supervisee that are clearly stated and understood by the supervisee in advance. A plan for supervision that is above and beyond a contract is also an important component of competency-based supervision.

It may be useful for BCBA supervisors working within an organization or operating their own agency to form a relationship with universities with BACB-approved course sequences and establish an internship or mentoring affiliation at the applied site. Students are often seeking a setting to gain experience and to receive supervision as part of the experience requirement needed for certification. Such an approach, which is a standard in other fields, such as psychology, can help with providing a high quality of supervision and experience for the supervisee as they will be working directly within the supervisor's organization. There are many advantages for all concerned, including training competent practitioners, which will improve and enhance behavior analysis services to the public in general.

Evaluation Systems and Process

The supervisor also needs to evaluate the process, which can have several components, including:

• A plan for supervision with goals and objectives for the supervisee • A method for evaluating supervisee progress and skills • A list of skills the supervisee needs to learn and master, starting with a baseline

Page 8

There should be a remediation or correction plan with target dates for completion, a summary report for the supervisee to review on a regular schedule based on the number of weeks of supervised experience, and an opportunity for the supervisee to provide feedback to the supervisor, which can also be incorporated into a rating scale along with open-ended written feedback. Recommended Process for Supervisor Evaluation Devise a plan for supervision to include:

1. Goals and objectives for the supervisee to achieve 2. Time frames for completion 3. A regular schedule for supervision sessions and observation times 4. A method for evaluation, including supervisor observation 5. Scoring of performance during observation using a data system 6. Verbal skills evaluation during supervision meetings 7. Evaluation of permanent products (reports, behavior plans, notes, data presentation in

graphic and table form) 8. Plan for remediation of skills not meeting criteria with time frames for completion to

meet criteria 9. Feedback to supervisee on a regular basis 10. Feedback and evaluation of supervision and supervisor by the supervisee

Ongoing Professional Development

Supervisors need to engage in their own ongoing professional development. This can be facilitated by creating a learning community to:

• Enhance supervisory and training behavior • Raise awareness of available resources • Obtain training in various ways

This can be done by reading relevant literature on behavior analysis, attending conferences, workshops, online seminars or webinars, and internal training events within one's own organization. Some of these internal training events may be in the form of face-to-face training, webinars, staff meetings, clinics or clinical rounds in some hospital or clinic settings, mentoring meetings, and other forums. Often many of these options also may be available for continuing education credits, which are required for maintaining certification if the training events meet BACB standards and requirements.

Aspects of a learning community can include regular review of resources and research for best practices in supervision, forming supervisory study groups, participation in journal clubs, and Special Interest Groups (such as those sponsored by the Association for Behavior Analysis: International and/or local ABAI chapters). Also, it includes peer review or peer-mentoring

Page 9

groups, in which the supervisors seek out colleagues to review their cases or discuss challenging cases or situations or ethical dilemmas.

If the supervisee wants to develop expertise in a particular area, such as Precision Teaching, or Discrete Trials Training, the supervisor needs to be well-versed in the area before he/she can teach and supervise in these areas. As a supervisor, you should recognize situations in which you need to seek consultation about a case. There are several professional organizations that may be available as a resource for supervisors and supervisees. Some of these include:

• ABAI (Association for Behavior Analysis International) • APBA (Association of Professional Behavior Analysts) • BACB (Behavior Analyst Certification Board) • CCBS (Cambridge Center for Behavioral Studies) • NASP (National Association of School Psychologists) • APA (American Psychological Association) • NEA (National Education Association)

These organizations all have websites with a large amount of information and resources. ABAI and APBA support research and practice of behavior analysis, and APBA supports BACB certificants in the practice of behavior analysis. The BACB website provides listings of organizations authorized to provide Continuing Education (CE) training either in live events, such as conferences or workshops, and/or via online means. ABAI holds several conferences per year, including the long-running annual conference and the autism conference, among others. APBA also holds an annual conference. These conferences offer a variety of workshops, seminars, and lectures on various topics in the field for CE credits for BACB certificants and psychologists. There are also other websites that may serve as resources, but always check the validity of the information being provided on any of those websites.

An additional way to aid with ongoing professional development would be for supervisors to keep up to date with BACB standards, changes, updates, newsletters, ethical guidelines, continuing education requirements, course requirements, conferences, and other matters. Regularly checking the www.bacb.com website is critical to keeping current with requirements and standards, and information in the field. The supervisor needs to have an in-depth knowledge of the requirements and standards regarding supervision of those providing behavior analysis services and trainees receiving experience hours.

Supervisors also need to stay current with licensing laws for behavior analysts, given the increase in the number of states with such laws. In those states, a behavior analyst needs to meet state behavior analyst licensing board requirements, which can vary and may be under different boards in different states. In some states, retaining one's BCBA or BCaBA certification is necessary, along with completing the required number of Continuing Education credits from BACB-approved CE providers. Also, some states may grandfather in those with the BCBA or BCaBA certification, but after that period, other standards to obtain or retain licensure may be

Page 10

required. In addition, Continuing Education requirements may vary by state per the requirements of the board for the state in which the behavior analyst is practicing.

Licensure laws for a given state may require such licensure to practice as a behavior analyst and not simply having a BACB Certification alone. Another consideration is portability, which refers to the ability to use your certification or license to practice in another state. It is important to make sure that as a practicing behavior analyst and supervisor, you are aware of the requirements and laws in the state where you practice or intend to practice. For more information on licensure of behavior analysts read Weinberg et al. (2009) and Green & Johnston (2009).

Performance Feedback

Another critical element of effective supervision is performance feedback. As a supervisor, you need to learn effective, supportive, and educative approaches to feedback. This requires a delicate balance of maintaining a supportive environment, as well as a positive professional relationship with those you supervise. The elements of effective performance feedback that enhance the supervisee's skills and performance, and ultimately their competence as an independent practitioner, are presented in this lesson. It may be helpful to read some of the literature and resources on this in behavior analysis. Two of these resources that are highly recommended include the following books by Aubrey Daniels:

• Performance Management: Improving Quality and Productivity Through Positive Reinforcement (1989)

• Bringing Out the Best in People (2000)

Parsons et al., (2012) provides guidance and suggestions for evidence-based staff training that pertains to direct service staff but also can be adapted to the supervision of trainees pursuing BACB Certification. For more information on these training approaches, you can read Parsons & Reid (2011).

Parsons et al. (2012, pp. 2-3) point out that:

“Early behavioral research demonstrated that staff training programs relying on verbal- skill strategies (e.g., lectures, presentation of written and visual material) is effective for enhancing targeted knowledge, but often are ineffective for teaching trainees to perform newly targeted job skills.”

They conclude that programs focusing heavily on verbal skill training approaches are usually proven ineffective in their impact on job performance among human services staff. The protocol proposed by these authors for evidence-based staff training is based on the behavioral skills training (BST) approach (see Miles & Wilder, JABA, 2009, 42: 405-410) and includes the following components:

Page 11

1. Describe the Target Skill 2. Provide a Succinct Written Description of the Target Skill 3. Demonstrate the Target Skill 4. Require Trainee Practice of the Target Skill 5. Provide Performance Feedback During Practice

Components of Effective Performance Feedback

The authors describe performance feedback as:

Describing to the trainee exactly what s/he performed correctly and corrective feedback involves specifying what was not performed correctly. Corrective feedback also involves providing instruction about exactly how to perform any aspects of the target skill performed incorrectly in order to facilitate proficient future performance of the skill. Generally, we recommend providing feedback following completion of a given role play in contrast to interrupting an ongoing role-play activity to provide feedback. (Parsons, et al., 2012, p. 4)

The authors go on to say: “Observing trainees and providing feedback to each trainee requires time and effort on the part of trainers. This is another reason that it is often beneficial to have two trainers present, and especially if the number of trainees exceeds four or five. Providing individualized feedback is as critical to the training process as the trainee practice component and must involve each trainee” (Parsons et al., 2012, p. 4).

It should be mentioned that the authors point out that the time it took to complete these steps to criterion was a lot longer than traditional training approaches. Supervisors need to take the temporal aspect of training into consideration, especially for direct service staff. Other considerations include staff being taken out of their usual functions to receive the training, which may disrupt services to clients or students in a program.

If the supervisee receiving experience hours for certification has some level of prior experience, then this process may be more expedient and may be streamlined, depending on the level of skill, experience, and previous training and knowledge. The authors suggest including visual methods, such as videos or pictures, to demonstrate the correct implementation of a procedure, which may help expedite training and enhance the efficiency of the training. Other approaches may be to limit training session times to perhaps 1 hour (or as short as 30 minutes) to minimize service disruption.

Page 12

Goals and Characteristics of Effective Performance Feedback

Effective performance feedback will allow the supervisee to understand her/his current behavior or level of performance, the target skill, and the next steps toward mastering competency with that skill. The supervisee will know:

• What skill on which he/she is being provided feedback • What the observer sees in the supervisee's behavior while performing the skill • What the features of best practice performance for this skill include or entail • What the next steps are toward bringing the supervisee's behavior from current levels

to the goal or criterion performance levels

It should be mentioned that the supervisee may be performing many different types of tasks or activities of a budding independent behavior analyst practitioner. These activities might include the following:

• Teaching a child to match words to objects or pictures, follow verbal instructions, and address problem behavior

• Providing training to direct service staff on a behavior plan • Conducting a Functional Assessment interview with a teacher or parent • Writing the Functional Assessment report • Creating behavior graphs • Speaking with parents about their child's behavior concerns and making

recommendations • Reporting the results of the Functional Assessment to an interdisciplinary team

Effective feedback needs to be direct and clear, allowing for differential reinforcement of target performance and the use of a shaping process and open dialogue for the next steps. The feedback must clearly indicate areas of strength and for improvement in the supervisee's performance or implementation of a skill.

Direct feedback needs to be specific while avoiding losing the message via indirect communication. The feedback needs to maintain motivation for learning by reinforcing skill improvements and building a vision for how the next steps can be achieved and that they are achievable. The feedback need not be prescriptive. Also, the process of supervision should allow the supervisee to engage in the development of a professional development plan.

Challenges to Performance Feedback

Supervisors need to learn to provide performance feedback to supervisees without alienating them. This may not be as easy as it first seems, and it is important to get to know and develop a working professional relationship with the supervisee. The supervisor should understand the motivations and sensitivities of the supervisee to best interact and be effective

Page 13

as a mentor and supervisor to promote a positive work environment and, most importantly, to train a competent, ethical behavior analyst. The supervisor also needs to:

• Be clear regarding ethical and other best practice standards for behavior analysts • Provide feedback as soon as possible to ensure proper client care and reduce redundant

learning errors

Some resources and logistical considerations for supervision and feedback include time to observe and meet with the supervisee, as well as time for the supervisee to practice the skills being learned. Also, the supervisor needs to ensure access to any teaching materials needed, provide consistent procedures for maintaining contact, and use multi-dimensional approaches to teaching skills. It may be that some combination of methods, such as verbal explanation, modeling, and use of visual cues or methods, may all be very helpful to the supervisee. It may take some time to learn which of these works best to promote learning and progress for each supervisee.

Interpersonal Issues

The supervisor needs to be comfortable providing direct feedback about performance, whether it is positive or to point out errors or problems. Also, there is a need to maintain a positive working relationship and rapport with the supervisee. Ensuring the supervisee’s confidence in the supervision relationship is critical to the process. The supervisee must be motivated to move forward and must feel that he/she is being heard. In addition, the supervisee's efforts to perform well and progress made must be reinforced and acknowledged by the supervisor.

The supervisee should feel that he/she is in control but also supported and guided regarding his/her professional development. The supervisor needs to be confident in the role of supervisor and ensure that the feedback has not resulted in unclear or dishonest reporting, that the supervisee will clearly report needs and interests, and that the supervisee will independently solicit help and support.

A Collaborative Approach to Feedback

To establish a foundation for feedback, both the supervisor and supervisee must create a process for achieving effective feedback. This requires a dialogue between both parties that starts when supervision begins and defines what feedback is, how it functions, and what both parties should expect to get out of it. In this important phase of developing the supervisory relationship, the supervisor must ensure that he/she is utilizing a feedback model that is motivating for the supervisee. The supervision relationship should involve a dialogue that articulates the best strategies for achieving feedback between the parties involved. Some considerations include:

• When to provide feedback

Page 14

• What types and styles of comments provide honest yet supportive feedback and work best with the supervisee’s communication style.

• What types of materials or resources are needed or most supportive in the supervisory process? Among these resources, which ones are most useful to the supervisee in the feedback process?

• What the process is for dialoguing on the points of feedback • What methods will be used to ensure that the feedback has been heard and is helpful

Supervisors should be highly motivated to establish an effective process for supervision, particularly regarding feedback, as this can make or break the interaction and supervisory relationship with the supervisee. Supervisors must be prepared to adjust their style to match the needs of the supervisee and consider that a supervision relationship that lacks an effective feedback process is likely to fail.

Supervisor Goals

It is important to establish an effective feedback process to ensure a reliable and empowering method for providing feedback. Without this, supervisors are paralyzed, and forward progress can be halted. If feedback does not result in both improved understanding and motivation, then supervision goals may not be met. Feedback procedures that do not allow for dialogue to flow between both parties will not help supervisors to understand fully and successfully what must be done as a supervisor to be effective. The following steps for feedback can be helpful:

1. First, demonstrate the skill 2. Then observe 3. Review records and data 4. Ask for impressions and thoughts and/or feedback from the supervisee 5. Provide a clear and organized description of your impressions

These descriptions of impressions may include:

• General statements of the main skills observed. • Descriptions of the supervisee's goals and a clear path needed to achieve these • Positive statements regarding supervisee performance and particularly those linking to

previous points made when feedback was provided

Also, the supervisor needs to discuss the next steps needed in a collaborative manner. Performance feedback is only effective if it provides, or results in, a method for moving forward and ensuring that the supervisee becomes successful. Time needs to be allocated for planning the next steps. This may best be accomplished as a component of individual supervision meetings held privately. During these sessions, it is important to review the feedback and consider whether the feedback was understood, if it is relevant to the supervisee, and if there are questions to help better understand the feedback. Afterward, the supervisor should:

Page 15

• Discuss the next steps for achieving the goals • Outline the specific skills being referred to or discussed • Specify the plans for moving forward clearly and in objective, measurable terms • Decide what tasks the supervisee needs to complete to move forward • Determine the next time when the supervisee will need additional feedback • Stipulate what types of assistance the supervisor can provide

Points of Emphasis

Some additional aspects and points for supervision will be reviewed in this lesson. First, to deliver performance feedback, the supervisor should be able to describe and demonstrate the components of performance feedback. These include:

• Positive feedback involving contingent, descriptive praise • Corrective feedback

Corrective feedback should include the following elements:

• Provide an empathy statement that will help with keeping the supervisor-supervisee relationship as positive and supportive as possible.

• Describe ineffective performance to make sure the supervisee is clear on what the problems are with the performance or skill.

• Provide a rationale for the desired change in performance, including a discussion of evidence-based methods, effective intervention approaches and maximizing treatment effectiveness and efficiency as the goal of services.

• Provide instructions and demonstrate how to improve the designated performance to aid with promoting the desired outcome for the supervisee

• Provide opportunities to practice the desired performance • Provide immediate feedback about the performance after opportunities to implement

prior feedback have occurred

Finally, supervisors need to offer follow-up about performance, providing specific praise about the supervisee's improvements or continued implementation and how it meets the standards of best practices. Also, they should discuss the process and ensure that the supervisee is as comfortable as can be possible in the situation and how to improve the process to achieve a balance. Ongoing discussions about this aspect are critical to the process being effective and beneficial to the supervisee and to the clients or consumers being served. The following is a list of the methods of performance feedback delivery:

• Vocal • Written • Modeled • Video • Graphic

Page 16

• Self-monitored • Formal: such as in pre-arranged meetings or during a scheduled observation • Informal: ongoing, on-the-job, or in more casual situations

Defining Effective Feedback

Effective feedback is defined by its impact on future behavior. Feedback will be different in some regards in each supervisor-supervisee relationship, and each of these situations should incorporate and model the critical features of effective practice. The feedback that leaves the supervisee with a clear understanding of the competencies, their performance, and the next step toward reaching goals has the greatest potential to be highly motivating and to achieve the desired outcomes of the supervision process.

So far, we have discussed processes and procedures for preparing for supervision, establishing a process for ongoing professional development, creating a “learning community,” and means of staying up to date with current trends, issues, and methods, as well as professional issues in the field of behavior analysis. We also discussed current research methods pertaining to evidence-based approaches to performance feedback for supervisees. In the next section, the Behavioral Skills methodology, as it relates to supervision of behavior analysis services, will be presented and reviewed.

Behavioral Skills Training

We will now turn our attention to the topic of behavioral skills training. This approach has been used for several years, and those doing research on it include Dennis Reid and Marsh Parsons, among others. Their work has primarily focused on training direct-care personnel rather than those in supervision working to become independent practitioners. These authors did publish on training human services supervisors in positive behavior support methods (Reid et al., 2003), but not behavior analysis supervisors (which is the focus of this course) or direct service staff.

However, the information and what they learned from their studies are valuable for behavior analysts and direct service staff, and their supervisors. Their research is also relevant for this course since the course includes supervision of direct service staff and training them to implement behavior intervention programs and approaches directly with clients/consumers. Here are some main approaches to BST (Behavioral Skills Training):

• Didactic Training • Modeling by the trainer(s) • Role-play sessions

For each supervision session, the supervisor should provide a rationale for why the target skills are being trained. There are many reasons, but some may be more obvious, such as the skill needed as a best practice intervention or because it is most effective and efficient and

Page 17

it is adapted to the needs of the client and circumstances. The supervisor should provide a succinct, written description of the instructions for the target skills and provide a detailed vocal description of the target skills. Also, he/she needs to:

• Present a statement of the conditions under which the supervisee is expected to perform the desired skill

• Model the skill, then require the supervisee to practice each target skill • Give feedback to the supervisee

Parsons et al. (2012) also referred to the use of corrective feedback as a procedure involving interrupting the implementation as it is ongoing and making the correction on the spot and in the actual situation. This approach for a BCBA candidate, or even BCaBA or another person with training and experience, should be considered with caution. It may damage the supervisor- supervisee relationship and possibly result in significant, and even irreparable, damage to the relationship and halt the entire process, despite the supervisor's best intentions and efforts to repair any damage.

Following the prior step of providing corrective feedback, the supervisor should ask the supervisee to repeat this process of practicing and provide feedback until the supervisee performs each target skill correctly. Then, the supervisor needs to assess the application and generalization of skills to new targets, clients, and settings, as relevant and as appropriate. It should be noted that practicing the skill is a critical feature for the success of BST and should be required of each trainee to produce effective performance, as discussed by Nigro-Bruzzi & Sturmey (2010) and Rosales et al. (2009). Summary of the Steps for Behavior Skills Training

1. A statement of what the supervisee is expected to do (performance) 2. A statement of the quality or level of performance (criterion) 3. A statement of the conditions under which the trainee is expected to perform the

desired skill (conditions or context) 4. Model each of the target skills 5. Require supervisee to practice each target skill 6. Provide positive and corrective feedback to the supervisee (this may be best done

during the actual implementation of the skill with a client) 7. Repeat these as needed until each target skill is performed to criterion 8. Assess the application and generalization of the skills in both the original situation or

context or client and with new targets, clients, and settings as appropriate.

The following recommendations should be used by supervisors to create better skills during the training process:

• Have clear objectives

Page 18

• Provide training designed to allow the supervisee to draw on his/her own background of experience (curriculum matching)

• Ensure that supervisees have opportunities for practice • Give supervisees needed feedback. • Allow supervisees to learn by watching others. • Develop training programs that are properly coordinated and organized (arrange the

learning environment, provide the materials, etc.)

Tips for Providing Feedback

Behavior feedback should always describe the behavior, not state a conclusion or make an accusation. State the feedback in terms related to the behavior of the supervisee rather than making a judgment about the supervisee. Be specific about the skill or performance of the supervisee that may meet criteria or specifically why it did not. Then, proceed to work on how to improve the supervisee's skills. If there needs to be an improvement in the skill performed, include the impact on outcomes as a problem and how improved performance can alleviate this dilemma.

Ongoing Quality Management

Managing and supervising can best be drawn from the Organizational Behavior Management (OBM) Literature. See Aubrey Daniels' books Bringing out the Best in People and Performance Management books for a good review and understanding of OBM methods and processes. In an ongoing supervisory role, one must look at the continuous improvement of quality. At the systems level, a quality control system has five steps:

1. Identify critical points in the process 2. Design measures to monitor these points 3. Decide on the amount of inspection at these points 4. Decide on who will do the inspection 5. Implement the plan

• Ensuring the Generalization of Skills Learned Set the Climate for Transfer: Supervisors encourage and set goals for supervisees to use new skills and behaviors acquired in training.

• Using Task Cues: Characteristics of the supervisee's job prompt her/him to use the new skill and behavior acquired in training.

• Feedback Consequences: Supervisors support the application of new skills and behaviors acquired in training.

Approaches to Supervision

We now turn our attention to the types of approaches to supervision. The supervisor should be able to describe the following formats for providing behavioral skills training, including:

Page 19

• In-person using role-play and in-vivo assessment of skills. • The use of visual media, such as video, computers, web-based applications. and

software. This can be done with individual supervisees or in a group situation, either in the actual practice setting or meeting outside the practice setting.

The supervisor should also be able to apply behavior skills training across relevant skill areas, including but not limited to those outlined in the BACB® Task List. Some examples include:

• Fundamental behavior change procedures, such as prompting, shaping, fading, and effective use of reinforcement

• Specialized procedures, such as momentum, stimulus equivalence, and discrete trials training

• Data collection systems and methods and data analysis • Functional Assessment and Functional Analysis methods • Case Conceptualization • Problem Solving • Decision Making • Resource seeking (such as additional consultation, mentorship, and collaborative

efforts)

Providing Course or Supervision Objectives

• The behavior analyst provides a clear description of the objectives of a course or supervision, preferably in writing, at the beginning of the course or supervisory relationship.

• The behavior analyst provides a clear description of the demands of the supervisory relationship or course (e.g., papers, exams, projects, reports, intervention plans, graphic displays, and face-to-face meetings) preferably in writing, at the beginning of the supervisory relationship or course.

• The behavior analyst provides a clear description of the requirements for the evaluation of student/supervisee performance at the beginning of the supervisory relationship or course.

Supervision Contracts

The next area we will focus on is creating a contract so that the roles and responsibilities of the supervisor and supervisee are clearly stated from the start and agreed upon by both parties. This requires some detail in order to include all the elements needed. Samples can be created by the supervisor, in conjunction with the supervisee, and with support from management or supervisory staff of the setting where the supervisee is providing services. These contracts are used for the purposes of receiving supervised experience hours towards meeting the certification requirements of the BACB.

Page 20

The BACB has stated that the supervisee and supervisor must execute a written contract prior to the onset of the supervision experience. The purpose of the contract is to protect all involved parties and align the experience with the purpose of supervision described under Nature of Supervision. As a supervisor, you should understand that the supervisee has several potential objectives, which often include:

• Gaining practical experience in an applied setting and with a particular population, such as students with an autism spectrum disorder

• Wishing to learn behavior analysis methods that are effective • Having a BCBA supervisor sign their experience verification form upon completion of the

experience hours to enable the person to sit for the certification exam given successful course work completed in a BACB-approved graduate training program

As the supervisor, you need to provide an effective and sound program to train a qualified, competent, ethical behavior analyst practitioner who will produce effective outcomes with their clients in the future. In addition, before you sign the experience verification form, you have the responsibility of ensuring that this person you have supervised for several months is qualified to practice independently. This is a very large responsibility and one that should not be taken lightly. As supervisors, we have a critical role in the further advancement of the field and in ensuring that we are training competent practitioners who use evidence-based practices.

Contract Elements

• State the responsibilities of the supervisor and supervisee • Include a description of the appropriate activities and instructional objectives • Include the objective and measurable circumstances under which the supervisor will

sign the supervisee's Experience Verification Form when the experience has ended • Delineate the consequences should the parties not adhere to their responsibilities

(including proper termination of the relationship) • Include a statement requiring the supervisee to obtain written permission from the

supervisee's on-site employer or manager when applicable • Include an attestation that both parties will adhere to the Code of Ethics for Behavior

Analysts (based on the most current version posted on the ww.BACB.com website)

In addition, the BACB's Supervisor Standards state that the supervisee and supervisor are “responsible for retaining and providing a copy of the contractual agreement and supervision documentation to the BACB, if requested” (BACB, 2019, p.4.). This is another reason to ensure all the elements or components are included and adhered to and agreed to by both parties, with signatures and dates.

Page 21

Evaluation of Supervision Process

What to Evaluate

Evaluate the client(s) served by the supervisee. Ask if the client is making progress and review the data.

Interview Questions

• Does the client find value in the services? • Does the client have a positive view of services and the person delivering them? • Do the client's parent/guardian view services in a positive light and feel good about

these services?

The supervisor should also determine whether and how the interviews can be data-based. In addition, the supervisor should correlate the interview outcomes with the data from interventions as well as interviews.

Supervisee Skills

• Identify target skills in the contract • Measure progress of supervisee • Review data to track progress

Devise a plan to correct any skills deficits with objectives and criteria that both the supervisor and supervisee agree upon in writing. Use the competency-based Supervision Feedback Form for the supervision session, which includes feedback and points the supervisee earns for achieving target skills or products, such as Functional Assessment reports, data, and behavior plans. Review the form and points earned to ensure criterion performance for supervision is being met. Other areas to include in the evaluation process include:

• Identify experience standards per the BACB • Measure progress related to these standards and the contract • Conduct interviews with the supervisee • Use Social Validity data. This can be from a variety of available instruments that assess

social validity • Correlate the data with outcome data collected

Supervisor Evaluation

This is in the form of self-evaluation of your performance as a supervisor. Here are some questions to consider:

• Did you uphold your end of the contract?

Page 22

• Did you provide a quality, consistent supervision process? • Did you provide consistency with skills training, modeling, and the use of needed

materials and resources? • Did you arrange a supportive learning environment and provide effective feedback in

the context of a supportive learning environment? • Did you model, support, and promote the use of evidence-based, ethical practices? • Did you use peer observation and evaluation? • Did you allow for outside or third-party mentors to conduct observations and

evaluations?

As the supervisor, you should conduct an interview with the supervisee to ask him/her how you as a supervisor have been performing. There are several rating scales available for this, or you can create a Likert-type scale on your own that goes from low/disagree to high/strongly agree. It may be best to have an equal number of items on the scale to prevent answers in the middle. There are also formal evaluation forms available for supervisor evaluation purposes.

Summary of Evaluating the Effects of Supervision

Have a contract before beginning supervision and be sure to include measurable goals. Evaluate client outcomes, supervisee outcomes, and supervisor performance and seek feedback from your supervisee.

Summary

In this lesson, we discussed the processes and procedures for providing effective, competency-based supervision of behavior analysts seeking certification as a BCBA, as well as behavioral skills training, feedback procedures and suggestions, and evaluation processes and approaches. In addition, we looked at training, implementation, and the management of behavior analysis services, both for behavior analyst professionals and direct-care staff. Other aspects we covered in this lesson included:

• Preparing for supervision • Establishing a process for ongoing professional development • Creating a learning community and the means of staying up to date with current trends,

issues, and methods, as well as professional issues in the field of behavior analysis

Utilizing the methods reviewed here, in conjunction with the BACB's standards for supervisors of those providing supervision for BCBA candidates, should aid greatly in providing a high-quality, effective supervision process to train competent, ethical behavior analysts.

Page 23

References

Behavior Analyst Certification Board (2019). BCBA fieldwork requirements. Daniels, A. C. (2000). Bringing out the best in people. McGraw-Hill Education. Daniels, A. C., & Daniels, J. E. (1989). Performance management (Vol. 159). Tucker, GA:

Performance Management Publications. Dorsey, M. F., Weinberg, M., Zane, T., & Guidi, M. M. (2009). The case for licensure of applied

behavior analysts. Behavior Analysis in Practice, 2(1), 53-58.

Green, G., & Johnston, J. M. (2009). Licensing behavior analysts: Risks and alternatives. Behavior Analysis in Practice, 2(1), 59-64.

Nigro‐Bruzzi, D., & Sturmey, P. (2010). The effects of behavioral skills training on mand training by staff and unprompted vocal mands by children. Journal of Applied Behavior Analysis, 43(4), 757-761.

Parsons, M. B., & Reid, D. H. (2011). Reading groups: a practical means of enhancing professional knowledge among human service practitioners. Behavior Analysis in Practice, Fall 2011, 4(2):53-60.

Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-Based Staff Training: A Guide for Practitioners. Behavior Analysis in Practice, 5(2), 2-11.

Reid, D. H., Rotholz, D. A., Parsons, M. B., Morris, L., Braswell, B. A., Green, C. W., & Schell, R. M. (2003). Training human service supervisors in aspects of PBS: Evaluation of a statewide, performance-based program. Journal of Positive Behavior Interventions, 5(1), 35-46.

Rosales, R., Stone, K., & Rehfeldt, R. A. (2009). The effects of behavioral skills training on implementation of the picture exchange communication system. Journal of Applied Behavior Analysis, 42(3), 541-549.