Module 4 Reflection Paper

SPECIAL SECTION: SUPERVISION PRACTICES

Towards a Competency-Based, Ethical, and Socially Valid Approach to the Supervision of Applied Behavior Analytic Trainees

Laura B. Turner1 & Aaron J. Fischer2 & James K. Luiselli3

# Association for Behavior Analysis International 2016

Abstract Competency-based supervision of trainees has re- cently come to the forefront of behavior analytic practice; however, there are minimal data to support the effectiveness of various supervision practices on trainee outcomes. Accordingly, this paper is intended to spark further discussion and research activity regarding the supervision of those seek- ing to become Board Certified Behavior Analysts (BCBA). We present a practice model and considerations for supervis- ing applied behavior analytic trainees consistent with the Behavior Analyst Certification Board (BACB) Supervisor Training Curriculum Outline (Behavior Analyst Certification Board, 2012b), the Professional and Ethical Compliance Code for Behavior Analysts (Behavior Analyst Certification Board, 2014), and extant literature from behavior analysis and related fields. Inherent to the current model is a focus on bi- directional feedback and collaboration between the supervisor and trainee to frequently evaluate the acceptability of the pro- cedures, process, outcomes, and effectiveness of supervision. We present a Supervision Monitoring and Evaluation Form consistent with the current model and discuss the assumed

importance of objective and subjective self-assessment of su- pervisor competence to the ultimate advancement of the prac- tice of applied behavior analysis.

Keywords Applied behavior analysis . Competency-based supervision . Social validity . Supervision evaluation form .

Trainee

Supervision is a crucial activity of most Board Certified Behavior Analysts (BCBA) and is especially important given the growing number of applied behavior analytic trainees and the expanding demand for applied behavior analysis (ABA) services worldwide. It is generally assumed that quality super- vision will contribute to the development of effective and ethical practitioners, protection of clients, and advances in the field. Initial research also suggests that the intensity of supervision provided by applied behavior analysts is positive- ly related to clinically significant improvements in client per- formance (Eikeseth, Hayward, Gale, Gitlesen, & Eldevek, 2009). However, until recently, there were minimal standards on how to provide a valuable supervision experience for trainees. In 2011, the Behavior Analyst Certification Board (BACB) initiated a Supervision Task Force to develop defin- itive supervision guidelines and training standards for the field (Behavior Analyst Certification Board, 2011). Resulting from this task force were minimal competencies required of those providing supervision to individuals pursuing a BACB certi- fication or practicing as Board Certified Assistant Behavior Analysts (BCaBA) or Registered Behavior Technicians (RBT; Behavior Analyst Certification Board, 2012b). Relevant is- sues were designing a supervision contract, implementing be- havioral skills training (BST), and delivering frequent perfor- mance feedback.

Electronic supplementary material The online version of this article (doi:10.1007/s40617-016-0121-4) contains supplementary material, which is available to authorized users.

* Laura B. Turner [email protected]

1 University of Saint Joseph, 1678 Asylum Avenue, West Hartford, CT 06117, USA

2 University of Utah, 1721 Campus Center Drive, Salt Lake City, UT 84112, USA

3 Clinical Solutions, Inc. and North East Educational and Developmental Support Center, 1120 Main Street, Tewksbury, MA 01940, USA

DOI 10.1007/s40617-016-0121-4

Published online: 28March 2016

Behav Analysis Practice (2016) 9:287–298

Underlying this competency-based approach is the as- sumption that, rather than Bosmosis^ (Falender & Shafranske, 2012, p. 133), supervision is an independent area of practice and requires acquisition, refinement, and mainte- nance of skills; although strong clinical and research skills are essential to being a competent supervisor, these skills alone do not automatically equate to strong supervisory skills. Attention is now being given to the development of supervisor competencies and identification of effective components of supervision, a process mirrored by several other helping pro- fessions, such as social work (Kraemer-Tebes et al., 2011), psychology (American Psychological Association, 2015; Falender et al., 2004), and medicine (Frank et al., 2010).

The purpose of this paper is to present a practice model that integrates evidence-based training and supervision practices with the complex ethical and practical consider- ations regarding supervision. Because there are currently limited data on the effectiveness of supervision and specif- ic supervision practices in developing well-trained BCBAs, BCaBAs, and RBTs, the current model is guided by the supervisory experiences of the authors, extant liter- ature from behavior analysis and related fields, the BACB Supervision Training Curriculum Outline (Behavior Analyst Certification Board, 2012b), and the newly devel- oped Professional and Ethical Compliance Code for Behavior Analysts (Behavior Analyst Certification Board, 2014), which now more comprehensively reflects the pro- fessional responsibilities of supervisors. Some of these rec- ommendations may be time-consuming and go beyond the requirements of the BACB; however, these points of con- sideration on the application of existing supervisory and ethical guidelines may enhance supervisory practice. Note that we have adopted the term Btrainee^ throughout to refer to BCBA and BCaBA candidates, as well as prac- ticing BCaBAs and RBTs as they require ongoing super- vision and cannot practice independently.

Within this framework, we review the many competen- cies required of supervisors including fluency with the eth- ical guidelines regarding supervision, effective training techniques across basic and complex skill areas (e.g., clin- ical decision-making and professionalism), and perfor- mance feedback methods to support acquisition and main- tenance of skills. Our premise is that the implementation of an evidence-based approach within a positive, collabora- tive, and individualized environment fosters a well- rounded and effective trainee. We also provide examples and considerations for how to incorporate these competen- cy areas into a practice repertoire in which supervisor be- havior, in addition to trainee behavior, is frequently evalu- ated and modified based on trainee feedback and progress. Focus is placed on the use of social validity measures to aid in the ongoing adjustment of supervisory behavior and the ongoing self-assessment of the supervisor.

Practice Model for Behavior Analytic Supervision

Set the Occasion for Collaborative and Ethical Supervision

A strong and collaborative supervisory relationship is widely acknowledged as contributing to the quality and effectiveness of clinical supervision across several other disciplines (e.g., Bernard & Goodyear, 2014; Bordin, 1983; Falender et al., 2004). Although notoriously complex and difficult to objec- tively define and measure, the supervisory relationship is typ- ically defined in two parts. In behavior analytic terms, the first part can be described as the degree to which a supervisor and a trainee are mutual discriminative stimuli for generalized rein- forcement (i.e., Brapport^ as discussed by Carr and colleagues, 1997, p. 111). Carr and colleagues (1997; Magito, McLaughlin, & Carr, 2005) further discuss rapport as a setting event and demonstrate that the development of rapport is a key aspect to approaching skill development. Although their dis- cussion is specific to individuals with developmental disabil- ities, the basic behavioral principles behind their observations have clear implications for the supervisor-trainee relationship.

As an example, the relative rapport between a supervisor and trainee may impact the trainee’s approach behavior to- wards the supervisor. If a supervisor does not function as a cue for reinforcement (or the supervisor functions as a cue for punishment), it is likely that the trainee will avoid the super- visor and instead seek out others, including non-behavior an- alysts, for guidance and feedback. While the specific behav- iors required to develop rapport within the context of the su- pervisory relationship have not yet been systematically iden- tified, a few face valid examples that may assist supervisors in establishing themselves as reinforcers include being ap- proachable (e.g., visibility and nonverbal behaviors) and re- sponsive (e.g., to emails), using more positive statements rel- ative to negative (e.g., 4:1 ratio; Daniels & Bailey, 2014), and demonstrating undivided attention towards the trainee during meetings. Importantly, these behaviors certainly do not ensure that the supervisor will produce a competent trainee, and other variables are also likely to impact the degree to which the trainee approaches the supervisor with relevant issues and questions (e.g., supervisor competency).

The second component of the supervisory relationship can be described as the degree to which a supervisor and trainee generate mutually agreed upon expectations and goals for su- pervision and engage in behavior consistent with those expec- tations and goals. The supervision contract, a requirement of BACB approved supervision (Behavior Analyst Certification Board, 2015), can be used to guide the development of the supervisory relationship and the terms of supervision (see Guideline 5.05; Behavior Analyst Certification Board, 2014). The contract fosters an initial and collaborative discus- sion between a supervisor and trainee with regard to the nature

288 Behav Analysis Practice (2016) 9:287–298

of the professional relationship and the expectations and re- sponsibilities of each member. Supervisors can use this dis- cussion to model shared expectations and commitment to the supervision process.

Self-report data from related fields suggests that a poor supervisory relationship is related to a trainee’s decreased willingness to disclose relevant clinical information (Mehr, Ladany, & Caskie, 2015) and follow through with a supervi- sor’s recommendations (Ladany, Hill, Corbett, & Nutt, 1996), as well as increased likelihood of burnout (Gibson, Grey, & Hastings, 2009). However, to what extent the quality of the supervisory relationship influences the outcomes of supervi- sion is unknown and should be a focus of future research (see an extended discussion below). Until then, we hypothesize that the development of a technically skilled, ethical, and pro- fessional behavior analyst may potentially suffer without the presence of a strong supervisory relationship. Thus, an impor- tant first, and ongoing, step in the supervision process may be to build a strong supervisory relationship. As part of, or in addition to the supervision contract, we suggest supervisors participate in the following discussions with trainees to further establish mutual expectations for the supervision experience and ensure consideration of relevant ethical guidelines in an effort to avoid potential harm to trainees and the professional supervisory relationship.

Define the Relationship The first step in any professional relationship is to define basic and fundamental expectations between participating individuals. While there are clear exam- ples of inappropriate supervisory relationships that often do not necessitate a lengthy initial discussion (e.g., sexual relationships; see Guideline 1.07; Behavior Analyst Certification Board, 2014), subtle aspects of the supervisory relationship may be unclear to the trainee and should be clar- ified. The supervisory relationship can be confusing as the supervisor may serve as a teacher, mentor, evaluator, and fa- cilitator of self-assessment and growth, all of which are over- lapping roles and imply a slightly different style of profession- al interaction and power differential. For example, a supervi- sor is commonly defined as someone who oversees certain activities and holds evaluative power over the trainee, while a mentor is often described as a trusted advisor and confidant. Further, the everyday practice of behavior analysis can be stressful, often physically and mentally exhausting, making initial discussion of a supportive relationship beneficial.

While all these roles are supervisory in nature and holding these roles concurrently with a trainee does not necessarily constitute a multiple relationship as defined by the BACB (Behavior Analyst Certification Board, 2014), supervisors may find themselves differentially allocating their time across several of these roles depending on the current context of supervision and the trainee’s experiences. It may be beneficial to discuss these potentially changing roles at the outset of

supervision and to engage in self-assessment and monitoring, as well as peer consultation, to ensure the role fits the current needs of the trainee while still within the parameters of a professional relationship.

Further, trainees will already have negative and/or positive experiences working with people in these various roles and may have preconceived expectations about the nature of the current supervisory relationship. For example, trainees with past supervisors who did not encourage collaboration may be reticent to present contrasting ideas or raise concerns with their new supervisors. It may be fruitful for supervisors to ask trainees about their definition of supervision and their previ- ous experiences being supervised (whether or not in behavior analysis) and to discuss how their definition and experiences align with those of the supervisor and the most recent version of the BACB Experience Standards (i.e., Behavior Analyst Certification Board, 2015). Also important is the verbal rec- ognition of and sensitivity to the fact that the supervisory relationship can be impacted by beliefs, values, and interper- sonal biases shaped by previous experiences. Discussing such issues may reduce the likelihood of dissimilar views and ex- pectations regarding the supervision process.

Paying for supervision further complicates the role of the supervisor and trainee. Many individuals seeking a BACB credential do not have access to local or Bin-house^ BCBAs who are approved to provide supervision. This challenge re- quires those seeking supervision to identify approved BCBA supervisors in their community or even across great distances (through videoconferencing). When seeking supervision through this format, supervisors typically charge an hourly rate, which is determined by the supervisor and payment for supervision services must be discussed with the trainee at the onset of the supervisory relationship. Since trainees pay for services in this arrangement, the supervisor and trainee must consider any potential negative impact payment may have on the professional supervisory relationship.

Performance Expectations It is important to clearly define, outline, and agree upon the performance expectations and responsibilities of both the trainee and supervisor. Dual re- sponsibilities foster a give-and-take relationship and allow for shared accountability. To ensure a clear understanding of the expectations of the trainee and supervisor according to the BACB, we recommend reviewing the BACB Experience Standards (Behavior Analyst Certification Board, 2015) at the outset of supervision, even if the trainee has already ac- crued supervision hours with another supervisor. It is also important to identify any idiosyncratic expectations of the trainee and/or supervisor that, if not discussed, may place stress on the supervisory relationship. Such expectations may be that both parties must contact each other at least 24 h in advance to reschedule or that both parties will respond to email correspondence within 48 h.

289Behav Analysis Practice (2016) 9:287–298

The degree to which either a trainee or supervisor can con- sistently complete the agreed-upon responsibilities is often impacted by an individual’s time and competing commit- ments. Conducting an analysis of time for both parties is one method for outlining reasonable responsibilities. With regard to the supervisor’s time, consider a supervisor supervising one trainee who is accruing 30 h/week of Supervised Independent Fieldwork. Per the Experience Standards (Behavior Analyst Certification Board, 2015), this equates to 3 h of supervision for every 2-week supervisory period (i.e., 5 % of total accrued hours); however, 3 h of supervision almost always requires more than 3 h of time. Often overlooked in the initial calcula- tion of time is (a) travel time to and from the trainee’s work- place for direct observations (i.e., if different from the super- visor’s workplace and if audio-video recording or videocon- ferencing is not viable), (b) travel time to and from supervision meetings (i.e., if videoconferencing is not an option), (c) time to search for appropriate articles and resources for the trainee, and (d) time to read (or re-read) those articles. Additional time is required in responding to trainee emails, reviewing trainee permanent products, giving detailed feedback during the su- pervision meeting, completing performance evaluations, documenting various aspects of the supervision process (e.g., remediation), and seeking peer feedback, and/or consul- tation with regard to the supervisory behaviors of the supervi- sor. Taken together, these activities can quickly increase those 3 h every 2 weeks to 6 h or more every 2 weeks! We encour- age supervisors and trainees to complete this activity together to establish appropriate workloads and expectations for both parties (see Guideline 5.02; Behavior Analyst Certification Board, 2014).

Evaluation of Performance Expectations One of the main responsibilities of supervisors is to consistently evaluate and document trainee performance. This matter is often an area of trainee distress and an area without strict guidelines, so it is crucial to gain initial agreement about the behaviors to be evaluated (e.g., technical, professional, and ethical), the eval- uation process, and the consequences of poor performance evaluations. Agreeing on the terms of a poor evaluation of the trainee beforehand may make execution of the set conse- quences more straightforward and perhaps have fewer nega- tive side effects. Include in the guidelines clear, objective and measureable circumstances for all consequences. For exam- ple, at what point will a remedial plan be developed (e.g., after 3 consistent supervisory periods of unsatisfactory perfor- mance), what will the remedial plan consist of (e.g., an addi- tional individual meeting with the supervisor, extra readings), and what is the consequence of performing poorly during, or not completing, the remediation (e.g., supervisor terminating the relationship)? Lastly, if the relationship is ultimately ter- minated, it will be important to pre-determine what, if any, hours the supervisor will approve.

Supervisors are highly encouraged to seek feedback from trainees concerning their performance as a supervisor and their ability to follow through with the agreed upon expecta- tions. We recommend communicating the value of this ap- proach to the trainee at the outset of supervision to further create a sense of shared accountability with respect to obtaining the goals of the supervisory relationship. Obtaining candid information from trainees can be difficult (Mehr, Ladany, & Caskie, 2010), and supervisors must con- sider the inherent power differential; however, a strong super- visory relationship between the supervisor and trainee and an expectation of the reciprocal nature of feedback is hypothe- sized to counteract this limitation.

Confidentiality The BACB Professional and Ethical Compliance Code (Behavior Analyst Certification Board, 2014) states that behavior analysts have a primary obligation to protect the confidentiality of the individuals they supervise (Guideline 2.06). Confidential information in a supervisory relationship predominantly includes performance evaluations, such as written reviews or ratings on evaluation forms. It may be beneficial to initially communicate to trainees that this in- formation will be kept out of public sight and only discussed with others that have a clear, professional reason for being concerned with such matters, such as a trainee’s on-sight su- pervisor. It may also be beneficial to let trainees know that individual corrective feedback will be given in private and not during group supervision sessions.

Given the sometimes close, professional relationships that can develop through supervision, the trainee may disclose personal information germane to his or her behavior analytic work. As an example, a trainee may acknowledge an interper- sonal conflict with a professional colleague. Direct conversa- tion about the confidentiality of such disclosures may increase the likelihood that a trainee approaches their supervisor con- fidently and without fear of disclosure so that the issue may be discussed and an appropriate solution determined.

Remote Supervision With the advent of contemporary vid- eoconferencing technology, behavior analytic supervision can occur remotely by interacting through live two-way audio- video communication across the Internet or cellular networks. Supervision through videoconferencing allows supervisors to engage in supervision sessions with trainees without having to be in the same physical location, which results in reduced travel time, increased savings associated with travel costs, and the potential to provide supervision for individuals in rural or hard to reach areas who may not have access to locally qualified behavior analysts.

Despite these benefits, supervision conducted through vid- eoconferencing also poses many potential challenges. These challenges include breaches of client confidentiality, technical difficulties, and lack of physical presence of a supervisor

290 Behav Analysis Practice (2016) 9:287–298

during high-risk situations (Florell, 2016). Supervision con- ducted via videoconferencing must ensure client confidential- ity by facilitating sessions through HIPAA and FERPA com- pliant videoconferencing software (e.g., Adobe Connect, Vidyo, VSee); however, due to perpetual changes to video- conferencing software, supervisors should thoroughly scruti- nize potential software before using it during remote supervi- sion. Readers are also encouraged to review Cavalari, Gillis, Kruser, and Romanczyk (2015) for an extended discussion regarding digital sharing and storage of audio-visual recordings.

Regarding technical difficulties, supervisors and trainees should be familiar with the videoconferencing software that is used, ensure that they have the required hardware for high- quality videoconferencing (e.g., 720p or higher resolution webcam, high bandwidth Internet connection, computer/lap- top/tablet with fast processing speed), and become familiar with the evidence-based practice guidelines for telemental health services (cf. Grady et al., 2011). To guarantee support for trainees during high-risk situations, the supervisor should develop a plan with the trainee so that an on-site supervisor is available or the supervisor can be reached by phone (Abbass et al., 2011; Panos, Panos, Cox, Roby, & Matheson, 2002).

Termination of the Relationship Just as expected with cli- ents, the parameters of planned or unplanned termination of the supervisory relationship should be discussed at the outset of supervision (Guideline 2.15; Behavior Analyst Certification Board, 2014). The BACB encourages trainees to accrue experience hours with different supervisors. Accordingly, an initial conversation about the number of su- pervisors trainees have already worked with and the number of experience hours trainees have accrued is warranted to de- termine a mutually agreed upon duration for the current su- pervisory relationship. Additionally, if the trainee is transfer- ring to another supervisor, we encourage the previous super- visor to speak with the new supervisor about the strengths and weaknesses of the trainee and current goals. This dialogue should be initiated with approval from the trainee. In the event of an unplanned termination such as the supervisor changing employers, development of a contingency plan will ensure the trainee continues to receive supervision.

Baseline Assessment of Trainee Skills

Although there are no standard procedures to initially assess trainee skills, we can incorporate strategies from the field of behavior analysis to guide our evaluation. We suggest the fol- lowing methods as strategies to evaluate trainee baseline per- formance across various skills: behavioral observation, infor- mal assessment, review of course syllabi, discussion with pre- vious supervisors, and review of the BACB Fourth Edition Task List (Behavior Analyst Certification Board, 2012a).

Using these indirect and direct methods, the supervisor and trainee are able to collaboratively develop goals that are objec- tive, measureable, and obtainable (Cooper, Heron, & Heward, 2007; Daniels & Bailey, 2014). We will discuss each of these strategies and their contribution to assessing trainee skills.

Behavioral observation, the basis of all behavior analytic work, requires a behavior analyst to operationally define be- haviors of interest and subsequently record the occurrence of those behaviors. In the context of supervision, a supervisor should observe and record specific trainee behavior such as the extent to which an intervention or behavioral support plan is implemented with integrity (Gresham, Gansle, & Noell, 1993), verbal behavior of a trainee as they interact with staff, colleagues, clients, or parents, and reliability of trainee behav- ioral observation through assessment of inter-observer agree- ment (Cooper et al., 2007).

Informal (formative) assessment, although not behavioral, is useful to gather information from a trainee. This type of assessment is frequently used in education (Dunn & Mulvenon, 2009) and is achieved through conversations with trainees about their previous experiences, completion of checklists, and review of writing samples or portfolios. Such assessment should combine records review, semi-structured interview, and discussion about the areas of practice as a be- havior analyst. The informal assessment should be used to gather information about past experience but also specific areas of strength and weakness. Additionally, as an ongoing skills assessment, supervisors should ask questions of the trainees to evaluate their knowledge of behavioral procedures (e.g., BWhat were the four test conditions of the Iwata, Dorsey, Slifer, Bauman, and Richman (1994) functional analysis [FA]?^). Depending on trainees’ understanding of the FA lit- erature and their prior implementation history, supervisors should ask follow-up questions (e.g., BWhat are pros and cons regarding the manymodifications to the original—Iwata et al., 1994—FA?^). These types of questions allow supervisors to gauge trainees’ skills and trainees’ responses provide a base- line assessment of their ability to think critically and discuss behavior analytic theory broadly or more acutely.

Review of course syllabi with the trainee is quite useful when assessing skills. In a collaborative format within super- vision sessions, the supervisor and trainee may benefit from discussing the topics covered in the trainee’s BACB-approved coursework. Independent of the supervision session, the super- visor is encouraged to review the syllabi to ensure that relevant readings have been assigned—this action provides the oppor- tunity to introduce readings to the trainee that might not have been assigned. The benefit of assigning readings within the supervision context is, as a supervisor, you can work individ- ually with your trainee and critically analyze and discuss the behavior analytic literature. Subsequently, reviewing articles is assumed to teach critical thinking that is essential for applied work and preparation for the upcoming BACB exam.

291Behav Analysis Practice (2016) 9:287–298

Discussion with previous supervisors is a valuable way to assess a trainee’s skill set; however, the availability and ease of these options is variable depending on the practicum site. For example, if trainees receive their experience hours at a large human service provider with multiple schools within the or- ganization, trainees might change schools or classrooms. In this situation, the supervisors might change as well, leaving the opportunity for discourse between the prior and current supervisor.

Finally, it is recommended that the supervisor and trainee jointly review the BACB Fourth Edition Task List (Behavior Analyst Certification Board, 2012a) and use the document as a checklist of areas the trainee has already mastered and those that are still in need of instruction and practice.

Methods to Teach and Promote Skills

Acknowledging that trainees have varying levels of skills, supervisors can expect at some point that they will need to directly and systematically teach new skills. Importantly, the supervisor must attempt to match teaching techniques and performance expectations to the trainee’s current perfor- mance. For example, a novice trainee might not be fluent with all the different approaches to conducting preference assess- ments. In this case, the supervisor could indirectly discuss methods and provide relevant literature for the trainee to read about the topic and improve his or her knowledge. For the trainee who is familiar with the literature on preference assess- ments but has not actually conducted such an assessment with a client, the supervisor could model correct procedures. Lastly, for the trainee who has mastered designing and implementing preference assessments, the supervisor could speak with the trainee about possible modifications to implementing prefer- ence assessments and/or ask that the trainee uses behavioral approaches to teach the skill to another staff member or parent.

In each of the cases described above, the supervisor en- gages in different behaviors and those behaviors are appropri- ate to the trainee’s skill set. Taking the time to comprehensive- ly evaluate the current skill level of a trainee and set appropri- ate behavioral goals and expectations may allow for a more positive and successful supervisory experience for the trainee. As well, this strategy allows supervisors to provide ample amounts of positive feedback to trainees. If supervisors are unaware of trainees’ current skill set, supervisors might pro- vide a type and/or level of supervision that is inappropriate, and run the risk of confusing or overwhelming trainees, which in turn may potentially harm a client (see Guideline 5.03; Behavior Analyst Certification Board, 2014).

Within this individualized framework, it is expected that supervisors use behavioral skills training (BST) to develop novel skills among trainees (Behavior Analyst Certification Board, 2012b). BST is ubiquitous in the behavior analytic

literature (Rosales, Stone, & Rehfeldt, 2009) and shown to be effective for skill acquisition in typically developing chil- dren (Himle, Miltenberger, Flessner, & Gatheridge, 2004; Jones, Kazdin, & Haney, 1981; Yeaton & Bailey, 1978), chil- dren with autism (Gunby & Rapp, 2014), teachers and para- professionals (Lavie & Sturmey, 2002; Sarokoff & Sturmey, 2004), and parents (Himle & Wright, 2014; Seiverling, Williams, Sturmey, & Hart, 2012; Shayne & Miltenberger, 2013). In the context of supervision, BST involves providing a rationale to the trainee about the skills to be acquired, sys- tematic instruction of skills to the trainee, modeling the previ- ously taught skills to the trainee, rehearsal of the skills by the trainee (i.e., either in vivo or role-play), and supervisor posi- tive and corrective feedback to the trainee about his or her performance of the skills (Behavior Analyst Certification Board, 2012b; Parsons, Rollyson, & Reid, 2012). Despite the effectiveness of BST, to our knowledge, there are no be- havioral studies that have evaluated BST to teach skills (i.e., especially relatively complex skills) to individuals pursuing a BACB certification.

It is also expected that supervisors model technical, profes- sional, and ethical behavior (Behavior Analyst Certification Board, 2012b). The behavior of the supervisor is at the foun- dation of competency-based supervision. Ideally, a trainee looks up to the supervisor to learn how to behave in a variety of situations. As a supervisor, modeling technical behavior is important for many skills, such as the language we use to discuss behavior analysis and for the work we conduct with individuals. A supervisor should demonstrate mastery of the formal terminology of ABAwhile being competent with col- loquial speech when conversing with a layperson. Supervisors should teach their trainees both the formal and informal ways of discussing terminology, while explaining the limitations of using jargon with their constituents.

In addition to the technical language, supervisors should also model technical competency when working with individ- uals. Throughout the supervision process, the supervisor might need to Bjump in^ with the trainee and demonstrate how to implement a certain procedure with fidelity. As such, the supervisor must stay current on their clinical practice. Similar to professional athletes, a technically sound supervisor should be able to model technical skills and Bmake it look easy.^ Not only should supervisors model technical skills, they should also model professional and ethical behavior.

Professional and ethical behavior is guided in part by the BACB Professional and Ethical Compliance Code (Behavior Analyst Certification Board, 2014). The BACB ethical code, like other ethical codes (e.g., American Psychological Association), allows for interpretation and professional judg- ment. As a supervisor, it is important not to engage in egre- gious behavior including sexual relationships with a trainee (or a client/family member), accept gifts, fabricate data, or plagiarize. The previously described behaviors are clearer in

292 Behav Analysis Practice (2016) 9:287–298

the ethical code; however, there are other more subtle profes- sional and ethical behaviors that face fewer punishers for the supervisor, but that have the potential to negatively impact the development of a trainee. Common examples of those behav- iors include, answering phone calls, emails, and text messages during supervision (i.e., not providing undivided attention), being tardy and unprepared for meetings, as well as not mak- ing time available outside of supervision for emergencies, staying relevant with the literature, protecting the confidenti- ality of supervision, and modeling effective professional be- havior, interpersonal skills and collaboration with clients, con- stituents and other service providers.

Developing Case Conceptualization, Problem Solving, and Decision-Making Repertoires

Using effective teaching practices, supervisors should estab- lish fluency in trainees’ ability to behaviorally conceptualize a case, as this is a critical component of ABA (Behavior Analyst Certification Board, 2012b). The ability to effectively concep- tualize a case is influenced by mastery of behavior analytic theory and content, experience applying theory in practice, and the ability/willingness to engage in critical thinking. For trainees, all of the previously described components are de- veloping with varying levels across areas. The integration of these skills requires practice and the process of mastering case conceptualization is thought to be enhanced by supervision that includes active and critical conversations by supervisors. As mentioned earlier, engaging trainees in critical conversa- tion is a great way for trainees to gain competency in behavior analysis; likewise, it also allows trainees to carefully consider aspects of an individual’s behavior including, but not limited to, motivating operations, discriminative stimuli, conse- quences, and hypothesized function. As the supervision pro- cess progresses, the trainee should be more autonomous when conceptualizing cases and the role of the supervisor should be to confirm the trainee’s conclusion, while providing feedback as necessary.

Supervisors should also focus on developing problem solv- ing and decision-making abilities among trainees (Behavior Analyst Certification Board, 2012b). The practice of ABA is rooted in a problem-solving framework. Behavior analysts deal with a variety of problems and their ability to effectively manage those problems largely dictates their professional suc- cess. Nezu, Nezu, and D’Zurilla (2013) defined problem solv- ing as B…a self-directed process by which individuals attempt to identify, discover, and/or develop adaptive coping solutions for problems^ (p. 8). Problem solving involves identification of a problem, the formation of potential alternatives to solve the problem, a decision to choose one solution over others, and evaluation of the chosen solution. In supervision, the su- pervisor is responsible for fostering problem-solving skills in the trainee.

Like any behavior, problem solving can be taught using BST. The supervisor should discuss and model each of the steps of problem solving with the trainee (as needed), and particular focus should be on the decision-making step. In our experience, this step is often difficult for trainees to make because they are concerned that they will make the Bwrong^ choice and subsequently Bmess something up.^ Although making a contraindicated choice is a concern, the supervisor’s duty is to ensure that the trainee’s choice does not cause harm. Ultimately, the supervisor should shape independent problem solving as competency in problem solving and decision- making is a critical component to the independent practice of a behavior analyst.

Considerations in the Delivery of Performance Feedback

It is widely acknowledged that performance feedback is key to the development, maintenance, and enhancement of all skills (e.g., Daniels & Bailey, 2014; Mortenson & Witt, 1998). The effectiveness of frequent, genuine, positive, corrective, and specific feedback based on direct observations of behavior is commonly accepted. As such, the delivery of timely and ef- fective feedback (i.e., feedback that improves behavior) is required of all behavior analytic supervisors (Behavior Analyst Certification Board, 2012b; Guideline 5.06; Behavior Analyst Certification Board, 2014). Feedback can be given in a variety of modalities (e.g., verbal, written, graph- ic, informal) and has been shown to be most effective when combined with review of data, goal setting, and modifications to antecedents and consequences (Alvero, Bucklin, & Austin, 2001; Balacazar, Hopkins, & Suarez, 1985; DiGennaro, Martens, & Kleinmann, 2007; Sanetti, Luiselli, & Handler, 2007). Due to the importance of feedback to the supervision process, there are several variables to be considered, including but not limited to the individualization of feedback, ratio of positive to corrective feedback, use of rationales, delivery of corrective feedback regarding interpersonal or subjective be- havior, documentation of feedback, and a trainee’s current skill level.

The BACB Supervision Experience Form includes a sam- ple written evaluation of trainee performance across broad domains such as timeliness, professionalism, and acquisition of target behavior analytic skills. However, the form may be modified to better fit the individual needs of the trainee (but see the BACB Experience Standards [Behavior Analyst Certification Board, 2015] for the required components of this form). For example, items can be added (e.g., trainee-specific goals) and removed (e.g., items that are consistently rated as satisfactory) or the rating scale can be adjusted to better depict the breadth of trainee performance in any one area (e.g., 5- point rating scale). Individualizing the form collaboratively with the trainee at the outset of, and throughout, the supervi- sion process may increase the relevancy of the feedback for

293Behav Analysis Practice (2016) 9:287–298

the trainee, demonstrate the supervisor’s commitment to su- pervision, and allow the supervisor to focus feedback on the trainee’s current goals.

Feedback provided with a high positive to negative ratio is recommended within the performance management literature (e.g., Daniels & Bailey, 2014). Maintaining a supervisor’s value as a positive reinforcer may be enhanced by such a practice. We also suggest that positive feedback may be en- hanced by a rationale (i.e., a core component of both BST and corrective feedback, Behavior Analyst Certification Board, 2012b) that intimately links trainees’ behavior to their short- and/or long-term goals. For example, BVery impressive per- formance today with Johnny. Even though it was a stressful situation, you quickly jumped in to help the other staff imple- ment the reactive procedures of his behavior plan. The great thing was that all the other staff saw you keep your cool and implement the intervention as written, which is important as you’re trying to establish yourself as a role model in the class- room. The more you can get in there and model, just like you did today, the more others will start to look to you as a leader.^

Equally as necessary as positive feedback is the delivery of corrective feedback, which requires empathy, detailed review of incorrect and correct performance, modeling, and practice (Behavior Analyst Certification Board, 2012b). When giving corrective feedback, it is often recommended to remain objec- tive (e.g., do not correct when in a bad mood; Daniels & Bailey, 2014), do so privately, and keep in mind the trainee’s context prior to giving feedback (i.e., empathy). As a point of emphasis, supervisors are encouraged to be mindful of Bfavorites^ (i.e., potential side effect of mutual conditioned reinforcers) due to personal characteristics as these biases have the potential to overshadow difficulties of or mistakes made by the trainee.

While provision of corrective feedback for common behav- ior analytic activities such as graphing or implementing a preference assessment is relatively straightforward, delivering corrective feedback for personal or more subjective perfor- mance areas that are negatively impacting a trainee’s work can be significantly more challenging. These areas could be a trainee’s poor social interaction abilities, insensitivity to others, and inflexibility. Self-report data collected from re- searchers in related fields suggests that supervisors often with- hold corrective feedback regarding negative reactions to sub- jective aspects of a trainee’s performance or give higher rat- ings in fear of harming the supervisory relationship (Hoffman, Hill, Holmes, & Freitas, 2005; Gonsalvez & Freestone, 2007).

As supervisors, we are the gatekeepers of our field; failing to raise important feedback can have negative implications for clients and the advancement of the profession. When supervi- sors are faced with having to provide relatively more subjec- tive or difficult feedback, they are encouraged to seek peer consultation and discuss potential approaches and practice delivering corrective feedback prior to meeting with the

trainee. We recommend objectively documenting all relevant variables to the delivery of corrective feedback including dates/times, details of events, feedback-specific areas of diffi- culty, and objective responses and reactions from the trainee.

Lastly, tailoring the parameters in which supervisors pro- vide feedback should be considered as trainees progress through different skill areas. Novice trainees, or more ad- vanced trainees learning a new skill, often require a higher rate of performance feedback. As trainees begin to develop skills in each area, the ultimate goal is to have them accurately and reliably evaluate their own ability to demonstrate each skill. Asking trainees questions such as, BWhat did you do correctly?^ or BWhat would you do differently next time^ may help them reflect and think critically about their perfor- mance. One method for aiding this process is to have trainees view audio-visual recordings of themselves performing activ- ities such as leading a parent or staff training and implementing conditions of a functional analysis. Following observation of videos, the supervisor and trainee are able to discuss performance objectively. Another method that may promote this process is to have trainees evaluate their written work (e.g., FBA or BIP) relative to a rubric and compare their results to those of the supervisor.

Evaluation of the Process and Outcomes of Supervision

Behavior analytic supervisors should systematically evaluate the outcomes of their own supervision practices (Guideline 5.07, Behavior Analyst Certification Board, 2014, 2012b) using both objective and subjective measures. Direct and re- peated measurement of trainees’ behavior is required to ob- jec t ively evaluate the outcomes of supervis ion. Commensurate with the dimensions of ABA (Baer, Wolf, & Risley, 1968), the supervisor should define the skills to be acquired, collect data before, during and after skills training, and use these data to provide feedback to the trainee. For example, using a procedural fidelity form, supervisors can monitor and graph trainees’ performance implementing skill acquisition programs, behavior reduction plans, and behavior- al skills training. Direct measures of clearly defined, profes- sional behaviors (e.g., timeliness) can also assist in delivering feedback and evaluating outcomes. Additionally, behavior an- alysts are encouraged to publish the results in peer-reviewed journals to demonstrate the effectiveness of behavioral ap- proaches within the context of supervision and the relevant skills.

The review of permanent products also serves to objective- ly evaluate a trainee’s performance. Permanent products are the tangible results of behavior and are commonly used as a data recording method. They can be collected on academic tasks (e.g., completion of mathematics problems), problem behavior (e.g., count of holes in a wall), and social skills (count of phone numbers obtained) to name a few. In the case

294 Behav Analysis Practice (2016) 9:287–298

of supervision, materials developed by the trainee are the most obvious permanent products. Those materials include, for ex- ample, written behavior intervention plans, data sheets, skill- acquisition plans, assessment reports, graphs, and relevant goals and objectives in a student’s IEP. Moreover, there are other permanent products that should be reviewed such as notes/letters to parents and email correspondence with teachers/staff. When utilizing a rubric and clear criteria, all of these documents allow a supervisor to objectively evaluate a trainee’s growing competency regarding the quality of writ- ten materials that are critical for constituents that behavior analysts serve.

Complimentary to objective measures, social validity mea- sures (Wolf, 1978) can be used to evaluate socially significant changes in the behavior of a trainee’s clients and satisfaction with the performance of the trainee by parents and other pro- fessionals. Social validity measures can also be used to assess the trainee’s satisfaction with the goals, procedures, and out- comes of the supervision process. Besides the weekly (or bi- weekly) trainee rating forms we described earlier, supervisors are encouraged to obtain this type of feedback from their trainees. Although not mandated by the BACB, and while such measures may not be appropriate means to evaluate the effectiveness of the supervision process with regard to skill development and client and staff performance outcomes, they can establish more acceptable and mutually agreed-upon su- pervision practices and serve as a tool for monitoring the supervisor-trainee relationship. The objective is having the trainee identify the methods that are most acceptable, and allow supervisors to modify their approach during the trainee’s experience (if needed) and enhance the supervision process overall. The supervisor may disagree with the feed- back provided by the trainee; however, this activity opens a dialogue about what is and is not working and ideally, ensures that the supervisor and trainee speak openly to each other. Then, a plan can be created to alter the behavior of the trainee and/or the supervisor. To reiterate, the foundation for this type of feedback is grounded in the performance feedback litera- ture and has clear applications for BCBA supervision.

To assist with this type of evaluation, we developed a Supervision Monitoring and Evaluation Form as a project during training at a doctoral internship program in clinical psychology. The internship site had a strong behavioral orien- tation and employed/trained behavior analysts. Duringmonth- ly meetings with the Director of Training, the internship class discussed strengths and weaknesses of supervisors, commented about their previous experiences in supervision, and read literature on competency-based supervision from multiple fields. After aggregating all of the information, the group developed the Supervision Monitoring and Evaluation Form (see appendix).

After developing the form, the clinical interns piloted im- plementation with their supervisors several times throughout

the year. This implementation not only gave the trainees (i.e., behavioral psychology interns) the opportunity to Btest-drive^ the measure, but also allowed them to provide feedback to their supervisors. Supervisors, in fact, were uniformly positive about the process of evaluation and readily embraced the feed- back from their trainees.We suspect that this positive outcome was due, in part, to having supervisors participate in develop- ing the Supervision Monitoring and Evaluation Form at the earliest stages and being receptive to having interns objective- ly evaluate supervision practices. Again, it is the reciprocity between trainees and supervisors that is hypothesized to en- gender success and mutual satisfaction.

Continuing Education in Supervision

The above sections highlight the many responsibilities and considerations that are present to behavior analytic supervi- sors and the degree to which supervision of those pursuing a BACB certification can be complex, time consuming, and often requires skills beyond the research and practice skills taught in the formal education of behavior analysts. In recog- nition of this fact and in addition to maintaining competency in behavior analysis, the BACB now requires behavior ana- lysts to receive ongoing education in supervision (Behavior Analyst Certification Board, 2015), such as effective supervi- sion practices, ethical considerations regarding supervision, and the standards of BACB supervision.

While supervisors can obtain formal continuing education pertaining to supervision through conference presentations, workshops, and online seminars, supervisors should also seek out their own supervision (Behavior Analyst Certification Board, 2012b). The current model highlights the importance of reciprocal feedback and collaboration between the trainee and supervisor, but it is also important to develop this type of relationship among supervisors to encourage mentorship and consultation in an effort to maintain and enhance supervisor competence. In addition to consultation, to the extent possible, it is important to have other competent supervisors observe your supervision activities and provide feedback based on the objective criteria we outlined previously.

Social validity measures may further be used as a self- monitoring tool by providing a means for supervisors to con- tinually evaluate their performance as a supervisor, which provides an excellent model of continual self-assessment for trainees. Placing focus on self-assessment will enable super- visors to be aware of their strengths and weaknesses in super- vision practices as well as clinical and research activities. Using this self-assessment approach, competency-based su- pervision brings accountability to the practice of behavior analysis and sets the stage for quality supervision. A first next step is to conduct an evaluation of the impact of self- monitoring and evaluation on supervisor behavior and the

295Behav Analysis Practice (2016) 9:287–298

supervisory relationship, and in turn the impact on trainee skill development.

Concluding Remarks: Initiating a Research Agenda in Behavior Analytic Supervision

Competency-based models allow for systematic measurement of both supervisor and trainee behavior throughout the entire supervision process, which makes these approaches appropri- ate for applied research. As there is a dearth of research in this area, the specific direction of and topics that future research may take are vast; however, there we present a few consider- ations for research in this area. First, it will be essential to objectively define and task analyze core competencies of trainees and supervisors, especially those falling within the realm of professional and ethical behaviors (e.g., specific skills denoting professionalism, display of empathic behav- ior), so that we can make objective conclusions about an in- dividual’s competency. It will also be necessary to identify the validity and value of various outcome measures such as social validity, BACB exam pass rate, trainee observable behavior in contrived and naturalistic settings, and client behavior change.

Within this framework, future research should focus on the identification of effective methods for training generalizable and maintainable skills to trainees. While the training of basic procedural skills (e.g., implementing functional analysis con- ditions) may be a first step, methods for training the complex skills required of behavior analysts are needed (e.g., interviewing skills, problem solving, responding to ethical dilemmas). For example, what are the considerations for implementing BST for these skills? How many exemplars are needed to obtain competency and generalization? What are effective methods of prompting (e.g., video modeling) and teaching for generalization?

Further, there is a tentative hypothesis that the quality of the supervisory relationship as defined by the presence of rapport and mutually agreed upon expectations will impact the super- vision experience, notwithstanding the need for more thor- ough empirical validation. For example, does the supervisory relationship actually lead to quicker acquisition of skills and/ or a higher number of skill competencies obtained by the trainee? In order to answer these and many other questions, we must first objectively define the supervisory relationship. While we presented an initial definition of the relationship within this paper, objective indicators of such a relationship must be described, measured, and incorporated into supervi- sion practices. Thus, attempts to measure Bgood^ rapport have focused on the presence of smiling and laughing and the ab- sence of problem behavior (Magito et al., 2005) but other measures appear relevant. Trust, for example, can be observed as someone consistently following through on verbal state- ments. BLikeability,^ another researched construct, translates to a person who has a positive attitude, appears happy, speaks

in a pleasant tone, and helps others achieve their goals (Sanders, 2006). Given the focus of the supervisory relation- ship in all other models of supervision across disciplines, it would be advantageous for behavior analysts to systematically evaluate the impact of the relationship on trainee outcomes and disseminate to the larger supervision community. Taken together, comprehensive considerations of the supervisory guidelines, ethical code and establishing a research agenda in behavior analytic supervision will ideally lead to an effec- tive, ethical, and professional group of rapidly growing prac- ticing behavior analysts.

Compliance with Ethical Standards This article does not contain any studies with human participants or animals performed by any of the au- thors.

Conflict of Interest The authors declare that they have no conflict of interest.

Funding Funding was not obtained for the creation of this manuscript.

References

Abbass, A., Arthey, S., Elliott, J., Fedak, T., Nowoweiski, D., Markovski, J., et al. (2011). Web-conference supervision for advanced psycho- therapy training: a practical guide. Psychotherapy, 48(2), 109–118.

Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness and essential characteristics of performance feedback in organizational settings. Journal of Organizational Behavior Management, 21(1), 3–29. doi:10.1300/J075v21n01_02.

American Psychological Association. (2015). Guidelines for clinical su- pervision in health service psychology. American Psychologist, 70(1), 33–46. doi:10.1037/a0038112.

Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimen- sions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. doi:10.1901/jaba.1968.1-91.

Balacazar, F. E., Hopkins, B. L., & Suarez, Y. (1985). A critical objective view of performance feedback. Journal of Organizational Behavior Management, 7(3–4), 65–89. doi:10.1300/J075v07n03_05.

Behavior Analyst Certification Board. (2011). Online newsletter, September issue. Littleton, CO: Behavior Analyst Certification Board.

Behavior Analyst Certification Board. (2012a). Coursework require- ments for BACB credentials: fourth edition task list. Littleton, CO: Behavior Analyst Certification Board.

Behavior Analyst Certification Board. (2012b). Supervisor training cur- riculum outline. Littleton, CO: Behavior Analyst Certification Board.

Behavior Analyst Certification Board. (2014). Professional and ethical compliance code for behavior analysts. Littleton, CO: Behavior Analyst Certification Board.

Behavior Analyst Certification Board. (2015). Experience standards. Littleton, CO: Behavior Analyst Certification Board.

Bernard, J. M., & Goodyear, R. K. (2014). Fundamentals of clinical supervision (5th ed.). Boston, MA: Pearson.

Bordin, E. S. (1983). Aworking alliance based model of supervision. The Counseling Psychologist , 11(1) , 35–42. doi:10.1177/ 0011000083111007.

296 Behav Analysis Practice (2016) 9:287–298

Carr, E. G., Levin, L., McConnachie, G., Carlson, J. I., Kemp, D. C., & Smith, C. E. (1997). Communication-based intervention for prob- lem behavior: a user’s guide for producing positive change. Baltimore, Maryland: Paul H. Brookes Publishing Co., Inc.

Cavalari, R. N. S., Gillis, J. M., Kruser, N., & Romanczyk, R. G. (2015). Digital communication and records in service provision and super- vision: regulation and practice. Behavior Analysis in Practice, 8(2), 176–189. doi:10.1007/s40617-014-0030-3.

Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. Upper Saddle River: Pearson Education.

Daniels, A. C., & Bailey, J. S. (2014). Performance management: chang- ing behavior that drives organizational effectiveness (5th ed.). Atlanta: Aubrey Daniels International, Inc.

DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A com- parison of performance feedback procedures on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40(3), 447–461. doi:10.1901/jaba.2007.40-447.

Dunn, K. E., & Mulvenon, S. W. (2009). A critical review of research on formative assessment: the limited scientific evidence of the impact of formative assessment in education. Practical Assessment, Research & Evaluation, 14(7), 1–11.

Eikeseth, S., Hayward, D., Gale, C., Gitlesen, J., & Eldevek, S. (2009). Intensity of supervision and outcome for preschool aged children receiving early and intensive behavioral interventions: a preliminary study. Research in Autism Spectrum Disorders, 3(1), 67–73. doi:10. 1016/j.rasd.2008.04.003.

Falender, C. A., Erickson Cornish, J. A., Goodyear, R., Hatcher, R., Kaslow, N. J., Leventhal, G., et al. (2004). Defining competencies in psychology supervision: a consensus statement. Journal of Clinical Psychology, 60(7), 771–785. doi:10.1002/jclp.20013.

Falender, C. A., & Shafranske, E. P. (2012). The importance of competency-based clinical supervision and training in the twenty- first century: why bother? Journal of Contemporary Psychotherapy, 42(3), 129–137. doi:10.1007/s10879-011-9198-9.

Florell, D. (2016).Web-based training and supervision. In J. K. Luiselli & A. J. Fischer (Eds.), Computer-assisted and web-based innovations in psychology, special education, and health. New York: Academic Press/Elsevier.

Frank, F. R., Snell, L. S., tenCate, O., Holmboe, E. S., Carraccio, C., Swing, S. R., et al. (2010). Competency-based medical education: theory to practice. Medical Teacher, 32(8), 638–645. doi:10.3109/ 0142159X.2010.501190.

Gibson, J. A., Grey, I. M., & Hastings, R. P. (2009). Supervisor support as a predictor of burnout and therapeutic self-efficacy in therapists working in ABA schools. Journal of Autism and Developmental Disorders, 39, 1024–1030. doi:10.1007/s10803-009-0709-4.

Gonsalvez, C., & Freestone, J. (2007). Field supervisors’ assessments of trainee performance: are they reliable and valid? Australian Psychologist, 42(1), 23–32. doi:10.1111/j.17429544.2011.00033.x.

Grady, B., Myers, K. M., Nelson, E., Belz, N., Bennett, L., Carnahan, L., et al. (2011). Evidence-based practice for telemental health. Telemedicine and e-Health, 17(2), 131–148.

Gresham, F.M., Gansle, K. A., &Noell, G. H. (1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26(2), 257–263. doi:10.1901/jaba.1993.26-257.

Gunby, K. V., & Rapp, J. T. (2014). The use of behavioral skills training and in situ feedback to protect children with autism from abduction lures. Journal of Applied Behavior Analysis, 47(4), 856–860. doi: 10.1002/jaba.173.

Himle, M. B., Miltenberger, R. G., Flessner, C., & Gatheridge, B. (2004). Teaching safety skills to children to prevent gun play. Journal of Applied Behavior Analysis, 37(1), 1–10. doi:10.1901/ jaba.2004.37-1.

Himle, M. B., & Wright, K. A. (2014). Behavioral skills training to improve installation and use of child passenger safety restraints.

Journal of Applied Behavior Analysis, 47(3), 549–559. doi:10. 1002/jaba.143.

Hoffman, M. A., Hill, C. E., Holmes, S. E., & Freitas, G. F. (2005). Supervisor perspective on the process and outcome of giving easy, difficult, or no feedback to trainees. Journal of Counseling Psychology, 52(1), 3–13. doi:10.1037/0022-0167.52.1.3.

Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27(2), 197–209. doi:10.1901/jaba. 1994.27-197.

Jones, R. T., Kazdin, A. E., & Haney, J. L. (1981). Social validation and training of emergency fire safety skills for potential injury preven- tion and life saving. Journal of Applied Behavior Analysis, 14(3), 249–260. doi:10.1901/jaba.1981.14-249.

Kraemer-Tebes, J., Matlin, S. L., Migdole, S. J., Farkas, M. S., Money, R. W., Shulman, L., et al. (2011). Providing competency training to clinical supervisors through an interactional supervision approach. Research on Social Work Practice, 21(2), 190–199. doi:10.1177/ 1049731510385827.

Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature, extent, and importance of what psychotherapy trainees do not dis- close to their supervisors. Journal of Counseling Psychology, 43(1), 10–24. doi:10.1037/0022-0167.43.1.10.

Lavie, T., & Sturmey, P. (2002). Training staff to conduct a paired- stimulus preference assessment. Journal of Applied Behavior Analysis, 35(2), 209–211. doi:10.1901/jaba.2002.35-209.

Magito McLaughlin, D., & Carr, E.G. (2005). Quality of rapport as a set t ing event for problem behavior : assessment and intervention. Journal of Positive Behavior Interventions, 7(2), 68- 91.

Mehr, K. E., Ladany, N., & Caskie, G. I. L. (2010). Trainee nondisclosure in supervision: what are they not telling you? Counselling and Psychotherapy Research, 10(2), 103–113. doi:10.1080/ 1473314100371230d.

Mehr, K. E., Ladany, N., & Caskie, G. I. L. (2015). Factors influencing trainee willingness to disclose in supervision. Training and Education in Professional Psychology, 9(1), 44–51. doi:10.1037/ tep0000028.

Mortenson, B. P., & Witt, J. C. (1998). The use of weekly perfor- mance feedback to increase teacher implementation of a pre- referral academic intervention. School Psychology Review, 27(4), 613–627.

Nezu, A. M., Nezu, C. M., & D’Zurilla, T. J. (2013). Problem- solving therapy: a treatment manual. New York: Springer Publishing.

Panos, P. T., Panos, A., Cox, S. E., Roby, J. L., & Matheson, K. W. (2002). Ethical issues concerning the use of videoconferencing to supervise international social work field practicum students. International Social Work, 38(3), 421–437.

Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: a guide for practitioners. Behavior Analysis in Practice, 5(2), 2–11.

Rosales, R., Stone, K., & Rehfeldt, R. A. (2009). The effects of the behavioral skills training on implementation of the picture exchange communication system. Journal of Applied Behavior Analysis, 42(3), 541–549. doi:10.1901/jaba.2009.42-541.

Sanders, T. (2006). The likeability factor. New York: Crown Business. Sanetti, L. M. H., Luiselli, J. K., & Handler, M. W. (2007). Effects

of verbal and graphic performance feedback on behavior sup- port plan implementation in a public elementary school. Behavior Modification, 31(4), 454–465. doi:10.1177/ 0145445506297583.

Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete-trial teaching. Journal of Applied Behavior Analysis, 37(4), 535–538. doi:10.1901/jaba.2004. 37-535.

297Behav Analysis Practice (2016) 9:287–298

Seiverling, L., Williams, K., Sturmey, P., & Hart, S. (2012). Effects of behavioral skills training on parental treatment of children’s food selectivity. Journal of Applied Behavior Analysis, 45(1), 197–203. doi:10.1901/jaba.2012.45-197.

Shayne, R., &Miltenberger, R. G. (2013). Evaluation of behavioral skills training for teaching functional assessment and treatment selection skills to parents. Behavioral Interventions, 28(1), 4–21. doi:10. 1002/bin.1350.

Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11(2), 203–214. doi:10.1901/jaba. 1978.11-203.

Yeaton, W. H., & Bailey, J. S. (1978). Teaching pedestrian safety skills to young children: an analysis and one-year follow up. Journal of Applied Behavior Analysis, 11(3), 315–329. doi:10.1901/jaba. 1978.11-315.

298 Behav Analysis Practice (2016) 9:287–298