Module 7 case study

Vol.:(0123456789)

https://doi.org/10.1007/s40617-021-00675-4

DISCUSSION AND REVIEW PAPER

The Performance Diagnostic Checklist - Human Services: Guidance for Assessment Administration

Denys Brand1  · Tyra P. Sellers2  · David A. Wilder3  · James E. Carr2

Accepted: 17 November 2021 © Association for Behavior Analysis International 2022

Abstract The Performance Diagnostic Checklist - Human Services (PDC-HS) is an assessment designed to assess the environmental variables contributing to employee performance concerns in human-service settings. Recent research has demonstrated that interventions indicated by the PDC-HS result in improved employee performance across several human-service settings and that the assessment has acceptable reliability and validity. Although PDC-HS-indicated interventions have been effective at increasing employee performance, there is a need for additional guidance when using the assessment given the limited nature of the original administration guidelines. Thus, the purpose of the current manuscript is to introduce additional guidance for use of the PDC-HS across a variety of situations.

Key words functional assessment · human services · performance analysis · performance diagnostic checklist - human services · performance management · staff evaluation

Performance analysis is the organizational equivalent of the functional assessment of problem behavior that is often con- ducted in clinical and educational settings (Ditzian et al., 2015; Wilder et al., 2020). The goal of performance analy- sis is to identify environmental variables contributing to employee performance concerns and subsequently develop assessment-based interventions to address them (Austin, 2000). The most common performance analysis instrument in the behavior-analytic literature in recent decades is the Performance Diagnostic Checklist (PDC; Austin, 2000; Wilder et al., 2018). The PDC is an informant-based method for conducting a performance analysis and was designed specifically for use in business and industry (Austin, 2000).

In 2013, the PDC was adapted for use in human-ser- vice settings (e.g., clinics, residential treatment facilities, schools), resulting in the Performance Diagnostic Checklist – Human Services (PDC-HS; Carr et al., 2013). Common

employee performance concerns in human-service settings include poor attendance, inadequate data collection, and failure to implement treatment protocols with sufficient integrity, among others. Given the importance of services delivered in human-service settings, it is paramount that employee performance concerns are addressed efficiently and effectively. In addition, given the uniqueness of human- service settings, adapting the PDC to those environments was warranted. The PDC-HS includes 20 questions, 13 of which are informant-based, and 7 that require direct observa- tion. The questions are organized into four domains: Train- ing; Task Clarification and Prompting; Resources, Materi- als, and Processes; and Performance Consequences, Effort, and Competition. Interventions based on results from the PDC-HS have been referred to as indicated interventions, and those that are not have been referred to as nonindicated interventions; the former should be prioritized to maximize treatment efficiency.

Since its publication in 2013, the PDC-HS has been used in several empirical studies. A recent review by Wilder et al. (2020) showed that interventions indicated by the PDC-HS have been successfully implemented across a variety of human-service settings to improve employee performance, suggesting that the assessment is a valid and reliable tool for diagnosing the potential causes of performance concerns (see Wilder et al., 2020 for further

* Denys Brand [email protected]

1 Department of Psychology, California State University, 6000 J Street, Sacramento, CA 95819-6007, USA

2 Behavior Analyst Certification Board, Sacramento, CA, USA 3 School of Behavior Analysis, Florida Institute of Technology,

and the Scott Center for Autism Treatment, Melbourne, FL, USA

/ Published online: 4 February 2022

Behavior Analysis in Practice (2022) 15:951–957

discussion of the reliability and validity of the PDC-HS). Although PDC-HS-indicated interventions have been effective at improving employee performance, the origi- nal administration guidelines from Carr et al. (2013) are minimal. The original guidance appears sufficient when PDC-HS users operate under ideal or near-ideal condi- tions (Wilder et al., 2019). However, PDC-HS users do not always work under such conditions and may find themselves unsure of how to proceed when conducting the assessment. For example, in real-world consultations, PDC-HS questions are often followed with additional clarifying questions, and the answers to these follow-up questions may affect scoring and intervention selection (Cymbal et al., 2020). Based on our collective experience with the PDC-HS, it is our assertion that users may benefit from additional guidance and instructions when using the assessment. Thus, the purpose of the present manuscript is to provide updated guidance for administering the PDC- HS. We make several recommendations and suggestions for how to most effectively implement the PDC-HS based on available data, as well as our collective experience con- ducting related research, and training others to use the assessment. To that end, we discuss some considerations with respect to completing the PDC-HS, targeting perfor- mance concerns, and using data collected from PDC-HS assessments. Table 1 includes a summary of our main rec- ommendations and suggestions.

Completing the PDC‑HS

The PDC-HS (Carr et al., 2013) was designed to be imple- mented in an interview format, except for the items that require direct observation. That is, the consultant or manager interviews the direct supervisor of the employee exhibiting the performance concern and the direct supervisor answers PDC-HS questions about the employee’s performance con- cern. However, no additional guidance was provided on how to complete the assessment. Despite the authors’ inten- tion that the assessment be completed by behavior analysts, researchers have demonstrated that supervisors without training in behavior analysis can successfully use the PDC- HS (e.g., Bowe & Sellers, 2018; Smith & Wilder, 2018). In a recent review of the PDC-HS literature, Wilder et al. (2020) proposed that it might be valuable to explore having employees complete the assessment tool themselves. Given these varied applications and recommendations, supervisors and those responsible for training supervisors may benefit from some guidance related to when and how to implement the PDC-HS.

When to Use the PDC‑HS

The PDC (designed for use in business and industry) and PDC-HS (used to address performance concerns in human-service settings) were designed to help supervisors identify the environmental variables related to a specific

Table 1 Summary of PDC-HS Assessment Guidelines

User-Guidelines

How to Use the PDC-HS • Interview format with direct supervisor (except for direct observations) and ensure accuracy of report (e.g., others confirm, no presence of bias on part of supervisor)

• Where possible, use permanent product recording (e.g., videos, reports) to verify performance concern • Interview target staff directly to gather information of which supervisor may not be aware or to which

they may not have access Training Others to Use the PDC-HS Take a Behavioral Skills Training approach

• Instructions (i.e., Tailor direct instructions to the needs of individual trainees. Provide resources, such as articles)

• Modeling (i.e., Demonstrate how to administer the PDC-HS) • Rehearsal (i.e., Give trainees the opportunity to practice using the assessment) • Feedback (i.e., Trainer reviews trainee performance and provides feedback)

Describe Performance Concern • Accurately describe the performance concern in a way that is measurable and observable • Describe performance concern as either a behavioral excess or deficit

Conducting Direct Observations • Conduct direct observations during typical working conditions to ensure that data are representative of employee behavior

• Minimize employee reactivity • Collect enough data to obtain reliable information

Using PDC-HS Data • Every NO response is an opportunity for intervention • When several NO responses are scored across more than one domain, carefully consider the performance

concern, items scored as NO, and available organizational resources when selecting and designing inter- ventions

• Retain and aggregate data collected from the PDC-HS across employees and performance concerns to help organizations detect the need for possible systems-levels changes

952 Behavior Analysis in Practice (2022) 15:951–957

performance concern and subsequently design a matched intervention (Carr et al., 2013). Newer supervisors may have limited experience addressing staff performance concerns in a systematic and planful manner; therefore, they might use the PDC-HS each time they encounter an employee performance issue until they become proficient at (a) considering and identifying the relevant environ- mental factors that may be impacting performance and (b) matching an intervention to the identified barriers. Once proficient with this process, experienced super- visors may find they follow the process without using the PDC-HS for every performance issue. Conversely, consistently using the PDC-HS when addressing all per- formance concerns may increase the likelihood of suc- cessfully assessing and addressing the need early on and reduce the chances that the issue will worsen or resurface in the future. Thus, for inexperienced supervisors, it may be good practice to consistently implement the PDC-HS for persistent staff performance concerns.

If experienced supervisors cease using the PDC-HS for more common staff performance concerns, they may still consider using the PDC-HS under a few specific cir- cumstances. First, experienced supervisors may wish to periodically complete the PDC-HS to evaluate their own behavior when addressing staff performance concerns to detect any drift from carefully assessing potential envi- ronmental barriers in a systematic manner (e.g., Godat & Brigham, 1999). Second, it may be helpful to use the PDC-HS if a supervisor did not initially use it, but the performance issue was unresponsive to typical strategies for improving performance. In such cases, the supervisor may have overlooked a critical contributing factor requir- ing adjustments to the intervention to directly address the function of the performance issue. Third, if relatively new supervisors rarely encounter employee performance con- cerns, they may consider using the PDC-HS when those issues do arise, as they are likely not fluent in taking a structured, function-matched approach to assessment and intervention.

All supervisors should consider using the PDC-HS in instances in which they may be negatively influenced by their own perspective, history, or bias (e.g., Bernardin et al., 2016; Gonsalvez & Freestone, 2007; Kaplan , 2018) with the target employee, such that they find it difficult to remain objective when assessing the performance con- cern. Using the PDC-HS may provide sufficient structure to guide the supervisor through an objective evaluation process that minimizes attributing performance concerns to personal characteristics (e.g., laziness, inability to comprehend a concept or learn a skill) and maximizes critically assessing the environmental contributors.

How to Implement the PDC‑HS

In most instances, the employee’s direct supervisor will complete the PDC-HS (Carr et al., 2013). If the direct super- visor is well trained in using the assessment, they may com- plete the steps – answer questions; review data; and iden- tify, design, implement, and monitor the intervention – on their own. In other cases, the employee’s direct supervisor may need support from their own supervisor to complete the aforementioned steps (e.g., when the direct supervisor is new or unfamiliar with the performance concern). In either case, questions should be answered based on current, reli- able information about the employee. Many of the ques- tions (e.g., question 1 in the Training and Task Clarification and Prompting domains) can be answered based on what the direct supervisor already knows about the employee’s performance. However, it is important to critically evalu- ate the degree to which the direct supervisor’s recollection represents a current, accurate, and consistent pattern of the performance concern.

Many supervisors may report that they accurately remember details regarding an employee’s performance concern. Research in organizational behavior management, and applied behavior analysis more broadly, has provided some reason for skepticism about this claim. For example, Iwata et al. (2013) found that informant methods of assess- ment of problem behavior may be accurate in only about 64% of cases. Researchers have also assessed the extent to which supervisors can identify employee’s most preferred items, which requires them to remember details about employee behavior. Wilder et al. (2007) found that super- visors are not very good at identifying preferred items for their employees. Wilder et al. (2007) had supervisors in a variety of industries list some of their employees’ most preferred items, and then had those employees rank their preferences. Results showed that supervisors were good at identifying an employee’s most preferred item, but correlations between the manger-identified rank and the employee-identified rank were low. Wilder et al. (2011) replicated these findings with 100 participants. Therefore, it is important to ensure that the supervisor’s account is not influenced by a singular recent instance of poor perfor- mance, an overly sensitive perception of the performance concern, some other bias, or just a faulty recollection of the performance. This is especially important when the supervisor has limited direct knowledge about the employ- ee’s performance and is relying on the employee’s direct supervisor to provide answers to the assessment ques- tions. To address these concerns, the supervisor can ask whether there is a consistent performance concern (versus one that has only occurred once), and whether others also describe this pattern of behavior as concerning. In some cases, there may be permanent products (e.g., reports,

953Behavior Analysis in Practice (2022) 15:951–957

work areas, video footage) that the supervisor can review to assist in verifying the performance issue (e.g., Ditzian et al., 2015; Merritt et al., 2019).

Merritt et al. (2019) reported an alternative method of implementation in which the consultant directly interviewed the employees exhibiting the performance concern. These researchers individually interviewed four school-based direct-care staff members themselves, as well as their direct supervisors, using the PDC-HS to identify the variables con- tributing to tardiness among the direct-care staff members. Although the results obtained between the direct-care staff members and their supervisors were similar, 3 of the 4 par- ticipants showed some differences. This is not surprising, as responses from employees exhibiting the performance concern may be biased. That is, when asked questions about their performance, employees might not be skilled at self- observation and evaluation and, thus, might downplay the severity of the concern, or might provide inaccurate informa- tion to appear more capable.

Merritt et al. (2019) deviated from PDC-HS administra- tion guidelines because they were concerned that the direct- care staff members’ supervisors were unaware of some of the variables that may have impacted tardiness, which may be a common occurrence. One possibility for future research might be to interview both parties simultaneously or inter- view employees after interviewing supervisors, such that discrepancies between the two could be addressed in the interview with employees. Future research should formally evaluate PDC-HS outcomes obtained from employees exhib- iting performance concerns relative to their supervisors.

Conducting Direct Observations

Three of the PDC-HS domains include items requiring direct observation of employee behavior. When conducting direct observations, supervisors should do so during typical working conditions as it is important to obtain representative samples of employee performance (Sharp et al., 2015). For example, conducting direct observations on a day in which the employee is feeling ill, has to complete new or unfa- miliar tasks, or is working a shift outside of their typical schedule, may not produce data that are truly representative of their performance. Thus, conducting direct observations during times and under conditions that the employee is typi- cally expected to perform the task is recommended. Super- visors should be aware of employee reactivity; employees may behave differently when they know that they are being observed (Kazdin, 1979; Mowery et al., 2010). Moreover, direct observations of the employee should continue until enough data have been collected to obtain reliable informa- tion. It is likely that the minimum number of observations needed varies across employees and performance concerns.

Training Others to use the PDC‑HS

When supervisors are training others to become supervi- sors, or when they are mentoring new supervisors, they should consider incorporating the PDC-HS into their train- ing. It is recommended that supervisors follow a behav- ioral skills training (BST) approach in these instances (Miltenberger, 2003; Wurtele, 1986). Behavioral skills training is an evidence-based teaching procedure consist- ing of instructions, modeling, rehearsal, and feedback (Novak et al., 2019; Parsons et al., 2012). Researchers have demonstrated that BST can be used to train others to administer various types of assessment procedures with which they had no prior experience (e.g., Barnes et al., 2014; Shayne & Miltenberger, 2013). Thus, the evidence suggests that BST can be an effective procedure for train- ing supervisors to administer the PDC-HS.

When taking this approach, the supervisor-trainer should first identify whether the trainee or mentee is famil- iar with the PDC-HS, and if so, to what degree. The super- visor-trainer can then tailor the explicit instruction to the needs of the individual. Training activities might include any combination of the following components: providing articles to read, reviewing the PDC-HS in depth, discuss- ing prior experiences using the PDC-HS, role-playing, and arranging opportunities to collaboratively implement the PDC-HS using actual employee performance con- cerns or case scenarios. Training activities could include the trainee or mentee observing while the supervisor- trainer completes the PDC-HS themselves or interviews the employee’s direct supervisor. It may also involve the trainee or mentee completing the PDC-HS along with the supervisor-trainer, and then completing it on their own and reviewing the results with the supervisor-trainer, who may have also independently completed the assessment.

Considerations for Targeting Performance Concerns

Describing the Performance Concern

One of the first steps when completing the PDC-HS is to describe the performance concern. The original guide- lines indicate that performance concerns should be opera- tionalized as either a behavioral excess or deficit (Carr et al., 2013). When describing employee concerns, it is important to carefully pinpoint (i.e., accurately describ- ing behavior in a way that is measurable and observ- able) the behavior of interest (Daniels & Bailey, 2014; DiGennaro Reed et al., 2018; Rodriguez et al., 2016).

954 Behavior Analysis in Practice (2022) 15:951–957

Using well-crafted behavioral pinpoints is important to reduce the likelihood of any misunderstandings between managers and consultants when discussing performance concerns, which could lead to unreliable answers when completing the assessment (Wilder et al., 2019). Unreli- able answers are especially concerning given that interven- tions are selected based on the information obtained from the assessment (Carr et al., 2013). Clear descriptions of the performance concern can also help the PDC-HS user to frame questions in a way that helps them ask about the pinpointed behavior more clearly (e.g., “have you received training on how to properly close the door following the completion of your shift” versus “have you received train- ing on the task”).

Complexity of the Performance Concern

The PDC-HS is well suited for discrete performance con- cerns of varying complexity that can be easily observed and measured. As noted in the review by Wilder et al. (2020), researchers have applied the PDC-HS to tasks related to managing the environment (e.g., cleaning and securing doors), implementing instructional strategies (e.g., error cor- rection and discrete-trial instruction procedures, providing response opportunities), and other job-related duties (e.g., pricing items, arriving on time). However, it may be the case that the PDC-HS could be applied to more nuanced skills, such as issues related to interpersonal communication (e.g., being overly argumentative, failing to demonstrate good audience control by shifting verbal behavior, or demonstrat- ing low affect), problem solving, organizational and time management skills, or self-care (e.g., identifying stressors, engaging in strategies to manage stress and maintain health). These types of skills may not have discrete beginnings and ends, are not performed in the same way each time, and often require in-the-moment adjustments to responding based on contextual information (e.g., the way a commu- nication partner responds, the presence of risk of harm to a client, the emergence of an acute life stressor).

Completing the questions for the skills described above may necessitate flexibility on the part of the supervisor, requiring them to thoughtfully consider how the questions might support identifying related barriers or creative sup- ports. For example, consider an employee who consistently fails to use language that is easily understood by caregivers. In such an application the supervisor will need to be flex- ible when answering the questions, but in doing so might identify the relevant environmental variables that allow for a successful intervention. In our example, answering the first question of the PDC-HS (Has the employee received formal training on this task?) might reveal that the employee has never had high-quality instruction on how to develop a positive therapeutic relationship with caregivers, including

avoiding the use of overly technical jargon. Perhaps the skill was explained, but not directly modeled and practiced. Or maybe the skill was practiced, but the employee was never directly observed interacting with caregivers. Some ques- tions, such as those in Task Clarification and Prompting and Resources, Materials, and Processes domains may initially appear irrelevant to our example performance concern.

However, careful consideration of the how the questions might reveal information about the relationship between the performance concern and related environmental barriers is warranted. For example, Question 3 in the Task Clarification and Prompting domain of the PDC-HS asks about the pres- ence of a job-aid, which may appear irrelevant to the per- formance concern in the example. It may not be so strange to think that performing a complex skill like explaining behavior-analytic interventions in everyday language could be supported by a guide that includes some examples of common translations. Whereas, such a support is not techni- cally a job-aid, reflecting on this specific question could lead the thoughtful supervisor to create such a support and assess whether it helps alleviate the performance concern. Simi- larly, careful consideration of question 1 in the Resource, Materials, and Processes domain (Are there sufficient num- bers of trained staff available in the organization to complete the task?) might reveal that the employee’s colleagues also share the same performance concern, such that speaking in very technical terms with caregivers has become common- place. Therefore, application of the PDC-HS to performance concerns that are more nuanced should not result in skip- ping questions that may at first appear irrelevant. Instead, those questions should prompt the supervisor to engage in a thoughtful consideration of related possible environmental barriers relevant to the issue at hand.

Using PDC‑HS Data

There are several ways the data collected from the PDC-HS can be used. The most common application is to identify and implement performance improvement plans for individual employees or a small group of employees. In this typical approach, each item scored as a No on the assessment is an opportunity for intervention. When several items (within or across domains) have been scored as No, multiple interven- tions can be implemented either concurrently or consecutively based on the resources that are available within an organization (Carr et al., 2013). This approach has proven to be effective and has resulted in improved employee performance across a variety of settings (Wilder et al., 2020). However, there may be instances in which a supervisor uses the PDC-HS to derive an intervention that does not produce the desired result. For example, in two recent studies (Collier-Meek et al., 2021; Mer- ritt et al., 2019), some participants were not responsive to the

955Behavior Analysis in Practice (2022) 15:951–957

PDC-HS indicated interventions, requiring the researchers to modify some of their procedures or interventions.

Selecting interventions when several items across multi- ple domains have been scored as No can present some dif- ficulties. Previously, it was recommended that priority be given to domains with multiple No responses (Carr et al., 2013). However, selecting interventions may not always be that simple. Consider an example in which a supervi- sor administers the PDC-HS and scores that the materials needed to complete the task are not readily available (ques- tion 1 in Resources, Materials, and Processes) but does not mark any other items in that domain as problematic. In that case it may appear that this domain is acceptable, whereas the reality may be that the employee cannot complete the task without having easy access to materials. In such a case, an intervention would be required for that particular item to promote improved employee performance even if items within that domain had a smaller number of No responses relative to other domains. Once the intervention has been implemented (i.e., resources needed to complete the task have been made available) the supervisor may consider reas- sessing employee performance to determine whether further action is required. Thus, we recommend that the perfor- mance concern itself, items scored as No on the PDC-HS assessment, and available organizational resources be care- fully considered when selecting interventions.

The data collected from the PDC-HS can also be retained and aggregated across several employees and performance concerns (Wilder et al., 2020). As the use of the PDC-HS becomes more common within an organization, certain pat- terns within the data may begin to emerge. For example, if the Training domain is repeatedly identified as problem- atic across several employees and tasks, it may be that the organization needs to evaluate their training procedures and develop treatment integrity checks to ensure that high- quality training is maintained once provided. Thus, the data collected across all assessments conducted within an organ- ization could indicate the need for systems-level changes (Diener et al., 2009), which may assist organizations in tak- ing a more proactive approach when addressing employee performance concerns rather than always having to react every time an issue arises. We suggest that supervisors retain and aggregate the data every time the assessment is con- ducted and update their records as new data are collected. Future studies could investigate the utility of using PDC-HS data to inform systems-level changes.

Concluding Remarks

As the demand for effective behavior-analytic services in human service settings continues to increase, so does the need for effective supervision. The issues surrounding

effective supervisory practices within the applied behavior analysis profession become even more pronounced when considering the continued large increases in the number of certified behavior-analytic practitioners (Behavior Analyst Certification Board, n.d.). Thus, developing and refining per- formance analysis methods designed to assist supervisors with quickly and effectively addressing employee perfor- mance concerns represent an important line of investigation. For the most part, the PDC-HS appears to be relatively sim- ple and straightforward. However, users of the PDC-HS do not always operate under ideal conditions and may at times find themselves unsure of how to proceed when conducting the assessment. We provided a discussion of several issues involving the use of the PDC-HS with the intent of clarify- ing areas of concern. In addition, some recommendations, particularly those related to nontraditional implementation methods of the PDC-HS, warrant investigation by research- ers. We hope that users of the PDC-HS find our guidance and recommendations useful. Moreover, we hope that our discussion points will stimulate more research into these areas.

Funding No funding was received to assist with the preparation of this manuscript.

Disclosures

Conflict of interest The authors declare that they have no conflicts of interest.

References

Austin, J. (2000). Performance analysis and performance diagnostics. In J. Austin & J. E. Carr (Eds.), Handbook of applied behavior analysis (pp. 321–349). Context Press.

Barnes, C. S., Mellor, J. R., & Rehfeldt. (2014). Implementing the verbal behavior milestones and placement program (VB-MAPP): Teaching assessment techniques. Analysis of Verbal Behavior, 30(1), 36–47. https:// doi. org/ 10. 1007/ s40616- 013- 0004-5

Behavior Analyst Certification Board. (n.d.). BACB certificant data. https:// www. bacb. com/ bacb- certi ficant- data/

Bernardin, H. J., Thomason, S., Buckley, M. R., & Kane, J. S. (2016). Rater rating-level bias and accuracy in performance appraisals: The impact of factor rater personality, performance management competence, and rater accountability. Human Resource Manage- ment, 55(2), 321–340. https:// doi. org/ 10. 1002/ hrm. 21678

Bowe, M., & Sellers, T. P. (2018). Evaluating the performance diagnos- tic checklist - human services to assess incorrect error-correction procedures by preschool paraprofessionals. Journal of Applied Behavior Analysis, 51(1), 166–176. https:// doi. org/ 10. 1002/ jaba. 428

Carr, J. E., Wilder, D. A., Majdalany, L., Mathison, D., & Strain, L. A. (2013). An assessment-based solution to a human-service employee problem: An initial evaluation of the performance diag- nostic checklist - human services. Behavior Analysis in Practice, 6(1), 16–32. https:// doi. org/ 10. 1007/ BF033 91789

956 Behavior Analysis in Practice (2022) 15:951–957

Collier-Meek, M. A., Sanetti, L. M. H., Gould, K., & Pereira, B. A. (2021). Using the performance diagnostic checklist to evaluate and promote paraeducators’ treatment fidelity. Journal of School Psychology, 86, 1–14. https:// doi. org/ 10. 1016/j. jsp. 2021. 02. 005

Cymbal, D. C., Wilder, D. A., Thomas, R., & Ertel, H. (2020). Fur- ther evaluation of the validity and reliability of the Performance Diagnostic Checklist-Human Services. Journal of Organizational Behavior Management, 40(3-4), 249–257. https:// doi. org/ 10. 1080/ 01608 061. 2020. 17920 27

Daniels, A. C., & Bailey, J. S. (2014). Performance management: Changing behavior that drives organizational effectiveness ((5th ed.). ed.). Aubrey Daniels International, Inc..

Diener, L. H., McGee, H. M., & Miguel, C. F. (2009). An integrated approach for conducting a behavioral systems analysis. Journal of Organizational Behavior Management, 29(2), 108–135. https:// doi. org/ 10. 1080/ 01608 06090 28745 34

DiGennaro Reed, F. D., Novak, M. D., Erath, T. G., Brand, D., & Hen- ley, A. J. (2018). Pinpointing and measuring employee behavior. In B. Wine & J. K. Pritchard (Eds.), Organizational behavior management: The essentials. Hedgehog Publishers.

Ditzian, K., Wilder, D. A., King, A., & Tanz, J. (2015). An evalua- tion of the performance diagnostic checklist - human services to assess an employee performance problem in a center-based autism treatment facility. Journal of Applied Behavior Analysis, 48(1), 199–203. https:// doi. org/ 10. 1002/ jaba. 171

Godat, L. M., & Brigham, T. A. (1999). The effect of a self-manage- ment training program on employees of a mid-sized organization. Journal of Organizational Behavior Management, 19(1), 65–83. https:// doi. org/ 10. 1300/ J075v 19n01_ 06

Gonsalvez, C. J., & Freestone, J. (2007). Field supervisors’ assessments of trainee performance: Are they reliable and valid. Australian Psychologist, 42(1), 23–32. https:// doi. org/ 10. 1080/ 00050 06060 08276 15

Iwata, B. A., DeLeon, I. G., & Roscoe, E. M. (2013). Reliability and validity of the functional analysis screening tool. Journal of Applied Behavior Analysis, 46(1), 271–284. https:// doi. org/ 10. 1002/ jaba. 31

Kaplan, S. E. (2018). Further evidence on the negativity bias in perfor- mance evaluation: When does the evaluator’s perspective matter? Journal of Management Accounting Research, 30(1), 169–184. https:// doi. org/ 10. 2308/ jmar- 51698

Kazdin. (1979). Unobtrusive measures in behavioral assessment. Jour- nal of Applied Behavior Analysis, 12(4), 713–724. https:// doi. org/ 10. 1901/ jaba. 1979. 12- 713

Merritt, T. A., DiGennaro Reed, F. D., & Martinez, C. E. (2019). Using the performance diagnostic checklist - human services to identify an indicated intervention to decrease employee tardiness. Journal of Applied Behavior Analysis, 52(4), 1034–1048. https:// doi. org/ 10. 1002/ jaba. 643

Miltenberger, R. G. (2003). Behavior modification: Principles and procedures (3rd ed.). .

Mowery, J. M., Miltenberger, R. G., & Weil, T. M. (2010). Evaluating the effects of reactivity to supervisor presence on staff response to tactile prompts and self-monitoring in a group home setting. Behavioral Interventions, 25(1), 21–35. https:// doi. org/ 10. 1002/ bin. 296

Novak, M. D., DiGennaro Reed, F. D., Erath, T. G., Blackman, A. L., Ruby, S., & Pellegrino, A. J. (2019). Evidence-based performance management: Applying behavioral science to support practition- ers. Perspectives on Behavior Science, 42(9), 955–972. https:// doi. org/ 10. 1007/ s40614- 019- 00232-z

Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5(2), 2–11. https:// doi. org/ 10. 1007/ BF033 91819

Rodriguez, M., Sundberg, D., & Biagi, S. (2016). OBM applied: A practical guide to implementing organizational behavior manage- ment. ABA Technologies, Inc.

Sharp, R. A., Mudford, O. C., & Elliffe, D. (2015). Representativeness of direct observations selected using a work-sampling equation. Journal of Applied Behavior Analysis, 48(1), 153–166. https:// doi. org/ 10. 1002/ jaba. 193

Shayne, R., & Miltenberger, R. G. (2013). Evaluation of behavioral skills training for teaching functional assessment and treatment selection skills to parents. Behavioral Interventions, 28(1), 4–21. https:// doi. org/ 10. 1002/ bin. 1350

Smith, M., & Wilder, D. A. (2018). The use of the performance diag- nostic checklist - human services to assess and improve the job performance of individuals with intellectual disabilities. Behav- ior Analysis in Practice, 11(2), 148–153. https:// doi. org/ 10. 1007/ s40617- 018- 0213-4

Wilder, D. A., Rost, K., & McMahon, M. (2007). The accuracy of managerial prediction of employee preference. Journal of Organi- zational Behavior Management, 27(2), 1-14. https:// doi. org/ 10. 1300/ J075v 27n02_ 01

Wilder, D. A., Harris, C., Casella, S., Wine, B., & Postma, N. (2011). Further evaluation of the accuracy of managerial prediction of employee preference. Journal of Organizational Behavior Man- agement, 31(2), 130–139. https:// doi. org/ 10. 1080/ 01608 061. 2011. 569202

Wilder, D. A., Lipschultz, J., Gehrman, C., Ertel, H., & Hodges, A. (2019). A preliminary assessment of the validity and reliability of the performance diagnostic checklist - human services. Journal of Organizational Behavior Management, 39(3-4), 194–212. https:// doi. org/ 10. 1080/ 01608 061. 2019. 16667 72

Wilder, D. A., Cymbal, D., & Villacorta, J. (2020). The performance diagnostic checklist - human services: A brief review. Journal of Applied Behavior Analysis, 53(2), 1170–1176. https:// doi. org/ 10. 1002/ jaba. 676

Wilder, D., Lipschultz, J., King, A., Driscoll, S., & Sigurdsson, S. (2018). An analysis of the commonality and type of preinter- vention assessment procedures in the Journal of Organizational Behavior Management (2000–2015). Journal of Organizational Behavior Management, 38(1), 5–17. https:// doi. org/ 10. 1080/ 01608 061. 2017. 13258 22

Wurtele, S. K. (1986). Teaching young children personal body safety: The Behavioral Skills Training Program. Author

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

957Behavior Analysis in Practice (2022) 15:951–957