vi
THE RELATIONSHIP OF INTERNSHIP SETTING TO INTERNSHIP DIRECTORS’
PERSPECTIVES ON CULTURAL COMPETENCE IN PSYCHOLOGICAL ASSESSMENT
1
Chapter I: Introduction
Individual and Cultural Diversity: A Core Competency
According to the American Psychological Association’s (APA) Code of Ethics (2002),
psychologists must be aware and respectful of the following when working with their clients:
“age, gender, gender identity, race, ethnicity, national origin, religion, sexual orientation,
disability, language, and socioeconomic status” (p. 1063). In doing so, they are required to be
aware of their own biases, and to not “knowingly participate in or condone activities of others
based upon such prejudices” (p. 1560). Moreover, the Code of Ethics (2002) states that when
working with persons who differ in regard to the aforementioned considerations of diversity, it is
necessary for psychologists to obtain “training, experience, consultation, or supervision” that is
needed to provide effective and competent services or provide referrals (p. 1064). Thus, an
emphasis on training in diversity is essential, when providing clinical services to clients,
including psychological assessment.
The professional literature provides various descriptions of cultural competence (Leong
& Kim, 1991; Okazaki, 1998; Rogler et al., 1987; Sue et al., 1982; Stuart, 2004). Stuart (2004)
defines multicultural competence as one’s ability to “understand and constructively relate to the
uniqueness of each client” while considering “diverse cultures that influence each person’s
perspectives” (p. 6). Dana et al. (1992) described cultural competence as, “The ability to provide
services [clinical interventions and psychological assessment] that are perceived as legitimate for
problems experienced by culturally diverse persons” (p. 221). There is emerging literature on
implementing culturally competent psychological assessment for diverse groups (Okazaki,
1998). Specifically, researchers and clinicians are beginning to provide recommendations for this
task. Stuart (2004) provides 12 suggestions for achieving multicultural competence during the
provision of therapy and assessment services (See Figure 1).
2
Figure 1
Twelve Suggestions for Achieving Multicultural Competence
1. Develop skill in discovering each person’s unique and cultural outlook.
2. Acknowledge and control personal biases by articulating your worldview and evaluating
its sources and validity.
3. Develop sensitivity to cultural differences without overemphasizing them.
4. Uncouple theory from culture.
5. Develop a sufficiently complex set of cultural categories.
6. Critically evaluate the methods used to collect culturally relevant data before applying
the findings in psychological services.
7. Determine a means of determining a person’s acceptance of relevant cultural themes.
8. Develop a means of determining the salience of ethnic identity for each client.
9. Match any psychological tests to client characteristics.
10. Contextualize all assessments.
11. Consider clients’ ethnic and world views in selecting therapists, interventions goals, and
methods.
12. Respect clients’ beliefs, but attempt to change them when necessary.
According to Hansen (2002), when diversity is considered in the field of psychology,
emphasis is placed on the “four historically underserved ethnic groups” (p. 205). These groups
include: African American, Asian American, Latin American, and Native American persons.
Hansen (2002) argues that while individuals of these four ethnic backgrounds are representative
of diversity, this list is not comprehensive. Therefore, Hansen (2002) states that diversity training
should include emphasis on the eleven dimensions of difference that are listed in the APA Code
of Ethics (2002).
When considering cultural competence and its intersection with psychological
assessment, a number of recommendations have been made (Dana, 2002; Hansen, 2002; Stuart,
2004). For example, Hansen (2002) describes specific training for psychology doctoral students,
currently utilized at a doctoral program. This training consists of 15-hour didactics for training in
cultural competence in psychological assessment. These didactics facilitate extensive discussion
of students’ cultural identity and cultural countertransference they may experience at their
3
practicums. Additionally, students engage in discussion about the concepts of culture, race,
ethnicity, and biases within psychometric tests. Students are also trained in assessing their
clients’ cultural orientation and also training in the multicultural assessment model developed by
Dana (2002). This model functions under the premise that culture is heterogeneous, and thus
clients’ culture should be assessed, to determine if culturally-specific assessment instruments are
needed. Dana (2002) states that this practice increases the reliability of measures and results in
more accurate diagnoses.
A nationally representative study on the U.S. population (N = 9,282) examined the
utilization of mental health services within a 12-month period (Wang et al., 2005). Of the
participants who endorsed a history of mental illness, 41% reported the utilization of mental
health services. Further, 17.9% of the entire sample reported utilizing mental health services
within a 12-month period (Wang et al., 2005). Individuals of diverse populations, however,
utilized mental health services less frequently than individuals of European decent. This
disparity was attributed to diverse populations lacking resources (i.e., funding, health insurance),
which impeded their ability to attain services. Wang and coauthors (2005) posited that
individuals of diverse populations might also be apprehensive about seeking services due to
perceived prejudice or bias from healthcare professionals. The disparity can also be explained by
findings stating that some diverse populations may rely on spiritual, and communal networks to
manage mental health symptoms (Abe-Kim et al., 2004). Hence, when performing
psychological assessment, cultural competency is critical to effectively serve diverse populations
(Uzzell et al., 2013).
Fouad et al. (2009) developed a model to describe competency benchmarks for
professional psychology associated with different levels of training or development, from
practicum to pre-doctoral internship to postdoctoral practice. The model includes benchmarks
4
related to individual and cultural diversity. Proficiency in working with diverse populations is a
requirement of this model (see Figure 2), and the diversity-related expectations apply to
psychological assessment as well.
Figure 2
Competency Benchmarks: Individual and Cultural Diversity
5
1. Individual and Cultural Diversity: Awareness, sensitivity and skills in working professionally with
diverse individuals, groups and communities who represent various cultural and personal
background and characteristics defined broadly and consistent with APA policy.
READINESS FOR PRACTICUM READINESS FOR READINESS FOR ENTRY
INTERNSHIP TO PRACTICE
2A. Self as Shaped by Individual and Cultural Diversity (e.g., cultural, individual, and role differences,
including those based on age, gender, gender identity, race, ethnicity, culture, national
origin, religion, sexual orientation, disability, language, and socioeconomic status) and Context
Demonstrates knowledge, Monitors and applies Independently monitors and awareness, and
understanding of knowledge of self as a cultural applies knowledge of self as
one’s own dimensions of diversity being in assessment, treatment, a cultural being in
and attitudes towards diverse and consultation assessment, treatment, and
others consultation
2B. Others as Shaped by Individual and Cultural Diversity and Context
Demonstrates knowledge, Applies knowledge of others Independently monitors and awareness,
and understanding of as cultural beings in applies knowledge of others
other individuals as cultural beings assessment, treatment, and as cultural beings in
consultation assessment, treatment, and
consultation
2C. Interaction of Self and Others as Shaped by Individual and Cultural Diversity and Context
Demonstrates knowledge, Applies knowledge of the role
awareness, and understanding of of culture in interactions in
interactions between self and assessment, treatment, and diverse
others consultation of diverse others
2D. Applications based on Individual and Cultural Context
Independently monitors and
applies knowledge of
diversity in others as cultural
beings in assessment,
treatment, and consultation
Demonstrates basic knowledge of Applies knowledge, and
sensitivity to the scientific, sensitivity, and understanding
theoretical, and contextual issues regarding ICD issues to
work
related to ICD (as defined by APA effectively with diverse others
policy) as they apply to in assessment, treatment, and
professional psychology. consultation
Understands the need to consider
ICD issues in all aspects of
professional psychology work (e.g.,
assessment, treatment,
Applies knowledge, skills,
and attitudes regarding
dimensions of diversity to
professional work
research, relationships with colleagues)
6
According to the diversity-related competency benchmarks that Fouad et al. (2009)
proposed, readiness for internship involves more than knowledge, sensitivity, awareness, and
understanding. Internship readiness in professional psychology includes the ability to apply such
skills to treatment, assessment, and consultation of diverse others. Psychology interns must be
aware of themselves and what they contribute to the assessment experience through their own
individuality and culture, in order to effectively utilize psychological assessment when working
with diverse populations (Fouad et al., 2009). The authors identify three core areas that
demonstrate a psychologist’s expected skills in cultural sensitivity as awareness of:
Self as shaped by individual and cultural diversity… Others as shaped by individual and
cultural diversity…and Interactions of self and others as shaped by individual and
cultural diversity (e.g. cultural, individual, and role differences, including those based on
age, gender, gender identity, race, ethnicity, culture, national origin, religion, sexual
orientation, disability, language, and socioeconomic status) and context. (Fouad et al.,
2009, pp. S13-S14)
The emphasis on cultural sensitivity training in assessment courses is discussed in the
literature. Hansen (2002) notes that this specialized training is a requirement for doctoral
education in applied fields such as clinical psychology. Moreover, Hansen (2002) refers to
“culture-specific assessment procedures and tools,” which promote proficiency in assessing
issues that are significant to certain groups (p. 202). The author also refers to developing
proficiency in adjusting assessment tools contingent on the needs of the examinee (Hansen,
2002).
Puente and Agranovich (2004) attest to the need for cultural sensitivity in assessment.
Their article focuses on the lack of cultural awareness, specifically in neuropsychological
assessment. They posit that neuropsychology is often taught and practiced based on the notion
7
that all behavior is the same, which is not a culturally sensitive notion. Thus, an inaccurate
understanding of behavior is often perpetuated in psychology, which all too often is based on the
behavior of Caucasian, Western males. This is also an argument that privileges Western culture,
which is the presumed standard. Puente and Agranovich (2004) also observe that a frequent
approach in cognitive assessment is to attempt to enhance the cultural sensitivity of methods
through the development of non-verbal items and measures. Clearly, much more is needed for
neuropsychological assessment to truly be conducted in a culturally sensitive manner. Thus,
further discourse and investigation in this area is warranted to produce culturally sensitive
neuropsychology instruments and psychological assessments overall.
Nonetheless, too frequently it is falsely presented that all cultures think alike and utilize
strategies and styles of cognition that are based again on individuals who are Western, Caucasian,
and male (Ardila, 2007). Clients may have varying attitudes toward testing, which can impair an
assessor’s ability to attain valid test results/data. Further, it is noted that in assessment,
time/speed is also a factor, which is influenced by Western society, whereas moving slower can
be interpreted as a cognitive limitation (Ardila, 2007). In contrast, moving slowly to achieve a
task may be more culturally congruent for diverse populations. Hence, the presence of cultural
competency and cultural awareness while conducting psychological assessment is vitally
important for valid and ethical assessment with diverse clients (Puente & Agranovich, 2004;
Ardila, 2007). Hansen (2002) emphasizes the importance of interpreting results through a
cultural lens and acquiring skills in relaying assessment results in a “culturally sensitive manner”
(p. 201). As noted earlier, pre-doctoral interns are expected to show cultural sensitivity when
providing services to clients and also when selecting measures to administer (Fouad et al., 2009).
Krishnamurthy and colleagues (2004) touch on the importance of “cultural
selfawareness” in psychological assessment and the need to utilize measures that are
8
characterized by sensitivity to cultural diversity (p. 737). Cultural self-awareness is defined as
being aware of one’s biases and beliefs, which can impact the examiner’s views and interactions
with the client (Krishnamurthy et al., 2004). Krishnamurthy et al. (2004) posit that many
internship directors lack expertise in this area and they suggest that the training directors may
also require further training. López (2002) describes training that is provided to students at his
academic institution. He presents a discussion of bridging the gap between traditional assessment
and “multicultural assessment issues” (p. 227). He describes unique training strategies that he
provides to students to evoke an awareness of the need for training in multicultural/diversity
issues. López (2002) also discusses another goal of his assessment training, which is to support
students in becoming aware of their own biases and how they can further influence their
conceptualization and assessment of clients. Thus, training in diversity and multicultural issues
contributes to further meeting the needs of clients by preparing clinicians to view clients’
challenges through a culturally sensitive lens. It is likely that these are the types of experiences
that help graduate students become prepared for assessment with diverse populations at the
internship level and beyond.
The purpose of the present study was to explore predoctoral internship directors’
perspectives on psychological assessment in regard to diversity and other contemporary issues
across six major categories of internship setting. This archival study focused on directors’
appraisal of: beginning interns’ readiness for assessment of diverse populations; impact of
evidence-based practice on psychological assessment in internship programs; and both
experienced and desired changes in psychological measures utilized. Before additional details
are provided, relevant aspects of the literature will be considered.
9
Psychological Assessment: A Core Competency
Regarded as a core competency for clinical psychologists, psychological assessment has
been referred to as a “hallmark” of the field of professional psychology (Goldstein et al., 2004, p.
ix). The practice of psychological assessment is a competence area that is largely specific to
psychologists and is a distinguishing factor from other healthcare professionals (Groth-Marnat,
2009). Psychological assessment is also an integral component of the training of psychology
doctoral students (Fouad et al., 2009). Clemence and Handler (2001) conducted a survey that
reviewed the prominence of psychological assessment’s role at 329 pre-doctoral internship
programs in psychology. These internship programs included the following settings: university
counseling centers, community mental health centers, child facilities, Veterans Affairs Medical
Centers, state hospitals, private general medical centers, medical schools, and private psychiatric
hospitals. The study demonstrated that 41% of the respondents administered psychological
testing and assessment to clients at these settings. In addition, 99% reported that introductory
training in assessment was offered and provided to interns at their sites, indicating that
predoctoral interns are not always prepared for conducting psychological assessment. The study
also found that training in projective tests (i.e., Rorschach, TAT), is highly desirable, especially
in particular internship settings (e.g., psychiatric hospitals). These findings indicated that
psychological assessment plays a pivotal role in pre-doctoral internship sites and strongly
supported the practice of students receiving training in this area of specialization. Moreover, it
can be surmised that assessment-related measures, training, needs, etc., may vary across
internship settings.
Assessment is frequently utilized by psychologists who provide clinical services and is
considered a pertinent aspect of their training in psychology (Anderson, 2006; Schaffer et al.,
2013). Amongst professional psychologists, 10-25% of their work is comprised of conducting
10
psychological assessment (Camara et al., 2000; Watkins, 1991). It is also likely that
psychological assessment will be utilized throughout the careers of psychologists. This also
shows the relevance of a psychologist’s competency in psychological assessment due to its wide
utilization in clinical application and practice (Camara et al., 2000).
Krishnamurthy and colleagues (2004) discuss eight core competencies that are critical for
competency in psychological assessment (see Figure 3).
Figure 3
Core Competencies for Psychological Assessment
11
1. A background in the basics of psychometric theory
2. Knowledge of the scientific, theoretical, empirical, and contextual bases of psychological
assessment
3. Knowledge, skill, and techniques to assess the cognitive, affective, behavioral, and
personality dimensions of human experience with reference to individuals and systems
4. The ability to assess outcomes of treatment/intervention
5. The ability to evaluate critically the multiple roles, contexts, and relationships within
which clients and psychologists function, and the reciprocal impact of these on assessment
activity
6. The ability to establish, maintain, and to understand the collaborative professional
relationship that provides a context for all psychological activity including psychological
assessment
7. An understanding of the relationship between assessment and intervention, assessment as
an intervention, and intervention planning
8. Technical assessment skills
i. Problem and or goal identification and case conceptualization
ii. Understanding and selection of appropriate assessment methods including both
test and non-test data (e.g., suitable strategies, tools, measures, time lines, and
targets)
iii. Effective application of the assessment procedures with clients and the various
systems in which they function
iv. Systematic data gathering
v. Integration of information, inference, and analysis
vi. Communication of findings and development of recommendations to address
problems and goals
The authors report that competency in psychological assessment includes a specialized
set of skills to ensure that clients are receiving optimal care and services. It is also pertinent that
psychologists assimilate mindsets that facilitate the validity and usefulness of their assessments.
To help ensure that psychological assessment is ethical, psychologists must continually consider
their clients’ cultural and contextual backgrounds, which ultimately impact their lives. These
practices also facilitate the process of case conceptualization and psychologists’ ability to
establish a strong alliance with their clients (Krishnamurthy et al., 2004).
Although there is ongoing dialogue regarding the skills that should be specified as
benchmarks for competency in the field of psychology, the American Psychological Association
(APA) and the Association of State and Provincial Psychology Boards (ASPPB) agree that
12
psychological assessment is critical to a psychologist’s clinical training. Fouad and colleagues
(2009) provide benchmarks listing skills that should be exemplified at various stages of one’s
clinical training: practicum, internship, and professional practice (Fouad et al., 2009). This model
posits that trainees who display readiness for internship are skilled at selecting and administering
measures that are valid to an individual, given his/her historical and contextual background.
Trainees should also be aware of the strengths and weaknesses of measures that are administered,
show skill in scoring and interpreting measures, and display familiarity with technology that may
enhance the usefulness of these measures.
Additionally, trainees should display competence in obtaining information, writing
progress reports and assessment reports, selecting measures that are appropriate for their clients,
and utilizing knowledge regarding normal and abnormal behavior to inform case
conceptualization (see Figure 4). Furthermore, competency in psychological assessment is
determined by a trainee’s ability to conduct “…assessment and diagnosis of problems,
capabilities and issues associated with individuals, groups, and/or organizations” (Fouad et al.,
2009, p. S16).
Figure 4
Competency Benchmarks: Assessment
13
2. Assessment: Assessment and diagnosis of problems, capabilities and issues associated with
individuals, groups, and/or organizations.
READINESS FOR PRACTICUM
9A. Knowledge of Measurement
and Psychometrics
READINESS FOR
INTERNSHIP
READINESS FOR ENTRY
TO PRACTICE
Demonstrates basic knowledge of
the scientific, theoretical, and
contextual basis of test construction
and interviewing
9B. Knowledge of Assessment
Methods
Selects assessment measures
with attention to issues of
reliability and validity
Independently selects and
implements multiple methods
and means of evaluation in
ways that are responsive to and
respectful of diverse
individuals, couples, families,
and groups and context
Demonstrates basic knowledge of
administration and scoring of
traditional assessment measures,
models and techniques, including
clinical interviewing and mental
status exam
Demonstrates awareness of
the strengths and limitations
of administration, scoring and
interpretation of traditional
assessment measures as well
as related technological
advances
Independently understands
the strengths and limitations
of diagnostic approaches and
interpretation of results from
multiple measures for
diagnosis and treatment
planning
9C. Application of Assessment Methods
Demonstrates knowledge of Selects appropriate assessment
measurement across domains of measures to answer diagnostic
functioning and practice settings question
Independently selects and
administers a variety of
assessment tools and
integrates results to
accurately evaluate
presenting question
appropriate to the practice
site and broad area of
practice
9D. Diagnosis
Demonstrates basic knowledge Applies concepts of regarding the
range of normal and normal/abnormal behavior to abnormal
behavior in the context case formulation and
of stages of human development diagnosis in the context of
and diversity stages of human development
and diversity
Utilizes case formulation and
diagnosis for intervention
planning in the context of stages
of human development and
diversity
______________________________________________________________________________
14
READINESS FOR PRACTICUM READINESS FOR
INTERNSHIP
9E. Conceptualization and Recommendations
READINESS FOR ENTRY
TO PRACTICE
Demonstrates basic knowledge of Utilizes systematic approaches
formulating diagnosis and case of gathering data to inform
conceptualization clinical decision-making
9F. Communication of Assessment Findings
Independently and accurately
conceptualizes the multiple
dimensions of the case based
on the results of assessment
Demonstrates awareness of Writes assessment reports and models
of report writing and progress notes and
progress notes communicates assessment
findings verbally to client
Communicates results in
written and verbal form
clearly, constructively, and
accurately in a conceptually
appropriate manner
The Ethical Principles for Psychologists and Code of Conduct list guidelines for the
provision of ethical services for clients and further, identify expectations of sound professional
practice for clinicians who perform psychological assessment (APA, 2002). Specifically,
psychological assessment should be conducted for clinically-relevant reasons such as to address
diagnostic questions, to develop treatment recommendations, or to comply with court mandates,
etc. A key principle is that informed consent should be obtained from clients who are receiving
assessment services. The client’s confidentiality must be carefully maintained, and any individual
who conducts psychological assessment must be appropriately trained or receive supervision
from a clinician who is trained and competent in this aspect of clinical practice. Clinicians must
be continually aware of cultural, contextual, and historical factors that may impact a client’s
performance on psychological tests; they should strive to administer the most updated measures
available, and it is important that they provide appropriate feedback to their clients (APA, 2002).
These expectations listed in the Ethical Principles for Psychologists and Code of Conduct
support the maintenance of high standards for assessment in the field of psychology, and thus,
strengthen its role as a core competency.
15
Psychological Assessment Training and Practice
Training for psychological testing and assessment is evolving, which is to be expected for
all aspects of the doctoral-level curriculum in professional psychology. The Association of State
and Provincial Psychology Boards (ASPPB), developed the Examination for Professional
Psychology Practice (EPPP) in 1964 ensure that licensed psychologists adhere to optimal
standards and to facilitate the process of licensure within the United States (Hess, 1979).
Currently administered in 49 states, the EPPP is considered the best measure to determine one’s
knowledge of professional psychology and clinical application (Hess, 1979). According to Stigall
(1983), the EPPP examines six areas of competency, the first three of which focus significant
attention on psychological assessment: selecting and modifying test instruments, interpreting and
reporting assessment results, and devising and implementing a treatment plan based on an
interpretation of the results that were acquired during the testing process.
Watkins (1991) examined 30 years of survey findings regarding psychological
assessment training and practice. The studies Watkins analyzed reviewed findings from academic
program directors, pre-doctoral internship directors, and practicing psychologists. The surveys
considered were published between 1960 and 1990. One key finding that was mentioned by
Watkins (1991) is that psychological assessment is viewed by internship directors as a skill of
importance. Thus, the expectation is that trainees receive appropriate training in this area.
Watkins (1991) stated that students are sought out who possess these skills; however, upon the
start of internship, many are not prepared in this area. Watkins (1991) also stated that since
psychological assessment is regarded as an important skill, students who have received sufficient
training in this area are estimated to have an increased likelihood of securing an internship and
employment after graduation. Lastly, Watkins (1991) discussed common suggestions for training
16
in psychology graduate student coursework, noting that training recommendations are typically
based on practices of assessment within the professional field of psychology.
Literature is useful for showing the types of assessments that are more prominent within
the field of psychology, including which tests are used most often. Childs and Eyde (2002) note
that there is an abundance of literature which provides suggestions for clinical training in
psychological assessment, however there is a scarcity of literature that shows how this is actually
taught in doctoral programs. Childs and Eyde (2002) conducted a survey of American
Psychological Association (APA) accredited, clinical psychology doctoral programs and reported
on the assessment measures that are most frequently taught to doctoral students (see Figure 5).
17
Figure 5
Most frequently taught assessment measures
Instrument % of Programs
Wechsler Adult Intelligence Scale–III 93
Wechsler Intelligence Scale for Children–III 88
Minnesota Multiphasic Personality Inventory–2 86
Rorschach Inkblot Test 81
Thematic Apperception Test 71
Stanford-Binet Intelligence Scale: Fourth Edition 48
Bender Visual Motor Gestalt Test 46
Millon Clinical Multiaxial Inventory–III 38
Wechsler Pre-School and Primary Scale of Intelligence –Revised 37
Woodcock Johnson Test of Achievement –Revised 33
Minnesota Multiphasic Personality Inventory–Adolescent 30
Sentence Completion Test 29
Wechsler Memory Scale–Revised 26
Halstead-Reitan Neuropsychological Battery 25 Wide
Range Achievement Test –Third Edition 25
Kaufman Assessment Battery for Children 24
Projective Drawings 24
Wechsler Individual Achievement Test 20
Childs and Eyde (2002) identified the measures that are most prominently utilized in the
training of clinical psychology doctoral students. The following measures were those that
academic program directors most frequently listed as being included in doctoral training: the
Wechsler Adult Intelligence Scale–III (WAIS–III; Wechsler, 1997); the Wechsler Intelligence
Scale for Children–III (WISC–III; Wechsler, 1991); the Minnesota Multiphasic Personality
Inventory–2 (MMPI–2; Butcher et al., 1989); the Rorschach Inkblot Test (Exner & Erdberg,
18
2005); and the Thematic Apperception Test (TAT; Murray, 1943). Instruments that were reported
as less frequently utilized included the following: the Stanford–Binet Intelligence Scale fourth
edition (Thorndike et al., 1986), the Bender Visual Motor Gestalt Test (Bender, 1946), the
Millon Clinical Multiaxial Inventory–III (MCMI–III; Millon et al., 1994), the Wechsler
Preschool and Primary Scale of Intelligence–Revised (Wechsler, 1989), and the Woodcock–
Johnson Tests of Achievement–Revised (Woodcock, 1991). The authors reported that doctoral
courses in clinical psychology placed emphasis on administering, scoring, and interpreting
psychological instruments (Childs & Eyde, 2002).
While the literature indicates stability in what measures have been emphasized in recent
decades, newer measures such as the Personality Assessment Inventory (PAI) and the Millon
Clinical Multiaxial Inventory (MCMI-III/MCMI-IV) are being utilized more frequently (Belter
& Piotrowski, 1999; Camara et al., 1998). Camera et al. (2000) conducted a national survey of
clinical psychologists and neuropsychologists to determine what measures were being used by
practicing professionals. In regard to clinical psychologists, the findings showed many
similarities when compared to what Childs and Eyde (2002) reported was being taught in
academic programs. Specifically, clinical psychologists reported utilizing mostly
Intellectual/Achievement measures (34%) and Personality/Psychopathology measures (32%).
The third most frequently utilized measures consisted of Neuropsychological instruments, which
represented 13% of the sample of clinical psychologists. This differed slightly from the results of
Childs and Eyde (2002), which ranked the Bender-Gestalt test as the seventh most frequently
utilized assessment instrument. Figure 6 shows the measures most widely utilized by practicing
professionals, as well as the ranks.
19
Figure 6
Tests used by clinical psychologists and neuropsychologists
Studies have also focused on specific instruments that are utilized in internships that are
affiliated with the Association of Psychology Postdoctoral and Internship Centers (APPIC).
Piotrowski and Belter (1999) surveyed 84 APPIC-affiliated internships and reported a steady
emphasis on personality and intelligence measures, concurrent with an increasing emphasis on
neuropsychological assessment and a trend toward less emphasis on projective measures. The top
listed measures included the following: the MMPI/MMPI-2, Wechsler IQ Scales, and the
Rorschach Inkblot Test. Additionally, one of the Millon Inventories (i.e., the MCMI) appeared to
increasing in popularity, as it ranked fourth on the list of instruments that the internship directors
20
considered essential for psychological practice (Piotrowski & Belter, 1999). These findings
appeared consistent with other literature discussing the popularity of instruments utilized within
the field (Belter & Piotrowski, 2001; Butcher, 2006; Childs & Eyde, 2002; Durand et al., 1988;
Norcross & Karpiak, 2012; Piotrowski & Zalewski, 1993).
Psychological assessment has remained a prominent skill for professional psychologists,
with assessment applications being reported across various practice settings (Butcher, 2006;
Piotrowski & Belter, 1999; Stedman et al., 2000; Weiner, 2012). When clinical psychologists are
questioned, a vast majority report at least some utilization of psychological assessment measures
(Watkins, 1991). Moreover, when 412 clinical psychologists were surveyed, the majority of
respondents reported the utilization of psychological assessment in their clinical practices
(Watkins et al., 1995). Of these respondents, 90% report utilizing some form of assessment, with
66% reporting utilizing intellectual assessment, 15% reporting vocational and career assessment,
and 13% reporting utilizing assessments that measure abilities/aptitude (Watkins et al., 1995).
In another study, Meyer et al. (1998) highlighted the role of psychology in the training of
pre-doctoral level clinicians. Meyer and coauthors stated that a well-trained clinician can
integrate test data meaningfully. Moreover, the future of psychological assessment is contingent
on the training of future clinicians who can competently produce this quality of work. The
production of this quality of work (i.e., producing and integrating test-based assessments) is
reliant on rigorous clinical training and clinical supervision, thus raising the question of how well
are pre-doctoral level clinicians being trained in the area of psychological assessment.
Gains have been attained through the development of psychological assessment, thanks to
its continued importance in doctoral academic programs, practicum sites, and pre-doctoral
internships (Clemence & Handler, 2001; Weiner, 2013). Training in psychological assessment is
very prominent in psychology doctoral programs that focus on clinical application (i.e., clinical,
21
counseling, and school psychology). Given that psychology internship directors continue to
value pre-internship experience in assessment, developing competency in psychological
assessment is necessary to increase the competitiveness of doctoral students when applying for
pre-doctoral internships. Moreover, the pre-doctoral internship is a prominent opportunity for
students to further build and refine their skills in psychological assessment (Belter & Piotrowski,
2001; Clemence & Handler, 2001; Stedman et al., 2000; Weiner, 2012).
Pre-Internship Training
Although psychological assessment is a relevant skill within various clinical settings,
there is a growing concern regarding the methods in which assessment is taught and how
clinicians are trained. Weiner (2013) suggests there has been a decrease in assessment training
within clinical psychology doctoral programs, which he believes is a result of misunderstanding
its continued relevance to the field psychology. Thus, previous standards of excellence in
assessment training may have decreased among many clinical psychology doctoral programs
(Weiner, 2013). Weiner further surmises that having a limited understanding of the importance of
psychological assessment leads to more limited offerings of assessment courses to students by
their academic programs. Further, clinical psychology doctoral programs may be decreasing
requirements for assessment training, which decreases motivation for students to engage in
research related to assessment. Literature is also showing that there is a disparity between the
amount of quality assessment training that is provided in doctoral academic programs and
assessment being practiced amongst psychologists (Butcher, 2006; Childs & Eyde, 2002;
Weiner, 2013).
Earlier research indicates that assessment training has been a core component of the
curriculum in psychology doctoral programs (Piotrowski & Zalewski, 1993). However, this
trend may be decreasing, as research also reports some changes in the focus of training in
22
psychological assessment in recent years (Belter & Piotrowski, 2001). Belter and Piotrowski
(2001) conducted a survey of 82 academic program directors from APA-approved clinical
psychology doctoral programs. Their findings showed a slight decrease in emphasis on teaching
projective testing in doctoral programs. The academic program directors in this sample reported
they were retaining or increasing emphasis in all areas of psychological assessment, with the
exception of projective testing. Most of the program directors (65%) reported increased emphasis
in neuropsychological assessment, and 40% reported having increased their emphasis on
interviewing. Only 7% of the sample reported an increased emphasis in intelligence testing in the
prior five years, while 4% reported increasing their emphasis in projective testing over that same
time period (Belter & Piotrowski, 2001).
Research also includes feedback from psychology students. Stedman et al. (2001a)
reported that substantial numbers of clinical and counseling psychology doctoral students who
applied for pre-doctoral internships did not believe they received adequate training in
psychological assessment prior to internship. The study also noted that only 25% of this sample
of psychology graduate students reported they had sufficient assessment experience to meet
expectations at their internships. Additionally, 25% of these students reported minimal training in
assessment report writing prior to internship. According to Butcher (2006), there are doctoral
students who have challenges placing at an internship and find that their lack of assessment
training renders them less competitive during the internship application and match phases. When
competitiveness within the mental health care system is considered, applied psychology doctoral
programs (i.e., clinical, counseling, and school psychology programs) would do well to ensure
that their emphasis on assessment-related issues keeps pace with trends in the field. This will
also promote their students’ competitiveness for internship selection, given the high expectations
regarding assessment-related training (Robiner et al., 1994).
23
Internship Training
The pre-doctoral internship is a critically important aspect of applied doctoral degree
programs in psychology, including clinical psychology (Prinstein, 2013). The internship year is
usually considered the capstone of clinical training experiences at the doctoral level (Keilin &
Constantine, 2001). The pre-doctoral internship typically occurs during or near the final year of
doctoral training and usually takes place in an applied setting that emphasizes clinical practice
(Keilin & Constantine, 2001; Prinstein, 2013).
Stedman et al. (2001b) surveyed 324 internship-training directors and found most sites
provided interns with abundant opportunities for intellectual, objective personality, projective
personality, and neuropsychological test training. Moreover, consistent with Clemence and
Handler (2001), Stedman and coauthors (2001b) reported a lack of uniformity among responding
internship directors, as emphasis on test-based assessment training varied considerably across
settings. These two studies were critical of the adequacy of pre-internship assessment training.
They also raised questions about whether assessment training during internship could provide
enough professional development in assessment to meet the demands of clinical practice beyond
graduation. A national survey by Stedman et al. (2005) expanded on the aforementioned studies
by examining the assessment training patterns of 573 internship programs, all of which were
members of the Association of Psychology Postdoctoral and Internship Centers (APPIC). Their
data indicated that of the 21 internship specialty rotations reported on in the survey (e.g., serious
mental illness, trauma, forensics, substance abuse, etc.), an assessment rotation was the most
frequently offered specialty rotation. In fact, most of the internship directors surveyed (64%)
reported their programs offered an assessment rotation. Furthermore, this study found that major
rotations in assessment were most commonly offered in military (80% of 10 military sites) and
child (92% of 48 child sites) internship programs. Other noteworthy findings included that of the
24
105 university counseling center and 28 private hospital internship directors surveyed, none
offered a major rotation in psychological assessment. According to Stedman (2007), a
significant number of pre-doctoral internship programs may not provide enough emphasis in
assessment training to produce clinical psychology graduates with sufficient assessment
competency. The studies reviewed above further warrant additional examination of the training
available at specific categories or types of internship program. The research findings indicate
that important differences may occur across different types of internship program regarding
assessment-related expectations and practices.
Emerging Issues in Psychological Assessment
Use Across Different Settings
Over the years, there has been substantial growth in the range of settings where
assessment is conducted, including more assessment applications in forensic, healthcare, and
organizational settings (Weiner, 2013). Even though this growth is being seen, assessment
measures used across different settings have often varied little and typically have not been
sufficiently adapted for this broad range of applications. All too often, psychological measures
are being used with individuals and in various settings for which they were not originally
intended, and relevant norms have not been developed (Graham & Naglieri, 2003). Therefore, it
has been brought to light how important it is to examine whether accessible norms are
appropriate and if the interpretations made based on these norms are in fact valid and reasonable
for each setting and cultural group with whom the norms have been applied (Graham & Naglieri,
2003).
25
Impact of Culture and Diversity
The United States is continually changing, in terms of ethnicity, language/s spoken,
socioeconomic considerations, sexual orientation, age, and other considerations of diversity.
Latinas/Latinos have emerged as the largest ethnic minority in the U.S. (Marotta & Garcia,
2003). Most of the psychological assessment measures and tools currently being used in the
United States were normed on European American populations (Dana, 2000). This is an
important consideration when utilizing psychological assessments on individuals who identify
with different cultures. Moreover, the results might not be a valid representation of the
individual. Studies show that assessments developed and normed using the English language
should not be applied to individuals who do not speak English. This discourse is now being
integrated into the legal and ethical aspects of the practice of psychological assessment. It is
recommended that a translated and adapted version of the measure be used or that an attempt be
made to assess the individual through different methods, such as tests that are non-verbal in
nature (Frisby, 1999). This suggests a need to further investigate the use of assessment amongst
varying cultures. It also indicates that in clinical training, it is important to consider how well
graduate students are being prepared to conduct psychological assessment with diverse persons.
The Influence of Evidence-Based Practice
There is ongoing discourse on the pertinence for psychologists to utilize assessment tools
and strategies that are “economical, scientifically sound, and culturally sensitive” (Wood et al.,
2002, p. 519). Psychologists are largely experiencing the impacts of managed care, which
advocates for the utilization of cost-efficient and less time-consuming measures. Psychologists
are thus beginning to utilize briefer/abbreviated measures to address clinical and referral
questions (Hunsley & Mash, 2007; Wood et al., 2002). For example, projective measures are
being utilized less, due to questions of their efficacy, time requirements, and costs (Wood et al.,
26
2002).
Evidence-based assessment (EBA) is being utilized to address these concerns by
complimenting Evidence Based Practices (EBP) that are often utilized in clinical settings
(Hunseley & Mash, 2007; Jensen-Doss, 2011). EBA is defined as, “an approach to clinical
evaluation that uses research and theory to guide the selection of constructs to be assessed for a
specific assessment purpose, the methods and measures to be used in the assessment, and the
manner in which the assessment process unfolds” (Hunsely & Mash, 2007, p. 30). The utilization
of evidence-based assessment is increasing in the field, as emphasis is also being placed for
psychologists to remain updated on research and literature. Youngstrom (2013) states,
psychological assessment lacks “directness and clarity” that is greatly desired by psychologists
(p. 152). It is surmised that a shift in favor of complimenting EBP with EBA will promote
psychologists utilizing the most effective treatments for the populations they are working with.
Psychologists have increasingly been encouraged to take empirically-informed steps to ensure
that their clients are receiving the most appropriate care (Jensen-Doss, 2011). This shift towards
managed care and the use of EBA warrants investigation, as it may impact the future training
emphases in psychology doctoral programs, given demands in the field.
Parent Study
Recently, Bates, (2016), Faith (2016), and Shipley (2019) developed a 32-item
questionnaire to explore assessment-related trends and practices at the internship level. They
surveyed internship directors at APPIC-member, pre-doctoral internship programs throughout the
United States. Their study revealed important shifts in the reported usage patterns of specific
psychological tests and found potentially important differences across types of internships
regarding important aspects of psychological assessment practice. For example, Bates (2016)
identified some shifts in test usage across internship types. She reported a general increase in the
27
use of short, symptom-focused scales and some reduction in use of traditional projective
measures such as the Rorschach. Their findings also indicated that overall, directors of
APPICmember internship programs reported relatively high levels of satisfaction with entering
interns’ knowledge and preparation in psychological assessment. Bates (2016), Faith (2016), and
Shipley (2019) also found that internship directors, as a group, did not anticipate reduction in the
emphasis on psychological testing and assessment at the internship level. Instead, they tended to
report that the emphasis on assessment would stay the same or increase in the future. While
Bates (2016) examined test usage patterns across different types of internship setting, other study
findings were typically reported only for the sample as a whole. Important questions remain
about other potentially significant differences in psychological assessment practices or needs
across various types of internship programs (e.g., V.A. medical centers vs. university counseling
centers vs. prisons or correctional settings, etc.). For example, are there differences across
different types of internship programs in directors’ perceived satisfaction with the
assessmentrelated training and preparation of beginning interns? Are there differences across
categories of internship in directors’ perspectives on incoming interns’ preparation for
assessment of diverse clients? Does the use of technology to support assessment practices differ
across different types of internship program? Research on such questions is needed to fine-tune
our understanding of the specific assessment-related practices and experiences that may exist
across different types of internships.
Assessment continues to be a critical element of training at the pre-doctoral level and an
essential component for graduate students to be competitive for pre-doctoral internship
placement and for success at the internship level (Belter & Piotrowski, 2001; Clemence &
Handler, 2001; Stedman et al., 2000; Weiner, 2012). Because developing competency in
psychological assessment is considered to be a “complex, intensive, and multifaceted process”
28
that is noted to afford “numerous responsibilities and challenges to educators, trainers, learners,
and professional practitioners” (Krishnamurthy et al., 2004, p. 737), it is important to identify
and further explore differences that may exist across types of internship programs. The goal of
the present study was to attempt to shed light on differences in internship directors’ perspectives
that may exist across different categories of internship through re-analysis of an existing data set.
Purpose of the Study
The present study was an archival study that was conducted to shed light on how
internship directors’ perspectives on psychological assessment practices might vary across
different types of internship setting. This study was conducted as part of an Applied Scholarship
Community (ASC) at Pepperdine University that included the writer and two co-investigators,
Katlyn Grusecki and Cecilia Costa. Each of the three investigators took a subset of questionnaire
items from the parent study for particular emphasis. The primary focus of the present study was
to look at the questionnaire item from the parent study that addressed diversity issues in
assessment. This item examined internship directors’ satisfaction with their beginning interns’
preparation to conduct psychological assessment amongst diverse groups. The present study also
examined internship directors’ opinions on the extent to which the field of psychology’s current
emphasis on evidence-based practice has impacted testing and assessment in their internship
programs. Finally, this study also explored the responses to two open-ended items from the
questionnaire. The study utilized the data collected by Bates (2016), Faith (2016), and Shipley
(2019), and involved a reanalysis, exploring internship directors’ questionnaire responses as a
function of the six largest groupings of internship type that occurred in the original study.
Because this was an exploratory study, no specific hypotheses were made.
29
Chapter II: Method
Research Approach and Design
The current study was a non-experimental, descriptive, and exploratory study that utilized
archival data. Looking across different types of internship setting, the purpose of this study was
to gauge psychology internship directors’ satisfaction with their incoming interns’ preparation to
conduct psychological assessment with diverse populations. Additionally, emerging trends in
psychological assessment were examined, specifically: the impact of evidence-based practices
upon assessment; the introduction of new assessment measures in the internship program within
the prior five years; and any measures the directors would like to see introduced that were not
30
being used at the time of the survey. The researcher examined the results of an archival data set
that was collected from a national sample of psychology internship directors within the United
States (Bates, 2016; Faith, 2016; Shipley, 2019). The questionnaire developed by Bates (2016),
Faith (2016), and Shipley (2019) explored assessment-related themes and contained items in a
variety of response formats.
For the current study, the researcher worked in conjunction with two other researchers at
Pepperdine University, as part of an Applied Scholarship Community (ASC) research group.
One researcher (Grusecki, 2019) focused on the importance of psychological assessment across
internship settings; interns’ overall preparedness to conduct psychological assessment upon
commencing internship; and important considerations in intern selection. Another researcher
(Costa, 2019) investigated the role of technology in psychological assessment within internship
settings; the methods of test administration, scoring, and interpretation utilized in the internship;
the anticipated stability of funding for assessment in upcoming years; and whether emphasis on
assessment was likely to change in upcoming years. The researcher for the current study focused
primarily on questionnaire items addressing diversity, evidence-based practice, and trends in
measures used.
Participants and Clusters
The participants for this study were recruited for the dissertation research conducted by
Bates (2016), Faith (2016), and Shipley (2019) at Pepperdine University. Initially, the list of
potential participants was identified from the APPIC directory, which is readily available from
APPIC. The APPIC Directory is utilized as a resource for individuals in various stages of their
careers in psychology (i.e., students, recent graduates, training directors, faculty), provides
information about internships and postdoctoral training programs across the United States and
Canada, and is updated yearly. Internship programs included in the directory are those that have
31
received accreditation through the American Psychological Association (APA) or the Canadian
Psychological Association (CPA). In addition, non-accredited internship programs may qualify
for APPIC membership by meeting 16 criteria.
The researchers in the parent study contacted 741 of the eligible training directors via
electronic mail (e-mail) from a Pepperdine University account and invited them to participate in
the study. The number of participants was continuously refined based on the requirements listed
in the initial email and based on failure to provide a response. In the end, there were 182
participants that both consented and responded to at least some portion of the questionnaire,
which represented a 25% return rate.
Of the 182 internship directors represented in the original sample, 16% classified their
institutional settings as Veterans Affairs Medical Centers (VAMC), 15% as university counseling
centers (UCC), 14% as community mental health centers (CMHC), 12% as state/county/other
public hospitals (SCPH), 8% as consortiums (CON), 7% as prisons or correctional facilities
(PC), 5% as medical schools, 4% as child/adolescent psychiatric or pediatric clinics, 3% as
private outpatient clinics, 3% as private psychiatric hospitals, 3% as private general hospitals,
2% as Armed Forces medical centers, 2% as school districts, and 1% as psychology departments.
Seventeen participants (9%) responded as “other” sites; similar responses were collapsed under
the categories of Non-profit (2%), Residential Treatment (2%), Private Outpatient Clinic (1%),
Court/Forensic (1%), Prison or Correctional Facility (1%), University Counseling Center (<1%),
State/County/Other Public Hospital (1%), and Community Mental Health (<1%) (Bates, 2016).
The sample of 182 participants from the original study (Bates, 2016; Faith, 2016; and
Shipley, 2019) included 118 (66%) females and 62 (34%) males. Their mean age was 46.9 years
(SD = 10.6), with a range of 29 to 72 years old. With regard to ethnic or racial self-identification,
32
88% of the respondents identified as Caucasian, 4% as Latino, 3% as Asian, 2% as
AfricanAmerican, 2% as Multiracial, and 1% as American Indian or Alaskan Native. Three
participants
(2%) selected the “Other” category; they identified themselves as “Mediterranean,” “Middle
Eastern,” and “Hispanic,” respectively.
Regarding their highest academic degree, 62% of participants endorsed Ph.D., 37%
selected Psy.D., and 1% indicated Ed.D. One participant selected the “Other” category (1%) and
wrote that she or he had the following: “J.D., Psy.D.” The discipline or focus of their degrees
was also requested and results revealed that 76% identified Clinical Psychology, 16% indicated
Counseling Psychology, 4% reported School Psychology, and 2% indicated they had a Combined
Program focus. The “Other” category was selected by four participants (2%), who specified
“Experimental and later retrained in Clinical Psychology, also have a JD,”
Developmental Clinical,” “Clinical Neuropsychology,” and “General Psychology” as their
respective areas of study. Concerning licensure, 98% indicated they were licensed to practice
psychology, with 65% first becoming licensed before 2006, and 37% becoming licensed in 2006
or later (M = 2001; range =1965 to 2014). Four participants indicated they were not licensed
(2%) (Bates, 2016; see Table 1).
33
Table 1
Internship Directors’ Demographic Information
N
%
Age
180
--
Range = 29-72
Mean = 46.9 years
SD = 10.6
Gender
Male
62
35%
Female
118
65%
Transgender
0
0%
Other
0
0%
*Abstained from responding
2
<1%
Racial/Ethnic Identity
American Indian or Alaskan Native
1
1%
CHARACTERISTICS
34
Asian
4
3%
Black or African American
3
2%
Caucasian (White)
158
88%
Latino/a
7
4%
Native Hawaiian or other Pacific Islander
0
0%
Multiracial
4
2%
Other/Written-In Responses:
1. Hispanic
2. Mediterranean
3. Middle Eastern
3
2%
*Abstained from responding
2
<1%
Highest Academic Degree
Ph.D.
112
62%
Psy.D.
68
37%
Ed.D.
2
1%
Other
1
1%
Nature of Degree
Clinical Psychology
138
76%
Counseling Psychology
29
16%
Educational Psychology
0
0%
School Psychology
8
4%
Combined Program
4
2%
Other
4
2%
License Status
Licensed
178
98%
Not Licensed
4
2%
Mean = 2001.12
Standard Deviation 8.68
Range (Min-Max) = 41 years *Max
= 2014; Min = 1973
Note. Demographic information from parent study (Bates, 2016; Faith, 2016; & Shipley, 2019)
Given the present focus on response patterns and possible differences across categories of
internship, the present researchers examined the sample from the parent study and selected the
largest categories. The goal was to include as much of the original sample as possible, while also
having sufficient numbers of internship programs included in each category. The present
researchers selected the six largest groups or categories, which incorporated 124 (68%) of the
182 original respondents. The six clusters and corresponding percentages were as follows: 1)
35
Community Mental Health Centers (CMCH; n = 24; 19%); 2) Veterans Affairs Medical Centers
(VAMC; n = 27; 22%); 3) University Counseling Centers (UCC; n = 27; 22%); 4) State/Public
Hospitals (SCPH; n = 18; 15%); 5) Prison and/or Correctional Facilities (PC; n = 14; 11%); and
6) Consortium Programs (CON; n = 14; 11%). The demographic data and professional
background variables on this subset of participants are presented in the Results chapter.
Instrumentation
The survey instrument developed for the original/parent study was comprised of 32 items
that included differing response options: fixed-choice options, rating scale items, and open-ended
items (see Appendix A). The questionnaire was administered to participants at
www.surveymonkey.com. Specifically, the questionnaire contained items pertaining to
demographics of the directors (i.e., age, ethnic identification, and gender), their highest level of
education, and their licensure status. Questions also explored characteristics of the directors’
internship programs, including APA accreditation status, nature of the institutional setting,
theoretical orientation/s, types and numbers of trainees accepted, importance of testing and
assessment in the respondent’s internship, and how training, experience, and supervision in
testing and assessment are provided. The questionnaire also included items addressing the topic
areas that are explored by the researcher in the current study. Further, several open-ended items
allowed respondents to address assessment-related themes in their own words.
In an effort to increase validity and utility, Bates (2016), Faith (2016), and Shipley (2019)
emphasized closed-ended questions, in either multiple-choice or Likert-style response formats,
for their questionnaire, as well as several open-ended questions. When possible, they also
incorporated some opportunities for participants to offer comments or clarification of responses
through an “other” response option. This allowed for standardized data to be collected, while
36
still permitting for variability in the collected responses and for minimizing limitations placed on
respondents concerning their responses.
In order to address the goals of this study, the researcher focused upon select questions
from the original survey questionnaire. To consider assessment-related issues regarding diversity,
evidence-based practice, and emerging measures, the researcher selected the following items
from the original questionnaire:
Survey Item 20: How satisfied are you with incoming interns’ level of preparation for
conducting psychological assessment with diverse populations?
Survey Item 28: How much has the profession’s emphasis on evidence-based practice
impacted your program’s approach to psychological testing and assessment? Survey
Item 29: What new psychological tests or measures has your site begun using within
the last five years? (open-ended item)
Survey Item 30: Within your site, what psychological tests or measures would you like to
see used in the future that are not currently being used? (open-ended item)
Procedures
Data from the original web-based survey study were used for the present analysis. The
original survey used SSL (Secure Sockets Layer) encrypted software and all responses were
anonymously recorded by the principal investigators. The original investigators screened the data
file for answers that were out of the possible range of response options. Those types of error
responses were deleted from the data set to ensure they were not analyzed with the legitimate
37
data. Any edits made to the data set were recorded and kept by the original investigators. This
allowed for proper data analysis by the original researchers, as well as for subsequent reanalyses.
The researcher for the present study obtained permission from Bates, Faith, and Shipley
to utilize their data. The data were not accessed until the present study was approved by
Pepperdine University’s Institutional Review Board. The data that were reanalyzed for the
present study did not have any personally identifying information included.
Data Analysis
The data analyses included calculation of descriptive statistics, such as frequencies,
means, and standard deviations. Because the data from the rating scale items on the questionnaire
did not meet expectations for normal distribution, a non-parametric test was needed to examine
for any significant differences between groups on questionnaire items 20 and 28. Therefore, the
Kruskal-Wallis H test, which can be understood as a one-way ANOVA on ranks, was utilized. In
the event of any significant findings, the Dunn’s test was to be used to identify which pairwise
contrasts were significantly different.
Chapter III: Results
The current, archival study investigated assessment-related issues and themes across six
categories of predoctoral internships in psychology. In addition to examining demographic and
professional variables regarding the internship directors in the sample, the present study explored
internship directors’ satisfaction with their incoming interns’ cultural competence related to
psychological assessment with diverse populations. This study also examined internship
directors’ views on the impact of evidence-based approaches to assessment. The six categories of
internship for the present study were: Consortiums (CON), Prisons/Correctional Facilities (PC),
38
State/County Public Hospitals (SCPH), University Counseling Centers (UCC), Veterans Affairs
Medical Centers (VAMC), and Community Mental Health Centers (CMHC). Finally, the study
inquired about different trends that were occurring at these sites, especially regarding the
introduction of any new assessment measures in recent years and the desire to introduce
assessment measures not currently being utilized. These trends were examined by reviewing
internship directors’ responses to open-ended items on the questionnaire.
A total of 124 participant responses were drawn from the parent study’s original sample
of 182 internship directors. This subsample of 124 represented all of the internship directors in
the parent study that were from the six types of internship program listed above. These six
settings were the most frequently reported by internship directors in the parent study and were
thus selected for close examination in the present study. Thus, the goal of the study was to better
understand whether there were differences across these six settings in the variables of interest:
cultural competence for assessment, impact of evidence-based practice on assessment, and trends
in utilization of specific measures.
The results discussed in this section represent data that were reanalyzed for the current
study. Descriptive statistics were calculated on the demographic and professional background
variables of the 124 internship directors in the present study. This information was collected from
the first six items on the 32-item questionnaire developed for the original study. Questionnaire
item 20 asked internship directors to rate their satisfaction with their incoming interns’
preparation level for conducting psychological assessments with diverse populations. Internship
directors’ views on the impact of evidence-based practice on psychological assessment in their
internship programs was examined in questionnaire item 28. Responses to these two items were
analyzed, utilizing descriptive statistics to compare and contrast answers. Because assumptions
for the normal distribution of data were not met for items 20 and 28, the Kruskal-Wallis test was
39
utilized to determine any significant differences across groups on those two items. Directors
responses to two open-ended questions were also examined for additional information regarding
internship settings and trends for psychological assessment. Specifically, internship directors
were asked to list tests or measures that their sites had begun using within the past five years
(questionnaire item 29). Lastly, directors’ responses to questionnaire item 30 were examined,
which asked them to list tests or measures they would like to see used at their sites in the future,
that were not currently being used.
Participants and Demographic Information
For the current sample (N = 124), the mean age was 47.02 (SD = 10.31). Mean ages
were also calculated across settings. At CON settings (n = 14), the mean age was 46.21 (SD =
9.50). For PC settings (n = 14), the mean age was 43.5 (SD = 9.79). For SCPH settings (n = 18),
the mean age was 43.44 (SD = 7.96). For UCC settings (n = 27), the mean age was 46.74 (SD =
8.85). For VAMC settings (n = 27), the mean age was 48.66 (SD = 11.18). Lastly, for CMHC
settings (n = 24), the mean age was 50.66 (SD = 12.31).
Table 2
Internship Directors’ Mean Age by Setting
Descriptor Setting n Mean SD Range
Age
124
47.02 10.31 43
CON
14
46.21 9.50 33
PC
14
43.5 9.79 31
SCPH
18
43.4 7.96 31
UCC
27
46.74 8.85 33
VAMC
27
48.66 11.18 38
CMHC
24
50.66 12.31 43
40
Regarding gender, 70% of the sample was female (n = 87) and 30% of the sample was
male (n = 37). For CON settings, 64% of the sample was female (n = 9) and 36% were male (n
= 5). At PC settings, 79% were female (n = 11) and 21% (n = 3) were male. At SCPH settings,
72% (n = 13) were female and 28% (n = 5) were male. For UCC settings, 78% were female (n =
21) and 22% (n = 6) were male. At VAMC settings, 59% were female (n = 16) and 41% were
male (n = 11). For CMHC settings, 71% were female (n = 17) and 29% were male (n = 7).
Table 3
Internship Directors’ Gender by Setting
Descriptor(s) Setting n %
Gender
124
--
Male
37
30%
Female
87
70%
CON
14
11%
Male
5
35%
41
Female
9
64%
PC
14
11%
Male
3
21%
Female
11
79%
SCPH
18
15%
Male
5
28%
Female
13
72%
UCC
27
22%
Male
6
22%
Female
21
78%
VAMC
27
22%
Male
11
41%
Female
16
59%
CMHC
24
19%
Male
7
29%
Female
17
71%
In regard to racial/ethnic identity, 85% of the sample identified as Caucasian/White (n =
106), representing the majority of the sample. Additionally, 4% of the sample identified as
Latino/a (n = 5). Three percent of the sample identified as Asian (n = 4). Two percent of the
sample identified as African American (n = 3). Two percent of the sample identified as Native
Hawaiian or other Pacific Islander (n = 3), while just under 2% identified as Multiracial (n = 2).
Lastly, one internship director identified as American Indian or Alaskan Native (1%).
Table 4
Internship Directors’ Ethnic/Racial Identity by Setting
Setting Ethnicity/Race n %
42
CON
14
11%
Caucasian (White)
12
86%
Latino/a
1
7%
Multiracial
1
7%
PC
14
11%
Caucasian (White)
14
100%
SCPH
18
15%
Caucasian (White)
17
94%
Multiracial
1
6%
UCC
27
22%
American Indian or Alaskan Native
1
4%
Asian
2
7%
Black or African-American
2
7%
Caucasian (White)
19
70%
Latino/a
3
11%
VAMC
27
22%
Asian
1
4%
Black or African-American
1
4%
Caucasian (White)
23
85%
Latino/a
1
4%
Multiracial
1
4%
CMHC
24
20%
Asian
1
4%
Caucasian (White)
21
88%
Other
2
8%
Mediterranean
1
--
Middle Eastern
1
--
For racial/ethnic identity, the most variance was noticed in UCC settings. Of the
internship directors within this setting, 70% (n = 19) identified as Caucasian/White; 11% (n = 3)
identified as Latino/a; 7% identified as Black or African American (n = 2); 7% identified as
43
Asian (n = 2); and 4% identified as American Indian or Alaskan Native (n = 1). For VAMC
settings, 85% identified as Caucasian/White (n = 23); 4% identified as Latino/a (n = 1); 4%
identified as Black or African American (n = 1); 4% identified as Asian (n = 1); and 4%
identified as Multiracial (n = 1). For CON settings, 86% identified as Caucasian/White (n = 12);
7% identified as Latino/a (n = 1); and 7 % identified as Multiracial (n = 1). For CMHC settings,
88% identified as Caucasian/White (n = 21); 8% identified as Other (n = 2); and 4% identified as
Asian (n = 1). For SCPH settings, 94% identified as Caucasian/White (n = 17) and 6% identified
as Multiracial (n = 1). The least variance was noticed within PC settings, as all respondents
identified as Caucasian/White (n = 17, 100%).
Respondents were also asked to list their highest degree of attainment (questionnaire item
4), with the following options: Ph.D., Psy.D., Ed.D. and Other. Within the sample, 63% reported
having attained a Ph.D. (n = 78); 36% reported having a Psy.D. (n = 45); and just under 1%
reported having an Ed.D. (n = 1). Regarding the breakdown of academic degrees by setting: for
CON settings, 64% reported having a Ph.D. (n = 9) and 36% reported having a Psy.D. (n = 5).
In PC settings, 43% reported having a Ph.D. (n = 6) and 57% reported having a Psy.D. (n = 8).
In SCPH settings, 56% had a Ph.D. (n = 10) and 44% had a Psy.D. (n = 8). For UCC settings,
59% had a Ph.D. (n = 16); 37% had a Psy.D. (n = 10); and 4% had an Ed.D. (n = 1). For VAMC
settings, 85% had a Ph.D. (n = 23) and 15% had a Psy.D. (n = 4). Lastly, for CMHC settings,
42% had Psy.D. (n = 10) and 58% had a Ph.D. (n = 14).
Questionnaire item number 5 asked respondents to specify the nature of their highest
academic degree. The following options were provided: Clinical Psychology, Counseling
Psychology, Educational Psychology, School Psychology, Combined Program, and Other. Of the
total sample (n = 124), 73% reported having a degree in Clinical Psychology (n = 90). The
second most frequently selected option was Counseling Psychology, representing 22% of the
44
sample (n = 27). Directors from Combined Programs represented 1% of the sample (n = 1),
while persons with a doctorate in Educational Psychology represented 2% of the sample (n = 3).
Lastly, School Psychology represented 1% of the sample (n = 1). For those who marked Other,
their programs were listed as Clinical Neuropsychology (n = 1), Experimental and later trained
as Clinical Psychology/JD (n = 1), and Developmental Clinical Psychology (n = 1).
Questionnaire item 6 questioned if internship directors were or had ever attained a license
to practice in the field of psychology. According to the responses, 100% of the program
directors were licensed to practice psychology.
45
Table 5
Internship Directors’ Academic Degree and Discipline by Setting
Setting Academic Degree n %
Academic Discipline
CON
14
11%
Ph.D.
9
64%
Psy.D.
5
36%
Clinical
10
71%
Counseling
2
14%
Other:
2
14%
Clinical Neuropsychology
2
--
Developmental clinical
2
PC
14
11%
Ph.D.
6
43%
Psy.D.
8
57%
Clinical
12
86%
Counseling
2
14%
SCPH
18
15%
Ph.D.
10
56%
Psy.D.
8
44%
Clinical
18
100%
UCC
27
22%
Ph.D.
16
59%
Psy.D.
10
37%
Ed.D.
1
4%
Clinical
10
37%
Counseling
17
63%
VAMC
27
22%
Ph.D.
23
85%
Psy.D.
4
15%
Clinical
23
85%
Counseling
3
11%
Other:
1
4%
Experimental, clinical
also have a JD
1
--
CMHC
24
19%
46
Ph.D.
14
58%
Psy.D.
10
42%
Clinical
17
71%
Counseling
3
12%
School
3
12%
Combined
3
5%
Satisfaction with Incoming Interns’ Cultural Competence in Psychological Assessment
Questionnaire item 20 asked, “How satisfied are you with incoming interns’ level of
preparation for conducting psychological assessment with diverse populations?” Internship
directors were given the following options to select from: Extremely Satisfied (rating value of 5),
Very Satisfied (rating value of 4), Somewhat Satisfied (rating value of 3), Slightly Satisfied
(rating value of 2), and Not at All Satisfied (rating value of 1). Per the results, all six internship
director group means fell closest to the “Somewhat Satisfied” selection on the rating scale.
Specifically, CON internship directors (n = 14) obtained a mean score of 3.28 (SD = 0.73). PC
directors (n = 14) likewise obtained a mean score of 3.28 (SD = 0.47). CMHC directors (n = 24)
obtained a mean score of 3.04 (SD = 0.69). UCC directors (n = 27) obtained a mean score of
2.85 (SD = 0.98). SCPH directors (n = 18) obtained a mean score of 2.72 (SD = 0.75). Finally,
VAMC directors (n = 27) obtained the lowest mean, with a value of 2.59 (SD = 0.69). The
results are further described in the table below (Table 5).
Table 6
Internship Directors’ Response to Questionnaire Item 20 by Setting
CON
14
3.285
0.73
3
2
PC
14
3.28
0.46
3
1
SCPH
18
2.72
0.75
3
3
UCC
27
2.85
0.98
3
3
Setting
N
Mean
SD
Median
Range
47
VAMC
27
2.59
0.63
3
3
CMHC
24
3.04
0.69
3
4
To determine if there were any statistically significant differences across the six
internship settings in internship directors’ satisfaction with incoming interns’ preparation to
conduct psychological assessment with diverse populations, the Kruskal-Wallis test was utilized.
This test is also termed a one-way ANOVA on ranks. The results of the Kruskal-Wallis test was
in fact statistically significant at the .01 level of significance, c2 (5) = 15.281, p = 0.0092. The
Dunn’s Test was then conducted to determine whether any of the pairwise contrasts were
significantly different. Surprisingly, none of the pairwise contrasts reached the .05 level of
statistical significance. However, since the overall Kruskal-Wallis finding was significant, that
suggested that CON and PC directors, with their mean of 3.28, were more satisfied than other
internship director groups, most notably the VAMC directors with their mean of 2.59.
Emphasis of Evidence-Based Practice on Assessment
Questionnaire item 28 asked, “How much has the profession’s emphasis on
evidencebased practice impacted your program’s approach to psychological testing and
assessment?”
Respondents were given the options of the following: Extremely Impacted (rating value of 5),
Strongly Impacted (rating value of 4), Somewhat Impacted (rating value of 3), Slightly Impacted
(rating value of 2), and Not Impacted at All (rating value of 1). Per the data analysis, the mean
ratings for all six groups of internship directors fell closest to the “Somewhat Impacted” or
“Strongly Impacted” responses on the rating scale. Specifically, CON internship directors (n =
14) obtained a mean score of 3.21 (SD = 0.97). PC directors (n = 14) obtained a mean score of
3.57 (SD = 1.15). SPCH directors (n = 18) obtained a mean score of 3.72 (SD = 0.75). UCC
48
directors (n = 27) obtained a mean score of 2.85 (SD = 1.16). VAMC directors (n = 27) obtained
a mean score of 3.18 (SD = 1.001). Finally, CMHC directors (n = 24) obtained a mean score of
3.12 (SD = 0.89).
Table 7
Internship Directors’ Response to Questionnaire Item 28 by Setting
CON
14
3.21
0.97
3
4
PC
14
3.57
1.16
3
4
SCPH
18
3.72
0.75
4
3
UCC
27
2.85
1.17
3
4
VAMC
27
3.18
1.001
3
4
CMHC
24
3.12
0.90
3
4
To determine if there were any statistically significant differences among the six groups
of internship directors regarding the impact of evidence-based practices on their programs’
approaches to psychological assessment, the Kruskal-Wallis test was again utilized. The results
of the Kruskal-Wallis test did not indicate any statistically significant differences at the .05 level
for this questionnaire item, c2 (5) = 9.6082, p = 0.0871.
Open-Ended Items.
The remainder of the data analysis consisted of a review of responses for open-ended
questionnaire items 29 and 30. For these two items, internship directors were given the
opportunity to write their responses and to list measures recently introduced at their sites (item
29), as well as to list measures they would like to see utilized in the future (item 30). These items
provided some information on recent developments in assessment methods at the sites, as well as
Setting
N
Mean
SD
Median
Range
49
on internship directors’ aspirations regarding the incorporation of new assessment methods.
Given the emerging shifts in psychology (i.e., increase in diverse populations in the US,
managed care and evidence-based assessment within clinical settings), investigation in these
areas is warranted. This information may shed light on the direction of psychology doctoral
training currently and in the future.
Questionnaire item 29 asked, “What new psychological tests or measures has your site
begun using within the past five years?” Measures were organized into the following domains:
Cognitive Functioning, Emotional Functioning, Symptom Inventories/Behavioral Rating Scales,
Neuropsychological Functioning, Academic/Achievement, Forensic/Risk Assessment, and Other
Assessment.
For CON settings, a total of 40 responses were provided by internship directors. Of these
responses, most frequently mentioned measures fell within the Symptom Inventories/Behavioral
Rating Scales (27.5%). The second most frequently listed measures were in the
Academic/Achievement category (20%), followed by Emotional Functioning (17.5%), Cognitive
Functioning (15%), Neuropsychological Functioning (15%), Forensic/Risk Assessment (2.5%),
and Other Assessment (2.5%) categories.
It is worth noting what were the most commonly reported measures within each category.
For the Symptom/Inventories/Behavioral Rating Scales, the Autism Diagnostic Observation
Schedule (ADOS and ADOS-2) were most frequently listed. For the Academic/Achievement
category, the Connors Continuous Performance Test-Third Edition (CPT-3) and
WoodcockJohnson (WJ)- Cognitive and Academic tests were most frequently listed. For the
Emotional
Functioning category, the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) and Millon
50
Adolescent Clinical Inventory (MACI) were most frequently listed. For the Cognitive
Functioning category, the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) was most
frequently listed. For the Neuropsychological Functioning category, the Wechsler Memory
Scale-Fourth Edition (WMS-IV) was most commonly listed. For the Forensic/Risk Assessment
category, the Test of Memory Malingering (TOMM) was the only measure listed. Lastly, for the
Other Assessment category, the WIC-IC was the only measure listed.
Table 8
Testing/Assessment Instruments Recently Introduced at Consortium Program (CON) Internships
CON_______________________________________________________________________________________
Domain Measure Responses %
Cognitive Functioning
6
15%
Universal Nonverbal Intelligence Test (UNIT)
1
Wechsler Adult Intelligence Scale –Fourth Edition (WAIS-IV)
3
Wechsler Intelligence Scale for Children –Fifth Edition (WISC-V)
2
Emotional Functioning
7
17.5%
Millon Adolescent Clinical Inventory (MACI)
2
Minnesota Multiphasic Personality Inventory-2 (MMPI-2)
2
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
1
Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A)
1
Thematic Apperception Test (TAT)
1
Symptom Inventories/Behavioral Rating Scales 11 27.5%
Adaptive Behavior Assessment System (ABAS) 1
Adolescent Anger Rating Scale (AARS) 1
Autism Diagnostic Observation Schedule (ADOS) 2
Autism Diagnostic Observation Schedule (ADOS-2) 2
Autism Spectrum Rating Scale (ASRS) 1
Behavioural Assessment of Dysexecutive Syndrome (BADS) 1
Child Behavioral Checklist (CBCL) 1
Child Depression Inventory (CDI) 1
Multidimensional Anxiety Scale for Children (MASC) 1
51 CON__________________________________________________________________________________________
Domain Measure Responses %
Neuropsychological Functioning 6 15%
Bender-Gestalt Test 1
Developmental Neuropsychological Assessment-II (NEPSY-II) 1
Neuropsychological Assessment Battery (NAB) 1
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) 1
Wechsler Memory Scale –Fourth Edition (WMS-IV) 2
Academic/Achievement
Forensic/Risk Assessment
Career Thoughts Inventory (CTI)
Conners Continuous Performance Test –Third Edition (CPT-3)
Nelson-Denney Reading Test
Wechsler Individual Achievement Test –Third Edition (WIAT-III)
8
1
2
1
1
2
1
1
20%
2.5%
Woodcock-Johnson (WJ) –Cognitive and Academic
Woodcock-Johnson-III (WJ-III) -Cognitive and Academic
Test of Memory Malingering (TOMM)
Other Assessment
WIC-IC
1
1
1
2.5%
52
For PC settings, a total of 33 responses were provided. Of these responses, the most
frequently listed measures fell within the Emotional Functioning category (24%). The second
most frequently mentioned measures fell within the Symptom Inventories/Behavioral Rating
Scales (18%) and Neuropsychological Functioning (18%) categories. This was followed by the
Cognitive Functioning (15%), Academic Achievement (12%) and Forensic/Risk (12%)
categories.
Within the Emotional Functioning category, the Minnesota Multiphasic Personality
Inventory-2-Restructured Form (MMPI-2-RF) was most frequently listed. For the Symptom
Inventories/Behavioral Rating Scale category, there were six measures that were each listed once
(see Table 8 for a comprehensive list). Similarly for the Neuropsychological Functioning
category, there were six measures that were each reported once (see Table 8). For the Cognitive
Functioning category, the Wechsler Intelligence Scale for Children-Fifty Edition (WISC-V) was
most frequently listed. For the Academic/Achievement and Forensic/Risk categories, there were
four responses in each domain. However, for both categories, there was no measure that was
mentioned more than once. See Table 8 for a comprehensive list of the measures that were listed
and their frequencies.
53
Table 9
Testing/Assessment Instruments Recently Introduced at Prison/Correctional Facility (PC) Internships
PC___________________________________________________________________________________________
Domain Measure Responses %
Cognitive Functioning
5
15%
Kaufman Brief Intelligence Test, Second Edition (KBIT-2)
1
Montreal Cognitive Assessment (MoCA)
1
Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV)
1
Wechsler Intelligence Scale for Children –Fifth Edition (WISC-V)
2
Emotional Functioning
8
24%
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
2
Personality Assessment Inventory –Adolescent (PAI-A)
1
Rorschach Inkblot Test, Exner Manual
1
Rorschach Inkblot Test, Software Interpretation Program
1
Rotter Incomplete Sentences Blanks, 2nd Edition (RISB-2)
1
Social-Emotional Assets and Resilience Scales (SEARS)
1
Thematic Apperception Test (TAT)
1
Symptom Inventories/Behavioral Rating Scale 6 18%
Anger Regulation and Expression Scale (ARES) 1
Behavior Assessment System for Children, Third Edition (BASC™-3) 1
Childhood Trauma Questionnaire (CTQ) 1
Firestone Assessment of Violent Thoughts (FAVT) 1
Firestone Assessment of Violent Thoughts –Adolescents (FAVT-A) 1
Stress Index for Parents of Adolescents (SIPA) 1
54
PC____________________________________________________________________________________________________
Domain Measure Responses %
Neuropsychological Functioning 6 18%
Bender Gestalt Test 1
Developmental Neuropsychological Assessment-II (NEPSY-II) 1
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) 1
Wechsler Memory Scale-Fourth Edition (WMS-IV) 1
Wisconsin Card Sort 1
Stroop Color and Word Test 1
Academic/Achievement 4 12%
Test of Word Reading Efficiency –Second Edition (TOWRE-2) 1
Wide Range Achievement Test 4 (WRAT4) 1
Woodcock-Johnson NU Tests of Achievement 1
Woodcock-Munoz Language Survey (WMLS III) 1
Forensic/Risk 4 12%
Inventory of Offender Risks, Needs, and Strengths (IORNS) 1
Risk-Sophistication-Treatment-Inventory (RST-I) 1
Structured Interview of Reported Symptoms, 2nd Edition (SIRS-2) 1
Test of Memory Malingering (TOMM) 1
55
Within the SCPH settings, internship directors provided a total of 41 responses that listed
measures that had been introduced within the past five years. Of these responses, measures
within the Forensic/Risk category (27%) were most commonly listed. The second most
frequently listed measures fell within the Academic/Achievement (17%) category. This was
followed by the Cognitive Functioning (15%) and Emotional Functioning (15%) categories,
Neuropsychological Functioning category (12%), Other Assessment (7.3%) category, and finally
the Symptom Inventories/Behavioral Rating Scale category (7%).
Within the Forensic/Risk category, the Historical Clinical Risk Management-20, Version
3 (HCR-20, v3) was most frequently listed. For the Academic/Achievement category, the
Conners Continuous Performance Test-Third Edition (CPT-3) was most frequently listed. For the
Cognitive Functioning and Emotional Functioning categories, the Wechsler Intelligence
Scale for Children-Fifth Edition (WISC-V) and Minnesota Multiphasic Personality Inventory-2-
Restructured Form (MMPI-2-RF) were most frequently listed. For the Neuropsychological
Functioning category, the Wechsler Memory Scale-Fourth Edition (WMS-IV) was most
frequently listed. For the Other Assessment Category, there were three measures that were each
listed once (see Table 9 for a comprehensive list). Similarly, for the Symptom
Inventories/Behavioral Rating Scale category, there were three measures that were each listed just
one time.
56
Table 10
Testing/Assessment Instruments Recently Introduced at State/County/Other/Public Hospital (SCPH) Internships
SCPH______________________________________________________________________________________________________
Domain Measure Responses %
Cognitive Functioning
6
15%
Brief Cognitive Status Exam (BCSE)
1
Comprehensive Test of Nonverbal Intelligence, Second Edition (CTONI-2)
1
MATRICS Consensus Cognitive Battery
1
Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV)
1
Wechsler Intelligence Scale for Children –Fifth Edition (WISC-V)
2
Emotional Functioning
Minnesota Multiphasic Personality Inventory-2 (MMPI-2) 1
6
15%
Minnesota Multiphasic Personality Inventory-2-Restructured Form® (MMPI-2-RF)
4
Rorschach Performance Assessment System (R-PAS)
1
Symptom Inventories/Behavioral Rating Scale 3 7%
Adaptive Behavior Assessment System, Third Edition (ABAS-3) 1
Autism Diagnostic Observation Schedule (ADOS-2) 1
Childhood Autism Rating Scale, Second Edition (CAARS-2) 1
Neuropsychological Functioning 5 12%
Bilingual Verbal Abilities Test 1
Delis-Kaplan Executive Functioning System (D-KEFS) 1
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) 1
Wechsler Memory Scale –Fourth Edition (WMS-IV) 2
57
SCPH
Domain Measure Responses %
Academic/Achievement 7 17%
Conners Continuous Auditory Test of Attention (CATA) 1
Conners Continuous Performance Test –Third Edition (CPT-3) 2
Leiter International Performance Scale, Third Edition (Leiter-3) 1
University Performance-Based Skills Assessment (UPSA) 1
Vocabulary Assessment Scales–Expressive (VAS-E) 1
Vocabulary Assessment Scales–Receptive (VAS-R) 1
ACUTE Assessment Historical
Clinical Risk Management-20 (HCR-20) (Version not specified)
11
1
2
27%
7.3%
Historical Clinical Risk Management-20, Version 3 (HCR-20, v3)
Forensic/Risk
Other Assessment
Sex Offender Risk Appraisal Guide (SORAG)
Stable Assessment
Static-99R
Violence Risk Appraisal Guide (VRAG)
Violence Risk Screening-10 (V-RISK-10)
Safe Shooting Ability Assessment (SSAA)
Medication Management Ability Assessment (MMAA)
ACS Migration Skills Assessment
3
1
1
1
1
1
3
1
1
1
58
There were 35 responses for UCC settings. Of these responses, the most frequently
mentioned measures fell within the Symptom Inventories/Behavioral Rating Scales category.
These responses made up 34% of the total responses. The second most frequently listed
measures fell within the Emotional Functioning category (20%) and the Academic/Achievement
category (20%). These were followed by the Neuropsychological category (11%), Cognitive
Functioning category (8.5%), and Other Assessment (6%) category.
Regarding the measures that were most frequently listed for each category, the
Counseling Center Assessment of Psychological Symptoms (C-CAPS) was most frequently listed
for the Symptom Inventories/Behavioral Rating Scales. For the Emotional Functioning category,
the Millon College Counseling Inventory (MCCI) was most frequently listed. For the
Academic/Achievement category, the Woodcock Johnson-IV Test of Achievement was most
frequently listed. For the Neuropsychological category, the Delis-Kaplan Executive Functioning
System (D-KEFS) and Wechsler Memory Scale, Fourth Edition (WMS-IV) were each listed
twice. For the Cognitive Functioning category, the Wechsler Adult Intelligence Scale-Fourth
Edition (WAIS-IV) was most frequently listed. Lastly, for the Other Assessment category, the
Minimal Data Set Assessment (MDS) was the only measure listed.
59
Table 11
Testing/Assessment Instruments Recently Introduced at University Counseling Center (UCC) Internships
UCC
Domain Measure Responses %
Cognitive Functioning
3
8.5%
Test of Nonverbal Intelligence Fourth Edition (TONI-4)
1
Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV)
2
Emotional Functioning
7
20%
Millon College Counseling Inventory (MCCI)
3
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
2
Personality Assessment Inventory (PAI)
2
Symptom Inventories/Behavioral Rating Scales
12
34%
Adult-Attention Deficit Disorders Evaluation Scale (A-ADDES)
1
Bipolar Spectrum Scale
1
Conners’ Adult ADHD Rating Scale (CAARS)
1
Counseling Center Assessment of Psychological Symptoms (C-CAPS)
4
Eating Disorder Inventory, Third Edition (EDI-III)
1
Jesness Inventory-Revised (JI-R)
1
Quick Inventory of Depressive Symptomatology (QIDS)
1
Social Responsiveness Scale (self-report and other report)
1
Yale-Brown Obsessive Compulsive Scale
1
Neuropsychological
4
11%
Delis-Kaplan Executive Functioning System (D-KEFS)
2
Wechsler Memory Scale, Fourth Edition (WMS-IV)
2
60
UCC
Domain Measure Responses %
Academic/Achievement 7 20%
California Verbal Learning Test (CVLT) 1
Conners Continuous Performance Test (CPT) 1
Integrated Visual and Auditory Continuous Performance Test-2 (IVA-2) 1
Learning Style Assessment 1
Test of Word Reading Efficiency –Second Edition (TOWRE) 1
Woodcock Johnson-IV Tests of Achievement 2
Other Assessment 2 6%
Minimal Data Set Assessment (MDS) 1
61
The VAMC internship directors provided 34 responses that listed measures that have
been introduced at their settings over the past five years. Among these responses,
Neuropsychological measures were most frequently listed (38%), followed by Other Assessment
Measures (18%), and Emotional functioning measures (18%). The fourth most frequently listed
category were measures in the Cognitive Functioning category (12%), followed by the Symptom
Inventories/Behavioral Rating Scales (9%) and lastly, the Forensic/Risk (5.8%) category.
Within the Neuropsychological category, the following measures were listed most frequently:
Delis-Kaplan Executive Functioning System (D-KEFS), Neuropsychological
Assessment Battery (NAB), Repeatable Battery for the Assessment of Neuropsychological
Status (RBANS), Wechsler Memory Scale- Fourth Edition (WMS-IV). For the Forensic/Risk
category, the Test of Memory Malingering (TOMM) and the Hopkins Competency Assessment
Test were each listed once. For the Other Assessment category, there were six measures that
were each listed once (see table for comprehensive list). For the Emotional Functioning
category, the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
Scales was listed most frequently. For the Cognitive Functioning and Symptom Inventories/
Behavioral Rating Scales categories, no measure was listed more than once (see Table 11 for the
complete list).
62
Table 12
Testing/Assessment Instruments Recently Introduced at Veterans Affairs Medical Center (VAMC) Internships
VAMC_________________________________________________________________________________________
Domain Measure Responses %
Cognitive Functioning
4
Kokmen Short Test of Mental Status 1
Mini Mental Status Exam (MMSE) 1
St. Louis University Mental Status Exam (SLUMS) 1
12%
Wechsler Abbreviated Scale of Intelligence - Second Edition (WASI-II) 1
Emotional Functioning
6
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) 4
Minnesota Multiphasic Personality Inventory (MMPI) Restructure Clinical (RC) Scales 1
Rorschach Inkblot Test, Software Interpretation Program (R-PAS) 1
18%
Symptom Inventories/Behavioral Rating Scales 3 9%
Clinician-Administered PTDS Scale for DSM-5 (CAPS-5) 1
Geriatric Depression Scale (GDS) 1
Geriatric Anxiety Scale (GAS) 1
Neuropsychological 13 38%
Brief Visuospatial Memory Test-Revised (BVMT-R) 1
Behavioral Rating Inventory of Executive Functioning–Adult (BRIEF-A) 1
California Verbal Learning Test -Second Edition (CVLT-II) 1
Delis-Kaplan Executive Functioning System (D-KEFS) 2
Dementia Rating Scale (DRS) 1
Green's Word Memory Test 1
Neuropsychological Assessment Battery (NAB) 2
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) 2
Wechsler Memory Scale –Fourth Edition (WMS-IV) 2
63
VAMC
Domain Measure Responses %
Forensic/Risk
Other Assessment
Test of Memory Malingering (TOMM)
Hopkins Competency Assessment Test
Clock Drawing Test
Digit Vigilance Test
Independent Living Skills (ILS)
Tests for Attention Deficit/Hyperactivity Disorders in Adults: Ruff 2 and 7 Selective
2
1
1
6
1
1
1
1
1
1
5.8%
18%
Attention Tests, Adult Self-Report Scale, and Brief Test of Attention
The B Test
World Health Organizations Disability Assessment Schedule (WHODAS)
64
Lastly, for CMHC settings, the measures listed most frequently fell within the Cognitive
Functioning category (25%). The second most frequently listed measures fell within the
Emotional Functioning Category (22.5%) and that was followed by the Neuropsychological
category (20%). These were followed by the Symptom Inventories/Behavioral Rating Scales
(12.5%), Academic/Achievement (12.5%) and Other Assessment (7.5%) categories.
When reviewing the measures that were most frequently mentioned in each category, the
Wechsler Intelligence Scale for Children -Fifth Edition (WISC-V) was most frequently listed in
the Cognitive Functioning category. For the Emotional Functioning category, the Minnesota
Multiphasic Personality Inventory-Adolescent (MMPI-A) was most frequently listed. For the
Neuropsychological category, the Conners 3rd Edition (Conners-3) and Developmental
Neuropsychological Assessment-II (NEPSY-II) were most frequently listed. For the Symptom
Inventories/Behavioral Rating Scales, there were no measures that were listed more than once
(see Table 12). The same pattern was found for the Academic/Achievement and Other
Assessment categories, where no measure was listed more than once. In regard to cultural
diversity considerations, one internship director reported introducing a Spanish language measure
in the Academic/Achievement area (Batteria III Woodcock-Munoz).
65
Table 13
Testing/Assessment Instruments Recently Introduced at Community Mental Health Center (CMHC) Internships
CMHC____________________________________________________________________________________________
Domain Measure Responses %
Cognitive Functioning
10
25%
Montreal Cognitive Assessment (MoCA)
1
Wechsler Abbreviated Scale of Intelligence - Second Edition (WASI-II)
1
Wechsler Adult Intelligence Scale –Fourth Edition (WAIS-IV)
2
Wechsler Intelligence Scale for Children –Fifth Edition (WISC-V)
6
Emotional Functioning
9
22.5%
Millon Adolescent Clinical Inventory (MACI)
1
Millon Clinical Multiaxial Inventory-III (MCMI-III)
1
Minnesota Multiphasic Personality Inventory -Adolescent (MMPI-A)
3
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
1
Personality Assessment Inventory (PAI)
1
Rorschach Performance Assessment System (R-PAS)
2
Symptom Inventories/Behavioral Rating Scales
5
12.5%
Adult Clinical Symptoms Interpretation
1
Autism Diagnostic Observation Schedule -Second Edition (ADOS-2)
Behavior Assessment System for Children, Second Edition (BASC-2)
1
Clinical Report and Scoring
1
Conners’ Adult ADHD Rating Scale (CAARS)
1
Gillam Asperger’s Disorder Scale (GADS)
1
66
CMHC____________________________________________________________________________________________
Domain Measure Responses %
Neuropsychological
Academic/Achievement
Behavioral Rating Inventory of Executive Functioning–Adult (BRIEF)
8
1
1
1
2
2
1
5
1
1
1
20%
California Verbal Learning Test -Second Edition (CVLT-II)
Conners Continuous Performance Test 3rd Edition (CPT 3)
Conners 3rd Edition (Conners-3)
Developmental Neuropsychological Assessment-II (NEPSY-II)
Wechsler Memory Scale –Fourth Edition (WMS-IV)
Batteria III Woodcock-Munoz
Clinical Evaluation of Language Fundamentals (CELF)
Differential Ability Scales (DAS-II)
Leiter International Performance Scale, Third Edition (Leiter-3)
Other Assessment
Vineland Adaptive Behavior Scales (Vineland)
Health Dynamics Inventory
Instruments related to Autism Spectrum Disorders
Missouri Educator Gateways Assessment (MEGA)
1
1
3
1
1
1
12.5%
7.5%
67
Questionnaire item 30 asked, “Within your site, what psychological tests or measures
would you like to see used in the future that are currently not being used?” There were 15
written responses from CMHC sites and 12 written responses from VAMC settings. Eleven
responses were provided at UCC settings, and seven responses were provided at SCPH settings.
At PC settings, four responses were provided, and at CON settings, nine written responses were
provided for this survey item.
When the responses were examined based on each setting, there was variability in the
desire for the future introduction of measures. Specifically, for CON settings there were
measures listed for Cognitive Functioning (2), Emotional Functioning (2), Symptom
Inventories/Behavioral Rating Scales (2), Neuropsychological Functioning (2), and Academic
Functioning/Achievement (1). It is worth noting that among CON internship directors, the desire
to introduce Spanish language versions of the Minnesota Multiphasic Personality Inventory-
2Restructured Form (MMPI-2-RF) and the Wechsler Intelligence Scale for Children-Fourth
Edition (WICSC-IV) was mentioned.
For PC settings, emphasis was placed on introducing measures for Academic
Functioning/Achievement (3) and Forensic/Risk Assessment (1). At SPCH settings, there was a
desire for introducing measures within the following categories: Symptom
Inventories/Behavioral Rating Scales (3), Neuropsychological Functioning (2), Forensic/Risk
Assessment (1), and Other Assessments (1). It is worth noting that a respondent specified a
desire for the Bateria III Woodcock-Munoz, which is a Spanish version of the WoodcockJohnson
III. Additionally, several respondents did not list specific measures, but expressed a need for
neuropsychological batteries, symptom inventories, suicide assessment, and “More risk
assessment.”
68
At UCC settings, there was more of an emphasis on measures for Emotional Functioning
(6), Symptom Inventories/Behavioral Rating Scales (3), and Academic
Functioning/Achievement (3) measures. In addition, a Forensic/Risk Assessment measure was
listed, as was one measure in the Other Assessments category. One UCC internship director
stated, “None, we do not use tests.” Another UCC respondent wrote, “We would love to offer
formal ADHD assessment, but we don’t have the staffing to accommodate the potential demand.”
Other UCC category respondents stated, “Personality inventories,” and “Measures that accurately
assess for adult autism.”
Additionally, at VAMC settings, there was an emphasis on measures for Emotional
Functioning (4), Neuropsychological Functioning (4), Forensic/Risk Assessment (2), and Other
Assessments (1). One respondent expressed a need for “lots of briefer measures for medical
populations…” Another respondent expressed a desire for, “alternatives to the WAIS for
evaluation of IQ.” Respondents also noted their desires for aptitude and neuropsychological tests.
Lastly, for CMHC settings, there was a large focus on the introduction of measures for
Neuropsychological Functioning (9), Cognitive Functioning (5), Emotional Functioning (2),
Symptom Inventories/Behavioral Rating Scales (3), and Academic Functioning/Achievement
(1). Regarding written responses, the respondents expressed a desire for a broad range of tests.
Moreover, it is worth noting that the desire for “bilingual Spanish based tests,”
neuropsychological tests, and measures that assess the presence of autism were all mentioned by
CMHC directors. (See Table 13 for the comprehensive list of specific measures listed by each
setting.)
69
82
Table 14
Tests/Measures Directors Would Like to See Introduced in their Internships
CON Settings ________________________________
Domain Measure Responses %
Cognitive Functioning
2
22%
Cognitive Performance Test (CPT)
Wechsler Intelligence Scale for Children - Fourth Edition (WISC-IV)
1
Spanish Version
1
Emotional Functioning
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF)
2
22%
Spanish Version
1
Rorschach Performance Assessment System (R-PAS®)
1
Symptom Inventories/Behavioral Rating Scales
2
22%
Beck Depression Inventory-II (BDI-II)
1
Test of Everyday Attention for Children (TEA-Ch)
1
Neuropsychological Functioning
2
22%
Delis-Kaplan Executive Function System (D-KEFS)
1
Sensory Profile 2
1
Academic Functioning/Achievement
1
11%
83
Differential Ability Scales -II (DAS-II)
1
PC Settings
Domain Measure
Responses
%
Academic Functioning/Achievement
3
75%
Batería III Woodcock-Muñoz
1
Woodcock-Johnson Tests of Achievement
1
Woodcock-Johnson Tests of Cognitive Abilities
1
Forensic/Risk Assessment
1
25%
Miller Forensic Assessment of Symptoms Test (M-FAST)
1
SCPH Settings
Domain
Measure
Responses
%
Symptom Inventories/Behavioral Rating Scales 3 43%
Schedule for Affective Disorders and Schizophrenia (SADS) 1
Structured Clinical Interview for DSM-5 (SCID-5) 2
Neuropsychological Functioning 2 29%
Conners Continuous Performance Test 3rd Edition (Conners CPT 3) 2
Forensic/Risk Assessment 1 14%
Responses %
84
Miller Forensic Assessment of Symptoms Test (M-FAST) 1
Other Assessments 1 14%
DIS 1
UCC Settings
Domain Measure
Emotional Functioning 6 43%
Millon Clinical Multiaxial Inventory-III (MCMI-III) 2
Minnesota Multiphasic Personality Inventory (MMPI) 1
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) 1
Personality Assessment Inventory (PAI) 1
Rorschach Technique 1
Symptom Inventories/Behavioral Rating Scales 3 21%
Schedule for Affective Disorders and Schizophrenia (SADS) 1
Structured Clinical Interview for DSM-5 (SCID-5) 2
Academic Functioning/Achievement 3 21%
Conners Continuous Performance Test (Conners CPT) (Ed. Not specified) 1
Conners Continuous Performance Test (Conners CPT) 1
Wonderlic Scholastic Level Exam 1
85
Forensic/Risk Assessment 1 7%
Miller Forensic Assessment of Symptoms Test (M-FAST) 1
Other Assessments 1 7%
DIS 1
Responses %
86
VAMC Settings
Domain Measure
Emotional Functioning 4 36%
Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) 2
Rorschach Performance Assessment System (R-PAS) 1
Rorschach Technique 1
Neuropsychological Functioning 4 36%
Blessed Orientation Memory Concentration (BOMC) 1
Dementia Rating Scale–2 (DRS-2) 1
Neuropsychological Assessment Battery (NAB) 1
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) 1
Forensic/Risk Assessment 2 18%
Miller Forensic Assessment of Symptoms Test (M-FAST) 1
Structured Interview of Reported Symptoms (SIRS) 1
Other Assessments 1 9%
NBSI 1
Responses %
87
CMHC
Domain Measure
Cognitive Functioning 5 25%
Wechsler Adult Intelligence Scale-Fifth Edition (WAIS-V) 1
Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) 4
Emotional Functioning 2 10%
Rorschach Performance Assessment System (R-PAS) 2
Symptom Inventories/Behavioral Rating Scales 3 15%
Autism Diagnostic Observation Schedule (ADOS) 1
Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) 1
Millon Behavioral Medicine Diagnostic (MBMD) 1
Neuropsychological Functioning 9 45%
Behavior Rating Inventory of Executive Function (BRIEF) 1
California Verbal Learning Test (CVLT) 1
Conners Continuous Performance Test (Conners CPT) 1
Conners Continuous Performance Test 3rd Edition (Conners CPT 3) 1
Delis-Kaplan Executive Function System (D-KEFS) 2
Developmental Neuropsychological Assessment-II (NEPSY-II) 1
Weschler Memory Scale (WMS) 1
Responses %
88
Wisconsin Card Sorting Test (WCST) Computerized 1
Academic Functioning/Achievement 1 5%
Wechsler Individual Achievement Test (WIAT) 1
89
Chapter IV: Discussion
The current study utilized archival data from a previous study by Bates (2016) Faith
(2016) and Shipley (2019), which investigated internship directors’ perspectives on current
practices and emerging trends in psychological assessment at the internship level, including
interns’ overall preparedness for assessment. The data from this study was used to determine any
variability across six major categories of internship regarding the following questionnaire items:
Survey Item 20: How satisfied are you with incoming interns’ level of preparation for conducting
psychological assessment with diverse populations?
Survey Item 28: How much has the profession’s emphasis on evidence-based practice
impacted your program’s approach to psychological testing and assessment? Survey
Item 29: What new psychological tests or measures has your site begun using within
the past five years? (open-ended item)
Survey Item 30: Within your site, what psychological tests or measures would you like to
see used in the future that are currently not being used? (open-ended item)
Significant differences were found across internship settings when internship directors
were asked if they were satisfied with their incoming interns’ preparation to conduct
psychological assessment with diverse groups. Specifically, the PC and CON directors (M = 3.28)
were particularly more satisfied than other groups, with the VAMC directors (M = 2.59) reporting
the lowest mean satisfaction. This significant difference could be related to several factors. Based
on findings of the current study, there may be more emphasis on culture and diversity training for
psychological assessment in certain settings than others. Services may also be provided to more
diverse groups within certain internship settings than others, and the use of formal assessment no
doubt varies across internship settings. Literature has referred to the veteran population as a
90
specific culture that requires specialized care (Hobbs, 2008). Moreover, veterans’ challenges with
recovering from trauma-related disorders, substance abuse issues, etc., require culturally
competent care (Hobbs, 2008). Thus, VAMC internships may have higher expectations for
cultural competence than other sites. Further investigation is warranted to investigate the
emphasis on diversity and cultural competence across internship settings.
Overall, the majority of responses fell between the “Slightly satisfied” and “Very
satisfied” categories, with the most frequent response being “Somewhat satisfied.” This finding
suggests there may be insufficient training within doctoral programs on conducting psychological
assessment with diverse groups. This finding is also congruent with the research of Ready and
Veague (2014), who report that there is a need for cultural and diversity training within doctoral
programs. Ready and Veague (2014) state that within a sample of doctoral programs that were
surveyed to determine the role of assessment, multicultural training was “less than ideal”
although cultural competence was acknowledged to be “critical for competent care and sound
clinical science” (p. 282). This suggests a disconnect between stated goals or values and actual
practices in doctoral programs.
Ready et al. (2016) examined a broad cross section of internship directors’ views on their
interns’ preparedness for conducting psychological assessment at the predoctoral internship level.
Although numerous areas of concern were listed, only 1% of respondents reported concerns
regarding training in multicultural issues (Ready et al., 2016). This finding is noteworthy,
considering that cultural competence is a benchmark in the training of developing psychologists
(Krishnamurthy, 2004). The internship directors in the present sample may have shown a similar
pattern. While their responses indicated modest satisfaction with their interns’ preparedness to
conduct psychological assessment with diverse groups, there were few indicators that showed this
level of preparedness was a concern to internship directors. For example, in reviewing answers to
91
open-ended questions, it appeared that little attention was given to cultural and diversity factors.
One hypothesis for this finding could be the homogeneity among the respondents. For the current
study, there was a lack of diversity amongst the internship directors, as the group was mostly
representative of White/Euro, middle-aged females. Thus, ethnically and culturally diverse
respondents were largely underrepresented among directors across internship settings. It may be
that if there were a larger representation of diversity amongst internship directors, more attention
would be given to multicultural issues in psychological assessment within these settings.
An idea to consider is, if more internship directors were representative of diverse groups,
would more students of diverse backgrounds be attracted to the field? Literature shows that
diversity within training programs not only can impact student-faculty interactions, but promote
students’ intellectual development (Chang, 1996). Moreover, attention to diversity issues can also
promote the engagement of diverse students within their programs (Antonio, 2001; Chang, 1996;
Cole et al., 2003). Regarding race and gender demographics, the current study’s sample was
largely representative of the field of psychology. Specifically, ethnically diverse students have
been historically underrepresented in psychology doctoral programs (Callahan et al., 2018).
Further, regarding the overall psychology workforce, diverse groups (i.e., race, ethnicity,
disabilities) are underrepresented in comparison to the US population (Callahan et al., 2018; Lin,
et al., 2018). The lack of diversity within the current sample may provide a limited scope on
diversity pertaining to psychological assessment. Thus, more diversity amongst internship
directors and doctoral programs may also facilitate a larger representation of diversity within the
field of psychology overall. Such changes may also help direct more attention to cultural factors.
Another possibility is that the questionnaire used in the present study was not adequately
attuned to diversity issues. Additionally, the parent study was not primarily focused on diversity
issues. Thus, relying on archival data to examine diversity issues in assessment among internship
92
programs may represent a limitation of the present study. Had more specific questions been
devoted to diversity-related considerations, internship directors may have conveyed more of their
thinking on the matter and a different understanding of their perspectives may have emerged.
More research would be needed for this possibility to be explored.
Significant differences were not found across settings pertaining to the emphasis of
evidence-based assessment on the programs’ approaches to psychological assessment. Further,
settings showed they were either “Somewhat Impacted” or “Strongly Impacted,” with PC and
SCPCH settings falling most closely to being “Strongly Impacted.” This aligns with literature
that is stating that evidence-based assessment is being utilized more frequently within clinical
settings (Jensen-Doss, 2011). Further, evidence-based assessment is also being utilized as a tool
to address the impacts of managed care by developing assessment practices that are empirically
based, cost efficient, and applicable to the population that is being served (Hunsley & Mash,
2007; Wood et al., 2002).
It was interesting to note however, that UCC directors had the lowest mean on this item
(2.85), suggesting less impact of evidence-based approaches, while SCPH directors had the
highest mean (3.72), suggesting greater impact of this contemporary trend. More research with
larger samples would be needed to shed light on whether there are in fact any significant
differences among groups on this issue.
A trend worth noting in the present findings was that more neuropsychological assessment
measures are reportedly being utilized within certain internship settings. This trend was most
prominent within VAMC and CMHC settings. Additional literature reports that
neuropsychological training is increasing amongst doctoral programs (Mihura et al., 2017). In the
aforementioned study, assessment training was investigated within doctoral programs. Further,
neuropsychological assessment was the third most frequently endorsed type of assessment taught
93
(94%), while only 46% of the programs listed neuropsychological assessment as an actual
requirement of the training programs (Mihura et al., 2017). Thus, not only is neuropsychological
assessment utilized more prominently in certain internship settings, but doctoral programs are
also providing more training within this domain to address the rising needs of various clinical
settings.
Another trend that was noticed across domains was the use of symptom-focused
measures. These measures were most prominently found in CON and UCC settings. This finding
is noteworthy, as literature has also discussed the shift away from formal psychological testing
within managed care settings (Piotrowski, 1999). Piotrowski discusses the managed care model
of many clinical settings, which places emphasis on capitation. Further, it is stated that capitation
has significantly restricted psychological testing for clients, due to expenses and time required to
administer, score and interpret measures (Piotrowski, 1999). Moreover, Piotrowski (1999) states
that there is an ongoing devaluation of psychological assessment within managed care settings,
which is considered a direct threat to professional psychology. Similarly, Griffith (1997) reports
findings in which nine managed care agencies were surveyed and expressed that a clinical
interview was more efficacious in psychodiagnosis and treatment planning than psychological
assessment. Thus, shifts toward more symptom-based assessments may be indicative of these
practices and warrant further investigation. Symptom-based assessment measures may also be
more compatible with evidence-based approaches to practice, where specific symptoms are
targeted for intervention and monitored for responsiveness to treatment. Clearly more research is
needed on the ways that changes in assessment practices may be related to the growing influence
of evidence-based practice.
94
When asked to list measures that they would like to see introduced at their sites in the
future, a theme for abbreviated measures was noticed among internship directors. In particular,
the MMPI-2-RF was commonly listed, in addition to the Rorschach Performance Assessment
System (R-PAS). Both measures are abbreviated versions of the MMPI-2 and Rorschach Inkblot
Method, respectively. One respondent within a VAMC setting stated he/she would like to see,
“lots of briefer measures for medical populations.” This further attests to the shift to briefer,
symptom-based measures, which may also be related to the impact of managed care and its
emphasis on cost containment.
Another theme that was noticed was internship directors’ desire for versions of measures
in Spanish formats/versions to be introduced in their internship programs. This theme reflected
attention to, or awareness of, a diversity-related need in assessment practices on internship. It
may also reflect utilization of psychological services by increasingly diverse communities.
However, no other languages were mentioned in response to questionnaire item 30, and no other
suggestions for culturally sensitive measures. This finding is particularly interesting, given the
vast representation of diverse populations across the United States. Literature is showing that the
United States population is increasing in its diversity (Butcher, 2006). It is predicted that in
approximately 20 years, the United States will no longer have a single ethnicity which is
considered a majority of the population (Hempel, 2013). Moreover, the representation of
ethnically diverse populations continually increases. Specifically, the Latino population now
represents 16% of the population, the African American community represents 12% of the
population, the American Indian community represents 1.2% of the population, and the
Asian/Pacific Islander community represents 5.5% of the population (Wright et al., 2014). Thus,
psychologists may be serving diverse populations at higher frequencies and require training in
diversity, including training in multicultural issues that may be associated with assessment.
95
Many psychological assessments were created to suit homogenous groups (Naglieri & Graham,
2013). Thus, care must be taken to ensure that assessment tools are applicable to the populations
that psychologists are serving. Taking this information into consideration, for the current study, it
would be suspected that more emphasis would be placed on selecting measures that were most
suitable for diverse groups. These findings also attest to the need for further discourse in cultural
competence in assessment training overall.
Measures were also listed for younger consumers of psychological services, including
Spanish language versions of these measures. Interestingly, there were also no measures listed to
specifically assess geriatric populations. This is another group that is receiving more attention, as
the United States population is continually aging. The average life expectancy has largely
increased from 46 to an average of 76, specifically, 76.1 years for men and 81.1 years for women
(Fernandez-Ballesteros, 1999; Murphy et al., 2018).
Recommendations for future research
There are a number of opportunities for future research to further expand on this topic.
Results indicated a shift towards neuropsychological assessment and symptom-based measures
within certain settings. Research could investigate the client populations that are receiving
services at these sites to understand the needs and training that are beneficial for these
populations. Moreover, future research could further investigate the impacts of evidence-based
practice and managed care on assessment services within given settings. This information could
possibly provide information on the future of psychological assessment within these settings. It
could also be useful to psychology doctoral students who are considering career development and
training that is needed for specific practice settings.
In regard to psychological assessment, cultural competence, and cultural adaptations, a
new questionnaire could be developed and administered to internship directors nationwide that
96
first describes and explores all the dimensions of diversity. This step would be helpful, as
diversity is often considered to be limited to ethnic and racial backgrounds; however, the APA
code of ethical guidelines reports that diversity is reflected by a variety of dimensions (i.e.,
gender, gender identity, race, ethnic background, sexual orientation, national origin, age, religion,
socioeconomic status, language, disability). Further, by researchers operationalizing diversity,
respondents would be reminded to consider all aspects of diversity in their responses. It
would be beneficial to obtain the frequency of the directors’ sites in providing clinical and
assessment services to diverse groups. Specific questions first addressing trainees’ broad use
assessment at internship sites, followed by questions on the use of culturally sensitive measures
could also be utilized, particularly regarding the frequency of their utilization. It would be helpful
to have an estimate of the rates of diverse groups receiving services in comparison to their
interns’ abilities to provide assessment to these groups. This information could further attest to
the need for cultural sensitivity training and cultural adaptations of assessment. Lastly, openended
questions could be asked regarding the cultural adaptations of assessments and use of culturally
sensitive measures that directors would like to see introduced into their programs. This could also
attest to the trends of cultural adaptations for diverse groups in these settings and introduce
further needed discourse regarding selecting measures and interpreting data in a manner that is
culturally sensitive and optimally serves clients. Moreover, internship directors could be asked to
reflect upon their own staff members’ cultural competence in regard to psychological assessment
and what resources or additional development might be needed to strengthen their resources to
provide mentoring and training to interns in the assessment of diverse communities.
Limitations
Several limitations are identified in this study. The current study was an archival study,
which used data obtained from a parent study (Bates, 2016; Faith, 2016; Shipley, 2019). The
97
parent study researchers developed an online questionnaire that was administered nationwide to
internship directors. The current researcher’s area of interest was diversity and trends that were
prevalent amongst internship directors’ programs. However, the researcher was unable to add any
specific questions related to her interest area, and instead, had to utilize the data that was
previously collected. Additionally, the question that examined competence in working with
diverse groups did not list the dimensions of diversity, as defined in the APA Code of Ethics
(2002). Thus, internship directors may not have fully understood this survey item. Although an
abundance of data was interpreted, more information could be obtained from a questionnaire that
listed all dimensions of diversity and provided respondents the opportunity to address each
dimension, in relation to their interns’ cultural competency.
Additionally, the sample size of the current study was smaller for a number of reasons.
Primarily, the researcher analyzed data for the most frequently represented internship categories
from the parent study: VAMC, CMHC, SCPH, PC, CON, and UCC. However, other internship
settings were identified in the parent study. There may have been more discussion of diversity
issues if the full sample had been utilized for the present study.
Another limitation of the study was the response rate of the sample. Previous literature
has remarked on the difficulty of obtaining high response rates from internship and doctoral
training directors due to their heavy workloads of supervision, faculty duties, program
supervision, and personal caseloads (Shen-Miler et al., 2012). Moreover, obtaining a larger
sample size within this population appears to be increasingly difficult. For instance, in the parent
study,741 internship directors were invited to participate in the study, while 182 were consented
to participate, resulting in a 25% return rate (Bates, 2016; Faith, 2016; Shipley, 2019). Likewise,
in Ready and Veague’s (2014) study on doctoral training in psychological assessment, 233
academic program directors of clinical training were invited to participate, yet 77 responded,
98
resulting in a 33% response rate. Similarly, Shen-Miler and colleagues (2012) also examined
training in psychology doctoral programs, specifically related to addressing competency
challenges regarding diversity, and obtained a lower response rate. According to the authors, 64
programs were contacted, yet just 6 programs participated, resulting in responses from 22
participants, including faculty members (Shen-Miler et al., 2012). Literature has also shown a
mean response rate of 49.6% in published survey studies in counseling and clinical psychology in
the United States (Van Horn et al., 2009). Thus, the current study’s response rate fell under the
average response rate for survey studies that are ultimately published in peer review journals.
These outcomes attest to a challenging quest for researchers when attempting to obtain
information from internship directors, faculty members, and doctoral training programs overall.
There may be a need for some form of incentive or encouragement to participate in these studies,
facilitate this process, and also provide internship directors with support for them to participate in
larger numbers.
There are also limitations to the generalizability of the results of this study. Since only six
categories of internship were examined, the findings may not apply to other types of internships,
such as internships located in military settings or child guidance clinics. In addition, selection
factors may limit the generalizability of the findings. Internship directors with especially positive
or negative attitudes about assessment might have been more motivated to participate in the
original study. However, their views may have been less representative of internship directors in
general.
Conclusions
Findings for this and previous literature shows that diversity is continually growing and
impacting the practices of psychologists. Thus, training in diversity and cultural competence is
required as a competency benchmark for clinical practice. Although the U.S. population is
99
steadily growing in its numbers of diverse populations, there is still a paucity of literature on
moving forward in culturally adapting assessment measures and attending to diversity issues in
psychological assessment. Moreover, it is unclear how many training directors are showing
adequate interest in implementing and advocating for culturally-attuned measures that can be
used in their programs. This warrants further discourse and research, as psychologists are
providing more services to diverse populations, and thus trainees require guidance in this area.
Evidence-based assessment is also an emerging trend that is occurring amongst clinical
sites. With proper investigation, evidence-based assessment may also provide guidance for
implementing multicultural assessment and culturally adapting assessments for diverse groups.
Lastly, more emphasis is being placed on briefer assessments that require less time and are more
cost efficient. These trends may also impact the future training of doctoral students, including
their preparation for predoctoral internships.
“I. INSTRUCTIONS
The purpose of this questionnaire is to obtain psychology internship directors’ perspectives on
training and practice issues related to psychological testing and assessment. Please complete the
survey in one sitting; it should take no more than 10 to 12 minutes. We encourage you to respond
to every item, but you are free to omit items if you so choose. Click the “Next” button at the bottom
of each page in order to proceed. You may discontinue at any time by clicking the “Exit Survey”
button at the top of the page. After finishing, click the “Submit Responses” button. Please
complete the questionnaire only once.
For this study, psychological “assessment” refers to the broad competence that incorporates
multiple methods and sources of information to address referral questions and guide clinical
practice. The methods used may include interviews, record reviews, standardized and
nonstandardized tests, and behavioral observation. Psychological “testing” is defined as the use
of formal tests, such as standardized and norm-referenced measures, questionnaires, or checklists
(e.g., WAIS-V; MMPI-II, DKEFS).
Thank you for your participation!
II. DEMOGRAPHIC INFORMATION
1. What is your age?
100
2. What is your gender?
Male
Female
Transgender
Other (please specify)
3. Please select the category that best describes your ethnic or racial identity:
101
American Indian or Alaskan Native
Asian
Black or African-American
Caucasian (White)
Latino/a
4. What is your highest academic degree?
Ph.D.
Psy.D.
Ed.D.
Other (please specify)
5. What is the nature of your degree?
Clinical Psychology
Counseling Psychology
Educational Psychology
School Psychology
Combined Program
Other (please specify)
Native Hawaiian or other
Pacific Islander
Multiracial
Other (please specify)
6. Are you currently, or have you ever been, licensed to practice psychology?
Yes
No
1. If yes, what year did you first obtain licensure?
III. INTERNSHIP SITE & PROGRAM INFORMATION
7. Is your internship program APA accredited at this time?
Yes
No
In Process
8. Which of the following best describes the setting of your internship program? (Please select
ONE from the list below.)
Armed Forces Medical Center
Child/Adolescent Psychiatric or
Pediatric
Community Mental Health Center
102
Consortium
Medical School
Prison or Correctional Facility
Private General Hospital
Private Outpatient Clinic
Private Psychiatric Hospital
Psychology Department
School District
State/County/Other Public Hospital
University Counseling Center
Veterans Affairs Medical Center
Other (please specify)
9. Which of the following best describes the predominant theoretical orientation(s) of your
internship program’s site? (Please select UP TO THREE from the list below.)
Behavioral
Biological
Cognitive Behavioral
Eclectic
Humanistic/Existential
Integrative
Interpersonal
Psychodynamic
Systems
Other (please
specify)
10. On average, how many trainees do you typically accept each year in each of the following
categories?
a. Practicum Students:
N/A
b. Pre-doctoral Interns:
N/A
c. Postdoctoral Interns:
103
N/A
11. Does your site offer a PRIMARY rotation with an emphasis in psychological testing?
Yes
No
12. How much is psychological testing and assessment emphasized within your internship
program?
Extremely emphasized
Strongly emphasized
Somewhat emphasized
Slightly emphasized
Not at all emphasized
13. How is training in psychological testing and assessment provided within your internship
program? (Please SELECT ALL that apply.)
A dedicated assessment rotation
Across multiple rotations
Didactic seminars/training sessions
Structured trainings that yield certifications (e.g., with certified trainers)
Individual/one-on-one
Other (please specify)
14. How is supervision of psychological testing and assessment provided within your internship
program? (Please SELECT ALL that apply.)
Individual Supervision
Group Supervision
104
Other (please specify)
15. What functions do psychological testing and assessment serve at your internship site?
(Please SELECT ALL that apply.)
Psychoeducation
Differential diagnosis
Treatment planning
Monitoring response to treatment
Assessing treatment outcome
As a therapeutic intervention
Disability determinations
For accommodations/to access special programs
Research purposes
Other (please specifiy)
16. How important is clinical experience in psychological testing when selecting interns for
your program?
Extremely important
Very important
Somewhat important
105
Slightly important
Not at all important
17. How important is knowledge about psychological testing (gained from coursework and/or
didactic training) when selecting interns for your program?
Extremely important
Very important
Somewhat important
Slightly important
Not at all important
18. How satisfied are you with incoming interns’ level of clinical experience in psychological
assessment?
Extremely satisfied
Very satisfied
Somewhat satisfied
Slightly satisfied
Not at all satisfied
19. How satisfied are you with incoming interns’ level of theoretical knowledge about
psychological assessment?
Extremely satisfied
Very satisfied
Somewhat satisfied
Slightly satisfied
Not at all satisfied
20. How satisfied are you with incoming interns’ level of preparation for conducting
psychological assessment with diverse populations?
106
Extremely satisfied
Very satisfied
Somewhat satisfied
Slightly satisfied
Not at all satisfied
IV. PSYCHOLOGICAL TESTS AND MEASURES USED BY YOUR INTERNS
21. In your internship program, which of the following measures do interns use? (Please
SELECT ALL that apply)
COGNITIVE FUNCTIONING
Wechsler Intelligence Scales
(WAIS-IV, WISC-IV/V)
Stanford-Binet 5
TONI-3
Kaufman Assessment Battery for
Children (KABC)
SYMPTOM INVENTORIES
Beck Depression Inventory, 2nd
Edition (BDI-II)
Hamilton Depression Scale
Beck Anxiety Inventory (BAI)
Adult Manifest Anxiety Scale
DIAGNOSTIC INTERVIEW PROTOCOLS
SADS
SCID
DIS
NEUROPSYCHOLOGICAL
FUNCTIONING
Boston Diagnostic Aphasia Exam
Brief Rating Scale of Executive
Function (BRIEF)
Dementia Rating Scale-II
California Verbal Learning Test
Continuous Performance Test
Delis Kaplan Executive Function
System
Rey-Osterrieth Complex Figure
Bender Gestalt
Trail Making Test A & B
Wechsler Memory Scale III
Wide Range Assessment of Memory
and Learning
Wisconsin Card Sorting Test
EMOTIONAL FUNCTIONING
Millon Clinical Multiaxial Inventory,
3rd Edition (MCMI-III)
Minnesota Multiphasic Personality
Inventory, 2nd Edition (MMPI-2)
MMPI-2-Restructured Form
(MMPI2-RF)
Personality Assessment Inventory
Rorschach Inkblot Method
Rorschach Performance Assessment
System (R-PAS)
Thematic Apperception Test
Sentence Completion Test
Drawings (DAP, HTP, KFD, etc.)
NEO Personality Inventory-Revised
107
(NEO-PI-R)
ACADEMIC FUNCTIONING
Strong Interest Inventory
Wechsler Individual Achievement
Test (WIAT)
Woodcock Johnson-III
(Achievement; Cognitive)
Wide Range Achievement Test, 4th
Edition (WRAT-4)
FORENSIC/RISK ASSESSMENT
Psychopathy Checklist-Revised
(PCL-R)
Static 99
Violence Risk Assessment Guide
(VRAG)
History-Clinical-Risk 20 (HCR-20)
Validity Indicator Profile
Structured Interview of Reported
Symptoms (SIRS)
Miller Forensic Assessment of
Symptoms Test (M-FAST)
Rey 15- Item Test
(Please select up to 10)
COGNITIVE FUNCTIONING
Wechsler Intelligence Scales
(WAIS-IV, WISC-IV/V)
Stanford-Binet 5
TONI-3
Kaufman Assessment Battery for
Children (KABC)
SYMPTOM INVENTORIES
Beck Depression Inventory, 2nd
Edition (BDI-II)
Hamilton Depression Scale
Beck Anxiety Inventory (BAI)
Adult Manifest Anxiety Scale
DIAGNOSTIC INTERVIEW PROTOCOLS
SADS
SCID
DIS
NEUROPSYCHOLOGICAL
FUNCTIONING
Boston Diagnostic Aphasia Exam
Brief Rating Scale of Executive
Function (BRIEF)
Dementia Rating Scale-II
California Verbal Learning Test
Continuous Performance Test
Delis Kaplan Executive Function
System
Rey-Osterrieth Complex Figure
Bender Gestalt
Trail Making Test A & B
Wechsler Memory Scale III
Wide Range Assessment of Memory
and Learning
Wisconsin Card Sorting Test
EMOTIONAL FUNCTIONING
Millon Clinical Multiaxial Inventory,
3rd Edition (MCMI-III)
Test of Memory Malingering
(TOMM)
22. Please identify the measures most frequently used by interns at your internship program?
108
Minnesota Multiphasic Personality
Inventory, 2nd Edition (MMPI-2)
MMPI-2-Restructured Form
(MMPI2-RF)
Personality Assessment Inventory
Rorschach Inkblot Method
Rorschach Performance Assessment
System (R-PAS)
Thematic Apperception Test
Sentence Completion Test
Drawings (DAP, HTP, KFD, etc.)
NEO Personality Inventory-Revised
(NEO-PI-R)
ACADEMIC FUNCTIONING
Strong Interest Inventory
Wechsler Individual Achievement
Test (WIAT)
Woodcock Johnson-III
(Achievement; Cognitive)
Wide Range Achievement Test, 4th
Edition (WRAT-4)
FORENSIC/RISK ASSESSMENT
Psychopathy Checklist-Revised
(PCL-R)
Static 99
Violence Risk Assessment Guide
(VRAG)
History-Clinical-Risk 20 (HCR-20)
Validity Indicator Profile
Structured Interview of Reported
Symptoms (SIRS)
Miller Forensic Assessment of
Symptoms Test (M-FAST)
Rey 15- Item Test
109
Test of Memory Malingering
(TOMM)
23. Please indicate which measures you prefer your interns to have had clinical experience
with before starting internship? (Please SELECT ALL that apply.)
COGNITIVE FUNCTIONING
Wechsler Intelligence Scales
(WAIS-IV, WISC-IV/V)
Stanford-Binet 5
TONI-3
Kaufman Assessment Battery for
Children (KABC)
SYMPTOM INVENTORIES
Beck Depression Inventory, 2nd
Edition (BDI-II)
Hamilton Depression Scale
Beck Anxiety Inventory (BAI)
Adult Manifest Anxiety Scale
DIAGNOSTIC INTERVIEW PROTOCOLS
SADS
SCID
DIS
NEUROPSYCHOLOGICAL
FUNCTIONING
Boston Diagnostic Aphasia Exam
Brief Rating Scale of Executive
Function (BRIEF)
Dementia Rating Scale-II
California Verbal Learning Test
Continuous Performance Test
Delis Kaplan Executive Function
System
Rey-Osterrieth Complex Figure
Bender Gestalt
Trail Making Test A & B
Wechsler Memory Scale III
Wide Range Assessment of Memory
and Learning
Wisconsin Card Sorting Test
EMOTIONAL FUNCTIONING
Millon Clinical Multiaxial Inventory,
3rd Edition (MCMI-III)
Minnesota Multiphasic Personality
Inventory, 2nd Edition (MMPI-2)
MMPI-2-Restructured Form
(MMPI2-RF)
Personality Assessment Inventory
Rorschach Inkblot Method
Rorschach Performance Assessment
System (R-PAS)
Thematic Apperception Test
Sentence Completion Test
Drawings (DAP, HTP, KFD, etc.)
NEO Personality Inventory-Revised
(NEO-PI-R)
ACADEMIC FUNCTIONING
Strong Interest Inventory
Wechsler Individual Achievement
Test (WIAT)
Woodcock Johnson-III
(Achievement; Cognitive)
Wide Range Achievement Test, 4th
Edition (WRAT-4)
FORENSIC/RISK ASSESSMENT
Psychopathy Checklist-Revised
110
(PCL-R)
Static 99
Violence Risk Assessment Guide
(VRAG)
History-Clinical-Risk 20 (HCR-20)
Validity Indicator Profile
Structured Interview of Reported Symptoms (SIRS)
Miller Forensic Assessment of Symptoms Test (M-FAST)
Rey 15- Item Test
Test of Memory Malingering
(TOMM)
111
V. FUTURE DIRECTIONS OF PSYCHOLOGICAL ASSESSMENT
24. Currently, which methods of administration and scoring are typically used within your site?
(Please SELECT ALL that apply)
Traditional paper-based test administration
Traditional hand scoring
Computer-based test administration
Computer-based test scoring
Computer based test result interpretation
Tablet-based assessment (e.g., IPAD)
App-based assessment (e.g., on a smartphone or tablet)
Other (please specify)
25. How significant is the use of technology in the training and practice of psychological
assessment within your internship program?
Extremely important
Very important
Somewhat important
Slightly important
Not at all important
26. In the next five years, what do you expect regarding funding and resources for
psychological testing and assessment in your internship program?
Significant increase in funding/resources
Slight increase in funding/resources
No change in funding/resources
Slight decrease in funding/resources
Significant decrease in funding/resources
27. In the future, how do you expect your internship program’s emphasis on psychological
testing and assessment to change?
112
Significantly increase
Slightly increase
Stay the same
Slightly decrease
Significantly decrease
113
28. How much has the profession’s emphasis on evidence-based practice impacted your
program’s approach to psychological testing and assessment?
Extremely impacted
Strongly impacted
Somewhat impacted
Slightly impacted
Not impacted at all
29. What new psychological tests or measures has your site begun using within the last five
years?
None
30. Within your site, what psychological tests or measures would you like to see used in the
future that are not currently being used?
None
31. What recommendations do you have for academic programs regarding pre-internship
training in psychological testing and assessment?
None
114
32. Please add anything else you would like to offer regarding psychological assessment training
and practice at the internship level that was not covered in this survey.
None
Thank you for participating in this study!”
115
APPENDIX B
Group Coding by q8a for Data Analysis
116
Group Coding by q8a for Data Analysis
Please note the change in coding for the data below:
Original Code
Setting
2
Consortium Programs (CON)
7
Prison and/or Correctional Facilities (PC)
13
State/County/Other Public Hospital (SCPH)
14
University Counseling Centers (UCC)
15
Department of Veteran Affairs Medical Centers (VAMC)
20
Coding by q8
Community Mental Health Centers (CMHC)
q8 = 2
Consortium Programs (CON)
q8 = 7
Prison and/or Correctional Facilities (PC)
q8 = 13
State/County/Other Public Hospital (SCPH)
q8 = 14
University Counseling Centers (UCC)
q8 = 15
Department of Veteran Affairs Medical Centers (VAMC)
q8 = 20
Coding by q8a1s
Community Mental Health Centers (CMHC)
q8a 6
Consortium Programs (CON)
q8a 5
Prison and/or Correctional Facilities (PC)
q8a 4
State/County/Other Public Hospital (SCPH)
q8a 3
University Counseling Centers (UCC)
q8a 2
Department of Veteran Affairs Medical Centers (VAMC)
1
The settings were re-coded as “q8a” to perform the statistical analysis and as reflected in the subsequent
appendices. Each was assigned a number, not representational of a numerical value.
117
q8a 1
Community Mental Health Centers (CMHC)
APPENDIX C
Wilcoxon Scores (Rank Sums) for Variables
118
Wilcoxon Scores (Rank Sums) for Variables “Q” Classified by Variable q8a
Q q8a N Sum of Expected Std Dev Mean Score
1
Scores Under H0 Under H0
20 6 14 1101.00 875.00 112.245408 78.642857
5 14 1100.00 875.00 112.245408 78.571429
4 18 969.00 1125.00 124.938822 53.833333
3 27 1702.50 1687.50 146.378055 63.055556
2 27 1271.50 1687.50 146.378055 47.092593
1 24 1606.00 1500.00 140.124425 66.916667
28 6 14 860.50 875.00 119.955563 61.464286
5 14 1012.50 875.00 119.955563 72.321429
4 18 1440.00 1125.00 133.520890 80.000000
3 27 1367.00 1687.50 156.432787 50.629630
2 27 1673.00 1687.50 156.432787 61.962963
1 24 1397.00 1500.00 149.749594 58.208333
1
Average scores were used for ties.
119
APPENDIX D
Distribution of Wilcoxon Scores
120
Distribution of Wilcoxon Scores
Survey Item 20
121
Distribution of Wilcoxon Scores
Survey Item 28
APPENDIX E
122
Kruskal-Wallis Test
123
Kruskal-Wallis Test
Item Chi-Square DF Pr > Chi-Square
20 15.2813 5 0.0092
28 9.6082 5 0.0871
124
APPENDIX F
Group Comparisons
125
Group Comparisons
Questionnaire Item 201
Group
Group Comparison
Differences in
Cutoff at
Significant
Comparison
by q8a
by Setting
Average Ranks
Alpha =0.05
Difference
1-2
CMHC-VAMC
19.8241
29.5945
1-3
CMHC-UCC
3.8611
29.5945
1-4
CMHC-State/Public
13.0833
32.8924
1-5
CMHC-Prison/Correction
11.6548
35.4760
1-6
CMHC-Consortium
11.7262
35.4760
2-3
VAMC-UCC
15.9630
28.7108
2-4
VAMC-State/Public
6.7407
32.0997
2-5
VAMC-Prison/Correction
31.4788
34.7423
2-6
VAMC-Consortium
31.5503
34.7423
3-4
UCC-State/Public
9.2222
32.0997
3-5
UCC-Prison/Correction
15.5159
34.7423
3-6
UCC-Consortium
15.5873
34.7423
4-5
State/Public-
Prison/Correction
24.7381
37.5913
4-6
State/Public-Consortium
24.8095
37.5913
5-6
Prison/CorrectionConsortium
0.0714
39.8716
1
Questionnaire item 25: Chi-Square=15.281; DF=5; Pr>Chi-Square=0.0092
126
Group Comparisons
Questionnaire Item 281 (RE ENTER)
Group
Group Comparison Differences in Cutoff at
Significant
Comparison
by q8a
by Setting Average Ranks Alpha =0.05
Difference
1-2
CMHC-VAMC
3.7546
29.5945
1-3
CMHC-UCC
7.5787
29.5945
1-4
CMHC-State/Public
21.7917
32.8924
1-5
CMHC-Prison/Correction
14.1131
35.4760
1-6
CMHC-Consortium
3.2560
35.4760
2-3
VAMC-UCC
11.3333
28.7108
2-4
VAMC-State/Public
18.0370
32.0997
2-5
VAMC-Prison/Correction
10.3585
34.7423
2-6
VAMC-Consortium
0.4987
34.7423
3-4
UCC-State/Public
29.3704
32.0997
3-5
UCC-Prison/Correction
21.6918
34.7423
3-6
UCC-Consortium
10.8347
34.7423
4-5
State/Public-
Prison/Correction
7.6786
37.5913
4-6
State/Public-Consortium
18.5357
37.5913
1
Questionnaire item 28: Chi-Square=9.6082; DF=5; Pr>Chi-Square=0.0871
127
5-6
Prison/CorrectionConsortium
10.8571
39.8716
