Personality Tests in the Workplace

Art3_CPI.pdf

Home >Human Resource Management homework help >Personality Tests in the Workplace

Personality-Based Profile Matching in Personnel Selection: Estimates of Method Prevalence and

Criterion-Related Validity

John T. Kulas* Saint Cloud State University, USA

Profile matching refers to selection based on applicant similarity to a pre- specified pattern of standing across several mutually considered personality dimensions. Although many investigations support the use of personality data through univariate, linear-based selection methodologies, there is no evidence within the literature that supports (or refutes) the use of profile matching. Regardless, a phone survey revealed that 62 per cent of consultative vendor organisations implement some form of profile matching. The current study addresses this scientist–practitioner void by investigating the broad, cross- organisational viability of three different profile matching strategies (profile band specification, profile similarity estimation, and configural scoring). Although some specifications of profile matching came close (empirically) to challenging linear regression cross-validation estimates, the profile matching strategy is considered to be burdened with additional conceptual concerns (primarily resulting from a lack of formal model specification) as well as prac- tical limitations (for example, the likely creation of an artificial predictor ceiling). Linear regression is presented here as the more effective use of multi- trait information; however, if practitioners continue to utilise profile matching, it is suggested that they consider either adopting a configural scoring approach or referencing an index of profile similarity rather than retaining and applying desired profile bands.

INTRODUCTION

Personality assessment has a rich tradition of application within organisa- tional selection contexts. In addition to the perpetual stream of local valida- tion studies, several meta-analyses have investigated the generalisable relationship between personality and performance (i.e. Barrick & Mount, 1991; Judge, Bono, Ilies, & Gerhardt, 2002; Mount, Barrick, & Stewart,

* Address for correspondence: John T. Kulas, Department of Psychology, Saint Cloud State University, St Cloud, MN, USA. Email: jtkulas@stcloudstate.edu

The author would like to thank Briana Olson for conducting the phone survey, the phone survey respondents for their participation, and CPP, Inc. and the Center for Creative Leadership for providing access to the CPI and 360° data used in this research.

bs_bs_banner

APPLIED PSYCHOLOGY: AN INTERNATIONAL REVIEW, 2013, 62 (3), 519–542 doi: 10.1111/j.1464-0597.2012.00491.x

1998). This line of investigation has been so heavily researched that a second- order “meta” meta-analysis was performed (Barrick, Mount, & Judge, 2001) with the authors recommending that no further meta-analyses be conducted. This moratorium has not been observed (see, for example, Dudley, Orvis, Lebiecki, & Cortina, 2006; Hogan & Holland, 2003), suggesting that interest in personality–performance relationships is strong and continuing.

Within this literature, investigations consider outcome-based associations independently across personality dimensions (e.g. Barrick et al., 2001; Dudley et al., 2006; Hogan & Holland, 2003; Judge et al., 2002; Mount et al., 1998). Generally, through the lens of the five-factor model (FFM), it has been well established that Conscientiousness is moderately associated with perfor- mance across job applications, with other traits exhibiting relationships largely dependent on context or criteria (e.g. Barrick & Mount, 1991; Barrick et al., 2001). Although the literature is replete with traditional univariate,1

linear-oriented validation studies and meta-analyses, in practice personality information is also being applied in a “multivariate” manner. An example of this multivariate application (in a personnel selection context) is referred to here as profile matching. Through this strategy, applicants are not viewed independently (or incrementally) through the lens of individually considered personality traits but are rather evaluated based on congruence to a desired multi-trait template.

This profile matching approach to selection is presented in some personnel textbooks (e.g. Gatewood & Feild, 2001; Landy & Trumbo, 1980) but is absent from broad practice-based guidelines (i.e. Principles for the Valida- tion and Use of Personnel Selection Procedures [Principles]; Society for Industrial and Organizational Psychology, 2003) as well as the selection literature. Some published studies have used the term “profile” (i.e. Cook, Young, Taylor, O’Shea, Chitashvili, Lepeska, Choumentauskas, Vents- kovsky, Hermochova, & Uhlar, 1998; Detrick & Chibnall, 2006), but none has assessed the viability of selection-oriented profile matching. Rather, the term profile (as presented in the empirical literature-base) represents a descriptive reference to several independently validated scales (see, for example, Detrick & Chibnall, 2006), and is therefore concerned with tradi- tional univariate applications and not the procedural use of the term that is the focus of the current article.

This paper takes an initial step toward investigating the broader practice of profile matching by documenting estimates of regularity of application and criterion-related validity. The usage information is specifically addressed through a phone survey of selection-oriented vendors, while the validity of

1 The use of the term “univariate” here refers to a hiring decision-based focus on one personality dimension (even though the procedure implemented to collect validity evidence may have been, for example, bivariate).

520 KULAS

the methodology is assessed through a concurrent design, investigating 360° rating associations across three different procedural applications of profile matching. These three profile matching strategies are contrasted with a linear regression model through comparing single sample cross-validation esti- mates of hit, miss, correct rejection, and false alarm rates.

Predictive Selection Models

The traditional orientation toward the modeling of predictor–criterion asso- ciations is “more is better” (e.g. Barrick et al., 2001; Dudley et al., 2006; Hogan & Holland, 2003; Judge et al., 2002; Mount et al., 1998). In the simplest case, the magnitude of relationship between one predictor and one criterion is estimated through specification of a linear (commonly least squares) function (see, for example, Coward & Sackett, 1990; Society for Industrial and Organizational Psychology, The Principles, 2003). This asso- ciation is formally defined through the general linear model (GLM). The goal within the GLM is to maximise prediction (i.e. criterion variance). When patterns of relationship can be better described by nonlinear associations, nonlinear functions can be specified and estimated from within the GLM (through, for example, predictor transformations).

When combining information from more than one predictor to inform a selection decision, selection specialists have choices. The GLM can be retained and expanded to accommodate multiple predictors. Such a specifi- cation allows for the estimation of shared and unique predictor contributions to criterion variance. This GLM approach represents a compensatory model, whereby one low predictor score’s impact on a final selection decision can be overcome through obtaining higher scores on other predictors. Alternatively, a cut-score (non-compensatory) based integration procedure can be imple- mented, whereby a minimum standard of performance must be obtained for each predictor.

Gatewood and Feild (2001) provide five possibilities for the incorporation of more than one predictor in a selection context: (1) multiple regression, (2) multiple cutoffs, (3) multiple hurdle, (4) a combination of multiple cutoff and regression, and (5) profile matching. The first four are methodological speci- fications of the compensatory and non-compensatory GLM and cut-score procedures. The fifth procedure (profile matching) represents a creative application of predictor–criterion associations that does not fit neatly within either the GLM or cut-score models. Selection here is accomplished through identifying a desired (good performer) multitrait profile across predictors, and then selecting applicants based on closeness of fit to this template.

In addition to deviation from the GLM and cut-score models, the simple identification of a profile template would appear to be characterised by an atypical measurement model—one that can perhaps be best described as an

PROFILE MATCHING 521

extreme form of operationalism (whereby the object of measurement is fully defined by its process of measurement; see, for example, Bickhard, 2001). Because the goal of profile specification is to capture the unique pattern of construct interrelationships that characterise an identifiable group (for example, a good performer group), the procedure is presented here as being more consistent with the concept of “typing” (e.g. Jung, 1971) than to graded trait models of individual difference. Note that the defining characteristic of a personality profile is its focus on intra-individual differences (e.g. relative within-person trait score standing across multiple constructs) that are believed to collectively identify a unique person or category of people. This focus is not entirely novel, as individual difference researchers have histori- cally vacillated somewhat between emphasising the importance of intra- versus inter-person characteristics. Allport (1962), for example, noted that the focus of personality researchers on common dimensions of inter- individual differentiation may be ignoring something important—the within- person patterns that define uniqueness within an individual.

Allport’s interest was in advancing “morphogenic” approaches to person- ality that focus on individuality, and he believed that intra-individual com- parisons were key in this pursuit. Selection-oriented profile matching, it would seem, is an extension of this focus and an attempt to capture this same characteristic “uniqueness” (e.g. what is the specific combination of charac- teristics that collectively makes a “good manager” a good manager?). It is focused on the internal pattern or structure of personality. This is contrasted with generally espoused personnel selection practices, whereby “more” of an identified job-relevant construct (such as general cognitive ability or Consci- entiousness) is taken as a probable indicator of inter-individual differences in performance, and predictor interactions are the exception rather than the rule.

In summary, it is the contention of this paper that profiling entails an implicit creation of a superordinate formative categorical construct or “type” in the form of a desired pattern of multitrait standing across personality constructs. This represents a mutual consideration of intra- and inter- personal differences. The intra-individual focus entails a candidate being characterised by, for example, more Conscientiousness than Extraversion. The inter-individual consideration entails both aggregation of individual good performer profiles into a group template as well as a comparison of an individual candidate’s multitrait profile to the pre-identified desired template. In the context of personality assessment, it is unclear what construct the measured profile is identifying, and it is considered to therefore represent an extreme form of operationalism. Although this procedure deviates from the traditional procedural, measurement, and analysis model specifications in a selection context, the general procedure of developing and interpreting assessment profiles does have a well-documented history within clinical and educational assessment domains.

522 KULAS

Theoretical and Historical Specification of Assessment Profiles

The original focus on assessment profiles can be traced perhaps to Meehl (1950), who noted that patterns are extremely important for clinicians, refer- ring to the “configural character of clinical material” (p. 165). The basic premise behind Meehl’s perspective is that the mutual consideration of mul- tiple pieces of evidence can provide information where no information is yielded if the pieces are considered independently (sometimes referred to as Meehl’s paradox; Janarone & Roberts, 1984). The context of his discussion is the discrimination of groups—if individual items lack discriminating power, patterns of responses across items may prove more effective. Although Meehl (1950) presented the original configural scoring idea with items, he later extended this general idea to scales (e.g. Meehl & Dahlstrom, 1960). Within this approach an individual is assigned to a diagnostic category based on mutual consideration of multiple pieces of information—collectively identi- fied as a pattern or profile.

Livingston, Jennings, Reynolds, and Gray (2003) have more recently noted that “it is common practice in psychological assessment for clinicians to consider not only individual scores when interpreting tests, but also profiles containing multiple scores” (p. 488). In addition to common application within the intelligence testing domain (e.g. especially the Weschler intelli- gence scales, see for example, Glutting, McDermott, Watkins, Kush, & Konold, 1997), profiles have also been developed and applied to the Minne- sota Multiphasic Personality Inventory (MMPI). Harkness and McNulty (2006), for example, demonstrate MMPI-2 profile interpretation, presenting a fictional profile characterised by three elevated clinical scales, one elevated supplementary scale, and a low scale 0, noting that “. . . this combination of scores is associated with acting out, impulsive behavior” (p. 97). Similar multi-scale considerations are made to identify diagnostic groups in a relative scale elevation sense. For example, Sullivan and Welsh (1952) refer to the “psychosomatic V” which is a pattern of scores across three MMPI scales (Hs and Hy elevated relative to the D scale [which is located between the Hs and Hy scales in the MMPI report]). This pattern is recognised as being common in individuals who are classified as psychosomatic.

The rationale across these applications is that a summary behavioral or cognitive type can be identified through consideration of patterns of standing across multiple scales. The extrapolation of this general approach to personnel-based profile matching is logical—the specification of a pattern of responses may identify groups (such as, for example, good leaders) better than individual (and incremental) consideration of graded scale scores across predictors. The orientation of practitioners employing a form of profile matching in a selection context would therefore seem to be consistent with

PROFILE MATCHING 523

Meehl’s configural scoring perspective. However, Meehl’s presentation is isolated to the case in which an individual item or scale exhibits zero validity and the end goal is diagnostic classification. The personnel selection context is more typically characterised by predictors with some predictive power and criteria that reflect graded differentiation (for example, beyond gross cat- egorisation as psychosomatic, impulsive, or a “good” performer). The pro- priety of application of Meehl’s orientation to the personnel selection context is therefore considered here to be unknown.

Selection-Oriented Viability of Profile Specification

The current paper takes the perspective that profile matching introduces unique conceptual complications into the practice of personnel selection. These include a lack of specification regarding: (1) what it is that is being measured, (2) the properties of its measurement (e.g. error disturbances), and (3) the appropriate procedural applications of the technique. Although these conceptual issues could potentially be resolved, the practical concern with the approach is that it either fails to fully utilise criterion-associated gradations (e.g. all “good manager” individuals are considered equal for template definition) and/or make full appropriate use of bivariate predictor associations (e.g. profile inclusion of a linearly valid construct such as Con- scientiousness may neutralise the construct’s full predictive utility). Regard- less of individual component validities, there is a (likely) artificial ceiling induced by the method, whereby full congruence to the template is “as good as is possible” regarding a candidate’s predictor responses. There is no room within this method for applicants to “improve” beyond the iden- tified template.

In addition to the above conceptual and practical concerns, the profile matching selection strategy may be at odds with other organisational goals such as increasing organisational member diversity (e.g. Glomb & Welsh, 2005; Mohammed & Angell, 2003; Sackett, 2005). Schneider’s (1987) attraction-selection-attrition (ASA) model suggests a tendency for organisa- tions to develop member homogeneity over the course of time through various innocuous and hidden mechanisms. This effect may be so broad as to suggest a shared personality that is unique to an organisation (called a modal personality; Schneider, Smith, Taylor, & Fleenor, 1998).

The use of profile matching strategies would seem to represent a more aggressive and overt manifestation of increasing member homogeneity— selection based directly on similarity to an incumbent multitrait profile. While acknowledging that the possible reasons for member homogeneity are largely speculative, Schneider et al. (1998) note that “research on the ways such a combination of personal and organizational influences on personality homogeneity operates is warranted” (p. 467). Certainly the practice of speci-

524 KULAS

fying a particular desired personality profile represents an additional con- tributor to the development and sustainment of an organisation’s “modal” personality (and workforce homogeneity). In addition to being characterised by the aforementioned conceptual and practical issues, the application of the profile matching procedure may also therefore be at odds with other organi- sational pursuits (such as increasing member diversity).

Summary and Hypotheses

The specification and use of assessment profiles is fairly common in clinical and educational applications (e.g. Glutting et al., 1997; Livingston et al., 2003; Konold, Glutting, McDermott, Kush, & Watkins, 1999). In these contexts, however, the focus of assessment tends to be on the interpretation of profiles to provide diagnostic insight and the profile components are most commonly subtests of an identifiable overarching superordinate con- struct or type classification. In organisational contexts, intra-individual dif- ferences have most commonly been addressed in training and development applications. Selection has traditionally focused on inter-individual differ- ences. The current paper takes the position that the pursuit of applicant profile similarity within a selection context represents a mutual consider- ation of inter- and intra-individual differences and a tacit acknowledgement of “types” (i.e. the unique pattern of traits is both defined by and ultimately applied as a template for a good applicant or incumbent “type”). Collec- tively the multi-trait profile defines an “ideal candidate”. More of that profile (e.g. elevation) does not implicate a better candidate—any deviation from the template is indicative of incongruence with the identified ideal. Scores cannot be improved beyond a perfect match to this idealised tem- plate and it is therefore presented as introducing a (likely artificial) predic- tor ceiling (e.g. regardless of the objective adequacy of the template or nature of any predictor’s demonstrable bivariate relationship with the desired criterion, profile specification is the “best” that can be achieved). From a selection model perspective, it represents a shift from a linear, univariate model to an unspecified interactive nonlinear or multivariate predictive model.

Given these conceptual and practical concerns as well as the absence of attention given the topic within the selection literature, this current inves- tigation represents an initial attempt to evaluate both the frequency of practitioner application and the broader empirical efficacy of the profile matching approach. Questions regarding the prevalence of profile matching have not previously been attempted and are assessed exploratively (without specific hypotheses). Regarding validity, it is hypothesised that linear-based methods provide a more effective specification of personality scale infor- mation (than does profile matching). To test this premise, a series of four

PROFILE MATCHING 525

cross-validation investigations are undertaken to document the relation- ships between personality information specification and resulting hit, miss, false alarm, and correct rejection rates for three profile matching and one linear regression procedure.

PHONE SURVEY

Methods

Participants. One hundred and twenty individuals were contacted via telephone.

Procedure. Potential vendor organisations were identified through advertisements in the Society for Industrial and Organizational Psychology’s (SIOP) conference bulletins and a broader internet-based search for such organisations with headquarters within the United States. The internet search was limited to organisations offering psychological assessment-based consulting services. Together, these searches resulted in the identification of 144 organisations. Names and phone numbers were collected (when possible) based on product- or research-oriented job titles. Contact was eventually made with representatives of 120 of the 144 identified organisations.

Phone respondents were asked if their organisation used personality assessment, what percentage of clients who use personality information employ some form of profile matching, at what levels are profiles developed and used, how the profile matching is used, the staffing of their organisation (percentage I/O background, percentage non-I/O background, percentage clinical background), and the size of their organisation. If a representative was unavailable or did not answer the phone, they were phoned again (up to three times).

The definition of profile matching, if prompted by the phone participant was:

Profile matching is selection based on similarity with a desired pattern of person- ality traits. It is contrasted with a linear selection strategy, where more of a single trait is sought. Profile matching may result in a high score on one trait, moderate score on another, and low score on a third being the desired applicant profile. Selection here is a function of how closely an applicant fits a profile.

Results

Of the 120 individuals contacted, 15 declined to participate in the survey. Representatives from 55 of the 105 remaining organisations claimed that

526 KULAS

their organisation did not use personality assessment in a selection context.2

Fifty of the identified organisations with representatives willing to partici- pate did indicate that their organisation used personality assessment in a selection context. Thirty-one of these 50 vendor organisations (62.0%) implemented profile matching strategies with at least some clients. The average per cent of client organisations using profile matching for these vendors was 75.9 per cent (s = 26.2%). The majority of vendors (17 of 31; 54.8%) that used profile matching strategies applied them to both executive and non-executive positions, with five vendors using profiles exclusively at the executive level and nine vendors using profiles exclusively at the non- executive level.3

Information regarding the development of the profiles was shared by only 12 representatives. Although the most commonly stated procedure was to develop a profile based on identified top performers, profiles were also devel- oped through subject matter expert and hiring manager input, “job analysis”, and direct comparison of top performer trait scores with either a norm group or identified bottom performer trait scores. Only two representatives volun- teered information regarding the specific form of their profiles, with both of these individuals stating that profiles were defined by means and standard deviations (this specific information was not explicitly assessed, and was only recorded if volunteered by the contact representative). In summary, although reference to the procedure is absent within the selection literature, the prac- tice is being applied by a majority of US vendor organisations that use personality information for selection purposes.

VALIDITY ANALYSES

Methods

Participants. Twelve thousand and forty-one Center for Creative Lead- ership program participants completed 360° evaluations and/or personality self-assessments between January 2004 and September 2007. The multi- source ratings consisted of self (n = 11,916), boss (i.e. immediate supervisor; n = 11,535), superior (n = 8,586), peer (n = 43,775), direct report (n = 43,294), other (n = 18,866), and unknown (n = 21) rater groups. Eleven thousand seven hundred and fifty individuals shared both 360° ratings of their performance as well as personality assessment information. The average age of these

2 A significant percentage of these organisations can be classified as non-selection-based consultative organisations—the initial search that identified 144 organisations was purposefully broad, with the phone screen intended to further narrow the sample.

3 The script question asked: “At what organisational levels are profiles developed and used (for example, executive or non-executive?)”, which in retrospect was leadingly restrictive.

PROFILE MATCHING 527

11,750 respondents was 42.9 years. Most individuals were male (66%) and Caucasian (73.3% [African American = 5.7%, Asian or Pacific Islander = 4%, Hispanic = 3%]). Eight thousand eight hundred and thirty-five participants’ (75.2%) country of origin was the United States, with 128 other countries of origin being represented in the sample. The current location of the majority of these individuals was also the United States (n = 9,638; 82.0%), with the remaining individuals residing, at the time of assessment, in one of 107 different countries.

Materials. The CPI 260TM (Gough & Bradley, 1996) assesses 26 folk- concept scales across 260 dichotomously scored items. The CPI is used for selection purposes by many Fortune 500 companies, large local governments, the Federal Government, as well a number of leadership development organisations (R. Thompson, personal communication, 29 July 2011). It was chosen in the current study as a matter of convenience (the information was obtainable) as well as practicality (the large number of scales [relative to, for example, an FFM measure] allowed for flexibility with regard to the scope of profile specification) and history of documented validity within both Ameri- can and international managerial contexts (e.g. Anderson & Tett, 2009; Miller, Watkins, & Webb, 2009; Patwardhan, 2004). The 26 scales are: Domi- nance (Do), Capacity for Status (Cs), Sociability (Sy), Social Presence (Sp), Self-acceptance (Sa), Independence (In), Empathy (Em), Responsibility (Re), Social Conformity (So), Self-control (Sc), Good Impression (Gi), Commu- nality (Cm), Well-being (Wb), and Tolerance (To), Achievement via Con- formance (Ac), Achievement via Independence (Ai), Conceptual Fluency (Cf), Insightfulness (Is), Flexibility (Fx), Sensitivity (Sn), Managerial Poten- tial (Mp), Work Orientation (Wo), Creative Temperament (Ct), Leadership (Lp), Amicability (Ami), and Law Enforcement Orientation (Leo). Assess- ment respondents receive feedback relative to a normed sample of 6,000 (3,000 male, 3,000 female) respondents.

The Center for Creative Leadership’s Benchmarks® 360° instrument assesses 16 managerial competencies related to success and five possible areas of career derailment. The current study focused on Section One items (indi- cators of the 16 success competencies). The 115 Section One items aggregate to publisher-specified scales of: (1) Resourcefulness, (2) Doing whatever it takes, (3) Being a quick study, (4) Decisiveness, (5) Leading employees, (6) Confronting problem employees, (7) Building and mending relationships, (8) Compassion and sensitivity, (9) Straightforwardness and composure, (10) Balance between personal life and work, (11) Self-awareness, (12) Putting people at ease, (13) Differences matter, (14) Participative management, (15) Career management, and (16) Change management. Although these compe- tencies may be viewed as deviating from traditional indicators of “job per- formance”, it should be noted that the Benchmarks scales and items were

528 KULAS

developed based on CCL studies of factors that effectively differentiate suc- cessful and unsuccessful leaders (see, for example, Leslie & Peterson, 2011, for a comprehensive overview).

Procedure. Because there is no widely available reference for “how to” specify selection-oriented profiles, the current procedures were identified by speaking with selection-oriented practitioners, referencing selection text- books, and performing the historical literature review. The identified proce- dures grossly reflect one personnel-oriented application and two clinical- oriented applications (these “orientations” refer entirely to author opinion regarding procedural provenance). Although it is likely the profile matching strategy is being applied in numerous different ways, the focus of the current article is on a broad introduction to the investigation of profile matching practices, and these three were identified as either the most common or possessing the most historical documentation.

Overview of procedural models: Each of these methods is described again in greater detail within their respective results sections. The following is intended as a simplified overview of the procedural application of each approach. The first investigated (personnel selection-oriented) procedure entails the specification of mean and standard deviation bands across mea- sured traits. These means and standard deviations are derived from identified “top performers”. The second investigated (clinically oriented) procedure incorporates information from both top and bottom performers, and this approach is most consistent with Meehl’s specification of configural scoring (e.g. Meehl & Dahlstrom, 1960). Formally, it is known as the construction of “signed” profiles (e.g. Sullivan & Welsh, 1952). For the third investigated (clinically oriented) procedure, the profile is specified as in the personnel model (derived from identified top performers), but a graded coefficient of candidate similarity is produced (rather than a categorical determination of whether or not a candidate fits the desired template). For the final investi- gated (regression-oriented) procedure, a “criterion pattern” was identified through a modified regression application that permits conceptual compari- sons to profiles (e.g. Davison & Davenport, 2002).

To test the predictive effectiveness of each of these four procedures, the larger 11,750 person dataset was broken into eight separate data files (all ns = 1,468 or 1,469). Each of the procedures was then evaluated using two of these smaller files (one to develop a profile [referred to as a “calibration” file], and one to cross-validate the developed profile [referred to as a “validation” file]). As pointed out by an anonymous reviewer, this single-sample cross- validation method does not represent the most sophisticated procedural possibility with this data. It was done here as a passable but less-than failsafe guard for model overfit and generalisability of results. Across these proce- dures, the outcomes of interest were validation sample hit, miss, correct

PROFILE MATCHING 529

rejection, and false alarm rates using a trichotomisation of 360° assessment score as the criterion. Each method therefore yielded the percentage of clas- sifications falling into the four traditional signal detection theory-derived categories of focus, as well as those falling in between (because of the tri- chotomised criterion).4

Results

Principal axis factoring of the 360° items yielded one factor that accounted for 45.9 per cent of the shared item variance (across all 126,077 non-self respondents). The ratio of the first (52.8) to second (4.9; ratio = 10.8) eigen- values was substantially greater than the ratio of the second to third (2.1; ratio = 2.4) and subsequent associated factor eigenvalues. All 115 items were therefore used to construct a unit-weighted criterion score aggregate (com- prising other ratings). As a check of the split files’ similarity, a one-way ANOVA was conducted on the total Benchmarks rating across the eight calibration and validation files (F = .32, p > .05; h2 = 0.00) and a profile analysis was conducted across the 26 CPI scales and eight split files. Similar to the 360° score findings, the profile analysis yielded neither a main effect across files (i.e. levels effect, F = 1.03, p > .05; partial h2 = 0.00) nor a file by scale interaction (i.e. parallelism effect, Wilks’ L = .99, p > .05; partial h2 = 0.00). Based on these indices of file similarity, the eight files were retained for the four different investigations of criterion-related procedural validation.

Approach 1: Mean and SD Band Profiles. The prototype for personnel- based profile matching involves an identification of well-performing indi- viduals and developing an averaged personality profile thought to provide a descriptive template for this well-performing group. The current approach operationalised top perfomers as the highest 5 per cent of calibration file individuals on the Benchmarks instrument. Bands of plus and minus one standard deviation were constructed around scale means for these top per- formers and profile match was attained if validation file individuals’ person- ality scores fell within this profile band. Figure 1 shows this developed profile. Applying this template to the validation sample, no individuals had scores within the profile band across all 26 scales. Loosening the fit con- straints by approximately 25 per cent, individuals who fit 19 of the 26 profile dimensions yielded the non-differentiating contingency table presented in Table 1 ( χ2

2 1 00= . , p > .05). A similar modification was made with the width

4 These profiles were also developed and applied within managerial hierarchies of top execu- tive, upper middle, and middle managers, although the results of these analyses largely paralleled the omnibus effects. These position-specific analyses as well as estimates of race-based adverse impact are available from the author upon request.

530 KULAS

of the profile—using bands 1.5 standard deviations above and below com- ponent scale means resulted in 158 individuals fitting the full 26-component profile, but the associations with the trichotomised 360° score were again undifferentiated ( χ2

2 3 74= . , p > .05). Effectively there is no difference in categorisation of performance (as high, middle, or bottom) for either those who fit the loosened profiles or those who do not fit the profiles. Mean and standard deviation profile specification, with the current predictor data, is not effective in the dissociation of multisource ratings.

Approach 2: Sullivan and Welsh’s “Signed” Profiles. Both the Blum and Naylor (1968) and Landy and Trumbo (1980) selection texts criticise the lack of discriminant validity with the profile matching strategy (i.e. the identifi-

FIGURE 1. Desired profile band (defined by mean and standard deviation scores of the top 5 per cent Benchmarks performers within each of the 26 CPI scales).

TABLE 1 Top 5 per cent Profile Matching Prediction (Fitting 19 of 26 CPI Scales)

Benchmarks rating

Profile-based selection decision

Selected Not selected

Top third 208 (14.2%; H) 284 (19.3%; M) Middle third 204 (13.9%) 284 (19.3%) Bottom third 192 (13.1%; FA) 296 (20.2%; CR)

Note: H = Hit, M = Miss, FA = False Alarm, CR = Correct Rejection.

PROFILE MATCHING 531

cation of a top performer profile in the previous and following approaches does not necessarily indicate that poor performers deviate from the profile). Additionally, these authors (as well as two anonymous reviewers of an earlier draft of this paper) cite concerns regarding which predictors should be included in a developed profile. Application of the concept of “configural scoring” to the profile matching strategy is aimed at addressing these con- cerns. This approach retains profile shape information while ignoring differ- ences in elevation and retains only component information from scales that are shown to differentiate across identified groups.

Applying this procedure within the next calibration file, the top and bottom 5 per cent performers were identified, and these individuals’ assess- ment scores were subjected to 325 relative pair-wise comparisons. This was accomplished via the following procedure: For each identified (top or bottom) individual, simple comparisons of relative scale elevation were made (e.g. Is the Do score higher or lower than the Cs score? The Sy score? The Sp? and etcetera until all possible scale contrasts were recorded). Equal scale scores were then assigned a value of zero, a focal scale elevation was assigned a score of +1, and a focal score suppression was assigned a score of -1. T-tests were next conducted for each of the 325 pair-wise comparisons using these three ordinal “signed” values (across top and bottom performer groups), and comparisons were subsequently sorted by absolute magnitude of the t-scores. The contrasts with the largest magnitude t-scores provide the “profile”—these are the personality dimension contrasts (e.g. Do consis- tently higher than Cs) across which top performers deviate the most from low performers.

Within the validation file, fit was estimated for the 15, 20, and 25 greatest pair-wise contrasts identified via the above procedure. For example, Inde- pendence scale scores tended to be more elevated relative to Sensitivity scale scores across the identified top performers than across the identified bottom performers (d = .69). For a validation file individual to be considered as fitting the desired profile, the individual’s assessment scores would need to exhibit this relative difference (In > Sn) as well as 14, 19, or 24 other targeted scale comparisons. Splitting the criterion into upper, middle, and lower tertiles and applying these signs procedures within the validation sample results in the contingency tables reported in Table 2 (15-signs χ2

2 = 4.89, p > .05; 20-signs χ2

2 = 14.73 , p < .05, jc = .10; 25-signs χ2 2 = 6.46, p < .05). This signs profile

specification did show moderate 360° rating differentiation with the 20-signs profile exhibiting the largest effect size.

Approach 3: Cattell’s Coefficient of Profile Similarity (rp). Approach no. 3 is similar in specification to Approach no. 1, but rather than cat- egorically fitting or not fitting a profile, a graded index of association (from the desired profile) is generated. This coefficient of profile similarity is sen-

532 KULAS

sitive to discrepancies in profile shape (pattern across profile components), elevation (average component score), and scatter (sum of individual com- ponents’ deviation from the elevation estimate). The index is essentially distributed as a Pearson’s product-moment correlation, with values of 1 reflecting perfect profile agreement, -1 indicating mirror profiles, and 0 indicating no association between profiles (Cattell, 1949; Cattell, Coulter, & Tsujioka, 1966).

Procedurally, an averaged top 5 per cent performer profile was identified within the calibration file (identical to the profile development procedure in Approach 1—scale means defined the top performer profile) and indices of profile similarity were computed by correlating each validation sample profile with the top performer template. rp values ranged from a low of -.71 to a high of .85 (average rp = .36). These coefficients of similarity to the desired template were then compared to Benchmarks ratings (e.g. whether or not rp values were associated with Benchmarks ratings; r = .12, p < .05). Trichotomising both the rp and total 360° scores yielded the slightly differ- entiating categorical cells presented in Table 3 (e.g. misses and false alarms slightly suppressed; χ4

2 = 10.45 , p < .05; jc = .06).

Approach 4: Criterion Pattern Profiles. Davison and Davenport (2002) developed a multiple regression procedure to identify a pattern of predictor scores without categorising the criterion (for example, into “top” versus “bottom” performers). The procedure identifies both patterns and levels

TABLE 2 Configural Scoring (Signs) Prediction (15, 20, and 25 Scale Comparisons)

Benchmarks rating

Profile-based selection decision

Selected Not selected

Fifteen signs Top third 215 (14.6%; H) 277 (18.9%; M) Middle third 199 (13.6%) 288 (19.6%) Bottom third 180 (12.3%; FA) 309 (21.0%; CR)

Twenty signs Top third 109 (7.4%; H) 383 (26.1%; M) Middle third 98 (6.8%) 389 (26.5%) Bottom third 64 (4.4%; FA) 425 (29.0%; CR)

Twenty-five signs Top third 40 (2.7%; H) 452 (30.8%; M) Middle third 28 (1.9%) 459 (31.3%) Bottom third 21 (1.4%; FA) 468 (31.9%; CR)

Note: H = Hit, M = Miss, FA = False Alarm, CR = Correct Rejection.

PROFILE MATCHING 533

(elevation) effects, with the profile effect being conditional on the level esti- mation for a particular individual. The general application can be expressed as:

′ = −( ) −( ) + +∑ ∑Y b b X X b X ap v pv pv pv. .. .(1)

where X is a predictor value, Y’ is the predicted criterion score, p is a respondent row, and v is a variable column. The dot notation indicates means (i.e. Xp. refers to a mean computed within persons (across variables); b. refers to an average regression weight); (Xpv - Xp.) is the pattern effect; (bv - b.) is a least-squares regression deviation; and Xp. is the level effect. Davison and Davenport (2002) recommend rescaling the regression deviation coefficients (bv – b.)s by a constant, k, to facilitate graphical presentation of the identified pattern vector profile and this estimated calibration sample criterion pattern is presented in Figure 2 (k = 1,000).

Applying the pattern and levels predictors to the calibration sample Bench- marks criterion yielded bivariate correlations of .18 (ps < .05) for both the levels and pattern effects. The correlation of the levels and pattern effects was r = -.18 (p < .05). The relative contributions of the pattern versus level to 360° score explication were estimated through comparison with the full-model R2

(.08; Multiple R = .28; F = 61.82, p < .05; Note that the same omnibus R2 will be obtained whether the criterion pattern profile or individual scales are retained as predictors). Both the nested levels model (DR2 = .05; F(25, 1442) = 2.84, p < .05) and the nested pattern effect (DR2 = .05; F(1, 1442) = 66.87, p < .05) were deemed different from the full model, indicating that each predictor (pattern and level) contributed uniquely to performance rating prediction. The identified criterion pattern (Figure 2) yields higher scores for the Domi- nance, Amicability, and Well-being scales, and lower scores for the Good Impression, Independence, and Creative Temperament scales. The pattern effect alone accounts for 3 per cent of the variability in 360° scores. The level effect alone accounts for a similar amount of variability. Together, the

TABLE 3 Profile Matching Cross-Validation Prediction (rp method)

Benchmarks rating

Profile similarity

High similarity Moderate similarity Low similarity

Top third 179 (12.2%; H) 176 (12.0%) 137 (9.3%; M) Middle third 161 (11.0%) 152 (10.3%) 176 (12.0%) Bottom third 151 (10.3%; FA) 161 (11.0%) 176 (12.0%; CR)

Note: H = Hit, M = Miss, FA = False Alarm, CR = Correct Rejection.

534 KULAS

pattern and level information account for 8 per cent of the variability in the total Benchmarks score.5

Cross-validation of this developed profile was accomplished through applying the regression weights and constant estimated in the seventh dataset to the final validation dataset individuals. In this validation sample, both the pattern (r = .09, p < .05) and level (r = .18, p < .05) effects once again exhibited bivariate relationships, as did the full-model cross-validation estimate (r = .21, p < .05). Table 4 presents the contingency estimates created through trichotomising both the regression-predicted as well as actual Benchmarks ratings (Pearson’s χ4

2 = 44.24 , p < .05; jc = .12). Hits and correct rejections yield the highest percentages in the table, while misses and false alarms are minimised (relative to the other cross-categorisations).

Comparison of Results across the Four Procedures. Clearly the specifica- tion of profile bands (as defined across 19 or 26 dimensions by means and standard deviations) is not an effective use of the available personality infor- mation. In contrast, Cattell’s profile similarity index did evidence some potential utility, although the cross-validation coefficient was of a smaller

5 Note that the increased variance accounted for constitutes a prototypical case of “coopera- tive” suppression (negative correlation between pattern and level with a positive correlation between both pattern and total 360° score and level and total 360° score).

FIGURE 2. CPI 260 criterion pattern vector (demonstrating both criterion-related pattern and relative elevation of the 26 scales).

PROFILE MATCHING 535

magnitude than the cross-validation regression estimate (Fisher’s z = 2.49, p < .05). The “profile matching” strategy that provided the closest challenge to the regression’s cross-validation coefficient was the configural scoring procedure. These two methods, which both consider criterion information from top as well as bottom performers, result in only a slight regression advantage. The regression approach provided better classification differen- tiation (jc = .12) than the most differentiating (20 comparison) Sullivan and Welsh (1952) approach (jc = .10). Unfortunately, due to different degrees of freedom in the two chi-square analyses, direct inferential comparison between the 20-component profile (χ2 14.73

2 = ) and regression-based cross- validation estimate (χ4

2 = 44.24 ) is not possible. Note also that this signs procedure is more akin to Meehl’s specification of configural scoring than it is to the textbook concept of “profile matching” (i.e. Gatewood & Feild, 2001; Landy & Trumbo, 1980).

DISCUSSION

The use of personality information in selection contexts has a rich history of investigation. One unifying theme across these investigations, however, is the predominant consideration of personality–performance relationships in a strictly linear, independently specified manner. The possible relation- ships between outcome-based criteria and multivariate specifications of trait patterns have been largely ignored in the literature, but the phone survey indicates that the practice is commonplace in applied settings. The opera- tionalisations of profile matching specified here suggest that the strategy may not constitute a best practice for the incorporation of mutual pieces of personality information in a selection context, but the high rate of practi- tioner application as well as the limitations of the current investigation both suggest that the procedures warrant further research attention. Certainly the prevalence of vendor application suggests that the approach deserves more

TABLE 4 Profile Matching Cross-Validation Prediction (Davison & Davenport

[2002] Method)

Benchmarks rating

Predicted performance

Top third Middle third Bottom third

Top third 211 (14.4%; H) 163 (11.1%) 117 (8.0%; M) Middle third 150 (10.2%) 168 (11.4%) 171 (11.6%) Bottom third 130 (8.8%; FA) 158 (10.8%) 201 (13.7%; CR)

Note: H = Hit, M = Miss, FA = False Alarm, CR = Correct Rejection.

536 KULAS

attention and should perhaps be at least acknowledged in future editions of broad practice-based guidelines such as the Principles (Society for Industrial and Organizational Psychology, 2003).

Regarding the possible factors sustaining a high practitioner prevalence rate, it is possible that the findings of some published validity studies are being misinterpreted. Detrick and Chibnall (2006), for example, repeatedly use the term “profile” to describe effective police recruits, but only establish relationships independently within each profile dimension. Validating scale components and then discussing a desired profile may leave some confused regarding “what to use”—the individual dimensions in separate consider- ation or the full profile. Compounding this semantic obfuscation, the gener- ally accepted practice of developing and interpreting profiles within educational and clinical applications may be permeating into the organisa- tional selection domain. With the estimated prevalence of profile matching strategies in applied contexts and currently documented questions regarding conceptual, practical, and empirical propriety of the approach, it may be prudent for published validation articles to carefully distinguish between the terms facet, domain, and profile when disseminating results and offering personality-oriented application recommendations.

Future Directions

Although retention of the general procedure is not recommended (when the regression model suffices), the current empirical findings suggest that consid- eration of absolute discrepancy from the desired profile (the rp index) may prove a more effective use of profile information than consideration of whether or not an applicant falls within a desired band (especially when a fairly arbitrary convention is used for the development of the bands [i.e. � X standard deviations]). The procedure used to develop the profile would also ideally incorporate information from top and bottom performers. The Davison and Davenport (2002) procedure identified both level and pattern effects as contributing to the prediction of 360° ratings. This suggests that the signs approach, which only specifies relative component patterns, may have room for improvement. It is possible that a hybrid configural scoring proce- dure, whereby elevation as well as differentiating patterns are assessed, might provide greater predictive power than specification of the signs procedure alone.

The current study included predictors that both did and did not exhibit bivariate linear associations with the criterion. This is important to note because the full advantage of the profile matching strategy may perhaps come from situations in which there is no demonstrable bivariate relationship (e.g. Meehl’s paradox). That is, the selection specialist who forgoes univariate specification in favor of profile matching is at least implicitly assuming

PROFILE MATCHING 537

greater predictive power for the personality construct interaction than the main effect(s). The interactionist template, however, carries the potential of obviating the utility of the personality information when predictors exhibit some degree of bivariate validity (this would obviously not occur in situa- tions where the bivariate relationship is absent [e.g. there is “nothing to lose”]). Empirically mining for such interactions is not difficult. Predicting patterns of scale interactions, interpreting their meaning, and assessing the reliability of the larger profile is where complications arise and where the absence of a directive model is most noticeable.

Most importantly, therefore, the theoretical specification of personality profiles needs to be addressed. The starting point for this endeavor would be a specification of the profile measurement model (form of relationship between the construct and the measure). Additional clarification may be gained if the profile model could be contrasted with a properly specified regression model (including possible curvilinear relationships and interac- tions). Of paramount interest in the specification of a measurement model would be the nature of the errors—are these random or possibly correlated across components? Such a model needs to be developed that allows for comparisons to be made between individual component scale reliabilities and the superordinate profile reliability. These issues would ideally be resolved prior to continuation of the practice.

The lack of such specification can lead to confusion. Livingston et al. (2003), for example, citing a Gestalt perspective, state that “Profiles taken as a whole may be more reliable than their individual components, just as a subtest is more reliable than any of its single items” (p. 489). This perspective is trou- bling, as it confounds a general psychological theoretical orientation (Gestalt psychology) with fundamental principles of measurement. While it is true that the more assessments made of a unidimensional construct, the more random error is likely to cancel out, only under very limited circumstances would the specification of multiple distinct constructs (each of which possessing less-than perfect measurement) be predicted to yield greater stability than any one of the components considered independently. Indeed, the random error associated with component scores becomes more problematic as more components are added to a profile. In the multidimensional case, the absolute (or sum of squared) error term becomes the appropriate target of reliability estimation procedures. This “profiles may be more reliable” perspective is therefore dependent on the profile assessing a unique superordinate construct—an assumption that is unlikely to be met in the case of personality profiles.

Limitations

Regarding the estimated prevalence of the practice, the 62 per cent application rate from the phone survey is likely a conservative estimate. In

538 KULAS

addition to the imposed limitation of contacting only consultative ven- dor organisations, contact persons were identified by titles such as “research director” or “product manager”. Although these individuals should have knowledge of the assessments being used, they may not be fully aware of how the assessments are being utilised by all consultants for all clients. Identified organisations were further limited to North America. The prevalence of the use (and indeed validity) of profile matching strategies in different countries is therefore unknown and would be an interesting avenue for future research. Furthermore, the phone survey yielded very little description regarding the nature of the profiles. Future exploration would ideally document how the profiles are being defined and used. The current study investigated three likely proce- dures; however, the method is likely being applied in a number of different ways.

The tertile-split categorisation of the criterion was fully arbitrary and was used to facilitate hit- and miss-rate comparisons across procedures. Similarly, the criterion operationalisation was aggregated and it is possible that results would differ based on multidimensional criteria. Although the 360° scores were normally distributed, ratings were elevated across the scale—the range of the criterion measure was therefore slightly restricted. Furthermore, the number of top (and bottom) performers chosen in the profile applications could certainly be adjusted (with possible different results). Such an adjust- ment could even be taken to the extreme with the rp procedure—“cloning” a particular individual if the assessment dimensions are believed to adequately capture the individual’s uniqueness. However, it should be noted that the current datasets were explored fairly rigorously with different specifications of numbers of top and bottom performers with no resulting impact on results.

The distributions of CPI scores within each of the 26 scales were not greatly varied, and collectively the instrument yielded only a moderate multiple correlation (R = .28, cross-validated R = .21). Furthermore, in two of the investigated procedures, all components were given equal weight in desired template identification. It is possible that a different person- ality instrument, such as one assessing the five-factor model (or alterna- tively simple retention of different numbers of scales), may yield different results. It should be noted, however, that the CPI and the NEO-PI have been shown to share scale similarity (e.g. McCrae, Costa, & Piedmont, 1993). In addition, the potential advantages of the profile matching strategy would seem to come from Meehl’s paradox—situations in which individual scales provide no criterion separation, but mutual consideration of patterns across scales do exhibit such separation. A many-scale instru- ment such as the CPI certainly allows for greater possibilities to uncover such effects.

PROFILE MATCHING 539

Summary and Conclusions

Although it is possible that intra-organisational, position-specific, or other creative applications may bear more promise for the general procedure, the current investigation indicates that profile matching, when broadly viewed, is a suboptimal use of personality information compared to traditional linear methods. Landy and Trumbo (1980) state, regarding profile match- ing, that “given the conceptual and methodological problems and the dubious nature of some of the assumptions of this method, it is difficult to make a case for its continued use in selection” (p. 176). Roughly 30 years later, the estimate of vendor implementation is 62 per cent, but the recom- mendation remains “it is difficult to make a case for its continued use in selection”. This recommendation is again made more on conceptual and methodological rather than empirical grounds (the configural scoring and profile similarity indices did approach the predictive power of the regression model). Ultimately with this approach, it would seem difficult to overcome the very practical “false ceiling” limitation. Similarly (and relatedly) the utility of the procedure is ultimately bound by the characteristics of the group being “cloned”. The empirical results of the current investigation suggest that, if the procedure is retained absent the advocated conceptual remediation, both top and bottom performers be used in profile specifica- tion and/or the profile similarity index be considered (rather than a consid- eration of whether an applicant falls within a desired band across focal personality dimensions).

REFERENCES

Allport, G.W. (1962). The general and the unique in psychological science. Journal of Personality, 30, 405–422.

Anderson, M., & Tett, R. (2009). Specificity and multivariate prediction in personality–job performance linkages. Paper presented in J. Coaster (Chair), Trait, Criterion, and Situational Specificity in Personality–Job Performance Rela- tions. Symposium conducted at the 24th annual conference of the Society for Industrial and Organizational Psychology, New Orleans.

Barrick, M.R., & Mount, M.K. (1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26.

Barrick, M.R., Mount, M.K., & Judge, T.A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9, 9–30.

Bickhard, M.H. (2001). The tragedy of operationalism. Theory & Psychology, 11, 35–44.

Blum, M.L., & Naylor, J.C. (1968). Industrial psychology: Its theoretical and social foundations (revised edn.). New York: Harper & Row.

Cattell, R.B. (1949). rp and other coefficients of pattern similarity. Psychometrika, 14, 279–298.

540 KULAS

Cattell, R.B., Coulter, M.A., & Tsujioka, B. (1966). The taxonometric recognition of types and functional emergents. In R.B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 288–329). Chicago, IL: Rand McNally.

Cook, M., Young, A., Taylor, D., O’Shea, A., Chitashvili, M., Lepeska, V., Chou- mentauskas, G., Ventskovsky, O., Hermochova, S., & Uhlar, P. (1998). Person- ality profiles of managers in former Soviet countries: Problem and remedy. Journal of Managerial Psychology, 13, 567–579.

Coward, W.M., & Sackett, P.R. (1990). Linearity of ability–performance relation- ships: A reconfirmation. Journal of Applied Psychology, 75, 297–300.

Davison, M.L., & Davenport, E.C., Jr. (2002). Identifying criterion-related patterns of predictor scores using multiple regression. Psychological Methods, 7, 468– 484.

Detrick, P., & Chibnall, J.T. (2006). NEO PI-R personality characteristics of high- performing entry-level police officers. Psychological Services, 3, 274–285.

Dudley, N.M., Orvis, K.A., Lebiecki, J.E., & Cortina, J.M. (2006). A meta-analytic investigation of Conscientiousness in the prediction of job performance: Examin- ing the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91, 40–57.

Gatewood, R.D., & Feild, H.S. (2001). Human resource selection (5th edn.). Mason, OH: South-Western.

Glomb, T.M., & Welsh, E.T. (2005). Can opposites attract? Personality heterogeneity in supervisor–subordinate dyads as a predictor of subordinate outcomes. Journal of Applied Psychology, 90, 749–757.

Glutting, J., McDermott, P., Watkins, M., Kush, J., & Konold, T. (1997). The base rate problem and its consequences for interpreting children’s ability profiles. School Psychology Review, 26, 176–188.

Gough, H.G., & Bradley, P. (1996). CPI manual (3rd edn.). Palo Alto, CA: Consult- ing Psychologists Press.

Harkness, A.R., & McNulty, J.L. (2006). An overview of personality: The MMPI-2 Personality Psychopathology Five (PSY-5) scales. In J.N. Butcher (Ed.), MMPI-2: A practitioner’s guide (pp. 73–97). Washington, DC: American Psycho- logical Association.

Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job– performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88, 100–112.

Janarone, R.J., & Roberts, J.S. (1984). Reflecting interactions among personality items: Meehl’s paradox revisited. Journal of Personality and Social Psychology, 47, 621–628.

Judge, T.A., Bono, J.E., Ilies, R., & Gerhardt, M.W. (2002). Personality and leader- ship: A qualitative and quantitative review. Journal of Applied Psychology, 87, 765–780.

Jung, C.G. (1971). Psychological types (Collected works of C.G. Jung, Volume 6) (G. Adler & R.F.C. Hull, trans.). Princeton, NJ: Princeton University Press.

Konold, T., Glutting, J., McDermott, P., Kush, J., & Watkins, M. (1999). Structure and diagnostic benefits of normative subtext taxonomy developed from the WISC- III standardization sample. Journal of School Psychology, 37, 29–48.

PROFILE MATCHING 541

Landy, F.J., & Trumbo, D.A. (1980). Psychology of work behavior (revised edn.). Homewood, IL: The Dorsey Press.

Leslie, J.B., & Peterson, M.J. (2011). The Benchmarks sourcebook: Three decades of related research. Greensboro, NC: Center for Creative Leadership.

Livingston, R.B., Jennings, E., Reynolds, C.R., & Gray, R.M. (2003). Multivariate analyses of the profile stability of intelligence tests: High for IQs, low to very low for subtest analyses. Archives of Clinical Neuropsychology, 18, 487–507.

McCrae, R.R., Costa, P.T., & Piedmont, R.L. (1993). Folk concepts, natural lan- guage, and psychological constructs: The California Psychological Inventory and the five-factor model. Journal of Personality, 61, 1–26.

Meehl, P.E. (1950). Configural scoring. Journal of Consulting Psychology, 14, 165– 171.

Meehl, P.E., & Dahlstrom, W.G. (1960). Objective configural rules for discriminating psychotic from neurotic MMPI profiles. Journal of Consulting Psychology, 24, 375–387.

Miller, H.A., Watkins, R.J., & Webb, D. (2009). The use of psychological testing to evaluate law enforcement leadership competencies and development. Police Practice & Research: An International Journal, 10, 49–60.

Mohammed, S., & Angell, L.C. (2003). Personality heterogeneity in teams: Which differences make a difference for team performance? Small Group Research, 34, 651–677.

Mount, M.K., Barrick, M.R., & Stewart, G.L. (1998). Five-factor model of person- ality and performance in jobs involving interpersonal interactions. Human Perfor- mance, 11, 145–165.

Patwardhan, V. (2004). Exploration into personality of Indian women leaders. Gender & Behaviour, 2, 179–199.

Sackett, P.R. (2005). The performance–diversity tradeoff in admission testing. In W.J. Camara & E.W. Kimmel (Eds.), Choosing students: Higher education admissions tools for the 21st century (pp. 109–125). Mahwah, NJ: Lawrence Erlbaum Associates.

Schneider, B. (1987). The people make the place. Personnel Psychology, 40, 437–453. Schneider, B., Smith, D.B., Taylor, S., & Fleenor, J. (1998). Personality and organi-

zations: A test of the homogeneity of personality hypothesis. Journal of Applied Psychology, 83, 462–470.

Society for Industrial and Organizational Psychology (2003). Principles for the vali- dation and use of personnel selection procedures (4th edn.). Bowling Green, OH: Society for Industrial and Organizational Psychology.

Sullivan, P.L., & Welsh, G.S. (1952). A technique for objective configural analysis of MMPI profiles. Journal of Consulting Psychology, 16, 383–388.

542 KULAS

may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express

written permission. However, users may print, download, or email articles for individual use.