Personality Tests in the Workplace

Art_PersonalityAssessments.pdf

Home >Human Resource Management homework help >Personality Tests in the Workplace

ESSENTIALS OF PERSONNEL ASSESSMENT AND SELECTION

This second edition continues in the tradition of the first edition by giving man- agers and students the nuts and bolts of assessment processes and selection tech- niques. The book provides current and future managers with the knowledge and tools required to make informed personnel decisions based upon the results of tests and assessments. It emphasizes that good prediction requires well-formed hypotheses about personal characteristics that may be related to valued behavior at work and the need for developing a theory of the attribute one hypothesizes as a predictor—a thought process too often missing from work on selection pro- cedures. In addition, it explores such topics as team-member selection, situational judgment tests, nontraditional tests, individual assessment, and testing for diversity. The book covers both basic and advanced concepts in personnel selection in a straightforward, readable style intended to be used in both undergraduate and graduate courses in Personnel Selection and Assessment.

Scott Highhouse is a Professor and Ohio Eminent Scholar in the Department of Psychology, Bowling Green State University, USA. Scott is Founding Editor of the journal Personnel Assessment and Decisions and serves on the editorial boards of Journal of Applied Psychology and Journal of Behavioral Decision Making .

Dennis Doverspike is a Full Professor of Psychology at The University of Akron, USA, Senior Fellow of the Institute for Life-Span Development and Gerontology, and Director of the Center for Organizational Research. He is certified as a Specialist in Industrial-Organizational Psychology and in Organizational and Business Consulting Psychology by the American Board of Professional Psychology (ABPP) and is a licensed psychologist in the State of Ohio.

Robert M. Guion ( deceased ) was Distinguished University Professor Emeritus at Bowling Green State University, where he was on the faculty from 1952 until his death in 2012. Honors include the Distinguished Scientific Contributions Award, Society for Industrial and Organizational Psychology; Award for Lifetime Contribu- tions to Evaluation, Measurement, and Statistics, American Psychological Association (Div. 5); and the Stephen E. Bemis Memorial Award, International Personnel Man- agement Association Assessment Council.

This page intentionally left blank

ESSENTIALS OF PERSONNEL ASSESSMENT AND SELECTION Second Edition

Scott Highhouse, Dennis Doverspike, and Robert M. Guion

Second edition published 2016 by Routledge 711 Third Avenue, New York, NY 10017

and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

Routledge is an imprint of the Taylor & Francis Group, an informa business

The right of Scott Highhouse, Dennis Doverspike, and Robert M. Guion to be identified as authors of this work has been asserted by them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.

Trademark notice : Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

First edition published by Lawrence Erlbaum Associates, Inc., 2006

Library of Congress Cataloging-in-Publication Data Names: Guion, Robert M., author. | Highhouse, Scott, author. | Doverspike,

Dennis, author. Title: Essentials of personnel assessment and selection. Description: Second edition / by Scott Highhouse, Dennis Doverspike, and

Robert M. Guion. | New York, NY : Routledge, 2016. | Earlier edition published in 2006 written by Robert M. Guion and Scott Highhouse. | Includes index.

Identifiers: LCCN 2015038976| ISBN 9781138914575 (hardback : alk. paper) | ISBN 9781138914599 (pbk. : alk. paper) | ISBN 9781315690667 (ebook)

Subjects: LCSH: Personnel management—Decision making. | Prediction of occupational success. Employees—Rating of. | Employment tests.

Classification: LCC HF5549 .G793 2016 | DDC 658.3/11—dc23 LC record available at http://lccn.loc.gov/2015038976

ISBN: 978-1-138-91457-5 (hbk) ISBN: 978-1-138-91459-9 (pbk) ISBN: 978-1-315-69066-7 (ebk)

Typeset in Bembo by Apex CoVantage, LLC

http://lccn.loc.gov/2015038976

It is a fine thing to have ability, but the ability to discover ability in others is the true test.

— Elbert Hubbard (1856–1915)

This page intentionally left blank

CONTENTS

Preface ix

PART I Deciding What to Assess 1

1 Understanding Personnel Assessment 3

2 Analyzing Organizations and Jobs 16

3 Developing Predictive Hypotheses 44

4 Knowing What’s Legal (and What’s Not) 67

PART II Knowing How to Assess 95

5 Minimizing Error in Measurement 97

6 Predicting Future Performance 124

7 Using Multivariate Statistics 140

8 Making Judgments and Decisions 156

9 Analyzing Bias and Ensuring Fairness 172

viii Contents

PART III Choosing the Right Method 193

10 Assessing via Traditional Tests 195

11 Assessing via Inventories and Interviews 210

12 Assessing via Ratings 236

13 Individual and Group Assessment 258

Index 279

Robert Guion wrote a book, Personnel Testing, published in 1965, which was used as a textbook in undergraduate and graduate courses in testing and selection. A second book was later undertaken to be a reflection of changes in assessment methods and in selection problems that occurred subsequent to that first book, and it was also intended to be a textbook. That book, published in 1998 (2nd ed., 2011) as Assessment, Measurement, and Prediction for Personnel Decisions, had a much longer title; moreover, in an effort to be comprehensive, its content was also longer and more complex. It turned out to be more appropriate for professionals in the field, and those industrial and organizational psychology students preparing to become professionals, than for undergraduate students or master’s students prepar- ing for broader HR roles.

The first edition of this book, Essentials of Personnel Assessment and Selection, distilled from the bigger book the essentials that managers and other well-educated people should know about the assessment processes so widely used in contempo- rary society—and so widely not understood. By most accounts, the book suc- ceeded as a text for advanced undergraduates and master’s level students interested in becoming users of research-based assessment and selection information and techniques.

It is now 10 years later and much has changed. Robert Guion is no longer with us. He passed away on October 23, 2012, at the age of 88. Bob was a model of integ- rity and deeply believed that the waste of human resources should pain the profes- sional conscience of I-O psychologists. He worked tirelessly toward the development of a fundamental science that promotes human welfare at work. We are humbly moving forward with this Essentials text—which Bob made clear was his wish.

Like the earlier edition, this one emphasizes that good prediction requires well- formed hypotheses about personal characteristics that may be related to valued

PREFACE

x Preface

behavior at work. We continue to emphasize the need for developing a theory of the attribute one hypothesizes as a predictor, a thought process too often missing from work on selection procedures. New to this book is increased attention to topics such as managerial and executive assessment, advances on the legal front, and global testing, as well as technology and testing. We also consider topics that were not of much concern in 2006, such as unproctored online assessment and “big” data. Considerable attention was also given to updating the book to incorporate recent research findings. Realizing that professors who use our book as a textbook prefer not to make major changes to their syllabi, we have made only one major revision, switching the order of Chapters 11 and 12, so as to discuss ratings after we complete our discussion of other types of assessments.

Although we have updated the book in some respects, we also have tried to stay true to the original vision of Robert Guion. In particular, in the first edition, Bob emphasized the philosophical and historical basis behind personnel selection. He included a good deal of research reflecting the origins of personnel selection. Therefore, the current edition continues to reflect the work of many of the early innovators in the field of personnel selection.

As in the first edition, our goal was to produce an accessible guide to assessment that covers basic and advanced concepts in a straightforward, readable style. Evaluat- ing job candidates is an emotional topic, fraught with unsubstantiated claims from test publishers and baseless accusations from social critics. This book provides a review of the most relevant statistical concepts and modern selection practices that will equip readers with the tools needed to be competent consumers of assessment procedures and practices, and to be well-informed about the kinds of questions to be answered in evaluating them.

Finally, we would like to acknowledge the help of people who contributed their time and effort to make this book as good as we hope it is. A lot of people helped by critically reading parts of the earlier 2006 book. They include Neil Christiansen, Fritz Drasgow, Timothy Judge, Fred Oswald, and Charlie Reeve. A special thanks goes out to Catalina Flores, a graduate student at The University of Akron, who assisted with many of the administrative and editing tasks. On a personal note, Scott Highhouse would like to express his gratitude for distractions from his wife, Maggie, and their five kids: Carmen, Cole, Baye, Owen, and Willow. Dennis Doverspike would like to thank his wife, Ida, and sons, Dan and Tom, for keeping him centered and alive.

Thanks again to all of you.

— Scott Highhouse (Bowling Green State University) and Dennis Doverspike (The University of Akron)

PART I

Deciding What to Assess

This page intentionally left blank

• In 1921, applicants who answered a job advertisement anonymously posted by the world-famous inventor Thomas A. Edison arrived at the Menlo Park facil- ity only to find that they needed to answer a series of brainteasers such as “Is Australia larger than Greenland in area?” “If you were to inherit $1,000,000 within the next year, what would you do with it?” and “How is leather made?”

• Nearly 100 years later, applicants who made it through the initial screening process for a job with an Internet superstore were subjected to a grueling interview that included such oddball questions as “Why is a tennis ball fuzzy?” “Why are manhole covers round?” and “How many cows are in Canada?”

As these anecdotes show, employers are constantly inventing (or recycling) innova- tive methods for attempting to figure out if a job applicant has what it takes to succeed in their firm. What is vastly different between the two examples above is the public’s reaction to such innovative methods. The public reaction to Edison’s questions was almost uniformly negative (Dennis, 1984). The New York Times published 23 articles about the Edison questions in one month alone. Most of these articles ridiculed Edison for attempting to assess the fitness of job candidates with outrageous questions (“More Slams at Edison,” May 22, 1921). Today, com- panies such as Microsoft, Zappos, and Xerox are praised for using brainteaser interview questions, presumably because they enable candidates to provide atypical responses and demonstrate their creativity (e.g., Fuscaldo, 2014; Poundstone, 2012). Despite this, there is no evidence that such methods have any utility for predicting future job performance. For instance, the senior vice president of “people opera- tions” at Google commented, “On the hiring side, we found that brainteasers are a complete waste of time . . . They don’t predict anything. They serve primarily to make the interviewer feel smart” (Bryant, 2013).

1 UNDERSTANDING PERSONNEL ASSESSMENT

Assumptions of This Book, Validation and Its Limits, and Theory and Practice

4 Deciding What to Assess

Brainteaser questions are just one example of how employers often become enamored by their personal theories of what good applicants should be like in order to be successful at work. We believe that personnel assessment in practice will not be taken seriously by upper management until the people who use it become serious advocates for tests, acknowledge and master the complexities of selection, and thoroughly and persistently communicate the utility of using sound methods to reach decisions to key stakeholders.

Human Resource (HR) managers need to make a case to upper management for giving employee selection as much research and development (R&D) attention as is given to patent development. Staffing courses need to give the science of employee selection as much attention as they give to designing performance man- agement systems or strategizing about human capital. Getting a “seat at the table” is about proving to management that you can find diamonds in the rough, using state-of-the-art techniques in performance prediction. It is not about talking the right business lingo or rejecting proven methods as old-fashioned.

Wise Decisions

An organization functions through its members. New members are chosen in the belief that they will benefit the organization. Employees benefit the organization by accepting fairly specific organizational roles—fairly specific sets of functions, duties, and responsibilities. When existing members of an organization seek a new hire for a designated role, the dominant consideration is the suitability of the candidate for that role. Once in the organization, a person may keep the original role, be trans- ferred or promoted, be trained for a somewhat changed role, or be terminated. All are personnel decisions. All are based, if the organizational leaders are not too whim- sical and impulsive, on some sort of assessment of the person. Organizational decision makers hope to make wise decisions and competent assessments help.

Results of wise decisions can range from the mere absence of problem hires to the acquisition of genuine superstars, or top talent, who promote organizational purposes. Good hiring decisions can result in substantial increases in performance levels and productivity. Consequences of unwise decisions can range from incon- venience to disaster. An examination of past U.S. presidential elections or NFL draft choices can provide ready examples of good and bad hiring decisions.

Wisdom in selection decisions depends greatly on knowing the characteristics that are truly important in an anticipated role and on not being distracted by irrel- evant characteristics. Assessing relevant characteristics may be as easy as looking at a driver’s license and noting whether it is current, but most are more abstract and harder to assess. If it is inferred from job analysis that qualifications include skill in getting along with others, that skill might be assessed in an interview, or from per- sonal history information, but special efforts are needed to be sure that these assess- ments provide valid information related to future behavior on the job. Many qualifications are best assessed by tests or specially developed work samples.

Understanding Personnel Assessment 5

This book emphasizes work organizations and how they may improve the chances that their personnel decisions will be wise ones. Wisdom in decision mak- ing is elusive; there are opposing points of view about what is wise, desirable, and valued. In this book, we want to state our view explicitly and assist managers in refining and analyzing their own philosophy toward decisions concerning human resources.

Organizations exist when people join forces voluntarily to reach a common goal; they earn their existence by producing goods or services valued in at least a segment of the larger society. An organization, therefore, prospers according to its contribution to society (Eels & Walton, 1961), and individual members con- tribute by functioning well in their assigned roles. The interests of the consumers of the goods or services are compromised, no less than the personal interests of those in the organization, when a person who can function very well is denied a position given to one less qualified. Enough multiplication of such selection errors, and the organization fails—with resulting human and economic waste. If there are more applicants than openings, choices must be made. Choices could be random, or quasi-random, like “first come, first chosen.” Choices might be based on social values, giving preference to veterans, women, or minorities. The choices might be based on nepotism, prejudice, or a similar-to-me bias. Or they can be based on the science of selection and result in the proven prediction of future performance.

We believe the principal basis for personnel decisions should be merit . Some people reject merit as elitist. Some consider profit-oriented concepts of merit inimical to the interests of a broader society. Some dismiss the idea of merit in the belief that situational factors (e.g., having a good boss) influence work perfor- mance more than the personal characteristics people bring to the job. If the merit principle is accepted, however, methods for establishing relative merit are needed. We prefer psychometric methods that give standardized, even-handed assessments of all candidates, similar results from one time or situation to another, and demon- strable relevance to performance.

The term psychometric results from the combination of two Greek words and, literally translated, means “measurement of the mind.” The psy- chometric approach involves developing imperfect indicators of some underlying concept. They are imperfect because they are subject to mea- surement error.

It is wasteful to deny qualified people employment for invalid reasons, including whims known only as “company policy.” Wasting human resources is as inexcus- able as wasting physical resources. An organization has a responsibility to itself, to

6 Deciding What to Assess

the society that supports it, and to the people who seek membership in it, to be sure that it conserves and optimizes human talent.

The Role of Research in Staffi ng Decisions

The history of assessment for personnel selection is old. The ancient Chinese devel- oped civil service examinations (Bowman, 1989; DuBois, 1970). Plato devised pro- cedures for selecting the Guardians in his Republic. Another example is Biblical. Gideon had too many candidates for his army. On God’s advice, he used a two-stage personnel testing procedure. The first was a single-item preliminary screening test (“Do you want to go home?”); on the basis of the answers, he cut 22,000 candidates down to 10,000. A behavioral exercise—to observe candidates drinking from a stream—was used for those remaining; 300 were chosen. No one questioned the validi- ties of these procedures for they were given by God. Unfortunately, many contempo- rary testers behave as if they believe that they, too, have God-given tests and do not need to worry about research evidence. Selection researchers, however, recognize that tests and interpretations of results are fallible and that the validity of any given procedure for assessing candidate characteristics needs to be questioned. Such questioning has led to fairly standard procedures for evaluating (validating) selection procedures.

Fundamental Assumptions

Freyd (1923) identified five assumptions that were fundamental to the research process. With some updating, they are also fundamental to this book:

1. People have abilities and other traits: mental abilities, psychomotor abilities, knowledge, specifically learned skills (including social skills), and habitual ways of dealing with things and events (including personality or tempera- ment). We do not assume that traits are permanently fixed, either by heredity or early life experiences. We do assume, however, that some of them, espe- cially abilities, are reasonably stable for most adults, stable enough that the level of ability observed in a candidate will stay pretty much the same for some time. Thus, even if traits or characteristics cannot be directly observed, they can be inferred on the basis of their effects and are, thus, real. Psychome- tricians often refer to the existence of underlying latent traits .

2. People differ in any given trait. Those with higher levels of abilities relevant to the performance of a job are expected to perform better, other things being equal, than those with lower levels. Thus, individual differences exist on traits and characteristics.

3. Relative differences in ability remain pretty much the same even after train- ing or experience. People with higher levels of a required ability before being hired will be the better performers on that job after training or after a period of time has passed.

Understanding Personnel Assessment 7

4. Different jobs require different traits. For example, one job may require spe- cialized mathematical skills; another may require conscientious attention to procedural detail.

5. Required abilities can be measured. Cognitive abilities, for example, can be mea- sured with many different kinds of tests. Not only can traits or abilities be mea- sured, but the resulting scores or numbers have some real mathematical meaning.

Cognitive tests have been used successfully for employee selection and for many other purposes. The measurement of motivational requisites of successful perfor- mance has a less impressive record of success in employee selection. The record may be more impressive when the research effort expended on the definition and measurement of such traits approaches that expended on cognitive abilities.

Steps in Traditional Validation

Personnel research has traditionally focused on jobs that employ large numbers of people. For such jobs, traditional employment test validation follows steps like these:

Analyze Jobs and Organizational Needs . These procedures are sometimes casual, sometimes very systematic (see Chapter 2). Both job and organizational need analysis inform judgments of whether the need is for improved selection or some other sort of organizational intervention, such as redesigning the job or training current employees. Clearly, no new selection procedure can solve a problem that springs primarily from inadequate equipment or inept management.

Job analysis asks what a worker does, how it is done, and the resources (personal and organizational) used in doing it. Jobs are analyzed to get enough understand- ing of the job to know what applicant characteristics are needed to perform it effectively.

Choose a Criterion . The criterion in personnel research is that which is to be predicted: a measure of performance, of a limited aspect of performance, or of some valued behavior associated with the assigned job role. It might be a measure of trainability, production quality and quantity, attendance, or something else. Cri- terion choice is a matter of organizational values and organizational needs.

The predictor is what we use to assess the job candidate’s (future) suitabil- ity for the job. The criterion is the thing we use to assess the employee’s (current) performance on the job. If we used a test of personality to predict number of sales made by sales associates, the predictor would be the test of personality, and the criterion would be number of sales. Validation is the pro- cess of estimating the relationship between the predictor and the criterion.

8 Deciding What to Assess

Form Predictive Hypotheses . More than one kind of ability or trait likely must be measured if the criterion is to be predicted in all of its complexity. Each predictor– criterion pair is a hypothesis open to research (see Chapter 3). For example, an analysis of the job of potato chip sorter may have revealed that chip quality is an important work outcome to be predicted. One predictive hypothesis might be that individual differences in attention to detail should be related to better performance in monitoring chip quality. A predictive hypothesis may be rather casual and still prove to be a good one. More systematically developed, well-reasoned hypotheses ordinarily will be more likely to be supported by research.

Select Methods of Measurement . We tend to have more research on tests and questionnaires than on other methods—for good reasons. Practical research fol- lows success, and the predictive value of tests has been demonstrated more persua- sively and more frequently than for competing approaches to assessment. Further, testing is easily standardized, enabling a fairer assessment than is possible when the method of assessment varies from one person to another (as with an unstructured job interview). Test use is not, however, free from problems. One serious problem is the tendency to assess candidates only on traits for which tests are available, rather than to assess characteristics (such as interpersonal skills) not easily assessed by available testing procedures.

Design the Research . Good research tries to ensure that findings from the research sample can generalize to the population of interest, which is job applicants. One aspect of research design is the choice of research participants. Inappropriate par- ticipants may spoil the generalizability of results. In particular, incumbents and applicants may differ in motivation to do well on a test, in means and variances on the measured predictors, or in demographics. Demographic diversity has become a watchword in organizational staffing. The research implications of tapping cur- rently underused sources of job candidates in the search for diversity must be monitored carefully.

When the complexity of criterion performance calls for multiple predictors, some means of considering the predictors in combination is needed. Considering them in combination requires a choice of methods for forming a composite, and it is that composite of predictors that is to be evaluated. Sequential approaches to selection call for some rules for advancing from one step to the next. Any com- posite or sequence anticipated in operational use should be the composite or sequence used in research.

Collect Data . Predictors must be administered with both standardization and tact. The first of these is technical; the second is both technical and civil. Standardiza- tion of assessment procedure has long been accepted as a sine qua non of good practice; it has been virtually unquestioned throughout most of the history of

Understanding Personnel Assessment 9

personnel selection research. Everyone who is tested is given the same set of items, identically worded; any established time limits are rigidly followed whenever the test is given, and instructions are the same for everyone. With that said, appreciat- ing the apprehension of people being assessed is important. Standardization does not mean treating people in a way that is not courteous and respectful.

Evaluate Results . Freyd (1923) referred to evaluating measurement; the idea sub- sequently became known as validating the predictor as measured. Whether called evaluation or validation, the traditional procedure has been to correlate scores or ratings on predictor variables with numerical values on criterion measures. If the correlation is high, the predictor is said to be a good one (i.e., a valid one), and if the correlation is low, the predictor is said to be poor. High and low are relative terms, evaluated more against experience than against specified numbers. In employment testing, empirical evaluation of predictions traditionally has been deemed essential.

The tradition of empirical validation needs to be qualified in light of views developed later in Chapter 5. An even older psychometric tradition defines validity as how well the predictor (usually a test) “measures what it purports to measure” (Drever, 1952, p. 304). These views of validation are not the same. A test that purports to measure spelling ability may do so very well, but it is not likely to be very good at predicting how well mechanics repair faulty brakes. For this reason, we distinguish between the validity with which a trait or attribute is measured and the validity with which the measured trait predicts something else—between validity of measurement (psychometric validity) and validity as the job-relatedness of a predictor. Evidence for either concept of validity may be collected by any of several forms of empirical investigation.

Validation Designs

From the early days of employment testing, validation has followed one of two basic design methods: the present employee method, studying people already on the job, or the follow-up method, testing job applicants and getting criterion data later for those hired. The follow-up method is widely (but not universally) considered the better design because it tests actual applicants.

In an idealized follow-up design, sometimes called the Cadillac version, the tests are given to all applicants but not scored until criterion data are available for those who are hired. (This is to ensure that neither employment decisions nor subse- quent criteria are affected by knowledge of the test scores.) Decisions are made as if the tests were not available at all, using existing methods—application forms, interviews, references, tests, hunches, or whatever—whether previously validated or not. After a time, criterion data are collected for those hired; the tests are then scored, and the scores are compared to criterion data.

10 Deciding What to Assess

In the early days of employment testing, such ideal data collection procedures were rare; now they are virtually nonexistent. Nevertheless, the ideal provides a standard against which other designs can be discussed. Traditionally, the only other option was the present employee method where employees are taken off the job, tested, and the test scores are correlated with existing or concurrently obtained criterion measures. It is a faster method, and practical considerations often seem to favor it.

The two different approaches are referred to as “predictive” and “concurrent” research designs. These terms distinguish time spans for data collection, not the employment status of the research subjects. Predictive designs include a substantial time interval between the availability of predictor data and collection of subse- quent criterion data; in concurrent designs , both are collected at about the same time. Thus, a predictive design may use present employees if the data to be evalu- ated can be collected from them at one time and criterion data collected some weeks or months later.

Does it matter whether the research design is concurrent or predictive? Opin- ions differ. Barrett, Phillips, and Alexander (1981) argued that the importance of the issue has been exaggerated. Acknowledging that the design differences are potentially important, they presented arguments to show that the differences do not, in fact, have much impact on the results of studies. If anything, concurrent studies generally have given somewhat larger correlations (e.g., Gupta, Ganster, & Kepes, 2013). Moreover, abilities are enhanced through job training and experi- ence; people who do well on the job develop their abilities more than do those who do less well.

Concurrent and predictive designs are all variations on a single theme: the cor- relation between a predictor and a criterion. Validation research is not limited to that theme. This book considers other designs and considerations for assessing not only job-relatedness as an aspect of validity but also for assessing the meaning of scores on an assessment procedure. Because a predictor–criterion correlation is the traditional meaning of a “validity coefficient,” it serves as a way to introduce the problems and complexities of validation, but it is only an introduction.

Problems With Traditional Research

This recital of traditional personnel research is quite conventional, but it describes a paradigm that needs to be reexamined. It is subject to several potentially serious problems.

Numbers of Cases . Conventional research needs large numbers. “Large” once meant 30 or more; considerations of power in evaluating statistical significance have shown that “large enough” may require hundreds of research subjects. The power of statistical tests depends on the statistic. Generally, the more complicated the statistical analysis, the larger the sample needed. Major changes in the U.S.

Understanding Personnel Assessment 11

workforce have occurred and seem likely to continue. Most people do not work in large corporations on jobs performed by hundreds of coworkers. Technological growth has produced a wider variety of jobs. Many employment decisions must now be made where only a few people are to be hired (perhaps only one) from a relatively small group of candidates. Further, more hiring is being done in profes- sional, semi-professional, and managerial occupations, where one person must be chosen from perhaps as few as a half-dozen candidates. In short, the numbers for many decisions are too small for reliable correlation coefficients (i.e., less than 100). The traditional paradigm makes no provision for the small business, for choosing the replacement for a retiring manager, or for hiring a one-of-a-kind specialist.

Consideration of Prior Research . Traditional validation ignores prior research. Earlier, it was thought that validities were unique, specific to a situation at hand. Now it is known that validities often generalize well across different situations (see Chapter 7).

Need for Judgment . The traditional approach to selection is purely statistical; it leaves no room for judgment. In one sense, that is good. The idea that human judgment yields better predictions than statistical equations do is a myth (or a superstition based on hope) persisting in spite of overwhelming evidence to the contrary. Nevertheless, statistical prediction is often impossible, infeasible, or insuf- ficient; judgment is necessary (see Chapter 8). Even with research, the circum- stances for a candidate at hand may differ enough from the research circumstances that use of the research is questionable. The most obvious example lies in testing the skills of people with disabilities. One cannot intelligently (or legally in the United States) refuse to consider a blind applicant for a job in which visual acuity is not a genuine requirement just because the applicant does not match the research sample of people with sight. One can, of course, make some modification of the selection procedure (such as reading items orally), but the research does not apply to these nonstandard modifications (see Chapter 4). The decision maker must, therefore, make a judgment based on the applicant’s performance on a procedure of unknown validity, on interviewer judgments of unknown validity, prior work experience of unknown validity, or on a random basis known not to have any validity. To disqualify an applicant because the possible assessment procedures have not been validated is not very wise.

Global or Specifi c Assessments . A guiding theme of this book is that a predictive hypothesis can specify that people strong in a certain trait, or collection of traits, are likely to do well on the criterion. An alternative point of view is the whole person view—the idea that people are more than bundles of independent traits, that assessments should be holistic, looking globally at “the whole person.”

Dachler (1989) suggested that selection be considered a part of personnel devel- opment, considering patterns of behavior rather than scorable dimensions,

12 Deciding What to Assess

focusing more on probability of future growth and adaptability than on fitness for a particular job. There is much to recommend his position.

Accepting one of these views may not wholly exclude the other. Two major differences between them are not insurmountable. First, traditional correlation uses measures of dimensions, not patterns. This does not, however, preclude correlating X and Y where X is the degree to which people fit a designated pattern of behav- iors. Second, at least in the United States; the Uniform Guidelines (Equal Employ- ment Opportunity Commission [EEOC], Civil Service Commission, Department of Labor, & Department of Justice, 1978, Section 5I, p. 38298; see Chapter 4) follow traditional methods. Although holistic evaluation of people and their future growth are nowhere mentioned in the guidelines, we suspect that a well-reasoned, well-developed selection procedure with evidence that it improves productivity, without violating the values of the larger society, will be permitted by the courts. Traditional research may seem to preclude more holistic approaches because not enough traditional researchers have thought about holistic approaches often enough or deeply enough to develop a solid paradigm for its use.

Ethical Testing

• The person conducting the assessments must have knowledge and understanding of the psychometric instruments being used.

• The assessment process should be standardized and each candidate being assessed should be treated the same.

• Applicants should be informed of the purpose of the assessments and how the results will be used.

• Who will see the results of the assessments should be clearly explained to the candidate.

• The testing professional must take reasonable steps to ensure that the results are not misused by others in any way.

• Where feasible, the testing professional should respect the applicant’s desire for feedback.

Two important resources on ethical testing are the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (2014), Standards for educational and psychological testing . Washington, DC: American Educational Research Association, and the Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection proce- dures (4th ed.). Bowling Green, OH: Author. For further discussion of ethical issues in employee selection, see Lefkowitz, J., & Lowman, R. L. (2010). Eth- ics of employee selection. In J. L. Farr and N. T. Tippins (Eds.), Handbook of employee selection (pp. 571–591). New York, NY: Routledge.

Understanding Personnel Assessment 13

Theory and Practice

Good practice requires understanding of what one is doing. An existing, relevant theory can promote understanding, but its existence does not ensure it. We call for more attention to theory to promote understanding of what is done in practice. Too much of what we know about personnel assessment and decision making, and, therefore, too much of this book, is limited to techniques. Better theories of work and work effectiveness can sharpen, prune, and expand those techniques and improve decisions. If there is a theme to this book, it is that we need to develop much greater knowledge of how managers use assessment results to make selection decisions and that we need to provide managers with sufficient knowledge con- cerning assessment methods, so that they have a strong basis for making more informed, rational, and accurate selection decisions.

An unfortunate but growing gap seems to separate academic science from organizational practice. Academics often seem interested only in building theories. Practitioners tend to decry the triviality and impracticality they perceive in aca- demic theories, yet some of the theories they decry could inform many practical decisions in their organizations. There is, or should be, a symbiotic relationship between theory and practice and between basic and applied research. To be practi- cal, a theory has to be a good one, internally consistent, supported by solid data, and tested in practice to find out how well it works beyond the boundaries of an experimental situation.

A third member of this mutual relationship is society at large. Both science and practice must heed the social issues and problems they solve or exacerbate. Many scientific questions, especially in the behavioral sciences, stem from the concerns of that larger society. Practice within an organization is also practiced within that larger society; for many practical decisions, both the relevant scientific foundations and their social effects must be considered.

Research should not be limited to just one chosen criterion; decision outcomes are likely to be plural. They need to be understood. Understanding requires HR research and development programs at least on par with product and market research, and these programs work best if informed by competent theory. Out- comes and reasons for unexpected ones can be clarified through research, provid- ing further practical guidance for decision making. All of this occurs within a community (including the larger society) that experiences the effects of outcomes and seeks to influence them. With a well-funded R&D program, unspecified and unintended outcomes, whether relevant to community concerns or to organiza- tional needs, could be investigated much as medical research looks for side effects of medical interventions.

We must not, however, be so wrapped up in psychometric research, statistical analyses, and the contextual influences of the community that we forget that the purpose of all this is to optimize the process by which some people get rewards and opportunities and others do not. The central focus of this process—the one intended

14 Deciding What to Assess

to reach the best possible outcomes—is a decision. Decisions are based on assess- ments; they also imply judgment, preferably informed judgment. Some of the information comes from research and theory, some of it comes from knowing the organization’s needs, and some of it comes from community influences. We do, in fact, need more theory; and more theory needs to be informed by practice.

Discussion Topics

1. In the chapter, the authors argue for hiring based on merit. However, the def- inition of “merit” is open to interpretation; how would you define “merit”? Is it ever appropriate to hire on the basis of some other standard?

2. How do you think companies most commonly deviate from using psycho- metrically sound selection procedures? What are the consequences of this?

3. How does the selection approach of choosing the person who will be the best or highest performer in the job differ from choosing the person who has the best fit to the job, or is least likely to leave the job within a short time period? What are the implications of each?

4. What are some unique questions you have been asked when applying for jobs? If you have ever served as an interviewer, what are some of the more creative questions you have asked a job candidate?

References

(Note: In addition to citations contained in the text in Chapter 1, we have provided refer- ences that we believe are helpful to anyone involved in the practice of assessment.)

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.

American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073.

American Psychological Association (2010). Publication manual for the American Psychological Association (6th ed.). Washington, DC: Author.

Arthur, W., Jr., Doverspike, D., Barrett, G. V., & Miguel, R. (2013). Chasing the Title VII Holy Grail: The pitfalls of guaranteeing adverse impact elimination. Journal of Business and Psychology, 28, 473–485.

Barrett, G. V., Phillips, J. S., & Alexander, R. A. (1981). Concurrent and predictive validity designs: A critical reanalysis. Journal of Applied Psychology, 66, 1–6.

Bowman, M. L. (1989). Testing individual differences in ancient China. American Psycholo- gist, 44, 576–578.

Bryant, A. (2013, June 20). In head-hunting, big data may not be such a big deal. The New York Times . Retrieved from http://www.nytimes.com/2013/06/20/business/in-head- hunting-big-data-may-not-be-such-a-big-deal.html

http://www.nytimes.com/2013/06/20/business/in-head-hunting-big-data-may-not-be-such-a-big-deal.html

Understanding Personnel Assessment 15

Civil Rights Act of 1964 § 7, 42 U.S.C. § 2000e et seq (1964). Civil Rights Act of 1991 § 109, 42 U.S.C. § 2000e et seq (1991). Cohen, D. B., Aamodt, M. G., & Dunleavy, E. M. (2010). Technical advisory committee report on

best practices in adverse impact analyses . Washington, DC: Center for Corporate Equality. Dachler, H. P. (1989). Selection and the organizational context. In P. Herriot (Ed.), Assess-

ment and selection in organizations: Methods and practice for recruitment and appraisal (pp. 45–69). Chichester, England: Wiley.

Dennis, P. M. (1984). The Edison questionnaire. Journal of the History of the Behavioral Sci- ences, 20 (1), 23–37.

Drever, J. (1952). A dictionary of psychology . Baltimore, MD: Penguin. DuBois, P. H. (1970). A history of psychological testing . Boston, MA: Allyn & Bacon. Eels, R., & Walton, C. (1961). Conceptual foundations of business . Homewood, IL: Irwin. Equal Employment Opportunity Commission, Civil Service Commission, Department of

Labor, & Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43 (166), 38290–38315.

Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, Department of Justice (1979). Interpretation and clarification of the Uniform Employee Selection Guidelines. Federal Register, 44, 11996–12009.

Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, Department of Justice (1980). Adoption of additional questions and answers to clarify and provide a common interpretation of the Uniform Guidelines on Employee Selection Procedures. Federal Register, 45, 29529–29531.

Freyd, M. (1923). Measurement in vocational selection: An outline of research procedure. Journal of Personnel Research, 2, 215–249, 268–284, 377–385.

Fuscaldo, D. (2014, January 11). Why HR should consider asking oddball interview questions. Glassdoor . Retrieved from http://employers.glassdoor.com/blog/why-hr-should-consider- asking-oddball-interview-questions/

Gupta, N., Ganster, D. C., & Kepes, S. (2013). Assessing the validity of sales self-efficacy: A cautionary tale. Journal of Applied Psychology , 98 , 690–700.

Lefkowitz, J., & Lowman, R. L. (2010). Ethics of employee selection. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 571–591). New York, NY: Routledge.

More slams at Edison; Experts pronounce his questions only one-tenth effective in gaining their purpose. (1921, May 22). The New York Times . Retrieved from http://www. nytimes.com

Poundstone, W. (2012). Are you smart enough to work at Google?: Trick questions, Zen-like riddles, insanely difficult puzzles, and other devious interviewing techniques you need to know to get a job anywhere in the new economy . Oxford, England: Hatchet.

Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.

http://employers.glassdoor.com/blog/why-hr-should-consider-asking-oddball-interview-questions/

http://www.nytimes.com/

References

1 Understanding Personnel Assessment

(Note: In addition to citations contained in the text in Chapter 1, we have provided refer

ences that we believe are helpful to anyone involved in the practice of assessment.)

American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073.

American Psychological Association (2010). Publication manual for the American Psychological Association (6th ed.). Washington, DC: Author.

Barrett, G. V., Phillips, J. S., & Alexander, R. A. (1981). Concurrent and predictive validity designs: A critical reanalysis. Journal of Applied Psychology, 66, 1–6.

Bowman, M. L. (1989). Testing individual differences in ancient China. American Psychologist, 44, 576–578.

Bryant, A. (2013, June 20). In head-hunting, big data may not be such a big deal. The New York Times . Retrieved from

Civil Rights Act of 1964 § 7, 42 U.S.C. § 2000e et seq (1964).

Civil Rights Act of 1991 § 109, 42 U.S.C. § 2000e et seq

(1991).

Cohen, D. B., Aamodt, M. G., & Dunleavy, E. M. (2010). Technical advisory committee report on best practices in adverse impact analyses . Washington, DC: Center for Corporate Equality.

Dachler, H. P. (1989). Selection and the organizational context. In P. Herriot (Ed.), Assessment and selection in organizations: Methods and practice for recruitment and appraisal (pp. 45–69). Chichester, England: Wiley.

Dennis, P. M. (1984). The Edison questionnaire. Journal of the History of the Behavioral Sciences, 20 (1), 23–37.

Drever, J. (1952). A dictionary of psychology . Baltimore, MD: Penguin.

DuBois, P. H. (1970). A history of psychological testing . Boston, MA: Allyn & Bacon.

Eels, R., & Walton, C. (1961). Conceptual foundations of business . Homewood, IL: Irwin.

Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43 (166), 38290–38315.

Freyd, M. (1923). Measurement in vocational selection: An outline of research procedure. Journal of Personnel Research, 2, 215–249, 268–284, 377–385.

Fuscaldo, D. (2014, January 11). Why HR should consider asking oddball interview questions. Glassdoor . Retrieved from

Gupta, N., Ganster, D. C., & Kepes, S. (2013). Assessing the validity of sales self-efficacy: A cautionary tale. Journal of Applied Psychology , 98 , 690–700.

Lefkowitz, J., & Lowman, R. L. (2010). Ethics of employee selection. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 571–591). New York, NY: Routledge.

More slams at Edison; Experts pronounce his questions only one-tenth effective in gaining their purpose. (1921, May 22). The New York Times . Retrieved from http://www. nytimes.com

Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.

2 Analyzing Organizations and Jobs

Bartram, D. (2005). The Great Eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90, 1185–1203.

Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1992). A theory of performance. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 35–70). San Francisco, CA: Jossey-Bass.

Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. In W. Pasmore & R. Woodman (Eds.) Research in organizational change and development (Vol 1, pp. 129–169). Greenwich, CT: JAI Press.

Costa, P. T, McCrae, R. R., & Kay, G. G. (1995). Persons, places and personality: Career assessment using the revised NEO Personality Inventory. Journal of Career Assessment, 3, 123–139.

Cranny, C. J, & Doherty, M. E. (1988). Importance ratings in job analysis: Note on the misinterpretation of factor analysis in industry and the public sector. In S. Gael (Ed.), The job analysis handbook for business, industry and government (Vol. 2, pp. 1051–1071). New York, NY: Wiley.

Fine, S. A., & Cronshaw, S. F. (1999). Functional job analysis: A foundation for human resource management . Mahwah, NJ: Lawrence Erlbaum Associates.

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327–358.

Gottfredson, G. D., & Holland, J. L. (1994). Position Classification Inventory . Odessa, FL: Psychological Assessment Resources.

Harvey, R. J. (1993). Research monograph: The development of the CMQ . San Antonio, TX: The Psychological Corporation.

Inwald, R. (1992). Hilson Job Analysis Questionnaire . Kew Gardens, NY: Hilson Research.

Jeanneret, P. R., & Strong, M. H. (2003). Linking O*NET job analysis information to job requirement predictors. An O*NET application. Personnel Psychology, 56, 465–492.

Landy, F. J. (1989). Psychology of work behavior . Pacific Grove, CA: Brooks/Cole.

Levinson, H. (2002). Organizational assessment: A step-by-step guide to effective consulting . Washington, DC: American Psychological Association.

Lievens, F., Sanchez, J. I., & De Corte, W. (2004). Easing the inferential leap in competency modeling: The effects of task-related information and subject matter expertise. Personnel Psychology, 57, 881–904.

McCormick, E. J. (1959). Applications of job analysis to indirect validity. Personnel Psychology, 12, 402–413.

McCormick, E. J. (1979). Job analysis . New York, NY: AMACOM.

McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1969). The development and background of the Position Analysis Questionnaire (Contract NONR-1100(28), Report No. 5). West Lafayette, IN: Purdue University, Occupational Research Center.

Morgeson, F. P., Delaney-Klinger, K., Mayfield, M. S., Ferrara, P., & Campion, M. A. (2004). Self-presentation processes in job analysis: A field experiment investigating inflation in abilities, tasks, and competencies. Journal of Applied Psychology , 89 , 674–686.

Peterson, N. G., Borman, W. C., Hanson, M. A., & Kubisiak, U. C. (1999). Summary of results, implications for O*NET applications, and future directions. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21 st century: The development of O*NET (pp. 289–295). Washington, DC: American Psychological Association.

Raymark, P. H., Schmit, M. J., & Guion, R. M. (1997). Identifying potentially useful personality constructs for employee selection. Personnel Psychology, 50, 723–736.

Rounds, J. (1995). Vocational interests: Evaluating structural hypotheses. In D. Lubinski & R. V. Dawis (Eds.), Assessing individual differences in human behavior (pp. 177–232). Palo Alto, CA: Davies-Black.

Sackett, P. R., & Laczo, R. M. (2003). Job and work analysis. Handbook of Psychology: Industrial and Organizational, 12, 21–37.

Sanchez, J. I, & Levine, E. L. (2001). The analysis of work in the 20th and 21st centuries. In N. Anderson & D. S. Ones (Eds.), Handbook of industrial, work and organizational psychology: Volume 1 . Personnel Psychology (pp. 71–89). London, England: Sage.

Schippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eye, L. D., Hesketh, B., . . . & Sanchez, J. I. (2000). The practice of competency modeling. Personnel Psychology, 53, 703–740.

United States Depar tment of Labor. (1972). Handbook for analyzing jobs . Washington, DC: U.S. Government Printing Office.

United States Department of Labor. (1977). Dictionary of occupational titles: Definitions of titles (4th ed.). Washington, DC: U.S. Government Printing Office.

Van de Ven, A. H., & Ferry, D. L. (1980). Measuring and assessing organizations . New York, NY: Wiley.

3 Developing Predictive Hypotheses

Anderson, N., & Burch, G. St. J. (2003). The Team Selection Inventory. Windsor, England: ASE/NFER-Nelson.

Ash, R. A., Johnson, J. C., Levine, E. L., & McDaniel, M. A. (1989). Job applicant training and work experience evaluation in personnel selection. Research in Personnel and Human Resource Management, 7, 183–226.

Ashton, M. C., Lee, K., & & de Vries, R. E. (2014). The HEXACO Honesty-Humility, Agreeableness, and Emotionality Factors: A Review of Research and Theory. Personality and Social Psychology Review, 18, 139–152.

Barrett, G. V., Alexander, R. A., & Doverspike, D. (1992). The implications for personnel selection of apparent declines in predictive validities over time: A critique of Hulin, Henry, and Noon. Personnel Psychology, 45 (3), 601–617.

Barrett, G. V., Caldwell, M. S., & Alexander, R. A. (1985). The concept of dynamic criteria: A critical reanalysis. Personnel Psychology, 38, 41–56.

Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9, 9–31.

Bartram, D. (2005). The Great Eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90, 1185–1203.

Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478–494.

Borman, W. C. (1987). Personal constructs, performance schemata, and “folk theories” of subordinate effectiveness: Explorations in an Army officer sample. Organizational Behavior and Human Decision Processes, 40, 307–322.

Borman, W. C., & Motowidlo, S. M. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt & W. C. Borman (Eds.), Personnel Selection in Organizations (pp. 71–98). San Francisco, CA: Jossey-Bass.

Brief, A. P., & Motowidlo, S. J. (1986). Prosocial organizational behaviors. Academy of Management Review, 11, 710–725.

Burch, G. St. J., & Anderson, N. (2004). Measuring person-team fit: Development and validation of the team selection inventory. Journal of Managerial Psychology, 21, 406–426.

Buster, M. A., Roth, P. L., & Bobko, P. (2005). A process for content validation of education and experience-based minimum qualifications: An approach resulting in federal court approval. Personnel Psychology, 58, 771–799.

Campbell, J. P. (2012). Behavior, performance, and effectiveness—in the 21st century. In S.W.J. Kozlowski (Ed.), The Oxford Handbook of Organizational Psychology (pp. 159–194). Oxford, England: Oxford University Press.

Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. In N. Schmitt & W. C. Borman (Eds.), Personnel Selection in Organizations (pp. 35–70). San Francisco, CA: Jossey-Bass.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies . Cambridge, England: Cambridge University Press.

Cattell, R. B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54, 1–22.

Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90, 1241–1255.

English, H. B., & English, A. C. (1958). A comprehensive dictionary of psychological and psychoanalytic terms . New York, NY: Longmans, Green, and Co.

Farrell, J. N., & McDaniel, M. A. (2001). The stability of validity coefficients over time: Ackerman’s (1988) model and the General Aptitude Test Battery. Journal of Applied Psychology, 86, 60–79.

Fleishman, E. A., & Reilly, M. E. (1992). Handbook of human abilities: Definitions, measurements and job task requirements . Palo Alto, CA: Consulting Psychologists

Press.

Funder, D. C. (1991). Global traits: A neo-Allportian approach to personality. Psychological Science, 2, 31–39.

Gebhardt, D. L., & Baker, T. A. (2010a). Physical performance. In J. C. Scott & D. H. Reynolds (Eds.), Handbook of workplace assessment: Evidence-based practices for selecting and developing organizational talent (pp. 165–196). San Francisco, CA: Jossey-Bass.

Gebhardt, D. L. & Baker, T. A. (2010b). Physical performance tests. In J. L. Farr & N. T. Tippins (Eds.), Handbook of employee selection (pp. 277–298). New York, NY: Routledge.

Ghiselli, E. E. (1956). Dimensional problems of criteria. Journal of Applied Psychology, 40, 1–4.

Goldberg, L. R. (1993). The structure of phenotypic personality traits. American Psychologist, 48, 26–34.

Goldberg, L. R., Grenier, J. R., Guion, R. M., Sechrest, L. B., & Wing, H. (1991). Questionnaires used in the prediction of trustworthiness in pre-employment selection decisions: An APA task force report . Washington, DC: American Psychological Association.

Goleman, D. (1995). Emotional intelligence . New York, NY: Bantam Books.

Gough, H. G. (1985). A work orientation scale for the California Psychological Inventory. Journal of Applied Psychology, 70, 505–513.

Guion, R. M. (1965). Personnel testing . New York, NY: McGraw-Hill.

Guion, R. M., & Gottier, R. F. (1965). Validity of personality measures in personnel selection. Personnel Psychology, 18, 135–164.

Helmreich, R. L., Sawin, L. L., & Carsrud, A. L. (1986). The honeymoon effect in job performance: Temporal increases in the predictive power of achievement motivation. Journal of Applied Psychology, 71, 185–188.

Hofstee, W.K.B., de Raad, B., & Goldberg, L. R. (1992). Integration of the big five and circumplex approaches to trait structure. Journal of Personality and Social

Psychology, 63, 146–163.

Hogan, J. (1991a). Physical abilities. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 2, pp. 753–831). Palo Alto, CA: Consulting Psycholo g ists Press.

Hogan, J. (1991b). Structure of physical performance in occupational tasks. Journal of Applied Psychology, 76, 495–507.

Hogan, J., Hogan, R., & Busch, C. M. (1984). How to measure service orientation. Journal of Applied Psychology, 69, 167–173.

Hogan, J., & Quigley, A. M. (1986). Physical standards for employment and the courts. American Psychologist, 41, 1193–1217.

Hogan, R., & Hogan, J. (1992). Hogan Personality Inventory: Manual . Tulsa, OK: Hogan Assessment Systems.

Howard, A. (1986). College experiences and managerial performance. Journal of Applied Psychology, 71, 530–555.

Hulin, C. L., Henry, R. A., & Noon, S. L. (1990). Adding a dimension: Time as a factor in the generalizability of predictive relationships. Psychological Bulletin, 107, 328–340.

Humphreys, L. G. (1979). The construct of general intelligence. Intelligence, 3, 105–120.

Hurtz, G. M., & Donovan, J. J. (2000). Personality and job performance: The Big Five revisited. Journal of Applied Psychology, 85, 869–879.

Joseph, D., Jin, J., Newman, D., & O’Boyle, E. H. (2015). Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI. Journal of Applied Psychology, 100, 298–342.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/ aptitude treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657–690.

Kichuk, S. L., & Wiesner, W. H. (1998). Work teams: Selecting members for optimal performance. Canadian Psychology, 39, 23–32.

Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2004). Academic performance, career potential, creativity, and job performance: Can one construct predict them all? Journal of Personality and Social Psychology, 86, 148–161.

Matthews, G., Roberts, R. D., & Zeidner, M. (2004). Seven myths about emotional intelligence. Psychological Inquiry, 15 (3), 179–196.

Matthews, G., Zeidner, M., & Roberts, R. D. (2002). Emotional intelligence: Science and myth . Cambridge, MA: MIT Press.

Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? New York, NY: Basic Books.

McClough, A. C., & Rogelberg, S. G. (2003). Selection in teams: An exploration of the teamwork knowledge, skills, and ability test. International Journal of Selection & Assessment, 11, 56–66.

McCrae, R. R. (1992). The five-factor model: Issues and applications [Special issue]. Journal of Personality, 60, 175–532.

McDaniel, M. A., Schmidt, F. L., & Hunter, J. E. (1988). A meta-analysis of the validity of methods for rating training and experience in personnel selection. Personnel Psychology, 41, 283–314.

Mischel, W. (1968). Personality and assessment . New York, NY: Wiley.

Murphy, K. R. (1989). Is the relationship between cognitive ability and job performance stable over time? Human Performance, 2, 183–200.

Organ, D. W. (1988). Organizational citizenship behavior: The good soldier syndrome . Lexington, MA, England: Lexington Books/D.C. Heath and Com.

Organ, D. W., Podsakoff, P. M., & MacKenzie, S. B. (2006). Organizational citizenship behavior: Its nature , antecedents, and consequences . Thousand Oaks, CA: SAGE Publications.

Sack ett, P. R., Berry, C. M., Wiemann, S. A., & Laczo, R. M. (2006). Citizenship and counterproductive behavior: Clarifying relations between the two domains. Human

Performance, 19, 441–464.

Sackett, P. R., & Walmsley, P. T. (2014). Which personality attributes are most important in the workplace? Perspectives on Psychological Science, 9, 538–551.

Smith, F. J. (1977). Work attitudes as predictors of attendance on a specific day. Journal of Applied Psychology, 62, 16–19.

Society for Industrial and Organizational Psychology. (1987). Principles for the validation and use of personnel selection procedures . (3rd ed.). College Park, MD: Author.

Spearman, C. (1927). The abilities of man . New York, NY: Macmillan.

Spence, J. T., Helmreich, R. L., & Pred, R. S. (1987). Impatience versus achievement strivings in the Type A pattern: Differential effects on students’ health and academic achievement. Journal of Applied Psychology, 72, 522–528.

Stevens, M. J., & Campion, M. A. (1994). The knowledge, skill, and ability requirements for teamwork: Implications for human resource management. Journal of Management, 20, 503–530.

Stevens, M. J., & Campion, M. A. (1999). Staffing work teams: Development and validation of a selection test for teamwork settings. Journal of Management, 25, 207–228.

Thoresen, C. J., Bradley, J. C., Bliese, P. D., & Thoresen, J. D. (2004). The big five personality traits and individual job performance grow trajectories in maintenance and transitional job stages. Journal of Applied Psychology, 89, 835–853.

Thurstone, L. L. (1938). Primary mental abilities. Psychometric Monographs, (1).

Whyte, W. H., Jr. (1957). The organization man . New York, NY: Doubleday.

4 Knowing What’s Legal (and What’s Not)

Age Discrimination in Employment Act of 1967, 29 U.S.C. Section 621 (1967).

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (1999). Standards for educational and

psychological testing . Washington, DC: American Educational Research Association.

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (2014). Standards for educational and

psychological testing . Washington, DC: American Educational Research Association.

American Psychological Association, American Educational Research Association, &

National Council on Measurement in Education. (1954). Technical recommendations

for psychological tests and diagnostic techniques. Psychological Bulletin, 51, 201–238.

Bernard v. Gulf Oil Corporation, 841 F.2d 547 (5th Cir., 1988).

Civil Rights Act of 1964, 42 U.S.C. Section 2000e (1964).

Civil Rights Act of 1991, 42 U.S.C. Section 1981A (1991).

Connecticut v . Teal, 457, U.S. 440 (1982).

Doverspike, D., Taylor, M. A., & Arthur, W., Jr. (2000). Affirmative action: A psychological

perspective . Huntington, NY: Nova Science Publishers.

EEOC v . Joe’s Stone Crab, 136, F.2d 1311 (2001).

Equal Employment Opportunity Commission. (1970). Guidelines on employee selection

procedures. Federal Register, 35 (149), 12333–12336.

Equal Employment Opportunity Commission, Civil Service Commission, Department of

Labor, & Department of Justice. (1978). Uniform guidelines on employee selection

procedures. Federal Register, 43 (166), 38290–38315.

Equal Employment Opportunity Commission, Civil Service Commission, Department of

Labor, Department of Justice (1979). Interpretation and clarification of the Uniform

Employee Selection Guidelines. Federal Register, 44, 11996–12009.

Equal Employment Opportunity Commission, Civil Service Commission, Department of

Labor, Department of Justice (1980). Adoption of additional questions and answers to

clarify and provide a common interpretation of the Uniform Guidelines on Employee

Selection Procedures. Federal Register, 45, 29529–29531.

Fisher v . University of Texas, 570 U.S. ___ (2013).

Gratz v . Bollinger, 135 F.2d 790 (2001).

Gratz v . Bollinger, 539 U.S. 244 (2003).

Griggs v . Duke Power Co ., 401 U.S. 424 (1971).

Grutter v . Bollinger, 539 U.S. 306 (2003).

Guardians Association of the New York City Police Department, Inc . v . Civil Service Commission

of the City of New York, 630, F.2d 79 (1980).

Guion, R. M. (1998). Assessment, measurement, and prediction for personnel decisions . Mahwah,

NJ: Lawrence Erlbaum Associates.

Gutman, A., Koppes, L. L., & Vodanovich, S. K. (2011). EEO law and personnel practices

(3rd ed.). Thousand Oaks, CA: Sage Publications.

Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity gener

alization, minority issues, and the General Aptitude Test Battery . Washington, DC: National

Academy Press.

Highhouse, S., & Gutman, A. (2011, January). Was the addition of sex to Title VII a joke?

Two viewpoints. The Industrial Organizational Psychologist, 48, 102–110.

Hosanna-Tabor Evangelical Lutheran Church and School v . EEOC , 565 U.S. ___ (2012).

International Brotherhood of Teamsters v . United States, 431 U.S. 324 (1977).

Jeanneret, P. R. (1994, July). Accommodation: State of the research and practice when complying

with the Americans With Disabilities Act . Address to the American Psychological Society,

Washington, DC.

Johnson et al . v . City of Memphis, LEXIS 20644 (2014).

Latuga v . Hooters Inc ., WL 164427, (1996).

McDonnel-Douglas Corp . v . Green, 411 U.S. 792 (1973).

M . O . C . H . A . Society, Inc . v . City of Buffalo WL 604898 (2009).

Regents, University of California v . Bakke, 438 U.S. 265 (1978).

Ricci v . DeStefano, 557 U.S. 557 (2009).

Schuette v . Coalition to Defend Affirmative Action, 572 U.S. ___ (2014).

Smith v . City of Jackson, 544 U.S. 228 (2005).

Sterns, H. L., Doverspike, D., & Lax, G. A. (2005). The Age Discrimination in Employment

Act. In F. J. Landy (Ed.), Employment discrimination litigation: Behavioral, quantitative, and

legal perspectives (pp. 256–293). San Francisco, CA: Jossey-Bass.

Tenopyr, M. L. (2004, April). The University of Michigan cases: Promises and problems . Presenta

tion at the 19th Annual Conference of the Society for Industrial and Organizational

Psychology, Chicago, IL.

Wal-Mart Stores, Inc . v . Dukes, 131 S. Ct. 2541 (2011).

Wards Cove Packing Co . v . Atonio, 109 S. Ct., 2115 (1989).

Watson v . Fort Worth Bank & Trust, 108 S. Ct. 2777 (1988).

Weber v . Kaiser Aluminum & Chemical Corporation, 563 F.2d 2126 (1977). This page intentionally left blank

5 Minimizing Error in Measurement

American Psychological Association, American Educational Research Association, & National Council on Measurement Education. (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51, 201–238.

American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1966). Standards for educational and psychological tests and manuals. Washington, DC: American Psychological Association.

Boring, E. G. (1961). The beginning and growth of measurement in psychology. In H. Woolf (Ed.), Quantification: A history of the meaning of measurement in the natural and social sciences (pp. 108–127). Indianapolis, IN: Bobbs-Merrill.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.

Cardinet, J., Tourneur, W., & Allal, L. (1976). The symmetry of generalizability theory: Applications to educational measurement. Journal of Educational Measurement, 13(2), 119–135.

Cattell, J. M. (1890). Mental tests and measurements. Mind, 15, 373–380.

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443–507). Washington, DC: American Council on Education.

Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: Wiley.

Cureton, E. E. (1950). Validity. In E. F. Lindquist (Ed.), Educational measurement. (pp. 621–694). Washington, DC:

American Council on Education.

Drever, J. (1952). A dictionary of psychology. Baltimore, MD: Penguin.

Dunnette, M. D., & Borman, W. C. (1979). Personnel selection and classification. Annual Review of Psychology, 30, 477–525.

Guilford, J. P. (1959). Personality. New York, NY: McGraw-Hill.

Guion, R. M. (1980). On trinitarian doctrines of validity. Professional Psychology, 11, 385–398.

Guion, R. M. (2011). Assessment, measurement, and prediction for personnel decisions. New York, NY: Routledge.

Guttman, L. (1945). A basis for analyzing test–retest reliability. Psychometrika, 10, 255–282.

Hull, C. L. (1928). Aptitude testing. Yonkers-on-Hudson, NY: World Book.

Kuder, G. F., & Richardson, M. W. (1937). The theory of estimation of test reliability. Psychometrika, 2, 151–160.

Li, H., Rosenthal, R., & Rubin, D. B. (1996). Reliability of measurement in psychology: From Spearman-Brown to maximal reliability. Psychological Methods, 1, 98–107.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: American Council on Education & Macmillan.

Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Educational Measurement: Issues and Practice, 14(4), 5–8.

Richardson, M. W., & Kuder, F. (1939). The calculation of test reliability coefficients based upon the method of rational equivalence. Journal of Educational Psychology, 30, 681–687.

Schmidt, F. L., Le, H., & Ilies, R. (2003). Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs. Psychological Methods, 8, 206–224.

Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350–353.

Thorndike, R. L. (1949). Personnel selection: Test and measurement techniques. New York, NY: Wiley.

Thurstone, L. L. (1931). The reliability and validity of tests. Ann Arbor, MI: Edwards.

Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54, 229–249.

Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement: Issues and Practice, 15(1), 22–29.

6 Predicting Future Performance

Beatty, A., Barratt, C. L., Berry, C. M., & Sackett, P. R. (2014), Testing the generalizability of indirect range restriction corrections. Journal of Applied Psychology, 99, 587–598.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.

Coward, W. M., & Sackett, P. R. (1990). Linearity of ability-performance relationships: A reconfirmation. Journal of Applied Psychology, 75, 297–300.

Ghiselli, E. E. (1964). Dr. Ghiselli comments on Dr. Tupes’ note. Personnel Psychology, 17, 61–63.

Guion, R. M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum Associates.

Gulliksen, H. (1950). Theory of mental tests. New York, NY: Wiley.

Hawk, J. A. (1970). Linearity of criterion-GATB aptitude relationships. Measurement and Evaluation in Guidance, 2, 249–251.

Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594–612.

Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.

Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates.

Robie, C., & Ryan, A. M. (1999). Effects of nonlinearity and heteroscedasticity on the validity of conscientiousness in predicting overall job performance. International Journal of Selection and Assessment, 7, 157–169.

Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific

applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79, 680–684.

Sackett, P. R., & Yang, H. (2000). Correction for range restriction: an expanded typology. Journal of Applied Psychology, 85, 112–118.

Trafimow, D., & Marks, M. (2015). Editorial. Basic and Applied Social Psychology, 37, 1–2.

7 Using Multivariate Statistics

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). Washington, DC: American Council on Education.

Barrick, M. R., & Mount, M. K. (1993). Autonomy as a moderator of the relationships between the big five personality dimensions and job performance. Journal of Applied Psychology, 78, 111–118.

Bobko, P., Roth, P. L., & Buster, M. A. (2007). The usefulness of unit weights in creating composite scores: A literature review, application to content validity, and meta-analysis. Organizational Research Methods, 10, 689–709.

Claudy, J. G. (1978). Multiple regression and validity estimation in one sample. Applied Psychological Measurement, 2, 595–607.

Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.

Frederiksen, N., & Melville, S. D. (1954). Differential predictability in the use of test scores. Educational and Psychological Measurement, 14, 647–656.

Glaser, R. (1963). Instructional technology and the measurement of learning outcomes. American Psychologist, 18, 519–521.

Glaser, R., & Klaus, D. J. (1962). Proficiency measurement: Assessing human performance. In R. Gagné (Ed.), Psychological principles in system development (pp. 421–427). New York, NY: Holt, Rinehart, & Winston.

Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis. Newbury Park, CA: Sage.

Linn, R. L. (1994). Criterion-referenced measurement: A valuable perspective clouded by surplus meaning. Educational Measurement: Issues and Practice, 13(4), 12–14.

Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529–540.

Schmidt, F. L., & Hunter, J. E. (1981). Employment testing:

Old theories and new research findings. American Psychologist, 36, 1128–1137.

Wherry, R. J. (1931). A new formula for predicting the shrinkage of the coefficient of multiple correlation. Annals of Mathematical Statistics, 2, 446–457.

Witt, L. A., & Ferris, G. R. (2003). Social skill as moderator of the conscientiousnessperformance relationship: Convergent results across four studies. Journal of Applied Psychology, 88, 809–821.

8 Making Judgments and Decisions

Boudreau, J. (1996). The motivational impact of utility analysis and HR measurement. Journal of Human Resource Costing & Accounting, 1 (2), 73–84.

Breaugh, J. A. (2003). Effect size estimation: Factors to consider and mistakes to avoid. Journal of Management, 29, 79–97.

Brogden H. E. (1946). On the interpretation of the correlation coefficient as a measure of predictive efficiency. Journal of Educational Psychology, 37, 65–76.

Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 37, 65–76.

Brooks, M. E., Dalal, D. K., & Nolan, K. P. (2014). Are common language effect sizes easier to understand than traditional effect sizes? Journal of Applied Psychology , 99 , 332–340.

Cascio, W. F. (1993). Assessing the utility of selection decisions: Theoretical and practical considerations. In N. Schmitt & W. C. Borman (Eds.), Personnel selection in organizations (pp. 310–340). San Francisco, CA: Jossey Bass.

Cascio, W. F. (2000). Costing human resources: The financial impact of behavior in organizations (14th ed.). Boston, MA: Kent.

Cleveland, W. S. (1993). Visualizing data . Murray Hill, NJ: Hobart Press.

Cronbach, L. J., & Gleser, G. C. (1957). Psychological tests and personnel decisions . UrbanaChampaign: University of Illinois Press.

Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions (2nd ed.). Urbana-Champaign: University of Illinois Press.

Cronshaw, S. F. (1997). Lo! The stimulus speaks: The insider’s view on Whyte and Latham’s “The futility of utility analysis”. Personnel Psychology, 50, 611–615.

Highhouse, S. (1996). The utility estimate as a communication device: Practical questions and research directions. Journal of Business and Psychology, 11,

85–100.

Highhouse, S., & Kostek, J. A. (2013). Holistic assessment for selection and placement. In K. F. Geisinger, B. A. Bracken, J. F. Carlson, J-I. C. Hansen, N. R. Kuncel, S. P. Reise, & M. C. Rodriguez (Eds.), APA handbook of testing and assessment in psychology (pp. 565– 577). Washington, DC: American Psychological Association.

Hogan, R., & Hogan, J. (1992). Hogan Personality Inventory: Manual . Tulsa, OK: Hogan Assessment Systems.

Johnson, D., & Johnson, R. (1975). Learning together and alone: Cooperation, competition, and individualization . Englewood Cliffs, NJ: Prentice Hall.

Kahneman, D. (1992). Reference points, anchors, norms, and mixed feelings. Organizational Behavior and Human Decision Processes, 51, 296–312.

Kosslyn, S. M. (1989). Understanding charts and graphs. Applied Cognitive Psychology, 3 (3), 185–225.

Kosslyn, S. M. (1994). Image and brain: The resolution of the imagery debate . Cambridge, MA: MIT Press.

Kuncel, N. R. (2008). Some new (and old) suggestions for improving personnel selection. Industrial and Organizational Psychology, 1 (3), 343–346.

Kuncel, N. R., & Rigdon, J. (2013). Communicating research findings. In N. Schmitt & S. Highhouse (Eds.), Handbook of psychology (Vol. 12; pp. 43–58). New York, NY: W ile y.

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11 (1), 65–100.

Latham, G. P., & Whyte, G. (1994). The futility of utility analysis. Personnel Psychology, 47, 31–46.

Lewin, K. (1951). Field theory in social science . New York, NY: Harper.

Macan, T. H., & Highhouse, S. (1994). Communicating the utility of human resource activities: A survey of I/O and HR professionals. Journal of Business and Psychology, 8, 425–436.

McGraw, K. O., & Wong, S. P. (1992). A common language effect size statistic. Psychological Bulletin, 111, 361–365.

Meehl, P. E. (1954). Clinical versus statistical prediction . Minneapolis: University of Minnesota Press.

Muchinsky, P. M. (2004). When the psychometrics of test development meets organizational realities: A conceptual framework for organizational change, examples, and recommendations. Personnel Psychology, 57, 175–209.

Pinker, S. (1990). A theory of graph comprehension. In R. Frele (Ed.), Artificial intelligence and the future of testing (pp. 73–126). Hillsdale, NJ: Erlbaum.

Schmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow, T. W. (1979). Impact of valid selection procedures on work-force productivity. Journal of Applied Psychology , 64 , 609–626.

Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14, 47–69.

Slovic, P., Peters, E., Finucane, M. L., & MacGregor, D. G. (2005). Affect, risk, and decision making. Health Psychology, 24 (4S), S35.

Taylor, H. C., & Russell. J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection. Journal of Applied Psychology, 23, 565–578.

Tversky, A., & Kahneman, D. (1991). Loss aversion in riskless choice: A reference-dependent model. The Quarterly Journal of Economics, 106 (4), 1039–1061.

Vinchur, A. J., Schippmann, J. S., Switzer, F. S., III, & Roth, P. L. (1998). A meta-analytic review of predictors of job performance for salespeople. Journal of Applied Psychology, 83, 586–597.

Westen, D., & Weinberger, J. (2004). When clinical description becomes statistical prediction. American Psychologist, 59, 595–613.

Whyte, G., & Latham, G. (1997). The futility of utility analysis revisited: When even an expert fails. Personnel Psychology, 50, 601–610.

9 Analyzing Bias and Ensuring Fairness

Aguinis, H., Cascio, W., Goldstein, I., Outtz, J., & Zedeck, S. (2009). In The Supreme Court

of the United States: Ricci v . DeStefano : Brief of Industrial-Organizational Psychologists

as Amici Curiae in support of respondents.

Allen v . Alabama State Board of Education, No. 81–697-N (consent decree filed with United

States District Court for the Middle District of Alabama Northern Division, 1985).

Arthur W., Jr., & Doverspike, D. (2005). Achieving diversity and reducing discrimination

in the workplace through human resource management practices: Implications of

research and theory for staffing, training, and rewarding performance. In R. L. Dip

boye & A. Colella (Eds.), Discrimination at work: The psychological and organizational bases

(pp. 305–327). Mahwah, NJ: Lawrence Earlbaum Associates.

Arthur W., Jr., Doverspike, D., Barrett, G. V., & Miguel, R. (2013). Chasing the Title VII

holy grail: The pitfalls of guaranteeing adverse impact elimination. Journal of Business and

Psychology, 28, 473–485.

Arthur W., Jr., & Villado, A. J. (2008). The importance of distinguishing between con

structs and methods when comparing predictors in personnel selection research and

practice. Journal of Applied Psychology, 93, 435–442.

Civil Rights Act of 1964, 42 U.S.C. Section 2000e (1964).

Civil Rights Act of 1991, 42 U.S.C. Section 1981A (1991).

Cleary, T. A. (1968). Test bias: Prediction of grades of Negro and white students in inte

grated colleges. Journal of Educational Measurement, 5, 115–124.

Cohen, D. B., Aamodt, M. G., and Dunleavy, E. M. (2010). Technical advisory committee report on

best practices in adverse impact analyses . Washington, DC: Center for Corporate Equality.

Cole, N. S., & Moss, P. A. (1989). Bias in test use. In R. L. Linn (Ed.), Educational measurement

(pp. 201–219). New York, NY: American Council on Education/Macmillan.

Cullen, M. J., Hardison, C. H., & Sackett, P. R. (2004). Using SAT-grade and ability-job

performance relationships to test predictions derived from stereotype threat theory. Jour

nal of Applied Psychology, 89, 220–230.

Doverspike, D. (2014, October 1). Thoughts on Adverse Impact: Part 1. Assessment Services Review

Blog . Retrieved from

Drasgow, F. (1987). Study of the measurement bias of two standardized psychological tests.

Journal of Applied Psychology, 72, 19–29.

Einhorn, H. J., & Bass, A. R. (1971). Methodological considerations relevant to discrimina

tion in employment testing. Psychological Bulletin, 75, 261–269.

Equal Employment Opportunity Commission, Civil Service Commission, Department of

Labor, & Department of Justice. (1978). Uniform guidelines on employee selection

procedures. Federal Register, 43 (166), 38290–38315.

Golden Rule Insurance Company et al . v . Washburn et al ., No. 419–76 (stipulation for dismissal

and order dismissing cause, Circuit Court of Seventh Judicial Circuit, Sangamon

County, IL, 1984).

Guion, R. M. (1966). Employment tests and discriminatory hiring. Industrial Relations, 5,

20–37.

Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity gener

alization, minority issues, and the General Aptitude Test Battery . Washington, DC: National

Academy Press.

Hazelwood School District v . United States, 433, U.S. 299 (1977).

Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning . Hillsdale, NJ: Law

rence Erlbaum Associates.

Hunter, J. E. (1983). Test validation for 12,000 jobs: An application of job classification and validity

generalization analysis to the General Aptitude Test Battery (GATB) (Test Research Report

No. 45). Washington, DC: United States Employment Service, United States Depart

ment of Labor.

Ironson, G. H., Guion, R. M., & Ostrander, M. (1982). Adverse impact from a psychometric

perspective. Journal of Applied Psychology, 67, 419–432.

Lawshe, C. H. (1979). Shrinking the cosmos: A practitioner’s thoughts on alternative selection

procedures. In P. Griffin (Ed.), The search for alternative selection procedures: Developing a profes

sional stand (pp. 1–26). Los Angeles, CA: Personnel Testing Council of Southern

California.

Lawshe, C. H. (1987). Adverse impact: Is it a viable concept? Professional Psychology: Research

and Practice, 18, 492–497.

Nguyen, H.H.D., & Ryan, A. M. (2008). Does stereotype threat affect test performance of

minorities and women? A meta-analysis of experimental evidence. Journal of Applied

Psychology, 93, 1314–1334.

Ployhart, R. E., & Holtz, B. C. (2008). The diversity–validity dilemma: Strategies for reduc

ing racioethnic and sex subgroup differences and adverse impact in selection. Personnel

Psychology, 61, 153–172.

Sackett, P. R., Hardison, C. M., & Cullen, M. J. (2004). On interpreting stereotype threat as

accounting for African American–White differences on cognitive tests. American Psy

chologist, 59 (1), 7–13.

Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in

employment, credentialing, and higher education: Prospects in a post-affirmative-action

world. American Psychologist, 56, 302–318.

Steele, C. M., & Aronson, J. (1995). Stereotype threat and the intellectual performance of

Afr ican Amer icans. Journal of Personality and Social Psychology, 69, 797–811.

Stricker, L. J., & Ward, W. C. (2004). Stereotype threat,

inquiring about test takers’ ethnicity

and gender, and standardized test performance. Journal of Applied Social Psychology, 34,

665–693.

United States Commission on Civil Rights. (1993). The validity of testing in education and

employment . Washington, DC: U.S. Commission on Civil Rights. This page intentionally left blank

10 Assessing via Traditional Tests

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50, 179–211.

Arthur, W. A., Jr., Doverspike, D., Muñoz, G. J., Taylor, J. E., & Carr, A. E. (2014). The use of mobile devices in high-stakes remotely delivered assessments and testing. International Journal of Selection and Assessment, 22, 113–123.

Arthur, W. A., Jr., Glaze, R. M., Jarrett, S. M., White, C. D., Schurig, I., & Taylor, J. E. (2014). Comparative evaluation of three situational judgment test response formats in terms of construct-related validity, subgroup differences, and susceptibility to response distortion. Journal of Applied Psychology, 99, 335–345.

Below, S. (2014). New year, new workplace! SIOP announces top 10 workplace trends for 2015. Society for Industrial and Organizational Psychology, Inc . Retrieved from http://www. siop.org/article_view.aspx?article=1343

Brooks, M. E., & Highhouse, S. (2006). Can good judgment be measured? In J. A. Weekley & R. E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 39–55). SIOP Frontier Series. San Francisco, CA: Jossey Bass.

Campion, M. A. (1983). Personnel selection for physically demanding jobs: Review and recommendations. Personnel Psychology, 36, 527–550.

Campion, M. C., Ployhart, R. E., & MacKenzie, W. I., Jr. (2014). The state of research on situational judgment tests: A content analysis and directions for future research. Human Performance, 27, 283–310.

Carless, S. A. (2006). Applicant reactions to multiple selection procedures for the police force. Applied Psychology: An International Review, 55, 145–167.

Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143–159.

Chan, D., & Schmitt, N. (2002). Situational judgment and

job performance. Human Performance, 15, 233–254.

Dalessio, A. T. (1994). Predicting insurance agent turnover using a video-based situational judgment test. Journal of Business and Psychology, 9, 23–32.

Fleishman, E. A., & Reilly, M. E. (1992). Handbook of human abilities: Definitions, measurements and job task requirements . Palo Alto, CA: Consulting Psychologists Press.

Gebhardt, D. L., & Baker, T. A. (2010). Physical performance. In J. C. Scott & D. H. Reynolds (Eds.), Handbook of workplace assessment: Evidence-based practices for selecting and developing organizational talent (pp. 165–196). San Francisco, CA: Jossey Bass.

Guion, R. M. (1965). Personnel testing . New York, NY: McGraw-Hill.

Hattrup, K., Schmitt, N., & Landis, R. S. (1992). Equivalence of constructs measured by job-specific and commercially available aptitude tests. Journal of Applied Psychology, 77, 298–308.

Hedge, J. W., Teachout, M. S., & Laue, F. J. (1990). Interview testing as a work sample measure of job proficiency . AFHRL-TP-89–60. Brooks Air Force Base, TX: Air Force Systems Command.

Henderson, N. D. (2010). Predicting long-term firefighter performance from cognitive and physical ability measures. Personnel Psychology, 63, 999–1039.

Henderson, N. D., Berry, M. W., & Matic, T. (2007). Field measures of strength and fitness predict firefighter performance on physically demanding tasks. Personnel Psychology, 60, 431–473.

Hogan, J. (1991b). Structure of physical performance in

occupational tasks. Journal of Applied Psychology, 76, 495–507.

Hoover, L. T. (1992). Trends in police physical ability selection testing. Public Personnel Management, 21, 29–40.

Maher, P. T. (1984). Police physical ability tests: Can they ever be valid? Public Personnel Management Journal, 13, 173–183.

McCormick, E. J., & Ilgen, D. R. (1980). Industrial psychology (7th ed.). Englewood Cliffs, NJ: Prentice-Hall.

McDaniel, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9, 103–113.

Murphy, K. R. (2009). Content validation is useful for many things, but validity isn’t one of them. Industrial and Organizational Psychology, 4, 453–464.

Ployhart, R. E., & Ehrhart, M. G. (2003). Be careful what you ask for: Effects of response instructions on the construct validity and reliability of situational judgment tests. International Journal of Selection and Assessment, 11, 1–16.

Ryan, A. M., Greguras, G. J., & Ployhart, R. E. (1996). Perceived job relatedness of physical ability testing for firefighters: Exploring variations in reactions. Human Performance, 9, 219–240.

Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.

Schmit, M. J., Kihm, J., & Robie, C. (2000). Development of a global measure of personality. Personnel Psychology , 53 , 153–193.

Tippins, N. T. (2009). Internet alternatives to traditional proctored testing: Where are we now? Industrial and Organizational Psychology, 2, 2–10.

11 Assessing via Inventories and Interviews

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior & Human Decision Processes, 50, 179–211.

Anderson, C. W. (1960). The relation between speaking times and decision in the employment interview. Journal of Applied Psychology, 44, 267–268.

Baier, D. E., & Dugan, R. D. (1957). Factors in sales success. Journal of Applied Psychology, 41, 37–40.

Bentz V. J. (1967). The Sears experience in the investigation, description, and prediction of executive behavior. In F. R. Wickert & D. E. McFarland (Eds.), Measuring executive effectiveness (pp. 147–205). New York, NY: Appleton-Century-Crofts.

Bernadin, H. J. (1987). Development and validation of a forced choice scale to measure job-related discomfort among customer service representatives. Academy of Management Journal, 30, 162–173.

Bing, M. N., Davison, H. K., & Smothers, J. (2014). Item-level frame-of-reference effects in personality testing: An investigation of incremental validity in an organizational setting. International Journal of Selection and Assessment, 22 (2), 165–178.

Breaugh, J. A., & Dossett, D. L. (1989). Rethinking the use of personal history information: The value of theory-based biodata for predicting turnover. Journal of Business & Psychology, 3, 371–385.

Campion, M. A., Pursell, E. D., & Brown, B. K. (1988). Structured interviewing: Raising the psychometric properties of the employment interview. Personnel Psychology, 41, 25–42.

Carrier, M. R., Dalessio, A. T., & Brown, S. H. (1990). Correspondence between estimates of content and criterion-related validity values. Personnel Psychology, 43, 85–100.

Chapman, D. S., & Zweig, D. I. (2005). Developing a nomological network for interview structure: Antecedents and consequences of the structured selection interview. Personnel Psychology, 58, 673–702.

Conway, J. M., Jako, R. A., & Goodman, D. F. (1996). A meta-analysis of interrater and internal consistency reliability of selection interviews. Journal of Applied Psychology, 80, 565–579.

Daniels, H. W., & Otis, J. L. (1950). A method for analyzing employment interviews. Personnel Psychology, 3, 425–444.

Dean, M. A., Russell, C. J., & Muchinsky, P. M. (1999). Life experiences and performance prediction: Toward a theory of biodata. Research in Human Resources Management, 17, 245–281.

Dipboye, R. L. (1997). Structured selection interviews: Why do they work? Why are they underutilized? In N. Anderson & P. Herriot (Eds.), International handbook of selection and assessment (pp. 455–473). New York, NY: Wiley.

Dougherty, T. W., Ebert, R. J., & Callender, J. C. (1986). Policy capturing in the employment interview. Journal of Applied Psychology, 71, 9–15.

Dwight, S. A., & Donovan, J. J. (2003). Do warnings not to fake reduce faking? Human Performance, 16, 1–23.

Ellis, A.P.J., West, B. J., Ryan, A. M., & DeShon, R. P. (2002). The use of impression management tactics in structured interviews: A function of question type? Journal of Applied Psychology, 87, 1200–1208.

Furnham, A., & Jackson, C. J. (2011). Practitioner reactions to work-related psychological tests. Journal of Managerial Psychology, 26 (7), 549–565.

Gehrlein, T. M., Dipboye, R. L., & Shahani, C. (1993). Nontraditional validity calculations and differential interviewer experience: implications for selection interviewers. Educational and Psychological Measurement, 52, 457–469.

Gilovich, T. (1991). How we know what isn’t so: The fallibility of human reason in everyday life . New York, NY: The Free Press.

Goffin, R. D., & Christiansen, N. D. (2003). Correcting personality tests for faking: A review of popular personality tests and an initial survey of researchers. International Journal of Selection and Assessment, 11,

340–344.

Guion, R. M. (1965). Personnel testing . New York, NY: McGraw-Hill.

Guion, R. M. (1987). Changing views for personnel selection. Personnel Psychology, 40, 199–213.

Harris, M. M. (1989). Reconsidering the employment interview: A review of recent literature and suggestions for future research. Personnel Psychology, 42, 691–726.

Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639–683.

Highhouse, S. (2002). Assessing the candidate as a whole: A historical and critical analysis of individual psychological assessment for personnel decision making. Personnel Psychology, 55, 363–396.

Hollingworth, H. L. (1923). Judging human character . New York, NY: Appleton.

Hough, L. M., & Oswald, F. L. (2008). Personality testing and I-O psychology: Reflections, progress and prospects. Industrial and Organizational Psychology: Perspectives on Science and Practice , 1 , 272–290.

Hough, L., & Tippins, N. (1994, April). New designs for selection and placement systems: The Universal Test Battery. In N. Schmitt (Chair), Cutting edge developments in selection . Symposium at meeting of the Society for Industrial and Organizational Psychology, Nashville, TN.

Huffcutt, A. I., & Arthur, W., Jr. (1994). Hunter and Hunter (1984) revisited: Interview validity for entry-level jobs. Journal of Applied Psychology, 79, 184–190.

Huffcutt, A. I., & Culbertson, S. S. (2011). Interviews. In S. Zedeck (Ed.), APA handbook of industrial and organizational psychology (pp. 185–204). Washington, DC: American Psycholo g ical Association.

Huffcut, A. I., & Roth, P. L. (1998). Racial group differences in employment interview evaluations. Journal of Applied Psychology, 83, 179–189.

Jackson, D. N., & Messick, S. (1958). Content and style in personality assessment. Psychological Bulletin, 55, 243–252.

James, L. R. (1998). Measurement of personality via conditional reasoning. Organizational Research Methods, 1, 131–163.

James, L. R., McIntyre, M. D., Glisson, C. A., Bowler, J. L., & Mitchell, T. R. (2004). The conditional reasoning measurement system for aggression: An overview. Human Performance, 17, 271–295.

Janz, T., Hellervik, L., & Gilmore, D. C. (1986). Behavior description interviewing . Boston, MA: Allyn & Bacon.

Judge, T. A., Higgins, C. A., & Cable, D. M. (2000). The employment interview: A review of recent research and recommendations for future research. Human Resource Management Review, 10, 383–406.

Judge, T. A., Klinger, R., Simon, L. S., & Yang, I.W.F. (2008). The contributions of personality to organizational behavior and psychology: Findings, criticisms, and future research directions. Social and Personality Psychology Compass, 2, 1982–2000.

Kinicki, A. J., Lockwood, C. A., Hom, P. W., & Griffeth, R. W. (1990). Interviewer predictions of applicant qualifications and interviewer validity: Aggregate and individual analyses. Journal of Applied Psychology, 75, 477–486.

König, C. J., Klehe, U. C., Berchtold, M., & Kleinmann, M. (2010). Reasons for being selective when choosing personnel selection procedures. International Journal of Selection and Assessment , 18 (1), 17–27.

Kristof-Brown, A., Barrick, M. R., & Franke, M. (2002). Applicant impression management: Dispositional influences and consequences for recruiter perceptions of fit and similarity. Journal of Management, 28, 27–46.

Latham, G. P. (1989). The reliability, validity, and practicality of the situational interview. In R. W. Eder & G. R. Ferris (Eds.), The employment interview: Theory, research, and practice (pp. 169–182). Newbury Park, CA: Sage.

Latham, G. P., & Wexley, K. N. (1977). Behavioral

observation scales for performance appraisal. Personnel Psychology, 30, 255–268.

Latham, G. P., & Wexley, K. N. (1981). Increasing productivity through performance appraisal . Reading, MA: Addison-Wesley.

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575.

Lievens, F., Highhouse, S., & De Corte, W. (2005). The importance of traits and abilities in supervisors’ hirability decisions as a function of method of assessment. Journal of Occupational and Organizational Psychology, 78, 453–470.

Lilienfeld, S. O., Wood, J. M., & Garb, H. N. (2000). The scientific status of projective techniques. Psychological Science in the Public Interest, 1 (2), 27–66.

Lin, T. R., Dobbins, G. H., & Farh, J. (1992). A field study of race and age similarity effects on interview ratings in conventional and situational interviews. Journal of Applied Psychology, 77, 367–371.

Mael, F. A. (1991). A conceptual rationale for the domain and attribute of biodata items. Personnel Psychology, 44, 763–792.

Mael F. A. (1993). Rainforest empiricism and quasi-rationality: Two approaches to objective biodata. Personnel Psychology, 46, 719–738.

Mattioli, D. (2012, August 23). On Orbitz, Mac users steered to pricier hotels . Retrieved from http://online .wsj.com/ne ws/articles/

McCarthy, J. M., Van Iddekinge, C. H., Lievens, F., Kung, M. C., Sinar, E. F., & Campion, M. A. (2013). Do candidate reactions relate to job performance or affect criterion-related validity? A multistudy investigation of relations among reactions, selection test scores, and job performance. Journal of Applied Psychology, 98, 701–719.

McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599–616.

McMurry, R. N. (1947). Validating the patterned interview.

Personnel, 23, 263–272.

Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007a). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683–729.

Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007b). Are we getting fooled again? Coming to terms with limitations in the use of personality tests for personnel selection. Personnel Psychology, 60, 1029–1049.

Nolan, K. P., & Highhouse, S. (2014). Need for autonomy and resistance to standardized employee selection practices. Human Performance, 27 (4), 328–346.

Nowicki, M. D., & Rosse, J. G. (2002). Managers’ views of how to hire: Building bridges between science and practice. Journal of Business and Psychology, 17, 157–170.

Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge, T. A. (2007). In support of personality assessment in organizational settings. Personnel Psychology, 60, 995–1027.

Posthuma, R. A., Morgeson, F. P., & Campion, M. A. (2002). Beyond employment interview validity: A comprehensive narrative review of recent research and trends over time. Personnel Psychology, 55, 1–81.

Prewett-Livingston, A. J., Feild, H. S., Veres, J. G., III, & Lewis, P. M. (1996). Effects of race on interview ratings in a situational panel interview. Journal of Applied Psychology, 81, 178–186.

Reilly, R. R., & Chao, G. T. (1982). Validity and fairness of some alternative employee selection procedures. Personnel Psychology, 35, 1–62.

Ryan, A. M., & Sackett, P. R. (1987). Pre-employment honesty testing: Fakability, reactions of test takers, and company image. Journal of Business and Psychology, 1, 248–256.

Schmidt, F. L. & Zimmerman R. D. (2004). A counterintuitive hypothesis about employment interview validity and some supporting evidence. Journal of Applied Psychology, 89, 553–561.

Schmitt, N. (1976). Social and situational determinants of interview decisions: Implications for the employment interview. Personnel Psychology, 29, 79–101.

Shaffer, J. A., & Postlethwaite, B. E. (2012). A matter of context: A meta-analytic investigation of the relative validity of contextualized and noncontextualized personality measures. Personnel Psychology, 65, 445–494.

Shaw, J. (2014). Why “Big Data” is a big deal. Harvard Magazine, 3, 30–35.

Sitser, T., van der Linden, D., & Born, M. P. (2013). Predicting sales performance criteria with personality measures: The use of the general factor of personality, the big five and narrow traits. Human Performance, 26 (2), 126–149.

Society of Human Resource Management. (2012, January 4). SHRM Poll: Most Employers Don’t Use Personality Tests . Retrieved from http://www.shrm.org/hrdisciplines/ staffingmanagement/

Sparks, C. P. (1990). Testing for management potential. In K. E. Clark & M. B. Clark (Eds.), Measures of leadership (pp. 103–111). West Orange, NJ: Library of America.

Taylor, P. J., Pajo, K., Cheung, G. W., & Stringfield, P. (2004). Dimensionality and validity of a structured telephone reference check procedure. Personnel Psychology, 57, 745–772.

Tversky, A., & Kahneman, D. (1982). Judgment of and by representativeness. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 84–98). Cambridge, England: Cambridge University Press.

van der Zee, K. I., Bakker, A. B., & Bakker, P. (2002). Why are structured interviews so rarely used in personnel selection? Journal of Applied Psychology, 87, 176–184.

Viglione, D. J., & Hilsenroth, M. J. (2001). The Rorschach: Facts, fictions, and future. Psychological Assessment, 13, 452–471.

Villanova, P., Bernardin, H., Johnson, D. L., & Dahmus, S. A. (1994). The validity of a measure of job compatibility in the prediction of job performance and turnover of motion picture theater personnel. Personnel Psychology, 47, 73–90.

Vinchur, A. J., Schippmann, J. S., Switzer, F. S., III, & Roth, P. L. (1998). A meta-analytic review of predictors of job performance for salespeople. Journal of Applied Psychology, 83, 586–597.

Wagner, R. (1949). The employment interview: A critical summary. Personnel Psychology, 2, 17–46.

Watson v . Fort Worth Bank & Trust, 108 S. Ct. 2777 (1988).

12 Assessing via Ratings

Balzer, W. K. (1986). Biases in the recording of performance-related information: The effects of initial impression and centrality of the appraisal task. Organizational Behavior and Human Decision Processes, 87, 707–721.

Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77, 975–985.

Bernardin, H. J., & Beatty, R. W. (1984). Performance appraisals: Assessing human behavior at work . Boston, MA: Kent Publishing.

Bernardin, H. J., & Buckley, M. R. (1981). Strategies in rater training. Academy of Management Review, 6, 205–212.

Bernardin, H. J., & Smith, P. C. (1981). A clarification of some issues regarding the development and use of behaviorally anchored rating scales (BARS). Journal of Applied Psychology, 66, 458–463.

Borman, W. C. (1979). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology, 64, 410–421.

Borman, W. C. (1986). Behavior-based rating scales. In R. A. Berk (Ed.), Performance assessment: Methods and applications (pp. 100–120). Baltimore, MD: Johns Hopkins University Press.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.

Cole, M. (1953). Robert Owen of New Lanark . New York, NY: Oxford University Press.

Cooper, W. H. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218–244.

Cronbach, L. J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30, 116–127.

Day, D. V., & Sulsky, L. M. (1995). Effects of frame-of-reference training and information configuration on memory organization and rating accuracy. Journal of Applied Psychology, 80, 158–167.

Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York, NY: McGraw-Hill.

Judge, T. A., & Cable, D. M. (2004). The effect of physical height on workplace success and income: Preliminary test of a theoretical model. Journal of Applied Psychology, 89, 428–441.

Kane, J. S. (1987, April 22). Wish I may, wish I might, wish I could do performance appraisal right . Unpublished manuscript, School of Management, University of Massachusetts, Amherst, MA.

Kraiger, K. (1990, April). Generalizability of performance measures across four Air Force specialties (Technical Paper AFHRL-TP-89–60). Brooks AFB, TX: Air Force Systems Command.

Kraiger, K., & Ford, J. K. (1985). A meta-analysis of ratee race effects in performance ratings. Journal of Applied Psychology, 70, 56–65.

Landy, F. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72–107.

Latham, G. P., & Wexley, K. N. (1981). Increasing productivity through performance appraisal . Reading, MA: Addison-Wesley.

Lawshe, C. H., & Balma, M. J. (1966). Principles of personnel testing (2nd ed.). New York, NY: McGraw-Hill.

Lawshe, C. H., Kephart, N. C., & McCormick, E. J. (1949). The paired comparison technique for rating performance of industrial employees. Journal of Applied Psychology, 33, 69–77.

Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30, 881–905.

Murphy, K. R., Balzer, W. K., Lockhart, M. C., & Eisenman, E. J. (1985). Effects of previous performance on evaluations of present performance. Journal of Applied

Psychology, 70, 72–84.

Murphy, K., & Cleveland, J. (1991). Performance appraisal: An organizational perspective . Boston, MA: Allyn & Bacon.

Murphy, K., & Cleveland, J. (1995). Understanding performance appraisal: Social, organizational and goal-oriented perspectives . Newbury Park, CA: Sage.

Oppler, S. H., Peterson, N. G., & McCloy, R. A. (1994, April). A comparison of peer and supervisory ratings as criteria for the validation of predictors . Paper presented to the Society for Industrial and Organizational Psychology, Nashville, TN.

Putka, D. J., Hoffman, B. J., & Carter, N. T. (2014). Correcting the correction: When individual rater s offer distinct b ut valid perspectives. Industrial and Organizational Psychology, 7 (4), 543–548.

Sackett, P. R., & DuBois, C.L.Z. (1991). Rater-ratee race effects on performance evaluation: Challenging meta-analytic conclusions. Journal of Applied Psychology, 76, 873–877.

Schoorman, F. D. (1988). Escalation bias in performance evaluations: An unintended consequence of supervisor participation in hiring decisions. Journal of Applied Psychology, 73, 58–62.

Sisson, E. D. (1948). Forced choice—The new army rating. Personnel Psychology, 1, 365–381.

Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47, 149–155.

Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25–29.

Tinsley, H.E.A., & Weiss, D. J. (1975). Interrater reliability and agreement of subjective judgements. Journal of Counseling Psychology, 22 , 358–376.

Waung, M., & Highhouse, S. (1997). Fear of conflict and empathic buffering: Two explanations for the inflation of performance feedback. Organizational Behavior and Human Decision Processes, 71, 37–54.

13 Individual and Group Assessment

Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion

related validity of assessment center dimensions. Personnel Psychology, 56, 125–154.

Bentz V. J. (1967). The Sears experience in the investigation, description, and prediction of

executive behavior. In F. R. Wickert & D. E. McFarland (Eds.), Measuring executive effec

tiveness (pp. 147–205). New York, NY: Appleton-Century-Crofts.

Bingham, W. V., & Freyd, M. (1926). Procedures in employment psychology: A manual for devel

oping scientific methods of vocational selection . New York, NY: McGraw-Hill.

Camerer C. F., & Johnson E. J. (1991). The process-performance paradox in expert judg

ment: How can experts know so much and predict so badly? In K. A. Ericsson &

J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 195–217).

Cambridge, England: Cambridge University Press.

Collins, J. M., Schmidt, F. L., Sanchez-Ku, M., Thomas, L., McDaniel, M. A., & Le, H.

(2003). Can basic individual differences shed light on the construct meaning of assess

ment center evaluations? International Journal of Selection and Assessment, 11, 17–29.

Dilchert, S., & Ones, D. S. (2009). Assessment center dimensions: Individual differences cor

relates and meta-analytic incremental validity. International Journal of Selection and Assess

ment , 17, 254–270.

Freyd, M. (1923). Measurement in vocational selection: An outline of research procedure.

Journal of Personnel Research, 2, 215–249, 268–284, 377–385.

Freyd, M. (1925). The statistical viewpoint in vocational selection. Journal of Applied Psychol

ogy, 9, 349–356.

Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., III, & Bentson, C. (1987). Meta-analysis

of assessment center validity. Journal of Applied Psychology, 72, 493–511.

Hanson, C. P., & Conrad, K. A. (1991). A handbook of psychological assessment in business . New

York, NY: Quorum Books.

Highhouse, S. (2002). Assessing the candidate as a whole: A historical and critical analysis

of individual psychological assessment for personnel decision making. Personnel Psychol

ogy, 55, 363–396.

Highhouse, S. (2008). Stubborn reliance on intuition and subjectivity in employee selection.

Industrial and Organizational Psychology, 1 (3), 333–342.

Huse, E. F. (1962). Assessments of higher level personnel. IV. The validity of assessment

techniques based on systematically varied information. Personnel Psychology, 15,

195–205.

Jeanneret, R., & Silzer, R. (1998). An overview of psychological assessment. In R. Jeanneret

& R. Silzer (Eds.), Individual psychological assessment: Predicting behavior in organizational

settings (pp. 3–26). San Francisco, CA: Jossey-Bass.

Joyce, L. W., Thayer, P. W., & Pond, S. B., III. (1994). Managerial functions: An alternative to

traditional assessment center dimensions? Personnel Psychology, 47, 109–121.

Kuncel, N. R., Klieger, D. M., Connelly, B. S., & Ones, D. S. (2013). Mechanical versus clini

cal data combination in selection and admissions decisions: A meta-analysis. Journal of

Applied Psychology, 98, 1060–1072.

Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the

evidence . Minneapolis: University of Minnesota Press.

Meyer, H. H. (1956). An evaluation of a supervisory selection program. Personnel Psychology,

9, 499–513.

Miner, J. B. (1970). Executive and personnel interviews as predictors of consulting success.

Personnel Psychology, 23, 521–538.

Morris, S. B., Kwaske, I. H., & Daisley, R. R. (2011). The validity of individual psychological

assessments. Industrial and Organizational Psychology, 4 (3), 322–326.

Office of Strategic Services. (1948). Assessment of men . New York, NY: Rinehart.

Oldfield, R. S. (1947). The psychology of the interview . London, England: Methuen.

Reilly, R. R., Henry, S., & Smither, J. W. (1990). An examination of the effects of using

behavior checklists on the construct validity of assessment center dimensions. Personnel

Psychology, 43, 71–84.

Ryan, A. M., Daum, D., Bauman, T., Grisez, M., Mattimore, K., Nalodka, T., & McCormick,

S. (1995). Direct, indirect, and controlled observation and rating accuracy. Journal of

Applied Psychology, 80, 664–670.

Ryan, A. M., & Sackett, P. R. (1987). A survey of individual assessment practices by I/O

psychologists. Personnel Psychology, 40, 455–488.

Ryan, A. M., & Sackett, P. R. (1989). Exploratory study of individual assessment practices:

Interrater reliability and judgments of assessor effectiveness. Journal of Applied Psychology,

74, 568–579.

Sackett, P. R., & Dreher, G. F. (1982). Constructs and assessment center dimensions: Some

troubling empirical findings. Journal of Applied Psychology, 67, 401–410.

Sackett, P. R., & Wilson, M. A. (1982). Factors affecting the consensus judgment process in

managerial assessment centers. Journal of Applied Psycholog y , 67, 10–17.

Sarbin, T. L. (1943). A contribution to the study of actuarial and individual methods of

prediction. American Journal of Sociology, 48, 598–602.

Scott, W. D., & Clothier, R. C. (1923). Personnel management . Chicago, IL: A. W. Shaw.

Silverman, W. H., Dalessio, A., Woods, S. B., & Johnson, R. L., Jr. (1986). Influence of assess

ment center methods on assessors’ ratings. Personnel Psychology, 39, 565–578.

Silzer, R., & Jeanneret, R. (2011). Individual psychological assessment: A practice and sci

ence in search of common ground. Industrial and Organizational Psychology, 4 (3),

270–296.

Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the

construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47,

149–155.

Sparks, C. P. (1990). Testing for management potential. In K. E. Clark & M. B. Clark (Eds.),

Measures of leadership (pp. 103–112). West Orange, NJ: Leadership Library of America.

Thornton, G. C., III, & Byham, W. C. (1982). Assessment centers and managerial performance .

New York, NY: Academic Press.

Viteles, M. S. (1925). The clinical viewpoint in vocational selection. Journal of Applied Psy

chology, 9, 131–138.

Woehr, D. J. & Arthur, W. (2003). The construct related validity of assessment center rat

ings: A review and meta-analysis of the role of methodological factors. Journal of Manage

ment, 29, 231–258.

Zedeck, S. (1986). A process analysis of the assessment center method. In B. M. Staw & L. L.

Cummings (Eds.), Research in organizational behavior (Vol. 8, pp. 259–296). Greenwich,

CT: JAI Press. This page intentionally left blank

Cover
Title
Copyright
CONTENTS
Preface
PART I Deciding What to Assess
1 Understanding Personnel Assessment
References