Master forum 602

JOURNAL OF BUSINESS AND PSYCHOLOGY Volume 8, No. 2, Winter 1993

FAKABILITY OF A COMMERCIALLY PRODUCED PRE-EMPLOYMENT

INTEGRITY TEST

Steven G. LoBello Benjamin N. Sims

Auburn University at Montgomery

ABSTRACT: Three groups of male inmates were given the PSC Survey ADT, a commercially available integrity test, under differing instructional conditions. Subjects told to fake good were ahle to provide more favorable test profiles than those who were told to respond truthfully or who were given no specific response set to follow. However, no differences were observed among groups on the Drink- ing and Drugs subscale. Using only recommended cut scores as criteria, subjects in the fake good condition would have been hired in greater numbers than sub- jects in the other two conditions. Practical implications of this study include the use of social desirability scales along with integrity tests, and the avoidance of inducing inappropriate response sets by strictly adhering to test instructions.

The passage of the Employee Polygraph Protection Act of 1988 eliminated the routine screening of potential employees with polygraph examinations and led to increased reliance on paper-and-pencil tests of integrity. These tests are designed to identify employees or potential employees who may be unreliable or untrustworthy. Sackett and Harris (1984) reported that 5000 companies were using pre-employment integ- rity tests, primarily to screen job applicants who would assume respon- sibilities for handling money or merchandise. O'Bannon, Goldinger and Appleby (1989) reported that over 2.5 million integrity tests are admin- istered annually and summarized information on over 40 commercially available tests. Although Massachusetts has passed legislation banning the use of any type of predictor in hiring, and Rhode Island does not

This study was supported by a grant from Auburn University at Montgomery. We gratefully acknowledge tbe support and assistance of Warden Leoneal Davis and

Dr. Paul Van Wyk of the Bullock County Correctional Facility and tbe Alabama Depart- ment of Corrections in conducting tbis study.

Address correspondence to Steven G. LoBello, Ph.D., Auburn University at Mont- gomery, 7300 University Drive, Montgomery, Al 36117-3596.

265 © 1993 Human Sciences Press. Inc.

266 JOURNAL OF BUSINESS AND PSYCHOLOGY

allow the use of pre-employment integrity tests as the sole predictor of employee potential (Sackett, Burris, & Callahan, 1989), there is a grow- ing demand for such assessment instruments.

Integrity and honesty tests have been found valid and useful when used in the selection of personnel from among a group of applicants. Ones, Viswesvaran, & Schmidt (in press) performed a comprehensive meta-analysis on 665 validity coefficients (cumulative sample size ap- proached 600,000) and report a coefficient of .41 as the predictive val- idity of integrity tests in predicting supervisors' ratings. Their findings also revealed that integrity tests are better at predicting broadly de- fined counterproductive behaviors than the more narrow criterion of employee theft. The American Psychological Association (1991) issued a report by a task force commissioned to "gather and examine data rele- vant to the scientific and social policy considerations associated with the development and use of predictors of dishonesty, theft, and related be- haviors" (p. 4). This report supports the use of integrity tests in employ- ment selection, but points out that some test publishers adhere to stan- dards of test development and use, while others do not. Because of this, there is a continuing need to ensure that test publishers provide instru- ments of the highest quality.

One area of concern about integrity tests is the possibility of dissim- ulation on the tests. Hough, Eaton, Dunnette, Kamp and McCloy (1990) summarize research that shows that people instructed to do so can dis- tort responses to many types of personality inventories. Moore (1990) found that his samples of college students and behavioral scientists were both capable of producing favorable test profiles on the Phase II Profile when instructed to respond to the test items as though applying for a desired job. Ryan and Sackett (1987) randomly assigned college students to groups which were instructed to either fake good, respond truthfully, or respond as if applying for a job. Subjects then completed a 74-item scale that was patterned after existing honesty tests. Results indicated that subjects instructed to fake good earned more favorable scores on scales that measured attitudes toward theft, admissions of theft, and social desirability. However, subjects asked to respond as though applying for a job scored more like those asked to respond truth- fully, supporting the conclusion that dissimulation may not occur unless prompted.'

These studies indicate that subjects instructed to do so can easily distort their responses on integrity tests in order to create a more favor- able impression However, several questions remain unanswered. Nei-

'In this context, truthful responding is a response set induced by instructions or bias, whereas "honesty" is used to denote particular attitudes toward work and the workplace.

f •'•

STEVEN G. LOBELLO AND BENJAMIN N. SIMS 273

REFERENCES

American Psychological Association (1991). Questionnaires used in the prediction of trust- worthiness in pre-employment selection decisions: An APA task force. Washington, D. C: Author.

Ash, P. (1974). Convicted felons' attitudes toward theft. Criminal Justice and Behavior, 1, 21-29.

Hough, L. M., Eaton, N. K., Dunnette, M. D., Kamp, J.D., & McCloy, R.A. (1990). Crite- rion-related validities of personality constructs and the effect of response distortion on those validities. Journal of Applied Psychology, 75, 581-595.

Jastak, S. & Wilkerson, G. S. (1984). The Wide Range Achievement Test. Wilmington, DE: Jastak Associates.

Moore, R. W. (1990). Instructional effects of the Phase II Profile honesty test. Psychologi- cal Reports, 67, 291-294.

O'Bannon, M. R., Goldinger, L. A., & Appleby, G. S. (1989). Honesty and integrity testing: A practical guide. Atlanta: Applied Information Resources.

Ones, D. S., Viswesvaran, C, & Schmidt, F. (in press). Meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job perfor- mance [Monograph]. Journal of Applied Psychology.

Psychological Surveys Corporation (undated a). Validity. Oak Brook, IL: Author. Psychological Surveys Corporations (undated b). Evaluation of the Survey ADT. Oak

Brook, IL: Author. Rosenthal, R. & Rosnow, R. (1985). Contrast analysis: Focused comparisons in the analysis

of variance. New York: Cambridge University Press. Ryan, A. M., & Sackett, P. R. (1987). Pre-employment honesty testing: Fakability, reac-

tions of test takers, and company image. Journal of Business and Psychology, 1, 248-256.

Sackett, P. R., Burris, L. R., Callahan, C. (1989). Integrity testing for personnel selection: An update. Personnel Psychology, 42, 491-529.

Sackett, P. R. & Harris, M. M. (1984). Honesty testing for personnel selection: A review and critique. Personnel Psychology, 37, 221-245.

STEVEN G. LOBELLO AND BENJAMIN N. SIMS 267

ther study addressed the question of differences in the number of indi- viduals in the various conditions who would be considered desirable potential employees. Significant differences among groups in mean test scores do not reveal the proportion of individuals in the various condi- tions who would fall into a publishers' specified desirable and undesir- able score ranges (In many cases, cut scores may vary by industries and are determined in accordance with the base rate of the behavior in the particular population and the desired selection ratio). In the study by Ryan and Sackett (1987) a commercially produced test was not used, so cut scores were not available. Another question arises from the subject samples employed in each study. There is no question that college stu- dents are a relevant population for response bias studies of integrity tests. Ryan and Sackett (1987) reported that 84% of their sample either were or had been employed in occupational settings where integrity testing would be considered a reasonable pre-employment screening pro- cedure. Ash (1974) demonstrated that other populations (e.g. convicted felons) may be less likely to dissimulate because of an inability to adopt a frame of reference similar to those who adhere to social conventions and norms.

Convicted felons have been subjects in several studies of the con- struct validity of integrity tests (Sackett & Harris, 1984). Using the con- trasted groups method, it has been consistently demonstrated that more inmates score in the negative ranges of various integrity tests when compared to job applicants. Yet it would also be valuable to know if prison inmates, who have been shown to be relatively more dishonest that the general population in construct validity studies, are also capa- ble of dissimulation on integrity tests. Because many former inmates may be expected to find employment in various industry sectors, it is likely that a pre-employment integrity test would be included in the application process.

The purpose of this study is to test the hypothesis that convicted felons are capable of dissimulation on a commercially available integ- rity test. It is further hypothesized that, on the basis of publisher-recom- mended cut scores, a larger proportion of inmates instructed to fake good on the test would be considered desirable potential employees rela- tive to inmates who are instructed to respond truthfully to the test ques- tions or to inmates who are not given a particular response set.

METHOD

Subjects

Subjects were 60 male inmates at a medium security correctional facility in southeast Alabama. These subjects were randomly selected

268 JOURNAL OF BUSINESS AND PSYCHOLOGY

from a list of 201 inmates at the facility who had a reading level at the eighth grade or higher as measured by the Wide Range Achievement Test-Revised (WRAT-R, Jastak & Wilkerson, 1984). The mean WRAT-R reading grade equivalent scores of the groups was 10.07 (range = 8 to 13). The subjects had a mean age of 33.9 years (range = 21 to 53 years) and had completed an average of 11.98 years of education (range = 8 to 17 years).

Instruments

The dependent measure was the PSC Survey ADT (Strand & Strand, 1988), a pre-employment screening test which provides scores on three scales (Trustworthiness, Alienation, and Drinking and Drugs), a Total score, and a measure of evasiveness. The three primary scales are publisher-shortened versions of similar individual scales that are combined in the PSC Survey ADT. The test consists of 100 items to which the subject chooses either a yes, no or cannot say (?) response option. None of the items ask for direct admissions of theft, drug/alcohol use or interpersonal conflicts. Rather, the items are cast in the form of attitude statements, the purpose of which could be readily determined by the subject.

The PSC Survey ADT Trustworthiness scale is the largest of the scales, consisting of 60 items, and is an abbreviation of a 100 item ver- sion of the scale. Item content reflects attitudes toward theft, honesty and trustworthy behavior on the job. The correlation between the 100 item version and the 60 item version is .96 and the split-half reliability of the 100 item scale is .83. One study found that convenience store employees who had lower scores on the 100 item version generally re- ceived higher supervisor ratings (Psychological Surveys Corporation [PSC], undated a).

The Alienation scale is a 24 item abbreviation of a 60 item scale. The items are intended to measure attitudes toward rules and regula- tions, company policies, and respect for supervisors and other authority figures. The correlation between the long and short versions of the test is .96. The split-half reliability of the long version of the Alienation scale is .79. One study of employees of a retail grocery chain indicated that acceptable scores on the Alienation scale were associated with job retention (PSC, undated a).

The 16-item Drinking and Drugs scale is a short form of a 40-item scale designed to assess attitudes toward alcohol and drug use. The cor- relation between the long and short forms of this scale is .93 and the long version has a split-half reliability of .81. Again, PSC (undated a) reports that lower scores on the Drinking and Drugs scale were associ-

STEVEN G. LOBELLO AND BENJAMIN N. SIMS 269

ated with higher supervisor ratings in a study of convenience store man- agers.

The PSC Survey ADT Total score is the simple sum of the raw scores on the preceding three scales, including items endorsed cannot say. The Evasiveness score is the sum of cannot say responses on the total inventory. PSC (undated b) provides cut scores for each of the scales. These cut scores represent the median of the distributions of scores for the normative sample (M. Strand, personal communication, October 27, 1992). Scores above 7 on the Alienation scale, above 5 on the Drinking and Drugs scale, above 20 on the Trustworthiness scale, and above 31 for the Total score are considered to indicate low probability of success on the job. Evasiveness or deception is indicated by 10 or more endorsements of the cannot say option, or by extremely low scores on the 3 scales or the Total score. The inventory assumes a fifth grade reading level and administration time for the inventory is 20 minutes.

Procedure

Subjects were assembled in a classroom at the correctional facility and were given a consent from that was read to them before signing. Three inmates declined to participate after reading the consent forms, which accounts for the unequal group sizes. Scripts for each instruc- tional set were kept in an envelope and randomly selected after each group had signed the consent forms. Thus the experimenter was blind to the instructions to be received by each group until the independent vari- able was actually presented.

Group I (n = 19) was told to respond to the items as though they were applying for a job that they really wanted. Because the job was desirable, they were told to respond to the items in such a way as to make the most favorable impression possible. Group II (n = 20) was told to respond to the items truthfully. Subjects in this groups were asked to resist the impulse to create a favorable impression and to respond as truthfully as possible. Group III (n = 21) was given no particular instruc- tions beyond those received by all groups on how to correctly mark the answer sheets. These publisher-supplied standard instructions also stated that the cannot say option should be avoided unless absolutely necessary.

After receiving the instructional set, suhjects completed the inven- tories. All groups completed the inventories within the 20 minute time limit prescribed by the test authors (PSC, undated b).

270 JOURNAL OF BUSINESS AND PSYCHOLOGY

RESULTS

One-way analyses of variance were conducted on the data for age, educational level, and initial reading level. None of the F ratios were significant, indicating that the random assignment yielded three groups that were equivalent on these variables.

Table 1 summarizes the means, standard deviations and F ratios from one-way analyses of variance performed on each dependent vari- able. In each case, lower scores are considered more favorable indicators of employee potential. Significant differences were found among the three groups on the Trustworthiness, Alienation, and Total scores, while no differences were found on the Drinking and Drugs or Evasiveness scales.

For those scales in which significant F ratios indicated between groups differences, multiple comparisons using independent t-tests were conducted. No significant differences were found between Group II (truthful) and Group III (no instructions) means on the Trustworthiness, Alienation or Total scores. Table 2 summarizes the results of the multi- ple comparison tests and indicates that, consistent with prediction. Group I (fake good) was able to produce more favorable scores than the other two groups on the three scales. The exception was the comparison between Group I and Group III (no instructions) on the Alienation scale, which was not significant.

The second hypothesis concerned whether the test instructions would lead to between group differences in the proportion of individuals who fell below the recommended cut scores for each of the scales. Table 3 shows the proportions of subjects in each group who scored in the ac- ceptable ranges on the various scales. Tests for contrasts in proportions (Rosenthal & Rosnow, 1985) were conducted to determine if differences among groups were significant. Table 4 summarizes the results of these tests, which are expressed in z values. The results show that for each scale, except Drinking and Drugs, the response set significantly infiu- enced the proportion of individuals considered to be favorable potential employees. Individuals who were told to fake good would have been of- fered johs in higher numbers than individuals in the other two groups on the basis of cut scores alone.

DISCUSSION

This study demonstrated that prison inmates are capable of provid- ing favorable profiles on the PSC Survey ADT. Moreover, the profiles of subjects instructed to fake good would have led to more offers of employ- ment for this group than for the subjects instructed to respond truth-

STEVEN G. LOBELLO AND BENJAMIN N. SIMS 271

Table 1 Means, Standard Deviations and F-Ratios for PSC Survey ADT Scales

Scale

Trustworthiness

Alienation

Drinking & Drugs

Evasiveness

Total Score

*p<.05 Note. Standard

Fake Good

22.79 (7.74) 7.15

(4.69) 4.26

(2.75) 6.42

(10.47) 33,68

(12.61)

deviations appear

Truthful

29.85 (9.02) 11.85 (6.03) 6

(3.2) 14.6

(14.63) 47.2

(15.41)

in parentheses.

No Instructions

28.33 (9.03) 9.67

(4.39) 5.71

(3.52) 15.43

(13.52) 43.71

(13.94)

F

3.59*

4.06*

1.66

2.85

4.84*

Table 2 Pairwise Comparisons Between Group Means

Scale

Trustworthiness Alienation Total Score

*p < .05 **p < .01

Fake Good vs. Truthful

2.62* 2.71* 299**

Group Comparison

Fake Good vs. No Instructions

2.1* 1.75 2.38*

Truthful vs. No Instructions

.54 1.33 .76

Table 3 Proportion of Subjects in each Condition Meeting the Publisher-Recommended

Critical Values for Each Scale

Scale

Trustworthiness

Alienation

Alcohol & Drugs

Total Score

Truthful

.10 (2/20)

.25 (5/20)

.40 (8/20)

.10 (2/20)

Group

No Instructions

.14 (3/21)

.29 (6/21)

.43 (9/21)

.14 (3/21)

Fake Good

.37 (7/19)

.63 (12/19)

.63 (12/19)

.53 (10/19)

272 JOURNAL OF BUSINESS AND PSYCHOLOGY

Table 4 Contrasts of Proportions of Individuals Above and Below Critical Values

for Each Scale

Scale

Alienation Drinking & Drugs Trustworthiness Total Score

*p < .05 **p < .005 ***p < .0005

Fake Good vs. Truthful

-2.59** -1.47 -2.08* -3.23***

Group Comparisons

Fake Good vs. No Instructions

-2.28* -1.29 -1.72* -2.85**

Truthful vs. No Instructions

- .29 -1.27 - .43 -.40

fully or given no special instructional set. This pattern was not evident on the Drinking and Drugs scale. The prison in which the study was conducted operates a large alcohol and drug treatment program which, along with the brevity of the scale, may have created a restriction of range in the distribution of scores for this scale

Interpretation of the results are tempered by reality of the analogue situation in which no individuals were applying for actual employment. As in other studies (eg. Hough, et al, 1990; Sacket & Harris, 1987) the data indicate that the tendency to fake good was not activated unless specific instructions to do so were given. Hough, et al (1990) recommend a prudent approach to screening employees that includes the use of re- sponse validity scales and warnings to applicants that distorted re- sponse patterns will be detected.

Obviously, no employer should base a decision to hire on test scores alone and no author of an integrity or honesty test would recommend such a procedure. A multiple cutoff model using test scores, interview and background data is more acceptable, and further research in this area is needed.

Ryan and Sackett (1987) found that the inclusion of a social desir- ability scale was "reasonably effective" in differentiating honest from dishonest responders. The PSC Survey ADT lacks such a scale and would probably be a more powerful tool if one were included. The re- liance on numbers of cannot say items endorsed is apparently not effec- tive in differentiating honest from dishonest applicants. However, em- ployers using similar integrity tests may be encouraged that response distortion is not a problem if reasonable precautions are taken. Such precautions include the training of test administrators to rigorously ad- here to test instructions and to avoid coaching test takers.