Personality Tests in the Workplace

VALIDATION AND INTENSIFICATION OF THE SIXTEEN PERSONALITY FACTOR QUESTIONNAIRE

RA'i'MOND B. CATTELL

Laboratory of Personality Atse^svumt and Group Behavior Univertity of IUinoi»

T H E A I M OF IMPROVING AN INSTRUMENT

Although the ideal in personality measurement, as in ability measurement, is to deal with functionally unitary traits, there are as yet extremely few personality factor scales available. The clinical, educational or industrial jisyt hologi.st who is ready for the sophisticated and effective diagnosis and prediction which the use of factors—in the specification equation and in pattern func tions of factor profiles— make8 possible, finds available only one instrument of objective factor measure- ment"* and three or four questionnaires'"' ''• *'•'". Compared with the former, the latter have the virtue of brief and simple administration and the defei t of distorta- hility, which together permit a widespread usage, but with cooperative subjects only. Accordingly, though objective personality factor tests are on the march "• *• " ' , cooperative subjects are common enough to justify considering the pencil and paper questionnaire as a permanent part of the psychologist's equipment, and seeking to perfect it. This paper is an account of the concepts, methods and results in produc- ing a revision of the 16 P. F. Questionnaire.

The Sixteen Personality Factor Questionnaire, which consists of fifteen temper- amental or dynamic factors and one general intelligence factor, has been in use seven years"". During that time it has been translated for use in eight countries. It has accumulated valuable social validation data in the form of profiles for about thirty occupations''* and six clinical and delinquency syndromes"'*. Certain important regression weights of factors on criteria have also been determined, notably for pre- dicting certain occupational successes*"- " \ accident proneness"'*, success in var- ious kinds of leadership'^"', the selection of researchers and creative persons"*- *'", and the prediction of that part of educational achievement not due to ability <'*- **K

Although the 1948 factorization on which the construction of the 16 P. F. was based, and which we shall henceforth call "the original factor foundation", availed itself of the most advanced factor techniques then possible, and was based on an exceptionally wide area of items, it was the stated intention of the designers at the time to re-check the factor structure, later, by cross validation on different popula- tions, and by entirely independent rotations. The term validation in the present title applies to determining personality factor validity, i.e., internal concept or con- struct validity, while the term "intensification" is borrowed from photography, as a useful designation in psychometrics for the special, additional process of raising the saturation of items on required factors ancl reducing irrelevent correlations, i.e., correlations with factors other than the intended one. Thus, validation concerns the confirmation of a factor; while intensification connotes the development of items to express it more strongly and distinctly, as happens in intensifying a photographic negative. The notion of "homogenizing" a scale is not the same as "intensifying" it, for a test may be made more homogeneous without being made more factor-pure, and we have argued elsewhere""' that there are systematic psychological reasons why this may actually tend to lower factor saturations. Our discussion of test con- struction theory and its illustration by a particular case, though centered on valida- tion and intensification, makes a complete review of necessary principles in factored test construction.

T H E ORIGINAL 16 P. F. FOUNDATION IN THE FULL "PERSONALITY SPHERE"

Emphasis on certain particular standards and requirements in what follows can be understood only if the reader is first g^ven some perspective on the emergence of the 16 P. F. in relation to basic personality research. For this test is only one part

2 0 6 RAYMOND B. CATTELL

of a whole series of test developments which are unique in that the constructors are primarily concerned with basic personality structure and only secondarily with test "gadgets" yer se. In the first place, the adult 16 P. F. is developed in conjunc- tion with other questionnaire constructions, cross-sectioning personality at differ- ent age levels, notably at the level of early adolescence " ' ' ; at seven years; and at four years"*'. It thus implies dependence on research findings and emerging concepts about basic personality development. Secondly, at the level of adult cross-section, the 16 P. F. is an integrated part of a research advance conceived broadly in terms of three possible observation media—life records in situ, questionnaires and oh- jective-tests.This attention to breadth of manifestation increases our understanding of the primary personality structures in terms of different media and situational expressions.

The 16 P. F. can thus claim a somewhat more intensive and extensive research basis than the few excellent factored questionnaires otherwise available, notably in (1) the coverage of the personality sphere, which in this case is extended by cross- media factorings"'- ' " , with the wider reference of meaning thus ensured, and (2) a factor loading has been determined for every item, instead of for conglomerate blocks of items, by virtue of the special techniques of factoring invented for handling large numbers of variables. The resulting better selection of items permits measures of higher factor saturation though still with small numbers of items per factor.

The first of these advances can be briefly substantiated and given essential descriptive detail as follows. The original research on verbal responses from which the 16 P. F. emerged was based on a population of questionnaire items derived from:

(1) A complete survey'"' of all well known questionnaire, opinionnaire, interest and value scales. The evidence* thereof indicated that about twenty factors could be discerned as of 1946. Each of these factors was represented in the ensuing research by sufficient markers, and by newly invented items directly designed to measure the concepts better than by any existing tests.

(2) Evidence of entirely new personality factors, from non-questionnaire sources. In particular, new items were added to the pool of variables to cover the fourteen factors found in factoring rated behavior based on the complete personality sphere'*", as well as on objective tests. Parenthetically, the inter- factor studies"*' ^'' and Saunders' projection of questionnaire factors into be- havior rating space, have shown that the questionnaire factors can be matched with behavior rating factors much more closely and completely than with the objective test factors, at the present stage of the latter. Only life record factors D, J, and K are missing from the questionnaire factors and only questionnaire factors QS Q ,̂ Q* and Q* are missing from the life record factors. (Hence the unique Q designation for these four factors).

The outcome of the original factoring was a good confirmation both as to num- ber and kind of factors, agreeing with the hjrpothesized twenty from the above broad survey of evidence in questionnaire and rating media. Nevertheless it seemed to us appropriate and necessary research strategy at that time to drop the three or four most poorly defined factors in the original factoring and to build the 16 P. F. from intelligence and the clearest fifteen factors. Research should at some time pick up the four discards, but it has seemed sufficiently ambitious a task for our laboratory to concentrate on the definition of the fifteen, and their internal and social validities. And even these sometimes exceed the span of attention of certain applied psycho- logists!

Now the original structure was based on an 82 x 82 matrix, so the additions necessary for the 187 item A and B forms (giving either 10 or 13 items per factor per

'These factors resulted largely from the work of Guilford'"', Ferguson, Humphrey and Strong'*", Flanagan<»«), Layman<">, MosierW>, Reyburn and Taylor<»>, Thorndike<**>, ThurstoneW", ana VernoD<"> and covered, among many others, the data of such tests as the Bernreuter, BeU, Strong, Alli)ort-Vernon and other tests.

THE 16 P. F, QUESTIONNAIRE 207

scale) were made from items known to correlate with the factors in the survey**^ or picked up by item analyses against the separate factors. The mean coefficient of equivalence for all factors between the two forms was .51, and the mean consistency coefficient .68, which may be considered good for 10 to 13 items, but since the simple structure showed factors oblique up to about 0.3, better consistencies could be de- sired. Accordingly a series of re-factoringa, with search for more highly loaded items, was planned, as described here, to give the highest possible factor validities for a test of this length.

CANONS OF FACTORED T E S T CONSTRUCTION

An ideal factor scale differs from a Walker-GuUman scale^^^- **' only in that it yields a measure of a factor instead of a composite of factors. It should meet the conditions that (1) all items have the same factor composition, namely that of a pure, psychologically meaningful, simple structure factor, and (2) items are graded in degree of difficulty according to equal intervals on a normal distribution, since factors can be defined as normally distributed. (In the more general context of personality, which includes ability, "difficulty", which applies only to abilities, is better ex- pressed a8 "eccentricity of cut"). If the first condition is guaranteed, item compari- sons in all possible pairs as in Guttman scaling are unnecessary to ensure the second, since grading can be determined from cutting positions on the distribution. On the other hand, if it is not, application of the Walker-Guttman condition may, according to present experience, prevent anything broader than a scale for a factor of a relative- ly specific nature being formed.

The aim of a multiple factor questionnaire is to form distinct factor scales, in this case sixteen, with mutual obliquities no greater and no less than those discovered to exist among the simple structure factors themselves. Since perhaps only one item in a thousand initially tried ultimately turns out to be a pure factor measure it is likely to be several years before a sufficiency of items is discovered to make sixteen ideal factor scales. Accordingly at present the aim must be to obtain scales operat- ing with suppressors, i.e., obtaining the requisite degree of freedom from factor inter- correlation by using the principle of swnmated factor suppression. This states that the collection of items used for one factor scale should have the highest mean loading on the required factor consisteni with loadings on all other factors summing to zero. For example, in a two item (a and b) scale for Factor Fi, we should require that if a = xFi + sFj then b = yFi -sFj.

In any refined statistical work it is important not to lose sight of basic matters of psychology and common sense. For example a highly loaded and statistically perfect item in the given student sample is no good if it contains words obscure to non-students; contains a reference to an event likely to be unknown a year later; has an eccentricity of cut of 95% to 5%, and takes two minutes to read!

Accordingly the construction of the revised 16 P. F., from start to finish, follow- ed the following canons of procedure.

(1) A very large number of items (in this case 1552) is made up by at least six people (to avoid person-specific factors sometimes demonstrable in tests), in the light of all that is known (in questionnaire, rating and 16 P. F. criterion prediction data) about the number and nature of the primary person- ality factors.

(2) These are to be submitted to persons of different background, and to word count surveys, to eliminate uncommon words (Flesch word count), items that are too long, ambiguous or tied to matters too specific in place or time.

(3) Two population samples are to be taken, one toward the upper and one the lower half of the range for which the test is intended and correlation matrices are calculated among the items separately for these.

(4) Items with extreme cuts (under 10% in one end category of three), in either sample, are eliminated before the calculation of correlation matrices.

2 0 8 RAYMOND B. CATTELL

The phi co-«fficient or the tetrachoric is used. Phi divided by the maximum possible phi for the given extremity of cut has been used by us before and, like the tetrachoric, has the advantage of getting rid of 'difficulty factors'<**\ but since it is prone to yield non-Gramian matrices, and since the alternative tetra- < horic involves undue assumptions, the present study used phi.

(5) The two matrices are separately factored and rotated blindly to simple structure. It is very important that the latter be truly and thoroughly done.

(6) Items are picked for each factor having the highest loadings on the required factor and, if possible, suppressing, {i.e., cancelling) loadings on the others. At this point only those items are carried further which show emphatic consistency in their factor patterns in the two studies. For example, no matter how significant the positive loading on one study may be, the item would be rejected if it has insignificant or negative loading on the other.

(7) To get suitable means, variance and grading on each factor scale, the cuts (alternative response frequencies) must be examined. It is possible to pre- dict both mean and variance of the resultant scale, by certain assumptions*^", from the cuts on the included items. The choice of items by cuts should ac- cordingly give a mean that is central on the scale range and a maximum scatter (near-even cuts) (to an extent compatible with usefulness for extreme samples) as well as equal means and variances for the equivalent A and B forms.

(8) An even balance of 'Yes' and 'No' answers must be chosen, from the surviving items, to score positively on each factor, in order to abolish position or response set effeects.

(9) The items should be symmetrically divided between A and B forms, as to factor loading, mean, variance, yes and no answer, etc., as determined above. (Partly to ensure the kind of equivalence cited in (7) above). Then they need to be arranged in that form of cyclical order, avoiding several items in sequence for the same factor, most convenient for the scoring key.

(10) The scales must be standardized with the usual attention to stratified sampling, etc.

PROCEDURES FOLLOWED IN THE TWO FACTORINGS AND FINAL CONSTRUCTION

The present revision of the 16 P. F. has in principle followed the above ten canons, but economic compromises with the ideal have had to be made in steps 4 and 6 as will be described. Since at almost every step there is loss from a particular selec- tion process, one must start with a far larger number of items than the 374 which in this case are intended to constitute the final A and B forms of the 16 factor scale. There is no obstacle in starting with quite a large number to be submitted to the verdict of the first four canons, and in fact we began with 1552. But owing to the bottle neck created by the limits of size of factorizable matrices, every stage beyond step 5 tends to suffer more or less grievously from a dearth of items necessary to reach the desired standards. Indeed in past test construction, this has proved a well nigh insuperable obstacle to producing multi-factor scales with loadings known and confirmed for every item. Although the obstacle has not been completely overcome here, we feel that the device of "parcelled factor analysis," with the use of extension matrices, described here is perhaps the most important technological contribution of this article—apart from the finished instrument itself.

By this device, and the use of the electronic computer, larger initial matrices have here been factored than any hitherto reported, and the bottle neck partly eliminated. Accordingly, taking steps 1 through 4 as understood in current psycholo- gical practice we shall concentrate on the two factor analyses which have succeeded the original factor analysis <*̂ and which we shall henceforth refer to as the second and third checking analyses. The first of these was done in connection with produc- ing a "Basic English" Form C of the 16 P. F. and has been described elsewhere'*'.

THE 16 P. F. QUESTIONNAIRE 209

On 295 men and women undergraduates, it began with 720 items which were reduced by steps 1 and 2 to a set of 450, which we shall call Extension Questionnaire A and which was reduced to exactly 300 by step 4. At this point a further selection down to 126 was made (a 300 x 300 matrix being still unmanageable facti>ri;iily) by taking only those items showing significant (P = .01) correlations with factors in the existing 16 P. F. or with each other.

'Parcelled factoring' was now carried out for these 126 svirvivors and the 374 items of the existing 16 P. F. combined a.s follows, l'ach 16 P. F. factor was entered as two variables (the minimum for "marking" and recognizing the resultant fa< tors expected). One variable was the si ore on the 13 items of the given factor on the A form and the other the 13 on the B Form. The 120 nrw items were grouped in 'parcels' of three (and sometimes two) of a homogeneity guarantct'tl by original intercorrelations on a 126 x 126 matrix, and a relative factor purity indicated by correlations with the separate 16 P. F. factors. This gave 75 'parcel' variables (30 from the 15 personality factors of the 16 P. F., and 45 parcels of the new 126 items), a number readily factored. The saving in thus factoring a 75 x 7') matrix, despite the two preliminary special correlation jobs, over the 500 x 500 matrix otherwise nece.s- sary, is considerable and has been evaluated elsewhere'*'.

The blindly rotated factors agreed well with the original 16 P. F. factoring^*' as to number and nature, except for some confusion of the factors of neuroticism and anxiety, commonly labelled O and C. In view of these relatively modest loadings on the two less clear factors, every factor was accordingly estimated by the most exact method from Thomson'"' and the correlation of each item in the questionnaire de- termined with the factors (a 15 x 500 matrix). The results were used both to evaluate the existing 16 P. F. items and to construct the C Form. It is the first of these which is relevant to the present study. By eliminating deadwood from the original 16 P. F. it enabled us to start out with clearer, unconfused markers of the 15 factors for cross validation in the third factorization, and to guide the pulling in of new items from the second or B Extension Questionnaire, of 252 items described below, while it also supplied factor loadings for every item so that by the final factoring every load- ing would have a double check.

In the second experiment, with 408 subjects, (227 Air Force men and 181 undergraduates from four Illinois colleges) the markers for tbe known 16 P. F. factors were made up, not as previously, by taking all the factor items from one form to make one parcel, but by putting together only those 9-11 items for each factor shown by the second factoring above to be most highly loaded. Five "parcels" were made up in this case from tbe above proven items in the existing 16 P. F. to represent each factor. For the aim in this third factoring was to test the 16 P. F. structure more exactly than in tbe second factoring and to determine tbe correlations among simple structure factors with a high degree of exactitude. Also it aimed to get such well saturated factor estimates for each factor that they could be used to determine the loadings of new items, from the second extension questionnaire, with a precision comparable to a direct factoring, to permit replacement of any of the 169 existing items by any discovered in the extension having higher loadings.

The new 75 x 75 matrix was centroid-factored and rotated to simple structure with great care. Every hjT^erplane is above the .01 level of significance by Barg- mann's test^*'. It was gratifying to find that most of the cross loadings among the 0 and C factor (and to some extent the Qi, Q2 and Q, factor) parcels encountered in the second factoring disappeared in the better parcels of the third factoring. Fifteen factors were significant and w ere clearly ident.ifiable by their markers.^ The C = lambda^ x lambda matrix, giving cosines among the reference vectors when simple structure was reached, is set out in Table 1. It will be seen that the obliquities are moderate. A second order factoring of these inter-factor correlations is in press ̂ ".

•The unrotated, rotated and transformation matrices for this analyeifl are deposited with the American Documentation Institute, Library of Congress.

210 RAYMOND B. CATTELL

TABLE 1. COSINE MATRIX or CORRELATIONS AUONO REFERENCE VECTORS

16 P. F. Test

Fiiotors

A Ĉ E F G H I L M N 0 Qi Q j Q.

A

1.00 .13 .07

-.01 .00

-.36 -.29 -.01 .03 .OS 0 '

- 1 5 (17

..30

.04

C

1.00 .06 .05 .04

-.12 -.04 -.01

.14

.01

.19 .05

-.01 -.19

.09

E

1.00 .00

-.02 -.18 -.06 -.06 .06

-.11 00

-.31 .18 .12 (12

F

1.00 .05

-.26 .01

-.11 .20

-.12 - 0 4 - n ?

.().')

c;

1.00 .00 .00

-.04 -.04

.13

.02

.11 02

-.19 -.05

II

1.00 14

.2() .05

-.05 .13

-.09 .03

-.29 .01

I

1 00 -.04 -.14

.26 -.11

0?. -.11 -.02

.15

L

1.00 -.07 -.05 -.08

01 -.01

.11 -.15

M

1.00 -.02 -.06 -.19 -.11 -.04

-.01

N

1.00 .04 00 .03

_ ' » • >

-.07

0

1 00 .01 .08 .20

-.17

Q.

1 00 -.14

.00

.03

Q> Q. Q.

1 00 .12 1.00 .01 .23 1.00

Extension questionnaire B, of 512 items, was reduced by steps 1 to 4 to 252 items, which were then correlated, on 408 subjects, as above, with each of the 15 factors, estimated from the parcel variables and their loadings, by the method in- dicated before.

\\ henever existing items in the original 16 P. F. correlated, on both factorings, .20 or less with the factor they represented, they were cut out as 'deadwood' and re- placed by items found to be more highly loaded from Extension Questionnaire B. Unfortunately at this point more poor items were found in factors M, C, 0 and Qj than there were items to replace them. So a third Extension Questionnaire, C, was made, beginning with 320 and reducing to 200 items deliberately aimed at these factors. Thus, by direct correlation with the factors, on a sample of 200 men and women undergraduates, sufficient items loaded above 0.2 were found to supplant the unsatisfactory items in factors M, C, O and Q2.

In the following section the resultant structure of the 16 P. F. is illustrated by two items from each factor, one from the A form and one from the B form. These are neither the highest loaded items in the factorings nor the highest loaded among those which survived the ensuing selections of steps 5 through 10. They are selected instead to illustrate the degree of constancy of loading of particular items on particu- lar factors on two independent factorizations; the range of mean loadings; and the psychological nature of the items expressing each factor.

CONFIRMATION AND DEGREE OF INVARIANCE OF THE SIXTEEN FACTORS

Each item below is set out under the factor as usually symbolized by letter and contingent names. To the right are set out (1) the response—left or right for ((a) or (b) or yes or no)—which scores positively on the factor; (2) the loadings in the two factor studies (second and third); and (3) the frequency of the positive scoring, central and negative scoring responses in 408 subjects.

FACTOR A. CTCLOTHTMiA-vs-ScrazoTHTMiA

Test Form

1. A

2. B

Item If the earnings are the same I would rather be (A) a lawyer (B) a freight air pilot In a factory I would rather be: (A) in charge of mechanical matters (B) engaged in interviewing and hiring people.

Positive Response

Lt.

Rt.

Loadings

.36 .48

.57 .65

Response FrequeneiM

105 20 283

240 10 15S

THE 1 6 P. P. QUESTIONNAIRE 2 1 1

FACTOR B . OBNERAL INTELLIGENCE (Theee items were factored apart from the main study).

FACTOR C. EGO STRENGTH

Teit Positive Form Item Response Loadings Response Frequencies 3. A I occasionally have realistic dreams

that disturb my sleep. 4. B I sometimes feel compelled to count

things for no particular purpose.

FACTOR E . 5. A I occasionally tell strangers about the

things I am interested in and good at, without direct questions from them. Lt. .20 .21 22 13 373

6. B I have on occasion torn down a pub- lic notice forbidding me what I felt I had a perfect right to do. Lt. .24 .21 64 11 233

Rt. .48

Rt. .36

DOUINANCB

.46

.22

356

255

12

16

40

136

FACTOR F . 7. A I like a job that offers change, var-

iety and travel, even if it involves some dangers. Lt. .30 .31 327 27 51

8. B I would prefer the life of: (A) a master printer in a modern plant (B) an advertising man and pro- moter. Rt. .38 .47 294 22 92

FACTOR G . STJPEB EGO STRENGTH 9. A I think that good manners and res-

pect for law are more important than excessive freedom. Lt. .36 .47 324 23 43

10. B I adniire niore a person who: (A) is brilliantly intelligent and creative (B) has a strong sense of duty to the things he believes in. Rt. .28 .31 308 19 81

FACTOR H . IMMTTNITT (OB ADVENTUROUS CTCLOTHTMIA) 11. A I have at leaBt as many friende of the

opposite sex as of my own sex. Lt. .26 .35 212 23 103 12. B If people ia the street, or standing in

a store, watch me I feel slightly embarrassed. Rt. .39 .49 345 21 42

FACTOR I. SENSITIVITY-VS-TOUGHNESS 13. A I would rather spend a free evening:

(A) with a good book (B) working on a project with friends. Lt. .32 .28 24 25 359

14. B In art and music we should (A) give popular demand what it wanta, regardless of quality (B) try to raise standards, by giving experts a chance to control taste. Rt. .33 .26 57 23 328

FACTOR L. PARANOID TBEND 15. A If I am quite sure that a person is

unjust or behaving selfishly I show him up, even if it takes some trouble. Lt. .37 .34 112 34 262

16. B I suspect the honesty of people who are more friendly than I would nat- urally expect them to be. Lt. .29 .47 160 31 217

FACTOR 0 . FBBB ANXIBTT 17. A I feel grouchy and just do not want

to see people: (A) Occasionally

Rather often. Rt. .14 .32 41 23 344

2 1 2 RAYMOND B. CATTELL

FACTOR 0 . F R E E ANXIBTY (continued) Test Positive Form Item Response Loadings Response Frequencies 18. B I am moved almost to tears by some-

thing upsetting; !A) never B ) sometimes Rt. .10 .21 54 15 330

FACTOR Q I . RADICALISM-VS-CONSERVATISU

19. A I t would be better if we had more strict observance of Sunday, as a day to go to church. Rt. .32 .55 101 32 275

20. B In my work more troubles arise from men who:— (A) are constantly changing meth- ods that are already O. K. (B) refuse to employ up-to-date methods. Rt. .22 .46 197 68 143

FACTOR QJ. SELF SUFFICIENCY

21. A I like to take an active part in social affairs, committee work, etc. Rt. .23 .43 172 47 189

22. B I get as many ideas from reading a book myself as from discussing its topics with others. Lt. .19 201 16 190

FACTOR Q I . W I L L CONTROL

23. A When talking I like:— (A) to say things just as they occur to me. (B) to wait and say them in the most exact style possible. Rt. .17 .24 229 39 137

24. B However difficult and unpleasant the obstacles I always persevere and stick to my original intentions. Lt. .58 .54 324 39 45

FACTOR Q4. TENSION (SOMATIC ANXIETY)

25. A At times of stress or overwork I suffer from indigestion or constipation:— (A) practically never (B) occasionally Rt. .36 .36 09 28 281

26. B My nerves are sometimes on edge, so that certain sounds, e.g., a screechy hinge, are unbearable and "give me the shivers". Lt. .42 .38 112 13 283

SUMMARY

(1) A multiple factor-scale questionnaire, covering fifteen personality factors and the cognate factor of general inteUigence, based on an older factor analysis, has had its factor structure re-examined, the factor loading of every item determined, and items of low validity replaced by new items of improved validity. The process is defined as validation and intensifieation, since the conceptual factor validity of each item is determined, and the factor saturation and independence of the sixteen scales is intensified.

(2) Ten canons for multiple factor scale construction are laid down and ex- emplified in operations with the Sixteen P. F. Questionnaire.

(3) The principal innovation is the introduction of "parcelled factor analysis" in which a much larger number of items than could usually be handled is first group- ed, by clustering and correlation with existing factors, into a smaller number of homogeneous (but factor impure) "parcels" or short, rough scales. The factor struc- ture is determined on this relatively small matrix (75 x 7̂ 5 in this example) and the parcels are then "undone" and all constituent items correlated directly with the factors estimated in terms of parcels. An "extension questionnaire" of items hypoth- esized to be highly correlated with the factors is also correlated item by item with

THE 16 P. F. QUESTIONNAIRE 213

this same factor score, whereby weak items in the original questionnaire are replaced. This device gives the factor loading of every item in the original test and in the ex- tension with every substantial factor, and under comparable conditions, at a small fraction of the prohibitive labor for a matrix of order 1000 x 1000. The item loadings could be, but were not, corrected for attenuation by unreliability of factor estimate, since only relative goodness of items need be accurate in this procedure.

(4) Two parcelled factorizations, on 400 and 169 items, were carried out on independent population samples and with independent computing and blind rotation. However the process was iterative in that clearer factor definition was achieved in the second through entering with more factor-pure and homogeneous parcels as a result of the findings of the first. Both yielded, by existing tests of completion of factor extraction, 15 factors (i.e., 16 with intelligence) and, through the marker var- iables, they were confirmed to be the same factors in all three factorings, i.e., to be the same in both experiments here and the same as named in the original study.

It may be asked how far the inclusion of more items that are good measures of the factors found in the first factorization prejudges the structure of a second factor- ing. The writer would answer (1) the insertion of items high in one factor does not strengthen pre-existing hyperplanes for the other factors (unless factors are ortho- gonal and the items are factor-pure as well as highly loaded). (2) An infinity of rota- tion positions are still possible, so if the same is found again it is proof that the structure is inherent and that new items adhere to it for this reason, since they are not made to adhere for any other reason"'.

(5) A total of 1552 newly constructed items were brought into three extension questionnaires. Extension A began with 720, reduced to 450 before final correlation on a group of 295 men and women undergraduates. These items were relevant only to the initial factor structuring and were actually used for Form C, whereas ex- tensions B and C were used here for intensifsdng Forms A and B of the 16 P. F. Ex- tension B began with 512 items, reduced by the first steps to 252 and then correlated on a sample of 408 young men and women, half Air Force, half undergraduates. Ex- tension C began with 320, reduced to 200 before correlation on 200 men and women undergraduates. From the 1552 items, 110 eventually strengthened the original 16 P. F. (replacing the weakest 110 of the 374 items in A and B forms).

(6) Further work on the structure, psychological meaning and prediction value of the factors, in clinical and other work, is in progress. As to the re-standardization of the revised test it may be pointed out that one of the advantages of factor scales is that the clinical and occupational profiles, criterion regressions and specification equations found for standard scores on the older test continue to apply (with some attenuation correction) to the new. The meaning of the present factors in terms of second order factors is being determined from Table 1<'\ It is also an important aspect of the meaning of factors to determine whether they persist in different cul- tures and for that reason the present confirmation of constancy within a culture ia being extended by a similar comparison of factorizations on British, French, Italian, Indian and Chinese versions of the 16 P. F. ^"\

BIBLIOGRAPHY

1. BARGHANN, R . Significanzuntersuchungen der einfachen stniktur in der faktoren-anaiyse* MitteiLf. Math. Statist., Sonderdruck, Physica-Verlag, Wurzburg, 1954.

2. CATTELL, R . B . The Description and Measurement of Personality. New York: World Book Com- pany, 1946.

3. CATTELI,, R . B . A Guide to Mental Testing. Inst. Pers. & Abil. Test., 1602 Coronado Drive, Champaign, IUinoiB, 2nd Edition, 1948.

4. CATTELL, R . B . The main personality factors in questionnaire, self-estimate material. J. Soc. Psychol, 1950, 31, 3-38.

6. CATTELL, R . B . The principal replicated factors discovered in objective personality testa. TJ. Abnorm. Soc. Psychol, 1955 3, 291-314.

(J. CATTELL, H . B . et al. The Obiective-Analytic Personality Factor Batteries. Adult andJChild Forms. I. P. A. T,, 1602 Coronado Drive, Champaign, 111., 1955.

214 RAYMOND B. CATTELL

7. CATTELL, R . B . Occupational profiles on the I. P. A. T. 16 Personality Factor Questionnaire. Occupat. Psychol., Feb., 1956, in press.

8. CATTELL, R . B . A shortened "Basic English" version (Form C) of the 16 P. F. Queationnaire, J. Soc. Psychol., 1956, 30, 1-10.

9. CATTELL, R . B . A. determination of the second order personality factors among the 16 P. F. questionnaire primaries. J. consult. Psychol., 1956, in press.

10. CATTELL, R . B . Personality and Motivation Structure and Measurement. New York: World Book Co., 1956.

11. CATTELL, R . B . Are personality factors constant across cultures? In press. 12. CATTELL, R . B . , BELOFF, J., FLINT. D . and GRUEN, W . The Junior Personality Quiz. I. P. A. T.,

1602, Coronado Drive, Champaign, Illinois. 1954. 13. CATTELL, R . B . , DAT, M . ancf MEELAND, T . La standardisation du questionnaire de personalite

en 16 facteurs de l'l.P.A.T. Rev. de Psychol. Appliquee, 1953. 3, 67-83. 14. CATTELL, R . B . and DHEVDAHL, J. E. A comparison of the personality profile (16 P. F.) of em-

inent researchers with that of eminent teachers and administrators, and of the general popula- tion. Brit. J. Psychol. 1955, 46, 249-261.

15. CATTELL, R . B . ana GRUEN, W . G . Primary personality factors in the questionnaire medium for children eleven to fourteen years old. Educ. Psychol. Meas., 1954, 14, 50-76.

16. CATTBLL, R . B . and PBTBRSON, D . R . Personality factor structure in verbal responses o! four year olds. In preparation.

17. CATTELL, R . B . , SAUNDERS, D . R . and S T K E , G . F . The Sixteen Personality Factor Question- naire, I.P.A.P., 1602, Coronado Drive, Champaign, 111., 1949.

18. CATTELL, R . B . and SAUNDERS. D . R . Inter-relation and matching of personality factors from behavior rating, questionnaire, ana objective test data. J. Soc. Psychol., 1950, 31, 243-260.

19. CATTELL, R . B . and SAUNDERS, D . R . Beitrage zur Faktoren-Analyee der Personlichketi. Z. f, exper. u. angew. Psychol., 1953, 2, 325-356.

20. CATTELL, R . B . and STICE, G . F . Four formulae for selecting leaders on the basis of personality. Hum. Relat., 1954, 7, 493-507.

21. CATTELL, R . B . and SHOTWELL, A. M. Personality profiles of more successful and less successful psychiatric technicians. Amer. J. Ment. Def., 1954, 8, 496,499.

22. DHEVDAHL, J. E. Factors of Importance for creativity. / . din. Psychol, 1956,12, 21-26. 23. ETSENCK, H. J. The Scientific Study of Personality. London: Routledge, 1952. 24. FERGUSON, L . W . , HUMPHREY, L . G . and STRONG, E . K . A factorial analysis of interesta and

values. J". Educ. Psychol, 1939, 30, 151-156. 25. FLANAGAN, J. C. Factor Analysis in the study of Personality. Stanford Univ., Cal.: Stanford

University Press, 1935. 26. GuiLFORD, J. P. Psychometric Methods. 2nd Edit. New York: McGraw Hill, 1954. 27. GuiLFORD, J. P. & GuiLFORD, R. B. Personality factors D, R, T and A, / . Abnorm. Soe. Psychol,

1939, 34, 1-12. 28. GuiLFORD, J. P. and MARTIN, H . G . The Guilford-Martin Questionnaire. Sheridan Supply Co.,

Beverly Hills, Calif. 29. GUTTMAN, L . The Cornell technique for scale and intensity analjrsis. Educ. Psychol. Measmt.,

1947, 7, 247-279. 30. H!ADLET, S. T . A study of_ the predictive value of several variables for student teaching success

as measured by student teaching marks. Res. for the Teaching Profession. 1963. 31. LAYMAN, E . M . An item andysis of the adjustment questionnaire. / . Psychol, 1940,10, 87-106. 32. MCCLELLAND, D . C . Personality. New York; Sloane, 1951. 33. MosiER, C. J. A factor analysis of certain neurotic symptoms. Psychometrika, 1937,2,263-286. 34. O'HALLORAN, A. An investigation of Personality Factors associated with Under-achievement in

Arithmetic and Reading. M. S. Thesis. Purdue Univ. Library, Lafayette, Ind., 1954. 35. REYBUHN, H . A . and TATLOR, J. G. Some factors of temperament: a re-examination. Psycho-

metrika, 1943, 8, 91-104. 36. SuHR, V. W. The Cattell 16 P. F. aa a prognosticator of accident susceptibility. Proe. Iowa

Acad. Sd., 1953, 60, 553-561. 37. THOMSON, G . H . On estimating oblique factors. Brit. J. Psychol Stat. Sect., 1949, 2,1-2. 38. THORNDIKE, E . L . The interests of adulta: 2, The interrelations of adult interests. / . Educ.

Psychol, 1935, 26, 497-507. 39. THUKSTONE, L . L . The Thurstone Temperament Schedule. Chicago: Science Research Associatea,

Chicago, 1948. 40. THURSTONE, L . L . A multiple factor study of vocational interests. Person. J., 1931,10,198. 41. VERNON, P . E . The Assessment of Psychological Qualities by Verbal Methods. Indus. HeaUk

Res. Council, Rept. No. 81, London, H.M.S.O., 1938. 42. WALKER, D . A. Answer-pattern and score-scatter in tests and examinations. Brit. J. Psychol.,

1931, 22, 73-86; 1936, 26, 301-308; 1940, 30, 248-260. 43. WHERRY, R . J. and GAYLORD, R . H . Factor pattern of test item and tests as a function of the

correlation coefficient: content, difficulty and constant error factors. Psychometrika, 1944, 29, 37- 244.

44. WRIGHT, S. Some psychological and physiological correlates of certain academic undenchievers. Ph.D. Thesis, University of Chicago Library, 1955.