Scales, Norms, and Score Comparability

THE JOURNAL OF

EDUCATIONAL PSYCHOLOGY Volume. XVI October, 1925 Number 7

A METHOD OF SCALING PSYCHOLOGICAL AND EDUCATIONAL TESTS

L. L. THURSTOXE

University of Chicago

The idea for the method of scaling tests which I am about to describe occurred to me while trying to tease out the logic of some of the well-known educational scales and mental age scales. The authors of educational scale monographs seldom give an adequate discussion of the assumptions and of the logic of their scale constructions. Of course, it is not to be expected that any one method of scaling will apply to all types of educational and psychological test material, and I am not attempting to offer a single solution for all scaling problems.

My present method applies especially to test items that can be graded right or wrong, and for which separate norms are to be constructed for successive age- or grade-groups. By slight modification it is also applicable to tests that give a numerical score. The method is, there- fore, applicable to such test material as the Binet test questions. It is also applicable to educational test material such as arithmetic problems or spelling, and to other types of content in which answers can be definitely graded as right or wrong and for which we want separate tabulations for children of successive ages or of successive grades.

In order to explain the method I shall follow through an example with the Binet test questions. The method is by no means limited to mental age data, but the explanation of the method will perhaps be clearer if I explain it consistently with one type of material as an example. For the actual data I have selected the tables prepared by Burt (1) from his Binet test results on 3000 London school children.

The first part of the analysis will be similar to the methods already in general use. In Fig. 1 we have a frequency surface. Let it repre-

433

434 The Journal of Educational Psychology

sent the distribution of Binet test intelligence for seven-year-old children. I t is a normal surface, and this is about the only assumption that we shall make. It has been shown by Jaederholm (2) and others to be a reasonable assumption.

Those children who are represented toward the right end of the surface in Fig. 1 are bright for their age, while those toward the left side are below the average of their age. Evidently the average intelligence of seven-year-old children would be represented by the point d on the scale since that point is at the mean of the distribution. But the brightness of these children is judged by their answers to a lot of questions which are more or less roughly graded according to difficulty in the mental-age scales. We shall therefore locate these test questions on the scale as landmarks of different levels of intellec- tual growth. These questions, when so located on the scale, will serve

FIG. 1.

our purpose in arriving at a numerical measurement of scale distances. Let us assume that a particular test question has been located at the point d on the scale. This means that 50 per cent of the seven-year-old children answered it correctly. It is clearly an average question for that age since half of the children get it and half of them fail on it. Its difficulty represents the mid-point of the distribution of test intel- ligence for children of this age. This percentage rating of the tost question corresponds to the fact that one-half of the surface is to the right of the point d. If this is our basis of locating the test questions on the scale, it is clear that the difficult questions will be located toward the right of the mid-point, and that the easier questions will be located toward the left of that point. Consider, for example, the question which has been located at the point e. It is one of the more

A Method of Scaling Tests 435

difficult questions. It is correctly answered by only 30 per cent of the seven-year-old children and it is therefore located at such a point on the base line that 30 per cent of the whole surface is to the right of it.

If we imagine a slightfy older group of children whose mental growth has advanced a little beyond that of the seven-year-old children, we could imagine the average of this older group of children to be at the point e. They might have 50 per cent right answers for the question which we located at e on the scale for the seven-year-old children. It is clear that the distributions of test-intelligence for older children will move toward the right, and that the distributions for the younger children will, of course, be nearer the left. Now suppose that we find a test question which more than half of the seven-year-old children can answer correctly. Evidently it should be located at a point below the average for the children of this age. Such a question is shown at the point c. It is evident that the average seven-year-old child is one who is exceeded by 50 per cent of children of his own age and who exceeds 50 per cent of children of his own age. In other words, an average child is one who is at the middle of the distribution of intelligence for children of his age. Now, if a question is answered correctly by 60 per cent of seven-year-old children, such a question should be located below the average of that age group.

Let us now consider a test question which is answered correctly by 100 per cent of the seven-year-old children. It is so easy that all the children in this age group have passed it; the whole distribution is to the right of that question. The question, then, should be located at a point such as b, or still farther back such as the point a. In fact, if a test question is answered correctly by 100 per cent of a certain age group, we cannot locate the point definitely on the scale. All we know about it is that it is somewhere to the left, beyond the distribution.

The same reasoning applies to a test question which is so difficult that none of the seven-year-old children can answer it correctly. Such a question marks a point on the scale so far ahead that the distri- bution of intelligence of seven-year-old children has not yet covered it. Such a point might be at / or still farther on at the point g. When we find this type of question, we majr expect that an older age group, with its distribution of intelligence farther toward the right on the scale, will give a higher percentage of right answers and the question can then be definitely located on the scale. We see, therefore, that a question cannot be definitely assigned to a point on the scale if it is

436 The Journal of Educational Psychology

either so difficult that none of the children in a given age group can answer it or so easy that they can all give the correct answer. The percentages of right answers must be above 0 and below 100.

We can now summarize the procedure so far covered. We assume the distribution of intelligence of children of any given age group to be approximately normal. Since test-intelligence is indicated by the correctness of answers to test questions, it is legitimate to designate the points on the scale of test-intelligence by means of the questions as landmarks. Each test question is located at a point on the scale so chosen that the percentage of the distribution to the right of that pointis equal to the percentage of right answers to the test question for children of the specified age.

In order to facilitate the handling of such data, we give a numerical value to each test question for each age with the standard deviation of the distribution as a unit of measurement. All test questions that are located above the average of the group have positive values, while all test questions that are located below the average of the group have negative values. Thus a test question which has a numerical value of -f 1 a for seven-year-old children is answered correctly by about one-sixth of the children of that age. But its value for the next higher age group would, of course, be lower, because more of the eight-year-old children would answer it correctly We shall therefore designate numerically each test question for each age group. The numerical designation will he different for each age group because the distribution of intelligence moves to the right on the scale with increase in age.

Now, what we want is a series of frequency curves, all drawn on the same base line, that will truly represent the distributions of test- intelligence of children of successive chronological ages. On this base line we shall locate each of the test questions. The curves must be so drawn that no matter where a test question is located on this base line the proportion of any given age distribution to the right of that question will correspond to the actually observed percentage of children of that age group who answer the question correctly.

In order to draw a series of curves to represent the distributions of test-intelligence of children of successive ages we can draw the first one anywhere on our scale irrespective of the units or the origin. Our problem is then to draw the others on the same scale in such a way that their interrelations shall correspond with the actually observed percentages of right answers.

A Method of Scaling Tests 437

There are two facts that must be established for each curve before it can be drawn, namely its mean and its standard deviation. It does not matter at all where we locate the origin or zero. Let us arbitrarily locate it at the mean of the lowest age group that we are dealing with. It can later be shifted to any convenient place. Similarly, it does not matter what unit of measurement we use for the base line. Let us for convenience adopt as the unit of measurement the standard devia- tion of the lowest age group in any particular study. It can also be shifted later to any convenient scale.

In Fig. 2 let the frequency curve A represent the distribution of test ability for children of any specified age. Let the curve B represent the distribution for an older age group. The base line represents

X,-

M,

Fio. 2.

achievement, or relative difficulty of test questions, while the ordinates represent relative frequencies of children at each degree of achieve- ment. The means of the two distributions are designated Mt and M2, respectively. Naturally we should expect the higher age group to have a higher mean on the scale and, therefore, the two curves are so drawn that the mean Af2 is higher than the mean Mi.

Let the small circle represent any particular test question. The shaded area in the B surface represents the proportion of the older age group who can answer the question correctly. The remaining unshaded part of the distribution represents the proportion who fail on that question. The same reasoning applies to the A distribu- tion. There is a larger proportion of the older children who can answer the question, and that is reasonable because B represents children older than A.

438 The Journal of Educational Psychology

If we know the percentage of children of different ages who can answer each question, it is possible to locate the questions on an absolute scale, and it is also possible to locate the means of the succes- sive age groups on the same absolute scale, and to determine the standard deviations of the successive ages on the same scale. The present method assumes that the distribution of abilities is normal, but it does not assume that the standard deviations of the successive age or grade groups are the same.

Let A"i represent the deviation from the mean, Mi, of a particular question for children of a particular age. This is determined in the usual way from the percentage of these children who can answer the

v̂

I

-3 -£ -I O 5/gma values for set/en-t/eor -o/d cfn/dren

FIG. 3.

question correctly. Of course, X\ is measured in terms of the standard deviation, <n, of the distribution. In the same manner, let X2 repre- sent the deviation from the mean, M2, of the same question for an older age group. It is, of course, expressed in terms of the standard deviation, 02, of its own distribution.

It is clear in Fig. 2 that M\ plus XKTI must be equal to M2 plus .XV2 because they are measurements to the same point on the scale, both measurements representing the same test question by two different age groups. Hence,

Mi + X,iri = Mi + X*72 (1)

A Method of Scaling Tests 439

or, Mi v M2 , v o-2h A i = (- A 2—

Transposing,

This equation is linear if we plot the observed values of X\ against the corresponding values of X2 for the same test questions. The constants will be as follows:

-2 = Slope (3)

-]£LH_-'K1 =AVintercept (4)

In Fig. 3 the paired values of Xi and X2 have been plotted for two adjacent age groups and it is immediately apparent that the relation is linear. The two mean lines for Xi and X2 are determined from the following relations,

2 17ln = I

n in which nil is the average of the observed X-values for the lower age group, and ?«2 is, similarly, the average of the observed X-values for the upper age group, and n is the number of overlapping test ques- tions for the two age groups. It should be noted that m in this nota- tion represents the average of all the X-values in the data whereas M represents the mean of the distribution on the absolute scale.

The next step in the calculation is to determine the numerical values of the constants of the line in Fig. 3 because these two constants will enable us to draw the two distributions on the same base line. Let the deviations from mi and WJ2 be defined as follows:

xi = Xi — ?«i

x2 = X2 — mi

and let

«2 = * - - - = * - m2-

Then the line of relation, assuming r = 1., is as follows: S\

= -X2 52

440 The Journal of Educational Psychology

or

A i — ni\ — — ( A 2 — mo)

Transposing, l ( ) (5)(m, -

2 S2

But equations (2) and (5) represent the same linear plot. Hence: (To

I he slope = -- = $2

or 0-2 = S~<Tl (G)

S2

and, similarly, Mi — Mi Si — = mi m2

(Xl S 2

or

M2 = ciimi x-mi\-\-Mi (7)

Equations (6) and (7) are the fundamental equations for our scaling purposes. By means of equation (6) we can ascertain the standard deviation of one age distribution when the other is assumed or known, and by means of equation (7) we can locate the mean of one of the age distributions when the other is assumed or known. These two formulas should prove quite generally useful in scaling tests.

Let us assume an arbitrary origin at the mean of the distribution for three-and-one-half-year-old children in Burt's data, and let us assume as our unit of measurement on the absolute scale the standard deviation of the same age group. The scaling can later be changed into any desired unit of measurement and it can be easily adjusted to any desired origin.

In Table I we have the data from Burt for seven- and eight-year-old children. In the first column is the numerical order of the Binet questions as arranged by Burt. Note that the list does not extend below question 10 nor above question 59. The reason for this is that beyond this range one of the age groups received either 0 or 100 per cent right answers, which renders scaling for those questions impossible for these two age groups. The questions so eliminated can, however, be scaled for higher or lower age groups. The second column gives a brief description or designation of the Binet questions to facilitate

A Method of Scaling Tests 441

identification. The next column shows the actual percentage of seven- year-old children who answered each question correctly. The next column after that gives the similar facts for the eight-year-old children. These data are all copied directly from the tables of Burt for London school children.

The next two columns show the relative difficulty of each question for each age group expressed in terms of the standard deviation. These figures are obtained directly from the two percentage columns. Thus, for example, question 40 was answered correctly by 39.1 per cent of the seven-year-old children. Since less than half of the children in this age group answered this question correctly, that question belongs on the positive side of the difficulty scale. Its sigma standing is -f-28 which is read directly from a table of areas of the probability surface.1 In other words, at point +.28cr there remains 39.1 per cent of the surface to the right of that point.

The same question, 40, was answered correctly by 68.2 per cent of the eight-year-old children. Since more than half of the children in this age group answered this question correctly, the question belongs on the negative side of the mean for the eight-year-old children. Its exact location is —Ala, since 68.2 per cent of the whole surface lies above that point. It should be noted that the majority of the seven- year-old children fail on this question and that the majority of the eight-year children succeed with it. That is the reason for the difference in sign.

In the last two columns are recorded the squares of the sigma values. These are used in calculating the standard deviations S7 a n d Ss.

In Table II are shown the calculations corresponding to the data of Table I. Note that the scale values of M7 and of <r7 are here known because they have been derived from the similar calculations for the lower age groups.

Tables I and II show all the calculations for establishing the values of Ms and <rs. Similar tables can readily be prepared for any two adjacent age groups in order to extend the determinations to any desired age range.

In Table III we have the result of such calculations for the age range 3 to 14, inclusive, according to Burt's data. In the first column

1 1 found Kelley's tables of areas of the probability surface especially useful for these calculations. See also Table I, page 96, of Monroe, "Theory of Educa- tional Measurement."

442 The Journal of Educational Psychology

is recorded the average age of each successive age group. The second column shows the scale value of the mean test-intelligence of children of each age group, expressed in terms of the standard deviation of three and one-half-year-old children and with the mean of that age group as an origin. In the third column is shown the standard devia- tion of each age group with the same unit of measurement. The calculations for each age were carried out as shown in Tables I and II.

The first calculation is made for the lowest age groups, three and four, and the formulas are somewhat simplified for that determination in view of the fact that M3 is zero and o-3 is unity, these two values being assumed for the construction of an absolute scale.

In Fig. 4 the data of Table III have been represented graphically. The base line represents chronological age and the ordinates represent the absolute scale of Binet test intelligence. The middle curve is taken directly from column 2 of Table III and it shows the mean intel- ligence of children of successive ages. Note that the points on the curves are plotted at the half-year ages to correspond with the classifi- cation of the children in the original data of Burt according to their last birthday. A striking feature of this curve is that it continues to rise even at the age of 14 with no indication of reaching a level. It certainly looks as though the kind of intelligence which is measured by the Binet tests and their variations continues to grow as rapidly at the age of 14 as it does at the age of 9. This conclusion contradicts the statements frequently made to the effect that mental test intelli- gence approaches an adult and more or less stable level at 14 to 16 or 18 years. The appearance of these curves indicates that the growth of test intelligence continues beyond the age of 14. The continuity of the curves is such that one can hardly imagine that they will bump into an adult final level of some kind at 16 or 18 and then stay there. When we talk about adult intelligence as having been reached some- where in the middle of the 'teens, we are not sufficiently cautious in recognizing the mechanical limitations of the scale. If the scale stops at 16 so that no one can attain a rating higher than that, it is clear that the average for the whole population will be a little lower than the maximum point on the scale. Those who are below the maximum get their true rating, but those who are above the maximum point on the scale get only the maximum possible rating. When these ratings are pooled into one average, it will, of course, be a little below the maximum point of the scale.

A Method of Scaling Tests 443

The Binet scale is extended from one year to the next by inserting new tests on which the higher age groups do better than the lower age groups. It should be extended beyond the age of 14 or 16 by inserting tests on which older subjects succeed better than younger ones. I t is difficult to find test questions of the ordinary type in which such differentiation is possible, but our inability to find them does not prove that the development of intelligence stops somewhere in the 'teens. Common sense judgment certainly favors the assumption that the average man of 40 is more intelligent than the average boy of 20, but so far we have not been able to measure that difference.

r

5 0

!

1

' /

:

1

' " " ; ^

1 I

:

1

i 1

1

- ^

s°

"" i 1 i

I

i

1

i

1

1 1

1

1 r a 9 /o // /e /3 /»' /s

Fio. 4.

Instead of acknowledging this limitation in our measurement methods, we have not infrequently attempted to juggle with the definition of intelligence to make it fit the measuring devices that are accessible.

The validity of the method here described depends largely on the linearity of the plot of the X-values for any two adjacent age groups. In order to test the consistency of the method throughout the age range here considered, correlation tables were plotted for all the adjacent age groups with results as shown in Table IV. It will be readily seen that all the correlations are above +.97. If the distributions of test-

444 The Journal of Educational Psychology

intelligence for any of the age groups should not be normal, these plots would not be linear, and the correlations would be correspond- ingly reduced. The linearity of all the graphs is very conspicuous and similar to that of Fig. 3.

The upper curve in Fig. 4 shows the growth in Binet test intel- ligence of those children who rank -flo- with reference to children of their own age. The lower curve shows similarly the growth curve for those children who rank — l<r with reference to children of their own chronological age. It is interesting to note that these curves tend to separate with advance in age. The interpretation of this fact is that the absolute variability of intelligence increases with age.

Ohronofogico/

FIG. 5.

The next step in our analysis is to ascertain the chronological age at which each test question is at par. In other words, what is the average age of the children, 50 per cent of whom pass a certain test question? In Fig. 5 we have represented the per cent of children of successive ages who pass a given test question. Curves are shown for a random sample of questions. It was not possible to include curves for all the 65 test questions on one chart, because of the lack of space. But the curves shown indicate the nature of the function. The numbers on the lines indicate the numbers of the Binet test questions. Inspection of Fig. 5 shows that the functions are similar for practically all test questions with some noticeable variation in the slope of the curve.

A Method of Scaling Tests 445

A refined statistical method for ascertaining the age at par for each test question would be to fit an equation to the curve for each test question and then to ascertain the point at which the curve intersects the 50 per cent level. This would be rather laborious and it would not appreciably increase the accuracy of the short-cut that is here adopted because there is some error in the original percentages themselves.

In order to ascertain the age at which a particular test question is at par we draw a curve for that question similar to the curves in Fig. 5. We note the percentages immediately below and above 50. Call these percentages pi and p2 respectively. Then the interpolation is represented by the following formula:

A x . . . . 5 0 — pi Age at par = Y -\—

P Pi—Pi

in which Y = age which has Pi correct answers. Pi and pt are the two percentages immediately below and above 50. Applying this method to Question 8 in Burt's tables as an example,

we have

Age at par = 3.5 + ''33"!.—45 = 3 - 6

In Table V we have the test questions as listed by Burt together with the age allocation of each question by the method here described. The age allocation is so determined that a large random sample of children of the specified age at par would give approximately 50 per cent right answers to the particular question. Older children would give correct answers more often, and younger children would give correct answers less often. It was not possible to give a definite age allocation to some of the questions in the Binet tests at the lower end and at the upper end. The first few questions were so easy that the lowest age group in Burt's study, the three-3rear-old children, gave more than 50 per cent right answers and the last few questions were so difficult that less than 50 per cent of the 14-year-old children gave right answers. Of course one could resort to make-shift in scaling these test questions but it must be recalled that the functions shown in Fig. 5 are not all parallel. Hence the advisability of finding experimentally those age groups which get close to 50 per cent of the questions right.

446 The Journal of Educational Psychology

TABLE I1

Binet QUC8- tions

1 0 11 12 1 3 14 1 5 1 6 17 1 8 1 9 20 21 22 23 24 25 20 27 28 29 30 31 32 33 34 35 3G 37 38 3 9 4 9 4 1 4 2 4 3 4 4 4 5 4 6 4 7 4 8 4 9 50 51 52 53 54 55 56 57 58 5 9

Description of Binet Questions

Two lines Compunng faces . . . Triple order . . . . SQuare 10 syllables Age Morning and afternoon . 4 colors

2 weights Fingers 13 pennies Diamond Transcription Days of week . . . 4 coins . . . Divided card Definition (use) 5 numbers . Picture (description) . . 16 syllables . . . . Right and left . . Missing features Pence and halfpence Differences (concrete) Dictation . . Reading (2 facts) Easy questions . Counting 20 to 1 D a t e . . C h a n g e 6 n u m b e r s M o n t h s . . . 9 coins Heading (6 facts) . . . Definition (class) 5 weights Sentence building1 Memory drawing . . Absurdities . . . . D i f f i c u l t q u e s t i o n s . . . . 60 words 7 numbers Sentence building1 3 rhymes Mixed sentences Picture (interpretation) Suggestion . . Problems . 26 syllables . . .

Percent- age of seven-

year-olds who

answer correctly

Pi

99 2 99.0 98 4 98 2 99 6 97 3 97 8 97 8 96.1 92 2 95 3 96 6 95 6 94 8 93 9 94 0 86 2 87.2 94 0 84.7 85.3 83 8 87 3 80 5 70 6 67 3 58 2 51 7 53 6 36 4 39 1 42 5 35 3 33.3 19 3 23 6 21.0 14.7

9 . 8 5 2 6 3 7 6 5 8 3 1 3 4 0 9 4 6 6 9 0 9 1.8

Percent- age of eight-

year-olds who

answer correctly

P.

99 6 9 9 2 99 2 99 2 99 2 99 0 98 9 98 6 98 9 96 4 98.6 99 0 97 6 99 0 95 6 97 2 92 2 94 2 95 6 93 6 95 2 90 1 95 6 90 4 80 0 87 2 82 1 76 5 76 0 71 4 68 2 60 2 (il 6 57 2 44 5 40 4 36.1 34.4 28 9 24 4 13.1 21 G 18 8 16 5 20 2 15 3 11 2 23 6 2 2 4 3

Difficulty for seven- year-olds

Xi

— 2 41 - 2 33 - 2 14 — 2 10 - 2 65 - 1 93 - 2 01 - 2 01 — 1 76 - 1 . 4 2 — 1 67 - 1 83 - 1 71 - 1 . 6 3 - 1 55 - 1 55 - 1 09 - 1 14 — 1 55 - 1 . 0 2 - 1 05 - 99 - 1 14 - 86 - 54 - .45 — 21 - 04 - 09 + 35 + .28 + 19 + 38 + 43 + 87 + .72 + .81 + 1.05 + 1 29 + 1.63 + 1 53 + 1 43 + 1.57 + 1.87 + 1 83 + 2 37 + 1 68 + 1 48 + 2.37 + 2 10

Difficulty for eight- year-olds

X,

— 2 65 - 2 41 - 2 41 — 2 41 - 2 41 - 2 33 - 2 29 - 2 20 — 2 29 - 1 . 8 0 — 2 20 - 2 33 - 1 98 - 2 . 3 3 - 1 . 7 1 - I 91 - 1 42 - 1 . 5 7 - 1 71 - 1 . 5 2 - 1 66 - 1 29 - 1 71 - 1 . 3 0 - .84 - 1 14 - .92 - .72 - 71 - .57 - .47 - .26 - 29 - .18 + 14 + .24 + .36 + .40 + .56 + .69 + 1.12 + .79 + .89 + .97 + .83 + 1 02 + 1.22 + .72 + 2 01 + 1.72

Xi'

5 81 5 43 4 58 4 41 7 02 3 72 4.04 4 04 3 10 2 02 2 79 3 35 2 92 2 66 2 40 2 40 1.19 1 30 2 40 1 04 1 10

.98 1 30

.74 29

. 2 0 04 00

.01

.12 08 04 14

.18

.76 52 66

1 10 1 66 2 66 2 34 2 04 2 46 3 50 3 35 5 62 2 82 2 19 5 62 4 41

X.*

7 02 5 81 5.81 5 81 5 81 5 43 5 24 4 84 5.24 3 24 4.84 5 43 3 92 5 43 2 92 3 65 2 02 2 46 2 96 2 31 2 76 1 66 2 92 1.69

71 1.30

. 8 5 5 2 50

. 3 2 2 2 0 7 OS 0 3

. 0 2 0 6

. 1 3

. 1 6

. 3 1

. 4 8 1.25

.62

.79

.94 69

1 04 1.40

. 5 2 4 04 2 96

1 The children were classified according to their last birthday. The ages are handled as integers for convenience in the calculations

'Jaederholm, G. A : On the Measurement of Intelligence Scandinavian Scientific Reticle, April, 1923

> Burt, Cyril- "Mental and Scholastic Tests " P S. King, London, p 132.

(1) g , + 2A'7~

(2)

(3)

= +26.23 = -40.87 = - 1 4 . 6 4

SAT7

8 n

a - # S 8 = ^

4 Method of Scaling

TABLE II

-14.64 50

- 4 0 27 "50

xv , AT8

2 , n

+ = +13.68 _ = -53.95

= -40.27

- . 8 0 5

/119.32 50

858

- .6480

2A',2 = 2AV =

?i =

TO7 2 -

= 1.478

= 1.318

447

113 oo 119 32 50

.OSJS

6480

(4) il/7 = 4.06 (derived from similar calculations for the 6-7 age groups) <77 = 1.33 (derived from similar calculations for the 6-7 ago group'.)

1.06

4.S7

(5) (S7\ V.'Ss/

1.33 X J _ 4 7 8 "1.318

<78 = 1.496

It will be seen from Table V that the test questions are more numer- ous at certain ages than at others. For example, there are 12 questions that scale at par between the ages five and six, whereas there are only four questions that scale at par between six and seven.

If we desire to ascertain the scale value of each test question it can be done best by noting the two percentages just above and just below 50 in curves like those of Fig. 5. Let these two percentages be Pi and p2 as before, and let Xi and A'» be the corresponding sigma values. Then we have for the scale value the following determination.

Scale value of test question = }t[(Mi + X*n) + (3/2 + AV=)]

448 The Journal of Educational Psychology

Applying this procedure to Question 40 as an example, we have the following numerical values:

Vi = .391 pi = .682 X-, = +.277 Xs = -.473 M7 = 4.0G1 Ms = 4.875 0-7 = 1.333 erg = 1.496

These values give a scale value of 4.3 for Question 40. The scale value of particular test questions may also be determined by interpola- tion between the scale values of the ages corresponding to pi and p2. These two methods give practically the same numerical results. This latter method was used for the determination of the scale value of each test question. The result is shown in Fig. 6 which also brings out rather strikingly the fact that the questions are unduly bunched at certain ranges and rather scarce at other ranges.

SUMMAEY

An absolute method of scaling tests is here proposed which assumes that the distributions of ability in the several age or grade groups are normal but which allows freedom of variation for the means and for the standard deviations of the several age groups. The method is illustrated chiefly by Figs. 2 and 3 and Tables I and II. The type of result that is obtained by the method is illustrated by Fig. 4 and Table I I I .

The particular results of the method as applied to Burt's data for the Binet tests are of secondary value in this article but of some interest. Figure 4 shows that test-intelligence grows nearly as rapidly at the age of 14 as it does at the age of 9. This finding is not consistent with some current notions about so called " a d u l t " intelligence at the age of 14. It may be that this curve, if continued, would drop its acceleration to reach a limit in the early 20s or perhaps even at the age of 20 but it can hardly be extended to reach a limit much sooner than that.

It is also found that the absolute variability of test-intelligence increases noticeably with age. For example, the variability of test- intelligence for children at 14 is nearly twice that of children at three

The application of the present method of scaling to Binet test data shows that the distributions of intelligence for children can be assumed to be normal at least as far up as the age of 14.

A

, f

/̂̂ \\ S

Linear- Lfo'f Ortgtn 7hc

Hz?

© ©j

Abso/ute

€> © C

'1 ' -r-4- O

(a (7)©

6

Sco/e of

v ^ 7

Binet

meon of Sinet 7cs/ mfel/'gcce

M 0 O 0 0|2i©Jc a

© ©0 a

7esr Ques.'/ons Tesr of

/ntclfiqcncp of -3£ tjee •3?~ yeor o>d Cf°i k?rtn

3

9

r C'd C/'

© 1 cS

\ 1

A 3

o

Tio. 0.

4 CO

450 The Journal of Educational Psycoloyy

A O E M <r

SH o l 434 1.231 1.191 5}4 2.336 1.255 &H 3 372 1.255 73 2 4 061 1 333 8M 4 875 1.490 934 5.433 1.500

1034 6 190 1.453 113-2 0 776 1 446 1234 7 314 1 503 1334 8 040 1 532 UVz 8 084 1.792

TABLE IV.—CORRELATION BETWEEN SIGMA VALUES OF QUESTIONS FOR ADJACENT AGES

AGLS

- 434 - 534 - 634 - 734 - 834 - 934 -1034 -1134 -1234 -1334 -1434

N U M B E R OF O\ ERLAi'Fixo Q U E S T I O N S

20 33 41 48 50 44 31 30 26 25 23

r

+ 97 + 97 + 9S + 97 + 99 + 99 + 97 + .99 + .99 + 98 + .98

A Method of Scaling Tests

TABLE V

451

Binet test questions

(Burt's data)

8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

Year at par

3 6 3.8 3 7 4 0 4 1 4.1 4.5 4 4 4.5 4 5 4.7 4 9 5 1 5.4 5 3 5.6 5 7 5.5 5 6 5 7 5.8 5.5 5.6 5.5 6.0 6.7 6 4 6.7

Binet test questions

(Burt's data)

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Year at par

7.3 7 4 7.4 7 9 7.9 7 9 8.1 8 2 8 7 8.9 9 3 9 7 9 8

10.5 10 6 10 9 10.9 11.0 11 4 11.2 11 6 12.0 12.9 13.4 13.3