877025

Psychological Testing

Review of psychometric and statistical concepts

Jonathan Rich, Ph.D. CalSouthern University

To understand psychological testing it is important to have a solid understanding of a number of concepts:

Reliability Validity

Statistical significance Mean, standard deviation, standardized scores, percentile

Standard error of measurement Standard error of estimate

Factor analysis

Reliability

•Reliability is a measure of a test’s consistency. •One type is test-retest reliability, consistency over time. •Reliability is expressed as a number from .00 to 1.00. •Typically, good test reliabilities are at least .70.

Standard error of measurement is a concept that is closely related to reliability. It indicates how much a test score is likely to change when the test is readministered. It is a measure of score accuracy.

As a rule-of-thumb, a WAIS-IV Full Scale IQ will vary up to five points over a period of weeks. Index scores will vary more and longer periods of time result in greater changes.

Validity •Validity is a measure of a test’s utility. •One type is criterion-related validity, the correlation (relationship) between a test and some related criterion, such as IQ and school grades. •Validity is expressed as a number from .00 to 1.00. •Validities are often .20-.30 for personality tests; .40-.50 for intelligence and ability tests.

Standard error of estimate is a concept that’s closely related to validity. It indicates bands of error around a predicted score. It is a measure of predictive accuracy.

Score Comparisons

On the WAIS-IV you will need to compare index scores and subtest scores to see if they are significantly different. You do not need to do any calculations, since you will look up significant differences in a table.

“Significantly different” means that the difference is big enough that it would very likely hold if the test were readministered. Non-significant differences should not be interpreted because they might be due to random factors or test error.

For a 35-year-old, a VCI-PRI difference of 8.81 points is statistically significant at p < .05 [Table B1]. So a difference of 9 points or more could be expected to be stable -- a re- administration would probably show a difference in the same direction.

But note that a difference of 9 points is not uncommon – Table B2 shows that 25.5% of the normative sample had differences at least this large. This is the BASE RATE.

Statistically significance = HOW STABLE Base Rate = HOW COMMON

A statistically significant difference which is common should be described but does not always need to be explained. It may just be part of normal variation.

A difference which is uncommon, or smaller differences with a logical explanation, should be explained. Some possible reasons are listed below.

•Cultural or language differences •Effects of brain damage or disease •Emotional effects, including depression or anxiety •Attention deficit •Physical impairments, such as impaired vision, hearing, or motor skills •Fatigue •Distraction •Learning disability

Different problems have different implications and lead to different prognoses. This should be discussed.

Mean and Standard Deviation

Tests are normed by administering them to a representative group of people and finding their mean (average) score and the standard deviation of their scores.

When the test is given to individuals, their performance can be compared to the normative group. Their raw scores can be converted to z-scores. This is a form of standardized score that shows where an individual falls compared to a population.

A z-score shows how many standard deviations a score is above or below the mean. Almost all scores are between a z-score of -3 and +3.

Standardized Scores

Z-scores are a little “messy” since they have decimal places and can be negative. They can be converted to different scales that are more convenient. Z-scores use a mean of 0 and a standard deviation of 1; IQ scores use a mean of 100 and a standard deviation of 15. Wechsler scaled scores use a mean of 10 and a standard deviation of 3.

IQ/Index Scores Scaled Scores Z-scores Percentile

70 4 -2.00 2.3%

80 6 -1.33 9.2%

90 8 -0.67 25.1%

100 10 0.00 50.0%

110 12 0.67 74.9%

120 14 1.33 90.8%

130 16 2.00 97.7%

Factor Analysis

Factor analysis is a statistical technique that shows how variables cluster together, based on correlations between them.

Factor analysis of the WAIS-IV subtests show that there are four distinct abilities being measured: •verbal skills, •nonverbal reasoning, •processing speed, and •working memory.