Research paper writing

Reliability Notes

Reliability – the consistency with which test scores yield the same result in measuring whatever it measures. For test scores to be valid, test scores must be reliable. However, just because test scores are reliable, does not mean that the test scores are valid. Types of Reliability Coefficient of stability - test-retest method Coefficient of equivalence or alternate form reliability - parallel forms of a test Coefficient of internal consistency Split-Half KR-20 or KR-21 – Kuder-Richardson formula (multiple-choice items or items

scores as either correct or incorrect) Cronbach’s alpha (performance type items or items on a scale)

Appropriate Reliability Values for Use .60 to .70 okay for research purposes .80 school use to group classes .90 instructional decisions .90 to .95 important individual decisions

Index of reliability is the square root of reliability coefficient. The index of reliability informs one of the total variability that is true variability. Observer Agreement: Interrater reliability or interjudge reliability: Two or more judges over one occasion Intrarater reliability or intrajudge reliability: One judge over two occasions

Computations: Percentage of agreement Correlation coefficient

Standard Error of Measurement (SEM) allows one to generate a confidence interval where one would expect to find the individual’s true score from a single testing occasion.

r1σ xxxSEM −=

Standard Error of Estimate (SEE) allows one to estimate an individual’s expected score on a parallel form of the test, if taken. A confidence interval might also be generated around this estimate.

2

r xx−= 1σ 2

x SEE

Generate a confidence interval by taking the observed score plus and minus the multiplier times the SEM or SEE. Confidence interval = score ± Multiplier (e.g., 1.0, 1.96, or 2.576) * (SEM or SEE). (68%, 95%, or 99%)