Validity and Reliability

Explaination of Validity & Reliability of Measurement

· Validity

· the extent to which a measurement instrument measures what it is intended to measure

· Reliability

· the consistency with which a measurement instrument yields a certain result when the entity being measured hasn’t changed

Validity of Measurement Instruments (1 of 2)

· Face Validity

· Is extent to which an instrument looks like it measures a characteristic

· Relies on subjective judgment

· Content Validity

· Is extent to which a measurement instrument is a representative sample of the content area being measured

· Criterion Validity

· The extent to which the results of an assessment correlate with another, related measure

· Construct Validity

· The extent to which an instrument measures a characteristic that cannot be directly observed but is assumed to exist (such as intelligence)

Determining Validity (1 of 2)

· Table of specifications

· The researcher constructs a two-dimensional grid listing the specific topics and behaviors that reflect achievement in the domain.

· Multitrait-multimethod approach

· Two or more different characteristics are each measured using two or more different approaches. The two measures of the same characteristic should be highly related.

· Strive for consistency with a particular conceptual framework

· Conduct one or more pilot tests of your assessment strategy

· Judgment by a panel of experts

· Several experts in a particular area are asked to scrutinize an instrument to ascertain its validity for measuring the characteristic in question

Reliability

· Reliability is the consistency with which a measuring instrument yields a certain result when the entity being measured hasn’t changed.

· Instruments designed to measure social and psychological characteristics (insubstantial phenomena) tend to be even less reliable than those designed to measure physical (substantial) phenomena.

Determining the Reliability of a Measurement Instrument (1 of 2)

· Interrater reliability

· the extent to which two or more individuals evaluating the same product or performance give identical judgments

· Test-retest reliability

· the extent to which a single instrument yields the same results for the same people on two different occasions

· Equivalent forms reliability

· The extent to which two different versions of the same instrument yield similar results

· Internal consistency reliability

· The extent to which all of the items within a single instrument yield similar results