Data Collection in Quantitative Research & Measurements and Data Quality

irisdiaz1511

Polit10e_PPT_Ch14.ppt

Home >Nursing homework help >Data Collection in Quantitative Research & Measurements and Data Quality

Chapter 14

Measurement and Data Quality

Question

Tell whether the following statement is true or false:

Measurement involves assigning numbers to objects to represent the amount of an attribute.

Answer

True

Measurement involves assigning numbers to objects to represent the amount of an attribute, using a specified set of rules. Researchers strive to develop or use measurements whose rules are isomorphic with reality.

Measurement

The assignment of numbers to represent the amount of an attribute present in an object or person, using specific rules
Rules are necessary to promote consistency and interpretability.
Advantages
Removes guesswork
Provides precise information
Less vague than words

Theories of Measurement

Psychometrics is a branch of psychology concerned with the theory and methods of psychological measurement.
Two theories
Classical test theory (CTT)
Item response theory (IRT)

Errors of Measurement

Obtained Score = True score + Error

Obtained score: an actual data value for a participant

True score: value that would be obtained for a hypothetical perfect measure attribute

Error of measurement: represents measurement inaccuracies

Factors That Contribute to Errors of Measurement

Situational contaminants
Transitory personal factors
Response-set biases
Administration variations
Instrument clarity
Item sampling

Major Types of Measures

Generic
Specific
Static
Adaptive
Reflective scales
Formative indexes

Measurement Taxonomy

Four measurement property domains
Cross-sectional domains
Reliability
Validity
Longitudinal measurement domains
Reliability of change scores
Responsiveness

Correlation Coefficients

Correlation coefficients indicate direction and magnitude of relationships between variables.
Pearson's r
Range:

From −1.00 (perfect negative correlation)

Through 0.00 (no correlation)

To +1.00 (perfect positive correlation)

Reliability

Consistency—the absence of variation in measuring a stable attribute for an individual
Reliability assessments involve computing a reliability coefficient.
Most reliability coefficients are based on correlation coefficients.

Question

Tell whether the following statement is true or false:

Reliability coefficients usually range from .00 to 1.00, with higher values reflecting less reliability.

Answer

False

Reliability coefficients usually range from .00 to 1.00, with higher values reflecting greater reliability not less reliability.

Reliability

Replication approaches
Test-retest reliability: administration of the same measure to the same people on two occasions
Interrater reliability: measurements by two or more observers or raters using the same instrument or measurements by the same observer or rater on two or more occasions
Parallel test reliability: measurements of the same attribute using alternate versions of the same instrument, with the same people

Reliability Coefficient (R)

Represent the proportion of true variability to obtained variability:

R = VT
Vo

Should be at least .70; .80 preferable
Can be improved by making instrument longer (adding items)
Are lower in homogeneous than in heterogeneous samples

Internal Consistency

The extent to which all the instrument’s items are measuring the same attribute
Evaluated by administering instrument on one occasion
Appropriate for most multi-item instruments
Most widely used evaluation method is the coefficient alpha.

Measurement Error

Unless a reliability coefficient is 1.0 (virtually never happens), measurement error is present.
Used to estimate the range within which the true score lies
Standard error of measurement (SEM)
Limits of agreement (LOA)
Measurement error is routinely estimated for multi-item measures developed with item response theory (IRT) methods.

Validity

The degree to which an instrument measures what it is supposed to measure (resilience)
Four aspects of validity
Face validity
Content validity
Criterion-related validity
Construct validity

Face Validity

Refers to whether the instrument looks as though it is measuring the appropriate construct
Based on judgment, no objective criteria for assessment

Content Validity

The degree to which an instrument has an appropriate sample of items for the construct being measured
Relevance
Comprehensiveness
Balance
Evaluated by expert evaluation, via the content validity index (CVI)

Criterion Validity

The degree to which the instrument correlates with an external criterion or “gold standard”
Focal measures
Expense, efficiency, risk and discomfort, criterion unavailable, and prediction

Criterion Validity (cont.)

Two types of criterion-related validity:

Predictive validity: the instrument’s ability to distinguish people whose performance differs on a future criterion
Concurrent validity: the instrument’s ability to distinguish individuals who differ on a present criterion
Specificity, sensitivity
Predictive values
Likelihood ratios
Receiver operating characteristic curve (ROC curve), area under the curve (AUC)

Construct Validity

Concerned with the questions:

What is this instrument really measuring?
Does it adequately measure the construct of interest?

Methods of Assessing Construct Validity

Hypothesis-testing validity
Convergent validity
Known-groups validity
Divergent validity (discriminant validity)
Multitrait–multimethod matrix method (MTMM)
Structural validity
Cross-cultural validity

Measuring Change

Change score: represents the amount of change between two scores
Difference score: the difference between the randomized groups at posttest
Smallest detectable change (SDC): a change in scores that is beyond measurement error
Reliable change index (RCI): assesses the clinical significance of improvement during a psychotherapeutic intervention

Responsiveness

The ability of a measure to detect change over time in a construct that has changed, commensurate with the amount of change that has occurred
Whether a change score is truly capturing a real change in the construct

Question

Tell whether the following statement is true or false:

Reliability is the degree to which an instrument measures what it is supposed to measure.

Answer

False

Validity is the degree to which an instrument measures what it is supposed to measure. Reliability is the degree of consistency or accuracy with which an instrument measures an attribute.

Critiquing Data Quality in Quantitative Studies

Can I trust the data in this study?

Are the measurements of key constructs reliable and valid, and are change scores reliable and responsive?

Psychometric Assessment

Gather evidence:
Validity
Reliability
Other assessment criteria