Managerial Epidemiology: Assignment Week 4

Chapter 10

Data Interpretation Issues

Learning Objectives

• Distinguish between random and

systematic errors

• State and describe sources of bias

• Identify techniques to reduce bias at the

design and analysis phases of a study

• Define what is meant by the term

confounding and provide three examples

• Describe methods to control confounding

Validity of Study Designs

• The degree to which the inference drawn

from a study, is warranted when account it

taken of the study, methods, the

representativeness of the study sample,

and the nature of the population from

which it is drawn.

Validity of Study Designs

• Two components of validity:

– Internal validity

– External validity

Internal Validity

• A study is said to have internal validity

when there have been proper selection of

study groups and a lack of error in

measurement.

• Concerned with the appropriate

measurement of exposure, outcome, and

association between exposure and

disease.

External Validity

• External validity implies the ability to

generalize beyond a set of observations to

some universal statement.

• A study is externally valid, or

generalizable, if it allows unbiased

inferences regarding some other target

population beyond the subjects in the

study.

Sources of Error in

Epidemiologic Research

• Random errors

• Systematic errors (bias)

Random Errors

• Reflect fluctuations around a true value of

a parameter because of sampling

variability.

Factors That Contribute to

Random Error

• Poor precision

• Sampling error

• Variability in measurement

Poor Precision

• Occurs when the factor being measured is

not measured sharply.

• Analogous to aiming a rifle at a target that

is not in focus.

• Precision can be increased by increasing

sample size or the number of

measurements.

• Example: Bogalusa Heart Study

Sampling Error

• Arises when obtained sample values

(statistics) differ from the values

(parameters) of the parent population.

• Although there is no way to prevent a

non-representative sample from

occurring, increasing the sample size

can reduce the likelihood of its

happening.

Variability in Measurement

• The lack of agreement in results from

time to time reflects random error

inherent in the type of measurement

procedure employed.

Bias (Systematic Errors)

• “Deviation of results or inferences from the truth, or processes leading to such deviation. Any trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically different from the truth.”

Factors That Contribute to

Systematic Errors

• Selection bias

• Information bias

• Confounding

Selection Bias

• Refers to distortions that result from procedures used to select subjects and from factors that influence participation in the study.

• Arises when the relation between exposure and disease is different for those who participate and those who theoretically would be eligible for study but do not participate.

• Example: Respondents to the Iowa Women’s Health Study were younger, weighed less, and were more likely to live in rural, less affluent counties than nonrespondents.

Information Bias

• Can be introduced as a result of

measurement error in assessment of

both exposure and disease.

• Types of information bias:

– Recall bias: better recall among cases

than among controls.

• Example: Family recall bias

Information Bias (cont’d)

– Interviewer/abstractor bias--occurs

when interviewers probe more

thoroughly for an exposure in a case

than in a control.

– Prevarication (lying) bias--occurs when

participants have ulterior motives for

answering a question and thus may

underestimate or exaggerate an

exposure.

Confounding

• The distortion of the estimate of the effect of an exposure of interest because it is mixed with the effect of an extraneous factor.

• Occurs when the crude and adjusted measures of effect are not equal (difference of at least 10%).

• Can be controlled for in the data analysis.

Criteria of Confounders

• To be a confounder, an extraneous

factor must satisfy the following

criteria:

– Be a risk factor for the disease.

– Be associated with the exposure.

– Not be an intermediate step in the

causal path between exposure and

disease.

Simpson’s Paradox as an Example of Confounding

• Simpson’s paradox means that an

association in observed subgroups of a

population may be reversed in the entire

population.

• Illustrated by examining the data (% of

black and gray hats) first according to two

individual tables and then by combining all

the hats on a single table.

Simpson’s Paradox (cont’d)

• When the hats are on separate tables, a greater proportion of black hats than gray hats on each table fit.

– On table 1: • 90% of black hats fit

• 85% of gray hats fit

– On table 2: • 15% of black hats fit

• 10% of gray hats fit

Simpson’s Paradox (cont’d)

Simpson’s Paradox (cont’d)

• When the man returns the next day

and all of the hats are on one table:

– 60% of gray hats fit (18 of 30)

– 40% of black hats fit (12 of 30)

Note that combining all of the hats on

one table is analogous to

confounding.

Examples of Confounding

• Air pollution and bronchitis are positively

associated. Both are influenced by

crowding, a confounding variable.

• The association between high altitude and

lower heart disease mortality also may be

linked to the ethnic composition of the

people in these regions.

Techniques to Reduce

Selection Bias

• Develop an explicit (objective) case

definition.

• Enroll all cases in a defined time and

region.

• Strive for high participation rates.

• Take precautions to ensure

representativeness.

Reducing Selection Bias Among

Cases

• Ensure that all medical facilities are thoroughly

canvassed.

• Develop an effective system for case

ascertainment.

• Consider whether all cases require medical

attention; consider possible strategies to

identify where else the cases might be

ascertained.

Reducing Selection Bias

Among Controls

• Compare the prevalence of the exposure

with other sources to evaluate credibility.

• Attempt to draw controls from a variety of

sources.

Techniques to Reduce

Information Bias • Use memory aids; validate exposures.

• Blind interviewers as to subjects’ study status.

• Provide standardized training sessions and protocols.

• Use standardized data collection forms.

• Blind participants as to study goals and classification status.

• Try to ensure that questions are clearly understood through careful wording and pretesting.

Methods to Control

Confounding

• Prevention strategies--attempt to control confounding

through the study design itself.

• Three types of prevention strategies:

– Randomization

– Restriction

– Matching

• Two types of analysis strategies:

– Stratification

– Multivariate techniques

Randomization

• Attempts to ensure equal distributions of the confounding variable in each exposure category.

• Advantages:

– Convenient, inexpensive; permits straightforward data analysis.

• Disadvantages:

– Need control over the exposure and the ability to assign subjects to study groups.

– Need large sample sizes.

Restriction

• May prohibit variation of the confounder in the study groups.

– For example, restricting participants to a narrow age category can eliminate age as a confounder.

• Provides complete control of known confounders.

• Unlike randomization, cannot control for unknown confounders.

Matching • Matches subjects in the study groups according

to the value of the suspected or known confounding variable to ensure equal distributions.

• Frequency matching--the number of cases with particular match characteristics is tabulated.

• Individual matching--the pairing of one or more

controls to each case based on similarity in sex,

race, or other variables.

Matching (cont’d)

• Advantages:

– Fewer subjects are required than in

unmatched studies of the same hypothesis.

– May enhance the validity of a follow-up study.

• Disadvantages:

– Costly because extensive searching and

recordkeeping are required to find matches.

Two Analysis Strategies to

Control Confounding

• Stratification--analyses performed to evaluate

the effect of an exposure within strata (levels) of

the confounder.

• Multivariate techniques--use computers to

construct mathematical models that describe

simultaneously the influence of exposure and

other factors that may be confounding the

effect.

Advantages of Stratification

• Performing analyses within strata is a

direct and logical strategy.

• Minimum assumptions must be

satisfied for the analysis to be

appropriate.

• The computational procedure is

straightforward.

Disadvantages of Stratification

• Small numbers of observations in some

strata.

• A variety of ways to form strata with

continuous variables.

• Difficulty in interpretation when several

confounding factors must be evaluated.

• Categorization results in loss of

information.

Multivariate Techniques

• Advantages:

– Continuous variables do not need to be

converted to categorical variables.

– Allow for simultaneous control of several

exposure variables in a single analysis.

• Disadvantages:

– Potential for misuse.

Publication Bias

• Occurs because of the influence of

study results on the chance of

publication.

– Studies with positive results are more

likely to be published than studies with

negative results.

Publication Bias (cont’d)

• May result in a preponderance of

false-positive results in the

literature.

• Bias is compounded when

published studies are subjected to

meta-analysis.