Research Methods

RoseP05

CourseMaterial.ppt

Home >Business & Finance homework help >Management homework help >Research Methods

Chapter 8: Hypothesis Testing, Validity, and Threats to Validity

Graziano and Raulin

Research Methods (8th Edition)

Testing Hypotheses

Researchers tests specific hypotheses

Generated from the initial research idea through a series of steps

A research idea can stimulate dozens of research hypotheses depending on how

it is translated into a statement of the problem
the variables are operationally defined

Graziano & Raulin (1997)

The Initial Idea

The initial idea is the starting point

Often vague or general
It requires refining before research hypotheses can be generated

Refinement of the initial idea is based on

(1) a search of relevant research literature

(2) initial observations of the phenomenon

Narrow and formalize the initial idea into a statement of the problem

Graziano & Raulin (1997)

Statement of the Problem

In the form of a question

Clearly indicates an expected relationship
Nature of the question dictates the required level of constraint of a study
Causal questions will require experimental research
Questions about relationships can be answered with lower-constraint research

Convert into research hypotheses by operationally defining the variables

Graziano & Raulin (1997)

Generating Research Hypotheses

Ideas lead to

observations
library research

Statement of problem

Problem statements become research hypotheses when constructs are operationalized

Graziano & Raulin (1997)

Operational Definitions

Initially covered in Chapter 3

The procedures used to measure and/or manipulate variables

Most variables can be operationally defined in many different ways

Graziano & Raulin (1997)

Research Hypotheses

State clearly the expected relationship between the variables

The form is a declarative statement, but it is a tentative statement to be tested in research

Variables in research hypotheses are stated in operational definition terms

Graziano & Raulin (1997)

The Role of Theory

Theory guides all research planning

Often the primary source of the research hypothesis
Guides the selection of variables
Guides the operational definitions of variables

Most research is based on multiple, overlapping, and interacting theories

Graziano & Raulin (1997)

Testing Research Hypotheses

Actually testing three sets of hypotheses

The null hypothesis
The confounding variable hypotheses
The causal hypothesis

Accept causal hypothesis only if you

reject null hypothesis (statistical analysis)
rule out each potential confounding variable hypothesis (based on appropriate controls)

Graziano & Raulin (1997)

Evaluating Hypotheses

Graziano & Raulin (1997)

Types of Validity

Statistical Validity

Construct Validity

External Validity

Internal Validity

Graziano & Raulin (1997)

Statistical Validity

Are the statistical tests accurate?

Threatened by

Unreliable measures
Violations of statistical assumptions

Strengthened by

Using well validated measures
Having approximately equal sample sizes in each group (covered in Appendix D)

Graziano & Raulin (1997)

Construct Validity

Is our theory the best explanation for the results?

Threatened by

Any alternative explanation for the results

Strengthened by

Using well-validated constructs to build the theoretical predictions for the study

Graziano & Raulin (1997)

External Validity

Do the results apply to the broader population?

Threatened by

Unrepresentative samples
Generalizing beyond the limits of the sample

Strengthened by

Gathering a representative sample (if possible)
Clearly describing sample, so that other researchers will know the limits of generalization

Graziano & Raulin (1997)

Internal Validity

Is the independent variable responsible for the observed changes in the dependent variable?

Threatened by

Confounding variables

Strengthened by

Adding adequate controls to reduce or eliminate confounding

Graziano & Raulin (1997)

Avoiding Confounding

Confounding and internal validity

Many sources for confounding (covered next)
With proper controls, confounding can be virtually eliminated (see Chapter 9)

Confounding and construct validity

Make sure that you have considered alternative theoretical explanations for the anticipated phenomenon

Graziano & Raulin (1997)

Confounding Variables 1

Maturation

Changes due to growth or predictable changes

History

Changes due to an event that occurs during the study

Testing

Changes due to the effects of previous testing

Graziano & Raulin (1997)

Confounding Variables 2

Instrumentation

Any change in the calibration of the measuring instrument over the course of the study

Regression to the Mean

Tendency for participants selected because of extreme scores to be less extreme on a retest

Selection

Any factor that creates groups that are not equal at the start of the study

Graziano & Raulin (1997)

Confounding Variables 3

Attrition

Loss of participants during a study; are the participants who drop out different from those who continue?

Diffusion of treatment

Changes in participants’ behavior due to information they obtained about other conditions

Sequence Effects

Effects on performance in one condition due to experience with previous conditions

Graziano & Raulin (1997)

Subject Effects

Participants are not passive

They try to understand the study to help them to know what they “should do” (termed subject effects)
Respond to subtle cues about what is expected (termed demand characteristics)

Placebo effect: treatment effect due to expectations that the treatment will work

Graziano & Raulin (1997)

Experimenter Effect

Based on the expectations of the researcher

Can affect the outcome of studies if not controlled

May be due to the experimenter providing demand characteristics to the participant

Not the same as scientific fraud (which is deliberate)

Graziano & Raulin (1997)

Validity, Control, and Constraint

Three closely-tied concepts

Validity

The accuracy of the study or procedure
Increased by using appropriate control procedures

The more controls we employ, the higher the level of constraint

Graziano & Raulin (1997)

Ethical Principles

Risk is balanced by reward

A poorly designed study will provide no useful information; therefore, any risk would be unacceptable

Informed Consent

Virtually guarantees that you will have confounding due to selection because some people will refuse to participate
A small price to pay to maintain ethical standards

Graziano & Raulin (1997)

Summary

Start by building a research hypothesis

Testing the research hypothesis is actually testing three hypotheses

(1) null; (2) confounding-variable; (3) causal

Several types of validity

Many potential confounding variables

Subject and experimenter effects can also affect the outcome of the study

Graziano & Raulin (1997)