human factors

profileTomm.
HFchapter2.ppt
Home>Science homework help>human factors

Research Methods

It is actually way more exciting than it sounds!!!

Why do we have to learn this stuff?

Human Factors is first and foremost a science.

Thus it is based in research.

Before we delve into how to do research, you should be aware of three hurdles that tend to skew our logic.

What is Research?

  • Why do we do research?
  • To find truths! (or something close to it)

  • Involves:
  • Scientific gathering of observation or data
  • Interpretation of the meaning of this data

*

Hindsight Bias

  • The tendency to believe, after learning the outcome, that you knew it all along.

Monday Morning Quarterbacking!!!

After the OJ Simpson acquittal, my friend said that he knew he was going to get off. Did he really know??

Overconfidence

  • We tend to think we know more than we do.

  • 82% of U.S. drivers consider themselves to be in the top 30% of their group in terms of safety.
  • 81% of new business owners felt they had an excellent chance of their businesses succeeding. When asked about the success of their peers, the answer was only 39%. (Now that's overconfidence!!!)

The Barnum Effect

  • It is the tendency for people to accept very general or vague characterizations of themselves and take them to be accurate.

  • Eg: When a fortune teller is vague but you feel like they are nailing your life story.

Applied V. Basic Research

  • Applied Research has clear, practical applications.
  • YOU CAN USE IT!!!
  • Basic Research explores questions that you may be curious about, but not intended to be immediately used.

Research on therapies for drug addicts has a clear purpose.

Studying how kissing changes when you get older is interesting…but that’s about it.

Basic Vs. Applied Research

  • Basic Research
  • Driven by a scientist's curiosity or interest in a scientific question
  • Motivation is to expand man's knowledge, not to create or invent something.
  • No obvious commercial value to the discoveries that result from basic research
  • Applied Research
  • Designed to solve practical problems of the modern world and improve the human condition.
  • Basic research lays the foundation for applied research
  • Also differ in cost, time, feasibility, and safety

*

Terminology

Hypothesis

  • Expresses a relationship between two variables.
  • Participating in class leads to better grades than not participating.
  • A variable is anything that can vary among participants in a study.

What is a variable

  • What is a variable?
  • Something that may or does vary or change
  • Performance
  • Temperature
  • Weight
  • Type of programming language
  • The amount of air in a football (i.e., pressure)
  • The elevation of a mountain (i.e., altitude)

*

Independent Variable

  • Whatever is being manipulated in the experiment.
  • Hopefully the independent variable brings about change.

If there is a drug in an experiment, the drug is almost always the independent variable.

Dependent Variable

The dependent variable would be the effect of the drug.

  • Whatever is being measured in the experiment.
  • It is dependent on the independent variable.

Operational Definitions

  • Explain what you mean in your hypothesis.
  • How will the variables be measured in “real life” terms.
  • How you operationalize the variables will tell us if the study is valid and reliable.

Let’s say your hypothesis is that chocolate causes violent behavior.

  • What do you mean by chocolate?
  • What do you mean by violent behavior?

Sampling

  • Identify the population you want to study.

  • The sample must be representative of the population you want to study.

  • GET A RANDOM SAMPLE.

Experimental Method

  • Looking to prove causal relationships.
  • Cause = Effect
  • Laboratory v. Field Experiments

Smoking causes health issues.

Beware of
Confounding Variables

If I wanted to prove that smoking causes heart issues, what are some confounding variables?

  • The object of an experiment is to prove that A causes B.
  • A confounding variable is anything that could cause change in B, that is not A.

Lifestyle and family history may also effect the heart.

Random Assignment

  • Once you have a random sample, randomly assigning them into two groups helps control for confounding variables.
  • Experimental Group v. Control Group.

Hawthorne Effect

  • But even the control group may experience changes.
  • Just the fact that you know you are in an experiment can cause change.

Whether the lights were brighter or dimmer, production went up in the Hawthorne electric plant.

Experimenter Bias

  • Another confounding variable.
  • Not a conscious act.
  • Double-Blind Procedure.

Other Confounding Variables

  • Placebo effect

  • Order Effects

Survey Method

  • Most common type of study in psychology

  • Measures correlation

  • Cheap and fast

  • Need a good random sample

  • Low-response rate

Naturalistic Observation

  • Watch subjects in their natural environment.
  • Do not manipulate the environment.
  • The good is that there is no Hawthorne effect.
  • The bad is that we can never really show cause and effect.

Correlational Method

  • Correlation expresses a relationship between two variable.
  • Does not show causation.

As more ice cream is eaten, more people are murdered.

Does ice cream cause murder, or murder cause people to eat ice cream?

Correlation vs. Causation

  • Correlation
  • Two variables co-vary
  • One variable can be predicted by another
  • Causation
  • One variable causes a change in another variable
  • Requires an experimental manipulation to determine; cannot be determined through observation

*

Types of Correlation

Positive Correlation

  • The variables go in the SAME direction.

Negative Correlation

  • The variables go in opposite directions.

Studying and grades hopefully has a positive correlation.

Drug use and grades probably has a negative correlation.

Types of Correlation

  • Negative Correlation

No Correlation

  • Positive Correlation
  • Variables go in same direction
  • Taller people have higher self esteem
  • Variables go opposite directions
  • The more hours of TV people watch, the lower their GPA
  • No relationship

*

Correlation Coefficient

  • A number that measures the strength of a relationship.
  • Range is from -1 to +1
  • The relationship gets weaker the closer you get to zero.

Which is a stronger correlation?

  • -.13 or +.38
  • -.72 or +.59
  • -.91 or +.04

Case Studies

  • A detailed picture of one or a few subjects.
  • Tells us a great story…but is just descriptive research.
  • Does not even give us correlation data.

The ideal case study is John and Kate. Really interesting, but what does it tell us about families in general?

Statistics

  • Recording the results from our studies.
  • Must use a common language so we all know what we are talking about.

Descriptive Statistics

  • Just describes sets of data.
  • You might create a frequency distribution.
  • Frequency polygons or histograms.

Central Tendency

  • Mean, Median and Mode.
  • Watch out for extreme scores or outliers.

$25,000-Pam

$25,000- Kevin

$25,000- Angela

$100,000- Andy

$100,000- Dwight

$200,000- Jim

$300,000- Michael

Let’s look at the salaries of the employees at Dunder Mifflen Paper in Scranton:

The median salary looks good at $100,000.

The mean salary also looks good at about $110,000.

But the mode salary is only $25,000.

Maybe not the best place to work.

Then again living in Scranton is kind of cheap.

Normal Distribution

  • In a normal distribution, the mean, median and mode are all the same.

Distributions

  • Outliers skew distributions.
  • If group has one high score, the curve has a positive skew (contains more low scores)
  • If a group has a low outlier, the curve has a negative skew (contains more high scores)

Other measures of variability

  • Range: distance from highest to lowest scores.
  • Standard Deviation: the variance of scores around the mean.
  • The higher the variance or SD, the more spread out the distribution is.
  • Do scientists want a big or small SD?

Shaq and Kobe may both score 30 ppg (same mean).

But their SDs are very different.

Scores

  • A unit that measures the distance of one score from the mean.
  • A positive z score means a number above the mean.
  • A negative z score means a number below the mean.

Normal Distribution

Inferential Statistics

  • The purpose is to discover whether the finding can be applied to the larger population from which the sample was collected.
  • T-tests, ANOVA or MANOVA
  • P-value= .05 for statistical significance.
  • 5% likely the results are due to chance.

Significance

  • The Concept of Significance
  • How big a difference is significant?
  • Probability of error is used to estimate the meaningfulness of outcome
  • Statistical analyses are used to determine the meaningfulness of data
  • p = .05 commonly accepted
  • p = .01 more rigorous

*

APA Ethical Guidelines for Research

  • IRB- Internal Review Board
  • Both for humans and animals.

Animal Research

  • Clear purpose
  • Treated in a humane way
  • Acquire animals legally
  • Least amount of suffering possible.

Human Research

  • No Coercion- must be voluntary
  • Informed consent
  • Anonymity
  • No significant risk
  • Must debrief

Descriptive Research

  • Describes the characteristics of an existing phenomenon (e.g., U.S. Census/Labor statistics)
  • “Just the facts”
  • Provides a broad picture of a phenomenon
  • Pro: no test effects; Con: Can’t show causation
  • Example: Determine the number of civilian pilots currently flying over the age of 55

*

Observation

  • Identify variables to be measured
  • Determine how you will observe and record the variables
  • Determine the conditions under which the observation will occur
  • Example:
  • Observational Research Questions: Is new stop sign visible?
  • Dependent Variable: Whether or not people stop or not
  • Conditions: Day vs. night, congested vs. light traffic
  • You can draw conclusions about how visible the stop sign is based on driver behavior

*

Surveys and Questionnaires

  • Popular form of research in HF and Beyond
  • Quantitative vs. Qualitative
  • Pros:
  • Inexpensive, quick, easy to distribute
  • Cons:
  • Subjective in nature, low response rate, hard to ensure validity and reliability (best to use a VALIDATED SURVEY)
  • Anonymity often important to ensure truthful responses

*

Steps in Conducting an Experiment

Define the Problem and Hypothesis

Develop the Experimental Plan

Conduct the Study

Analyze the Data

Draw Conclusions

*

Pros and Cons of Experiments

  • Pros
  • You can control the environment and situation which allows you to draw more conclusions
  • Less confounding variables
  • Cons
  • Generalizability
  • Representativeness of subjects and environment
  • Hawthorne Effect/Test Effect
  • But even the control group may experience changes
  • Just the fact that you know you are in an experiment can cause change
  • Whether the lights were brighter or dimmer, production went up in the Hawthorne electric plant

*

1a. Define the Problem

  • What is the research question you want answered?

  • What are the variables of interest?

  • Example
  • Does talking on the phone and driving lead to distracted driving and how does it compare to drunk driving?
  • Variables
  • IV: secondary task/state: texting, intoxication, control
  • DV: driving performance, errors, accidents

*

1b.Develop a Hypothesis

Hypotheses

  • “an educated guess”--reflects the general problem statement or question that is being asked in the research
  • Based on theory/precise/must be testable
  • Expresses a relationship between two variables.

  • Examples:
  • Texting while driving will lead to similar performance decrements as drunk driving
  • Participating in class leads to better grades than not participating

*

1c. Operational Definitions

Explain what you mean in your hypothesis.

  • How will the variables be measured in “real life” terms.
  • How you operationalize the variables will tell us if the study is valid and reliable.

Let’s say your hypothesis is that chocolate causes violent behavior.

  • What do you mean by chocolate?
  • What do you mean by violent behavior?

*

2. Develop an Experimental Plan

  • Develop a detailed experimental plan
  • Who will your participants be?
  • What task will they perform?
  • What apparatus/equipment will they perform on?
  • How will you manipulate your Independent Variable (IV) and how many levels will there be?
  • How will you measure your Dependent Variable (DV)?
  • How will you analyze your data
  • Develop testing protocol/script

*

2a. Develop a plan – Who?

  • Identify the population you want to study.
  • You will select a “sample” of the population to study – to key is to ensure the sample is representative so the results are generalizable.
  • If you can, GET A RANDOM SAMPLE
  • Sampling Methods
  • Simple random sampling--each member has an equal and independent chance of being selected
  • Systematic sampling--less random
  • Convenience sampling
  • Samples and sampling error
  • Estimating sample size--power analysis

*

2b. Develop a plan –
Experimental Design

  • Number of groups within your IV
  • Number of IVs
  • Number of DVs
  • Number of times the DV is collected
  • Whether a subject is within or between

*

Two Group Design

  • One IV
  • Two Groups within the IV
  • Example:
  • Does drivers education lead to better driver performance?
  • IV = Training
  • Group 1 = no training (control)
  • Group 2 = training
  • DV = Driver Performance

IV = Drivers Ed Training
G1 = No Training G2 = Training

*

Multiple Group Design

  • One IV
  • Multiple Groups within the IV
  • Between groups
  • Example:
  • What is the optimal amount of sleep to get the night before a test?
  • IV = Amount of sleep
  • Group 1 = 4 Hrs
  • Group 2 = 6 hrs
  • Group 3 = 8 hrs
  • Group 4 = 10 hrs
  • DV = Test Performance

IV = Amount of sleep
G1 = 4 Hrs G2 = 6 hrs G3 = 8 hrs G4 = 10 hrs

*

Factorial Design

  • Multiple IVs
  • Can have multiple groups within each IV
  • Can assess interactions
  • Example:
  • What type of computer is easier to operate by different age groups?
  • IV 1 = Type of Computer
  • PC
  • Mac
  • IV 2 = Age group
  • <18
  • 18-25
  • >25
  • DV = Time to complete task

Type of Computer
PC Mac
Age <18 G1 G2
18-25 G3 G4
<25 G5 G6

*

Between Subjects vs.
Within Subjects Design

  • Between subjects
  • Different people in the different groups

  • Within subjects
  • Same people in the different groups
  • Experience different IV levels at different times
  • Benefits:
  • Requires fewer participants
  • More sensitive

IV = Drivers Ed Training
G1 = No Training G2 = Training
IV = Drivers Ed Training
G1 Trial 1 = No Training Trial 2 = Training

*

Mixed Design

  • Uses both Within and between subject IVs
  • Example:
  • Which driver interface if more effective?
  • IV 1 (Within)
  • Driver Interface
  • IV 2 (between)
  • Gender
  • Order Effects
  • Need to counterbalance

I Interface
Trial 1 = Interface 1 Trial 2 = Interface 2
Gender Male G1 G1
Female G2 G2

*

Multiple Dependent Variables

  • May measure more than on DV
  • Example:
  • What is the optimal amount of sleep to get the night before a test?
  • IV = Amount of sleep
  • Group 1 = 4 Hrs
  • Group 2 = 6 hrs
  • Group 3 = 8 hrs
  • Group 4 = 10 hrs
  • DV1 = Test Performance
  • DV2 = Confidence (Survey)
  • DV3 = Arousal (Heart Rate)

IV = Amount of sleep
G1 = 4 Hrs G2 = 6 hrs G3 = 8 hrs G4 = 10 hrs

*

2c. Develop a plan –
Random Assignment

  • Once you have a random sample, randomly assigning them into two groups helps control for confounding variables.

  • Experimental Group vs. Control Group.

*

  • Whatever measures are collected, they must be reliable and valid…
  • Reliability--consistency or stability of the measures of a variable over time
  • Validity--determines the extent to which different variables actually measure what was intended
  • Face validity--extent to which a measure looks as though it measures what is intended
  • Content validity--extent to which a measure of some variable samples a domain (e.g., ATC should include all facets of their performance rather than just a single aspect)
  • Construct--extent to which a measure is really tapping the underlying psychological “construct” or theory of interest (e.g., theory of intelligence)

2d. Develop a plan – Measures

*

2e. Develop a plan – Considerations

  • Confounding Variables
  • The object of an experiment is to prove that A causes B.
  • A confounding variable is anything that could cause change in B, that is not A.
  • Examples
  • Experimenter Bias - not a conscious act
  • Single Blind vs. Double-Blind Procedure
  • Standardized Script
  • Placebo effect/Test Effect/Hawthorne Effect
  • Order Effects

*

3. Conduct the Study

  • Prepare the materials
  • Conduct a pilot study
  • Adjust as necessary
  • Start actual data collection
  • Make sure methods remain constant
  • Inter-rater reliability
  • Calibration

*

4. Analyze the Data

  • DV is measured for each participant
  • Data are analyzed to evaluate the research questions and hypotheses (e.g., to see if there are significant differences between)
  • Descriptive Statistics
  • Inferential Statistics

*

Descriptive Statistics

  • Summarize the dependent variables for the different groups
  • Quantitative
  • Central Tendency
  • Variability
  • Visual
  • Frequency distribution
  • Histograms

*

Significance

  • The Concept of Significance
  • How big a difference is significant?
  • Probability of error is used to estimate the meaningfulness of outcome
  • Statistical analyses are used to determine the meaningfulness of data
  • p = .05 commonly accepted
  • p = .01 more rigorous

*

5. Draw Conclusions

  • Type I Error – Concluding IV had an effect when it was due to chance
  • At p = .05, will make this error 1 in 20 times
  • Type II Error – Concluding IV did NOT have an effect when it actually did
  • Probability inversely related to p value
  • Must balance the pros/cons of each
  • Power influenced by number of participants
  • The more participants, the more likely you are to find an effect
  • Statistical significance vs. Practical significance

*