Statistics 100 Exam prep- for Final

Lesson 2: Statistics: Benefits, Risks, and Measurements

Assignments 

· See your Course Syllabus for the reading assignments.

· Work through the Lesson 2 online notes that follow.

· Complete the Practice Questions and Lesson 2 Assignment.

Learning Objectives

Chapters 1 and 3

After successfully completing this lesson, you should be able to:

· Identify the three conditions needed to conduct a proper study.

· Apply the seven pitfalls that can be encountered when asking questions in a survey.

· Distinguish between measurement variables and categorical variables.

· Distinguish between continuous variables and discrete variables for those that are measurement variables.

· Distinguish between validity, reliability, and bias.

Terms to Know

From Chapter 1

· statistics

· population

· sample

· observational study

· experiment

· selection bias

· nonresponse bias

From Chapter 3

· data (variable)

· categorical variables

· measurement variables

· measurement (discrete) variables

· measurement (continuous) variables

· validity

· reliability

· bias

2.1 What is Statistics?

Section 2.1. Chapter 1

Overview

What is statistics? If you think statistics is just another math course with many formulas and lifeless numbers, you are not alone. However, this is a myth that hopefully will be debunked as you work through this course. Statistics is about data.  More precisely, statistics is a collection of procedures and principles for gaining and processing information from collected data. Knowing these principles and procedures will help you make intelligent decisions in everyday life when faced with uncertainty. The following examples are meant to illuminate the definition of statistics.

Example 2.1. Angry Women

Who are those angry women? (Streitfield, D., 1988 and Wallis, 1987.) In 1987, Shere Hite published a best-selling book called Women and Love: A Cultural Revolution in Progress. This 7-year research project produced a controversial 922-page publication that summarized the results from a survey that was designed to examine how American women feel about their relationships with men. Hite mailed out 100,000 fifteen-page questionnaires to women who were members of a wide variety of organizations across the U.S. These organizations included church, political, volunteer, senior citizen, and counseling groups, among many others. Questionnaires were actually sent to the leader of each organization. The leader was asked to distribute questionnaires to all members. Each questionnaire contained 127 open-ended questions with many parts and follow-ups. Part of Hite’s directions read as follows: “Feel free to skip around and answer only those questions you choose.” Approximately 4500 questionnaires were returned. Below are a few statements from this 1987 publication.

· 84% of women are not emotionally satisfied with their relationships · 95% of women reported emotional and psychological harassment from their partners · 70% of women married 5 years or more are having extramarital affairs

You should notice that this study is an example of a sample survey. The sample is comprised of individuals who actually provided the data while the population is the larger group from which the sample is chosen and whom the sample is to represent. In this example, the population is all American women (although some people may say all American women who have relationships with men), while the sample is the 4500 respondents who returned the questionnaire.

As you might expect, a sample should appropriately represent the population. However, in this instance, the sample does not represent the population because of two problems. The first problem found in this study is that only “joiners” were allowed to be a part of the sample. Even though Shere Hite tries to defend her methods by saying she sampled from a variety of organizations, the fact remains that only people who were involved in some organization had a chance to be in the sample. This problem is an example of selection bias.

The other problem found with this sample is nonresponse bias. Nonresponse bias can occur when a large number of people who are selected for the study elect to not respond to the survey or key questions on the survey. This is clearly evident because the response rate was only 4500/100,000 = 4.5%. (Note: most researchers like response rates to be at least 60% or 70%). Moreover, the directions encouraged the participants to“skip around” and answer only questions that they liked. As you might expect, only people with strong opinions would take the time to answer a questionnaire that contained 127 open-ended questions. In fact, Shere Hite estimated that, on the average, participants took about 4.4 hours to answer the questionnaire. Also the use of the group leader to distribute the questionnaires meant that there was a gatekeeper who had power to affect both who responded and the response rate for each organization.

So with Example 2.1, the overall conclusion is that even though the sample size is quite large, the sample does not adequately represent the population. Unfortunately, the results from this 7-year study are of little value.

Example 2.2 Pets and Marriage

Does owning a pet lead to less marital problems? (Rubin, 1998)

Karen Allen, a researcher at the University of Buffalo, conducted a study to determine whether or not couples who own cats or dogs have more satisfying marriages and experience less stress than couples who don’t own pets. Allen compared 50 pet-owning couples with 50 pet-free couples. The volunteers completed a standard questionnaire that assessed both their relationships and attachments to pets. Each couple also kept track of their social contacts for two weeks. Allen examined stress levels by monitoring the heart rates and blood pressure readings while couples discussed sensitive topics. Pet-owning couples not only started out with lower heart rates and lower blood pressure readings, but also had smaller increases in heart rates and blood pressure readings when they quarreled.

The study described above is an example of a comparative study. In this instance, the couples who owned pets are compared with couples who do not own pets as shown in Figure 2.1.

Figure 2.1. Illustration of a Comparative Study

A comparative study can either be an observational study or an experiment.  Observational studies collect data on participants in their naturally occuring settings/groupings, while withexperiments, the participants are randomly assigned to one of two groups before the data is collected.  

The study found in this example is an observational study because participants are observed in their naturally occuring groupings as either a pet-owning or pet-free couple.  It would be difficult to conduct an experiment because the researcher would have to randomly assign couples to either have or not have pets.  It is not ethical to impose pet-ownership on the couples, nor would it necessarily be good for the pet.  So what are the statistical differences between observational studies and experiments? With an experiment, appropriate evidence can support cause and effect conclusions. This is not possible with observational studies.

In this study one cannot say that owning pets causes married couples to have less stress and more satisfying marriages because randomization was not used to cancel out other factors that may affect stress level and marital satisfaction. Factors such as income, number of hours spent working, where you live (i.e., suburbs versus inner city), whether or not there are children, etc., may also be responsible for changes in stress level and marital satisfaction. We will never know because the study is not a randomized experiment.

The researcher correctly stated the conclusion by indicating that there was a difference in the two groups when considering stress level and marital satisfaction. Appropriately, no “cause and effect” language was used. However, it is not uncommon for people who have no statistical background to incorrectly infer a “cause and effect” conclusion from observational studies. So as you examine other studies that are found in the daily news, first determine if the study is an experiment or an observational study. Next decide if the conclusions are appropriate for the type of study that was conducted.

Example 2.3. Heights of Males and Females

One of the major points brought out in Chapter 1 is that the number of people in a study is an important factor to consider when designing a comparative study. To help you understand this concept, consider the following two samples of five heights in inches.

Sample of Female Heights in Inches:    61 64 68 66 63 Sample of Male Heights in Inches:       76 64 70 68 71

Do we have enough evidence to say that there is a difference in heights when comparing a sample of five female heights with a sample of five male heights? In order to answer this question, look at Figure 2.2.    

Figure 2.2. Heights for Sample Size 5

As you examine the graph you will probably decide that the evidence may not be strong enough to clearly say that there is difference in the two genders with regard to height. So suppose we instead obtain a sample of 15 female heights and 15 male heights. The results are found in Figure 2.3.

Figure 2.3. Heights for Sample Size of 15

What you should notice is that it is easier to distinguish between the two groups with the larger sample size. If the sample size were increased to a value even greater than 15, the differences in the two groups would be easier to detect.  Therefore, sample size is an important factor to consider when trying to detect differences between groups.

The overall conclusion from Chapter 1 is that in order to conduct a proper study, one must:

· Get a representative sample

· Get a large enough sample

· Decide whether the study should be an observational study or an experiment

2.2 Asking Research Questions

Section 2.2. Chapter 3

Overview

Suppose you desire to do a study or administer a survey. As an investigator, the most challenging task that you will confront is to decide what questions to ask and/or what measurements to obtain. In this chapter you will be introduced to some key definitions associated with obtaining measurements. You will also learn about possible pitfalls found with survey questions.

It’s All in the Wording

Chapter 3 lists seven possible pitfalls that can occur when asking questions in a survey or study. Of all the possible pitfalls, the one that is most commonly found is deliberate bias. People who use a form of deliberate bias often desire to gather support for a specific cause or opinion. It is also possible that more than one type of pitfall can happen at the same time. Examine the following examples.

Example 2.4. Deliberate Bias (One-Sided Statements)

Consider two different wordings for a particular question:

Wording 1: It is hard for today’s college graduates to have a bright future with the way things are today in the world.

a. agree

b. disagree

Wording 2: Today’s college graduates will have a bright future.

a. agree

b. disagree

Although Wording 1 and Wording 2 are contradictory statements, when both questions are used in the same survey, it is not uncommon to find that people answer “agree” to both questions. This is because respondents tend to agree to one-sided statements. Listed below are revised wordings for these two questions. These choices are preferred because the statements are now at least two-sided.

Revised Wording 1: Do you agree or disagree that it is hard for today’s college graduates to have a bright future with the way things are today in the world?

Revised Wording 2: Do you agree or disagree that today’s college graduates will have a bright future?

Example 2.5. Deliberate Bias (Filtering)

Consider two different choices of answers for a particular question:

Choice 1: What is your opinion of our current president?

a. favorable

b. unfavorable

Choice 2: What is your opinion of our current president?

a. favorable b. unfavorable c. undecided

This example illustrates the problem of “filtering.” Filtering exists when certain choices such as “undecided” or “don’t know” are not included in the list of possible answers. People tend to provide an answer of “undecided” or “don’t know” only when these choices are included in the list of possible answers.

Example 2.6. Deliberate Bias (Importance of Order)

Consider two different wordings for a particular question:

Wording 1: Pick a color: red or blue?

Wording 2: Pick a color: blue or red?

The results in Table 2.1 are from a study conducted in a Statistics class.  As you can see the results vary somewhat based on the order in which the colors are presented. Even though many people probably have a preference for one color over the other, if order does not matter, the percents should be same with each wording.

Table 2.1. Deliberate Bias (Order of Comparisons)

Color Choice	Wording 1	Wording 2
Red	59%	45%
Blue	41%	55%

Example 2.7. Deliberate Bias (Anchoring)

Consider two different wordings for a particular question:

Wording 1: Knowing that the population of the U.S. is 270 million, what is the population of Canada?

Wording 2: Knowing that the population of Australia is 15 million, what is the population of Canada?

This survey was conducted in Stat 100 classes where both wordings of the question were randomly distributed.  The students did not know that there were two versions of this question so each only answered the question that they received. The results for this survey are found in Figure 2.4.

Figure 2.4. STAT 100 Survey Results

As you can see, the students were influenced by the wording of the question that they were asked to answer. People’s perceptions can be severely distorted when they are provided with a reference point or an anchor. People tend to say close to the anchor because of either having limited knowledge about the topic or being distracted by the anchor. You should also consider the following three points:

· The sample sizes were large enough to detect a difference in the two groups (recall the point made in Chapter 1)

· Canada’s population is about 30 million

· The anchor might be less distracting if the following wording were used: “What is the population of Canada, when knowing that the population of the U.S. is 270 million?”

Example 2.8. Unintentional Bias

Consider two different wordings for a particular question:

Wording 1: Do you favor or oppose an ordinance that forbids surveillance cameras to be placed on Beaver Avenue?

Wording 2: Do you favor or oppose an ordinance that does not allow surveillance cameras to be placed on Beaver Avenue?

People will tend to answer “oppose” or “no” to a question that contains words such as forbid, control, ban, outlaw, and restrain regardless of what question is actually being asked. People do not like to be told that they can’t do something. So the responses to the two questions would not provide similar results. Wording 2 would be preferred over Wording 1.

Example 2.9. Unnecessary Complexity (“Double-Barreled” Problem)

Consider the following question.

Question: Do you think that health care workers and military personnel should first receive the smallpox vaccination?

The problem with this question is that the respondent must consider both health care workers and military personnel at the same time. The following rewording is much better.

Revised Question: Who should first receive the smallpox vaccination?

a. health care workers

b. military personnel

c. both Health care workers and Military Personnel

d. other

Example 2.10. Asking the Uninformed and Unnecessary Complexity (Double Negative Problem and List Problem)

Consider the following question.

Question: Do you agree or disagree that children who have a Body Mass Index (BMI) at or above the 95th percentile should not be allowed to spend a lot of time watching television, playing computer games, and listening to music?

The first concern with this question is that many people may not clearly understand what the Body Mass Index (BMI) represents. BMI is a measure that is used to identify obesity and is calculated by dividing a person's weight (in kilograms) by the square of their height (in meters). (Note: many Web sites have BMI calculators.) In children and adolescents, obesity is defined as a BMI for age and gender at or above the 95th percentile. This definition should be included prior to the listing of the question on a survey.

This question can also cause problems because of a possible “double negative”. Specifically, the problem is with the “disagree” choice. This choice produces a double negative because “disagree” and “should not” are both in the statement. Many respondents will not understand what they are really saying. (It is easy to make the mistake of the double negative).

Revised Question-First Revision: Do you agree or disagree that children who have a Body Mass Index (BMI) at or above the 95th percentile should spend less time watching television, playing computer games, and listening to music?

As you examine this revised question you should also note that there still is a list of three choices embedded in the questions. Since respondents sometimes can get hung upon the list of choices; the second revision would be preferred.

Revised Question-Second Revision: Do you agree or disagree that children who have a Body Mass Index (BMI) at or above the 95th percentile should spend less time in sedentary activities?

A follow-up question could be asked to clarify which sedentary activities should be reduced.

2.3 Defining a Common Language

In the previous examples we mostly considered problems associated with questions that measure opinion.  In order to discern what we want to measure, we also need to understand some basic definitions. Data is a collection of a number of pieces of information. Each specific piece of information is called an observation.  The observations are measurements of certain characteristics which we call "variables".   The word “variable” is used because the pieces of information, the observations, vary from one person to the next.

Figure 2.5. Types of Data

Example 2.11. Variables

Consider the following variables:

Table 2.2. Classification of Variables

Number	Variable	Type of Variable
1	Which are you? Near-sighted, far-sighted, neither	Categorical
2	What is your height?	Measurement and Continuous
3	How many phone calls do you typically make in a day on a cell phone?	Measurement and Discrete
4	What is your cholesterol level?	?

Hopefully, you find the classification of the first three variables easy to understand.

Variable #1 is a categorical variable because the possible choices are “words” or“categories.”

Variable #2 is a measurement variable because the possible choices are “numbers.” This variable is also called a continuous variable because it can assume a range of values.  You need an instrument, such as a tape measure or a ruler, to determine height. With measurement variables that are continuous, it is often necessary to use an instrument to determine the value of the variable.  Measurement variables that are continuous can be subdivided into fractional parts (subdivided into smaller and smaller units of measurement).  Typically, a measurement-continuous variable is expressed as "an amount of " something.

Variable #3 is a measurement variable because the possible choices are numbers. It is also a discrete variable because one can simply count the number of phone calls made on a cell phone in any given day. The possible numbers are only integers such as 0, 1, 2,….50, etc. (Some of you probably make a lot of cell phone calls.)   Measurement-Discrete variables cannot be subdivided into fractional parts (smaller and smaller units of measurement).  Typically, a measurement-discrete variable is expressed as "a number of " something.

Variable #4 is somewhat ambiguous. Obviously the variable is a measurement variable. But the question that remains is whether this variable is discrete or continuous. One person may state that their cholesterol level is 169 points, while a physician may report the cholesterol level as 169 milligrams per deciliter (mg/dL). Which is correct? Cholesterol levels must be determined by a blood test where an instrument is used to determine the final value. The reported value represents the concentration of cholesterol in the blood. The appropriate units are milligrams per deciliter (mg/dL). What typically happens is that the value of the cholesterol level is rounded to the nearest whole number. Consequently, the cholesterol level starts to be viewed as a discrete variable.  However, this perception is incorrect because cholesterol levels cannot be counted.

Example 2.12. Best Way to Determine Heart Rate

Consider an experiment where heart rate (heart beats/minute) is measured by three different methods. Let’s consider three different methods to determine heart rate.

Method 1: Count heart beats for 6 seconds & multiple by 10 to get heart beats/minute

Method 2: Count heart beats for 30 seconds & multiply by 2 to get heart beats/minute

Method 3: Count heart beats for 60 seconds

We collected six measurements on an individual for each of the three methods. These results are found in Table 2.3.

Table 2.3.   Results from the Heart Rate Experiment

Method	Six Results	Heart Rate (HeartBeats/Minute)	Minimum and Maximum Heart Rate	Average Heart Rate
1	7, 7, 7, 7, 7, 7	70, 70, 70, 70, 70, 70	70, 70	70
2	36, 35, 37, 38, 37, 37	72, 70, 74, 76, 74, 74	70, 76	73
3	73, 76, 74, 75, 74, 75	73, 76, 74, 75, 74, 75	73, 76	74.5

In this example, we will not explore whether or not heart rate is a valid measure of overall health and fitness. Obviously, it does provide some information about whether or not a person may have some health problems. But by itself, it usually does not provide a complete picture. The questions that we should pose are the following:

Question 1: Which method is the most reliable?

Question 2: Which method is the most biased?

What may surprise you is that the answer to both questions is method 1.   Method 1 is the most reliable because every time we took the measurement we observed 7 beats in 6 seconds. The results are consistent.   Results from method 1 are also the most biased because it consistently underestimates the individual's true heart rate.  If you look at the results from method 3, which is really the best method to determine heart rate, you find that the individual's average heart rate is 74.5 beats/minute. The results from method 1 always fell below this value. What this means is that even though method 1 is reliable, it still can have other problems, which in this case, is biasedness.

Example 2.13. Validity or Reliability

Suppose you are interested in knowing whether the average price of homes in a certain county had gone up or down this year in comparison with last year. Would you be more interested in having a valid measure or a reliable measure of sales?

Ideally you would like the measure to be both valid and reliable. However, a reliable measure that is not valid, can still often provide some meaningful information. Since the goal is to make a comparison of the average price of homes over two years, the measure must be reliable. So, even if the measure is not the most valid, the amount of change from one year to the next may be sufficient information to make a comparison.

Lesson 2 Practice Questions

Answer the following Practice Questions.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

Use the following scenario with Questions 1 and 2. Suppose you have 30 blueberry bushes and want to know if fertilizing them will help them produce more fruit. You randomly assign fifteen of them to receive fertilizer and the remaining fifteen to receive none.

1. What type of study is being implemented?

a. observational study

b. experiment

c. neither a or b

d. both a and b

2. If the fertilized bushes produce significantly more fruit than the unfertilized bushes, can you conclude that the fertilizer caused the bushes to produce more fruit?

a. yes

b. no

3. Consider the following research question. "Do you agree that constituents should be allowed to recall elected officials?" Which of the following pitfalls associated with research questions applies in this instance?

a. deliberate bias

b. unintentional bias

c. ordering of questions

d. unnecessary complexity

4. Consider the following variable: the number of words in a textbook. What type of variable is this?

a. categorical

b. measurement & discrete

c. measurement & continuous

5. Which of the following characteristics could be measured as either discrete or continuous depending on the units used?

a. brand of car you own

b. caffeine consumption

c. your favorite color

Lesson 3: Measurement Data: Summaries, Displays, and Bell-Shaped Curves

Assignments

· See your Course Syllabus for the reading assignments.

· Work through the Lesson 3 online notes that follow.

· Complete the Practice Questions and Lesson 3 Assignment.

Learning Objectives

Chapters 7 and 8

After successfully completing this lesson, you should be able to:

· Interpret any of the four graphs used with measurement data.

· Distinguish between a measure of center and a measure of spread.

· Determine when sensitive statistics or resistant statistics should be used to describe a data set.

· Interpret a five-number summary.

· Apply the empirical rule to variables that are normally distributed.

Terms to Know

From Chapter 7

· dotplot

· stemplot

· histogram

· symmetric

· skewed 

· boxplot

· mean 

· median

· percentiles

· five-number summary

· outlier

· interquartile range (IQR)

· standard deviation (SD)

· sensitive measure 

· resistant measure

From Chapter 8

· normal (bell-shaped) distribution

· empirical rule

· Z-score

Commentary

Section 3.1. Chapter 7

Overview

The goal of this lesson is to learn about different ways to display and summarize measurement data. These methods will be appropriate for all measurement variables regardless of whether the variable is discrete or continuous.

Figure 3.1. Flow Chart for Display and Summarization of Measurement Data

3.1 Graphs: Displaying Measurement Data

Here are four different graphs that can be used to describe measurement data. These graphs include:

1. Dotplots

2. Stemplots (Stem and Leaf Plot)

3. Histograms

4. Boxplots

Example 3.1. Graphs

Consider the following sample. Sample: The ages of forty selected PSU Tenured Faculty (n = 40 ages)

Ages

45 59 51 62 58 54 56 42 59 49 47 52 63 40 53 61 47 54 58 53 32 61 39 51 37 43 53 46 59 56 58 48 55 50 57 60 54 63 60 55

Dotplots

The dotplot is the first graph that will be used to display this sample of 40 ages. The purpose of the dotplot is to represent each observation as a dot. On this dotplot you will find that the ages range from 32 to 63 years. You should also notice that there are more tenured faculty at older ages.  

Figure 3.2. Dotplot (Ages of 40 PSU Tenured Faculty)

Stemplots (Stem and Leaf Plot)

The second graph that is possible with measurement data is a stemplot.  Stemplots concisely display the data in order from smallest to largest. Below is the list of the 40 ages in order from youngest to oldest.

Ages (Sorted) 

32 37 39 40 42 43 45 46 47 47 48 49 50 51 51 52 53 53 53 54 54 54 55 55 56 56 57 58 58 58 59 59 59 60 60 61 61 62 63 63  

These forty observations are displayed in the stemplot found in Figure 3.3.  In this stemplot we again find that the range of ages span from 32 year to 63 years. We also find that there are more tenured PSU faculty at older ages.  Stemplots can provide useful information about small data sets.

Figure 3.3. Stemplot (Ages of PSU Tenured Faculty)

Histograms

The third graph is called a histogram. Of all the graphs presented so far, the histogram may be the most valuable. A histogram is essentially a bar graph for measurement data. The difference, however, between a histogram and a bar graph, is that with a histogram the categories are a range of numbers rather than words. The requirement is that each numerical category must have the same width.

Recall that the ages span from 32 to 63 years. The range of these ages is (63-32) = 31 years. Looking at the stemplot, one finds that there are more tenured faculty at the older ages. We want to be able to show this trend. There needs to be enough categories to properly display any trend. If we only choose 4 categories (since we have tenured faculty in their 30s, 40s, 50s, and 60s) we would not be able to detect this trend as well. If we choose 9 categories the trend will become more obvious. Statistical software will usually make this determination for you.

Width of each Category = Range/ number of categories = 31/9 = 3.4 (rounded to 4 years)

The starting point is always below our lowest observed value and the ending point is always above our highest observed value. In this instance, our starting point of 30 years is below the lowest observed age which is 32 years. The ending point of 66 years is above our highest observed age which is 63 years. In order to make sure that an observation only falls into one category, we construct our 4 year categories as shown in Table 3.1 below. Our first category includes ages starting at 30 years and ending just below 34 years. An observed age of 34 is not included in the first category, but rather is included in the second category. Using this method, no observed age can fall into more than one category. The observed ages are then placed into the appropriate category and the histogram is constructed.

Recall Ages (Sorted)

32 37 39 40 42 43 45 46 47 47 48 49 50 51 51 52 53 53 53 54 54 54 55 55 56 56 57 58 58 58 59 59 59 60 60 61 61 62 63 63

Table 3.1. Summary of Ages for 40 PSU Tenured Faculty

Numerical Category	Ages that Fall into the Category	Number of observed Ages in that Category	Percents
1.  30≤ Age < 34	Ages 30 to 33	1	1/40 = .025 (2.5%)
2.  34≤ Age < 38	Ages 34 to 37	1	1/40 = .025 (2.5%)
3.  38≤ Age < 42	Ages 38 to 41	2	2/40 = .05   (5.0%)
4.  42≤ Age < 46	Ages 42 to 45	3	3/40 = .075 (7.5%)
5.  46≤ Age < 50	Ages 46 to 49	5	5/40 = .125 (12.5%)
6.  50≤ Age < 54	Ages 50 to 53	7	7/40 = .175 (17.5%)
7.  54≤ Age < 58	Ages 54 to 57	8	8/40 = .20   (20.0%)
8.  58≤ Age < 62	Ages 58 to 61	10	10/40 = .25 (25.0%)
9.  62≤ Age < 66	Ages 62 to 65	3	3/40 = .075 (7.5%)
		n = 40	40/40 = 1.0 (100%)

The information from Table 3.1 is used make the histogram found in Figure 3.4. The horizontal axis displays the categories while the vertical axis displays the percent of the observations (ages) found in each category. (Note: You will not be asked to make histograms. You will only be asked to interpret them. However, it is important to see how one is made so that you understand the interpretation.)

Figure 3.4. Histogram (Ages of PSU Tenured Faculty)

As you look at the histogram, you should notice that there are more ages on the upper half of the graph. In statistics, when data on a histogram is off-center, the data is labeled as skewed. In this case, the data is skewed to the left because a larger percent of the ages are found in the upper tail.

Table 3.2. Possible Histogram Interpretations                                                                               Figure 3.5 Data Shapes

Histogram Interpretations	Explanation	Graph
Skewed to the right	Larger percent of data is found on lower tail of the histogram
Skewed to the left	Larger percent of data is found on upper tail of the histogram
Symmetric	Equal percent of data on each tail of the histogram

3.2 Numbers: Summarizing Measurement Data

There are two general ways to summarize measurement data. These include:

1. measures of center

2. measure of spread (variation)

There are two ways to represent the center of measurement data. These include:

1. mean (won’t be asked to calculate)

2. median

Example 3.2. Measures of Center

Consider the following sample:

Sample:  The Number of Movie Rentals/Month for 5 Selected PSU Students (n = 5)

1     5     1     4     2

Suppose you want to find a number to represent the center of the data. The first choice would be the mean. The mean is also known as the average. The mean is found by obtaining a sum of all the observations and dividing by the sample size (n). In this instance:

mean = (1 + 5 + 1 + 4 + 2) / 5 = 13 / 5 = 2.6 movie rentals/month

Another possibility is the median. The median is the middle value of a sample when the observations are sorted from smallest to largest.

Sorted Sample:

1     1     2     4     5

In this example, the middle observation is 2 so the median = 2.0 movie rentals/month.

As you examine how the mean and median were calculated, hopefully you notice that the two methods are very different. The mean is an example of a sensitive measure because all observations were used in the calculation.  In contrast, the median is an example of a resistant measure because only the middle observation was used to determine its value.

Example 3.3. Which Measure of Center to Use

Consider the following sample:

Sample: The Annual Salaries ($) for 20 Selected Employees at a Local Company

Salaries (Sorted)

30000 32000 32000 33000 33000 34000 34000 38000 38000 38000 42000  43000 45000 45000 48000 50000 55000 55000 65000 110000

The mean for this sample is $45,000 while the media is $40,000. (Note: because the sample size is an even number, the median is the average of the middle two numbers, which in this case are $38,000 and $42,000). Even though we can always determine both the mean and median, one must determine which measure is more appropriate to use when there is a large difference between the two measures of center. In this instance, there is a difference of $5,000 between the two measures, so one should decide which measure of center is more appropriate to use. To help you understand what is happening, look at the histogram found in Figure 3.6.

Figure 3.6. Histogram (Salaries)

As you can see, the histogram is right-skewed because a larger percent of the salaries are located on the lower tail. The very large salary of $110,000 is largely responsible for the histogram being right skewed. With right-skewed histograms, the mean will be greater than the median, because the mean is sensitive to the large salary of $110,000 and is pulled in the direction of the unusually large observation.   In contrast, the median, which is the middle value of the data set, is resistant to any extreme observations because these observations are not used to determine its value.  Table 3.3 summarizes the link between the two measures of center and histogram shape.

Table 3.3. Link between Measures of Center and Histogram Shape

Compare Two Measures Of Centers	Histogram Shape
If mean and median are approximately equal	symmetric
If mean is greater than median	right skewed
If mean is less than median	left skewed

Figure 3.7. Different Distributions

So, getting back to the question of which measure of center is more appropriate to use. When you have skewed data, the mean is somewhat misleading. The mean can be pulled in one direction or the other by outliers. Generally, when the data is skewed, the median is more appropriate to use as the more typical measure of center. We generally use the mean for the typical measure of center when the data is symmetric.

However, it is important to be given both measures of center. The difference between the mean and median is important since the direction and magnitude of that difference will determine the shape of the data as indicated in Table 3.3. and in the plots shown in Figure 3.7.

The question being asked can also affect which measure of center can be considered more typical and therefore, more appropriate. Although we would normally use the median with skewed data, there may be cases where we might use the mean as a more typical measure of center. It all depends on the question being asked and on the shape of the data. For example, given the right-skewed data for the company in Example 3.3:

· If you are applying for a higher level position at the company in Example 3.3, the mean might represent the typical salary figure better than the median since it accounts for some of the higher salaries in the company.  In this case, the median salary figure may not be as appropriate as the mean salary figure.

· However, if you are applying for an entry level position within the company in Example 3.3, the median salary figure would represent the typical salary figure better than the mean and be more appropriate to use.

3.3 Five Useful Numbers (Percentiles)

A percentile is the position of an observation in the data set relative to the other observations in the data set. Specifically the percentile represents the percentage of the sample that falls below this observation. For example, the median is also known as the 50th percentile because half of the data or 50% of the observations lie below the median.  Table 3.4 displays three percentiles that will be of interest to us.  Figure 3.8 shows these percentiles (quartiles) graphically.

Table 3.4. Percentiles of Interest

Percentile	Alternate Names	Interpretation
25th percentile	· Lower Quartile (QL) · First Quartile (Q1)	25% of the data falls below this percentile
50th percentile	· Median · Second Quartile ( Q2)	50% of the data falls below this percentile
75th percentile	· Upper Quartile (QU) · Third Quartile (Q3)	75% of the data falls below this percentile

 

Figure 3.8. Quartiles for a Distribution

A five-number summary is a useful summary of a data set that is partially based on selected percentiles. Below are the five numbers that are found in a five-number summary. 

Figure 3.9. Five-Number Summary Flow Chart

Example 3.4. Five-Number Summary

Recall the sample that was used in the previous example.  

Sample: The Annual Salaries ($) for 20 Selected Employees at a Local Company

Salaries (Sorted)

30000     32000     32000     33000     33000     34000     34000     38000     38000     38000     42000 43000     45000     45000     48000     50000     55000     55000     65000     110000

Table 3.5. Five-Number Summary of Salaries

Lowest	Lower Quartile (QL)	Median	Upper Quartile (QU)	Highest
$30,000	$33,500	$40,000	$49,000	$110,000

Below are possible questions that can be answered with this five number summary.

1. What percent of the salaries lie below $49,000?

Answer: 75% Reason: $49,000 represents the 75th percentile or upper quartile

2. What percent of the salaries lie above $40,000?

Answer: 50% Reason: $40,000 represents the 50th percentile so 50% of the observations lie below this percentile and 50% lie above this percentile

3. What percent of the salaries lie between $33,500 and $49,000?

Answer: 50% Reason: asking for percent of observations that lies between the 25th percentile and the 75th percentile  (75% - 25% = 50%)

Boxplots

The five-number summary is also of value because it is the basis of the boxplot.  Figure 3.10 is a vertical boxplot of the variable salaries. The most important part of this graph is the box. The ends of the box locate the lower quartile and upper quartile, which in this case are $33,500 and $49,000 respectively. The line in the middle of the box is the median. As you examine the box portion of the box, you should notice that the median is closer to the lower quartile than to the upper quartile. This suggests that data set is skewed and specifically skewed to the right. In this instance the largest observation is represented with an asterisk. Since this observation is an unusually large salary of $110,000, the graph identifies this observation as an outlier or unusual observation.  Appropriate statistical criterion is used to determine whether or not an observation is an outlier.  Lines called 'whiskers' extend from the box out to the lowest and highest observations that are not outliers.

Figure 3.10. Horizontal Boxplot of Salaries

One of the most important uses of the boxplot is to compare two or more samples of one measurement variable.

Example 3.5. Using Boxplots for Comparisons

Recall Example 2.7 from Lesson 2. Consider two different wordings for a particular question:

Wording 1: Knowing that the population of the U.S is 270 million, what is the population of Canada?

Wording 2: Knowing that the population of Australia is 15 million, what is thepopulation of Canada?

The results from these questions are displayed on side-by-side boxplots found in Figure 3.11. 

Figure 3.11. Boxplots of Canada’s Population by Wording

Four comparisons can be made with side-by-side boxplots. One can compare the

1. centers: medians

2. amount of spread (variation): lengths of the box

3. shape: position of the median in the box relative to the quartiles

4. number of outliers

With this example, the median for those who had Wording 1 is larger than the median found with Wording 2. One also finds that the length of the box for Wording 1 is also larger than that found with Wording 2. This suggests that there is more spread or variation in the responses for Wording 1. The median is also not positioned in the same place in each box that indicates that the two samples do not have the same shape.  Finally, there are two outliers with Wording 2 while there are none with Wording 1.   Overall, these findings suggest that the wording of the question does affect the responses that are obtained.

3.4 Measures of Spread or Variation

Two ways to represent the spread or variation are:

1. Interquartile Range (IQR)

2. Standard Deviation (SD)

Example 3.6. Measures of Spread or Variation

Recall the five-number summary from Example 3.4.

Table 3.6. Five-Number Summary of Salaries

Lowest	Lower Quartile (QL)	Median	Upper Quartile(QU)	Highest
$30,000	$33,250	$40,000	$49,500	$110,000

With the five-number summary one can easily determine the Interquartile Range (IQR). The IQR = QU - QL. In our example,

IQR = QU - QL = $49,500 - $33,250 = \$16,250

What does this IQR represent? With this example, one can say that the middle 50% of the salaries spans $16,250 (or spans from $33,250 to $49,500).  The IQR is the length of the box on a boxplot. Notice that only a few numbers are needed to determine the IQR and those numbers are not the extreme observations that may be outliers. The IQR is a type of  resistant measure.

The second measure of spread or variation is called the standard deviation (SD). The standard deviation is roughly the average distance that the observations in the sample fall from the mean. The standard deviation is calculated using every observation in the data set. Consequently it is called a sensitive measure because it will be influenced by outliers. The standard deviation for the variable “salaries” is $17,936 (Note: you will not be asked to calculate a SD). What does the standard deviation represent?  With this example, one can say that the average distance of any individual salary from the mean salary of $45,000 is about $17,936.  Figure 3.12 shows how far each individual salary is from the mean. 

Figure 3.12. Dotplot of Salaries

What you notice in Figure 3.12 is that many of the observations are reasonably close to the sample mean. But since there is an outlier of $110,000 in this sample, the standard deviation is inflated such that average distance is about $17,936. In this instance, the IQR is the preferred measure of spread because the sample has an outlier.

Table 3.7 shows the numbers that can be used to summarize measurement data.

Table 3.7. Numbers used to Summarize Measurement Data

Numerical Measure	Sensitive Measure	Resistant Measure
Measure of Center	Mean	Median
Measure of Spread (Variation)	Standard Deviation (SD)	Interquartile Range (IQR)

· If a sample has outliers and/or skewness, resistant measures are preferred over sensitive measures. This is because sensitive measures tend to overreact to the presence of outliers.

· If a sample is reasonably symmetric, sensitive measures should be used. It is always better to use all of the observations in the sample when there are no problems with skewness and/or outliers.

3.5 Predictable Patterns

Section 3.2. Chapter 8

Many measurement variables found in nature follow a predictable pattern. The predictable pattern of interest is a type of symmetry where much of the data is clumped around the center and few observations are found on the extremes. Data that has this pattern are said to be bell-shaped or have a normal distribution.

Example 3.7. Normal Curves

Consider the following three variables from data that was collected from a sample of Stat 100 students:

· Variable #1: Heights (inches)

· Variable #2: Grade Point Average

· Variable #3: Number of Tattoos

Figure 3.13. Histogram of Height (Mean = 66.3 inches & Median = 66 inches)

Variable #1 is a great example of a normal distribution as shown in Figure 3.13. Since a normal distribution is a type of symmetric distribution, you would expect the mean and median to be very close in value. With this example, the mean is 66.3 inches and median is 66 inches.

Figure 3.14. Histogram of GPA (Mean = 3.25 & Median = 3.3)

Variable #2 reasonably follows a normal distribution as shown in Figure 3.14.  The only problem is that found with the upper tail where the data is clumped which is partially explained by the fact that GPAs at Penn State cannot exceed 4.0.  However, since the sample size is large (n = 198 students) and the mean and median are very close, one can assume that this sample is reasonably normal. It also helps that this variable is continuous.

Figure 3.15. Number of Tattoos (Mean = .23 & Median = 0)

Variable #3 is not normally distributed as shown in Figure 3.15.  The major problem with this variable is that it is discrete rather than continuous. Ideally, normal distributions should be based on measurements of variables that are continuous. As you can see, the graph has gaps because this variable is discrete. Even when ignoring this fact, the distribution is skewed because most people do not have any tattoos. The only reason that the mean and median are so close is because of the large sample size.

Empirical Rule

The empirical rule is a guideline that can be applied when you know that the sample is normally distributed. The empirical rule helps one to understand what the standard deviation represents.

The empirical rule says that for any normal (bell-shaped) curve, approximately:

· 68% of the values (data) fall within 1 standard deviation of the mean in either direction

· 95% of the values (data) fall within 2 standard deviations of the mean in either direction

· 99.7% of the values (data) fall within 3 standard deviations of the mean in either direction

Figure 3.16 The Empirical Rule

Example 3.8. Empirical Rule

Recall the variable heights used in Example 3.7. Since the histogram shows that this data is normally distributed, the empirical rule can be applied. The mean and standard deviation (SD) for this sample are 66.3 inches and 4 inches, respectively. Below are the calculations for the sample of heights.

Mean ± 1(SD) = 66.3 ± 4 inches = (62.3 to 70.3 inches) Mean ± 2(SD) = 66.3 ± 2(4) inches = 66.3 ± 8 inches = (58.3 to 74.3 inches) Mean ± 3(SD) = 66.3 ± 3(4) inches = 66.3 ± 12 inches = (54.3 to 78.3 inches)

Because the sample of heights is normally distributed, one can say that approximately

· 68% of the heights lie between 62.3 and 70.3 inches

· 95% of the heights lie between 58.3 and 74.3 inches

· 99.7% of the heights lie between 54.3 and 78.3 inches

One would not expect someone in this sample to be smaller than 54.3 inches or taller than 78.3 inches.

Standardized Scores (Z-Scores) used with Normal Distributions

A standardized score is simply a way to “standardize” data that is normally distributed.  By “standardize”, we mean that we convert the normal data into normal data that has a mean of 0 and a standard deviation of 1.0.   This normal distribution is then called the “Standard Normal” distribution.  Standardizing the data enables you to use the Z-Table (Table 8.1 on page 157 of the text) to determine percentiles for data values found in the data set.  

Example 3.9 Standardized Scores (Z-Scores)

If we know that the gas mileage for compact SUVs follows a normal distribution with a population mean of 28 mpg and a population standard deviation of 2 mpg.   What is the percentile for a compact SUV that gets 30 mpg?   The percentile will give us an idea of how the gas mileage of this compact SUV compares to the gas mileage of all other compact SUVs.  

We first have to compute the Z-Score (standardized score) which is found by using the following formula:

      Z-Score = (observed value - population mean)/population standard deviation

In our problem, the observed value we want to find the percentile for is 30, the population mean is 28, and the population standard deviation is 2. 

     Our Z-Score is:   Z = (30-28)/2 = 2/2 = +1.0

Our Z-Score is +1.0 which indicates that our observed value is 1 standard deviation above the population mean.  Negative Z-Scores indicate that the observed value is below the population mean.

We go to Table 8.1 in the text and look for our Z-Score.  Since the Z-Score is positive, we look at the fourth column from the left.  We go down the rows and see that a Z-Score of +1.0 is at the 84th percentile.  This indicates that 84% of compact SUVs have lower gas mileage than this particular brand of compact SUV.   Conversely, only 16% of compact SUVs get better gas mileage, so our brand of interest is in the top 16% for gas mileage.

Lesson 3 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. The following question was asked of a sample of STAT 100 students: How many times a month do you usually drink at least two beers? Which of the following graphs cannot be used to describe the resulting data?

a. boxplot

b. dotplot

c. stem and leaf

d. pie chart

e. histogram

2. The following histogram displays the number of CDs owned from a sample of STAT 100 students. What shape is displayed on this histogram? 

a. symmetric

b. right skewed

c. left skewed

3. Find the median for the following sample of five numbers: 2 7 6 4 3

a. 2

b. 7

c. 6

d. 4

d. 3

4. Which of the following cannot be determined from a five-number summary of a data set?

a. lowest (minimum value)

b. lower quartile QL

c. mean (or average)

d. interquartile range

5. Which of the following is not a measure of spread or variation?

a. interquartile range

b. median

c. standard deviation

6. Given a population mean for September temperatures in State College, PA of 75 degrees and a population standard deviation of 3 degrees. What percentile is a value of 72 degrees?

a. 84th

b. 16th

c. 99.87th

d. .13th

Lesson 4: How to Get a Good Sample

Assignments

· See your Course Syllabus for the reading assignments.

· Work through the Lesson 4 online notes that follow.

· Complete the Practice Questions and Lesson 4 Assignment.

Learning Objectives

Chapter 4

After successfully completing this lesson, you should be able to:

· Distinguish between a population, sample, and sampling frame.

· Interpret and identify the factors that affect the margin of error.

· Identify types of probability samples and judgment samples.

· Apply the “Difficulties and Disasters” in sampling to real world problems.

· Identify all steps used and issues addressed by the Gallup Poll.

Terms to Know

Chapter 4

· sample surveys

· experiments

· observational studies

· case studies

· unit (sampling unit)

· population

· sample

· sampling frame

· census

· margin of error (ME)

· sample size (n)

· probability sampling

· judgment sampling

· simple random sample

· stratified sampling

· cluster sampling

· systematic sampling

· voluntary sample

· haphazard (convenience) sample

· gallup poll

· nonresponse (no response or voluntary response)

· random-digit dialing

· selection bias

· sample percent

· population percent

Commentary

Section 4.1. Chapter 4 in Textbook

Overview

In this lesson, we will add to our knowledge base by explaining ways to obtain appropriate samples for statistical studies.

4.1 Common Research Strategies

Chapter 4 Section 4.1

The following research strategies are described in this section of the textbook.

1. Sample Surveys

2. Experiments

3. Observational Studies

4. Meta-Analyses (also covered in Chapter 25--not required for the course)

5. Case Studies

Terms Used with Sample Surveys (Chapter 4 Section 4.2 in Textbook)

It is first necessary to distinguish between a census and a sample survey.  A census is a collection of data from every member of the population, while a sample survey is a collection of data from a subset of the population.  A sample survey is a type of observational study. Obviously, it is much easier to conduct a sample survey than a census.  The remaining sections of this lesson (Chapter 4) will discuss issues about sample surveys.

Of the many terms that are used with sample surveys, the following four need the most clarification because of how they are connected to each other.

· Sampling Unit: The individual person or object that has the measurement (observation) taken on them / it

· Population: The entire group of individuals or objects that we wish to estimate some characteristic's (variable's) value   

· Sampling Frame: The list of the sampling units from which those to be contacted for inclusion in the sample is obtained. The sampling frame lies between the population and sample. Ideally the sampling frame should match the population, but rarely does because the population is not usually small enough to list all members of the population.

· Sample: Those individuals or objects who provided the data collected

Figure 4.1 Relationship between Population, Sampling Frame and Sample

Example 4.1. Who are those angry women?

(Streitfield, D., 1988 and Wallis, 1987)

Recalling some of the information from Example 2.1 in Lesson 2, in 1987, Shere Hite published a best-selling book called Women and Love: A Cultural Revolution in Progress. This 7-year research project produced a controversial 922-page publication that summarized the results from a survey that was designed to examine how American women felt about their relationships with men. Hite mailed out 100,000 fifteen-page questionnaires to women who were members of a wide variety of organizations across the U.S.   Questionnaires were actually sent to the leader of each organization. The leader was asked to distribute questionnaires to all members. Each questionnaire contained 127 open-ended questions with many parts and follow-ups. Part of Hite’s directions read as follows: “Feel free to skip around and answer only those questions you choose.” Approximately 4500 questionnaires were returned.

In Lesson 2, we determined that the

· population was all American women.

· sample was the 4,500 women who responded.

It is also easy to identify that the sampling unit was an American woman.  So, the key question is “What is the sampling frame?” Most people think the sampling frame was the 100,000 women who received the questionnaires.  However, this answer is not correct because the sampling frame was the list from which the 100,000 who were sent the survey was obtained.  In this instance, the sampling frame included all American women who had some affiliation with an organization.  There is no statistical term to attach to the 100,000 women who received the questionnaire.  However, if the response rate had been 100%, the sample would have been the 100,000 women who responded to the survey.

You should also remember that ideally the sampling frame should include the entire population. If this is not possible, the sampling frame should appropriately represent the desired population. In this case, the sampling frame of all American women who were “affiliated with some organization” did not appropriately represent the population of all American women.  InLesson 2, we called this problem selection bias.

Chapter 4 of your text also lists three difficulties that are possible when samples are obtained for surveys. These three difficulties, which happen to be possible with this example, include:

1. Using the wrong sampling frame. We just discussed this problem in the  preceding paragraph. This problem is also called selection bias.

2. Not reaching the individuals selected.   Because the questionnaire was sent to leaders of organizations, there is no guarantee that these questionnaires actually reached the women who were supposed to be in the sample.

3. Getting “no response” or a “volunteer response.”   In Lesson 2, we learned that this survey has a problem with nonresponse bias because of the low response rate. This problem can also be called “no response” or “volunteer response.”

4.2 The Beauty of Sampling

Sample surveys are generally used to estimate the percentage of people in the population that have a certain characteristic or opinion.  If you follow the news, you will probably recall that most of these polls are based on samples of size 1000 to 1500 people.  So, why is a sample size of around 1000 people commonly used in surveying?  The answer is based on understanding what is called the margin of error.

The margin of error:

· measures the accuracy of the percent estimated in the survey

· is calculated using a formula that includes the sample size (n)

For a sample size of n = 1000, the margin of error is 1n√=11000√=0.03 , or about 3%.

Even though you will not be asked to calculate a margin of error in this course, you should remember the margin of error formula and that the margin of error formula depends only on the size of the sample. The size of the population is not used in the calculation of the margin of error.  So, a percentage estimated by a selected sample size will have the same margin of error (accuracy), regardless of whether the population size is 5,000 or 5 billion.   It also helps that pollsters believe that an accuracy of ± 3% is reasonable with surveys.

So what does the margin of error represent?  The following statement represents the generic interpretation of a margin of error.

Generic Interpretation:  If one obtains many samples of the same size from a defined population, the difference between the sample percent and the true population percent will be within the margin of error, at least 95% of the time.

Key Features of the Interpretation of the Margin of Error

· Statistical theory is often based on what would happen if the survey were repeated many times.  So, even though a pollster usually obtains only one sample, the pollster must remember that the margin of error interpretation is based on doing the survey repeatedly under identical conditions.

· The margin of error represents the largest distance that would occur between the sample percent, which is the percent obtained by the poll, and the true population percent, which is unknown because we have not sampled the entire population.

· In statistics, when talking about the margin of error, it is just not possible to say that we are 100% certain that with all samples the difference between the sample percent and the population percent will be within the margin of error.  So, statisticians work with reasonable conditions so that one can say that at least 95% of the time, the difference between the sample percent and the population percent will be within the margin of error.

Example 4.2. Margin of Error

Suppose a recent poll based on 1000 Americans finds that 55% approve of the president’s current educational plan.  Since the sample size is 1000, the margin of error is about 3%.  These poll results suggest that 55% ± 3% of all Americans approve of the president’s current economic plan. What is the correct interpretation of this margin of error?

Margin of Error Interpretation

The difference between our sample percent and the true population percent will be within 3%, at least 95% of the time.  This means that we are almost certain that 55% ± 3% or (52% to 58%) of all Americans approve of the president’s current educational plan.   Because the range of possible values from this poll all fall above 50%, we can also say that we are pretty sure that a majority of Americans support the president’s current educational plan.  If any of the range of possible values would have been 50% or less, then we would not have been able to say that the majority supported the plan.  The range of values (52% to 58%) is called a 95% confidence interval.   We will go into further detail about confidence intervals in Lesson 7.

4.3 Relationship between Sample Size and Margin of Error

There is a predictable relationship between sample size and margin of error. The numbers found in Table 4.1 help to explain this relationship.

Table 4.1. Calculated Margins of Error for Selected Sample Sizes

Sample Size (n)	Margin of Error (M.E.)
200	7.1%
400	5.0%
700	3.8%
1000	3.2%
1200	2.9%
1500	2.6%
2000	2.2%
3000	1.8%
4000	1.6%
5000	1.4%

From this table, one can clearly see that as sample size increases, the margin of error decreases. In order to add additional clarity to this finding, the information from Table 4.1 is also displayed in Figure 4.2.

Figure 4.2 Relationship Between Sample Size and Margin of Error

In Figure 4.2, you again find that as the sample size increases, the margin of error decreases.  However, you should also notice that the amount by which the margin of error decreases is substantial between samples sizes of 200 and 1500.  This implies that the accuracy of the estimate is strongly affected by the size of the sample.  In contrast, the margin of error does not substantially decrease at sample sizes above 1500.  Therefore, pollsters have concluded that it is not worth it to spend additional time and money for samples that contain more than 1500 people.

4.4 Simple Random Sampling and Other Sampling Methods

Sampling Methods can be classified into one of two categories:

· Probability Sampling: Sample has a known probability of being selected

· Judgment Sampling: Sample does not have known probability of being selected

Probability Sampling

In probability sampling it is possible to both determine which sampling units belong to which sample and the probability that each sample will be selected. The following sampling methods, which are listed in Chapter 4, are types of probability sampling:

1. Simple Random Sampling (SRS)

2. Stratified Sampling

3. Cluster Sampling

4. Multistage Sampling

5. Random-Digit Dialing

6. Systematic Sampling

Of the five methods listed above, students have the most trouble distinguishing between stratified sampling and cluster sampling.

Stratified Sampling is possible when it makes sense to partition the population into groups based on a factor that may influence the variable that is being measured.   These groups are then called strata.  An individual group is called a stratum.  With stratified sampling one should:

· partition the population into groups (strata)

· obtain a simple random sample from each group (stratum)

· collect data on each sampling unit that was randomly sampled from each group (stratum) 

Stratified sampling works best when a heterogeneous population is split into fairly homogeneous groups.  Under these conditions, stratification generally produces more precise estimates of the population percents than estimates that would be found from a simple random sample. Table 4.2 shows some examples of ways to obtain a stratified sample.

Table 4.2. Examples of Stratified Samples

	Example 1	Example 2	Example 3
Population	All people in U.S.	All PSU intercollegiate athletes	All elementary students in the local school district
Groups (Strata)	4 Time Zones in the U.S. (Eastern,Central, Mountain,Pacific)	26 PSU intercollegiate teams	11 different elementary schools in the local school district
Obtain a Simple Random Sample	500 people from each of the 4 time zones	5 athletes from each of the 26 PSU teams	20 students from each of the 11 elementary schools
Sample	4 × 500 = 2000 selected people	26 × 5 = 130 selected athletes	11 × 20 = 220 selected students

Cluster Sampling is very different from Stratified Sampling. With cluster sampling one should

· divide the population into groups (clusters).

· obtain a simple random sample of so many clusters from all possible clusters.

· obtain data on every sampling unit in each of the randomly selected clusters.

It is important to note that, unlike with the strata in stratified sampling, the clusters should be microcosms, rather than subsections, of the population.   Each cluster should be heterogeneous. Additionally, the statistical analysis used with cluster sampling is not only different, but also more complicated than that used with stratified sampling.

Table 4.3. Examples of Cluster Samples

	Example 1	Example 2	Example 3
Population	All people in U.S.	All PSU intercollegiate athletes	All elementary students in a local school district
Groups (Clusters)	4 Time Zones in the U.S. (Eastern,Central, Mountain,Pacific.)	26 PSU intercollegiate teams	11 different elementary schools in the local school district
Obtain a Simple Random Sample	2 time zones from the 4 possible time zones	8 teams from the 26 possible teams	4 elementary schools from the l1 possible elementary schools
Sample	every person in the 2 selected time zones	every athlete on the 8 selected teams	every student in the 4 selected elementary schools

Each of the three examples that are found in Tables 4.2 and 4.3 were used to illustrate how both stratified and cluster sampling could be accomplished. However, there are obviously times when one sampling method is preferred over the other. The following explanations add some clarification about when to use which method.

· With Example 1: Stratified sampling would be preferred over cluster sampling, particularly if the questions of interest are affected by time zone. Cluster sampling really works best when there are a reasonable number of clusters relative to the entire population. In this case, selecting 2 clusters from 4 possible clusters really does not provide much advantage over simple random sampling.

· With Example 2: Either stratified sampling or cluster sampling could be used.  It would depend on what questions are being asked.  For instance, consider the question “Do you agree or disagree that you receive adequate attention from the team of doctors at Sports Medicine when injured?”  The answer to this question would probably not be team dependent, so cluster sampling would be fine.  In contrast, if the question of interest is “Do you agree or disagree that weather affects your performance during an athletic event?”  The answer to this question would probably be influenced by whether or not the sport is played outside or inside.  Consequently, stratified sampling would be preferred.

· With Example 3: Cluster sampling would probably be better than stratified sampling if each individual elementary school appropriately represents the entire population.  Stratified sampling could be used if the elementary schools had very different locations (i.e., one elementary school is located in a rural setting while another elementary school is located in an urban setting.)  Again, the questions of interest would affect which sampling method should be used.

Judgment Sampling

The following sampling methods that are listed in your text are types of judgment sampling:

1. volunteer samples

2. haphazard (convenience) samples

Since judgment sampling is based on human choice rather than random selection, statistical theory cannot explain what is happening.   In your textbook, the two types of judgment samples listed above are called “sampling disasters.”

Section 4.2. Article: “How Polls are Conducted”

The article is exceptional and provides great insight into how major polls are conducted. When you are finished reading this article you may want to go to the Gallup Poll Web site and see the results from recent Gallup polls. Check your Course Schedule for the address.

It is important to be mindful of the final point that is made in this article. We all need to remember that public opinion on a given topic cannot be appropriately measured with one question that is only asked on one poll.  Such results only provide a snapshot at that moment under certain conditions.  The concept of repeating procedures over different conditions and times leads to more valuable and durable results.

Lesson 4 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. Which of the following is not an example of probability sampling?

a. simple random sampling

b. cluster sampling

c. convenience sampling

d. stratified sampling

2. Which of the following surveys would have the smallest margin of error?

a. a sample size of n = 1,600 from a population of 50 million

b. a sample size of n = 500 from a population of 5 billion

c. a sample size of n = 100 from a population of 10 million

3. Suppose a recent survey finds that 80% of Penn State students prefer that fall semester begins after Labor Day. The results of this survey were based on opinions expressed by 200 Penn State students. Which of the following represents the calculation of the margin of error for this survey?

a. 200

b. 1/200

c. 1/ √200

d. √200

4. Suppose a margin of error for a poll is 4%. What is the correct interpretation of the margin of error for this poll? In about 95% of all samples of this size, the ________________.

a. difference between the sample percent and the population percent will be within 4%.

b. probability that the sample percent does not equal the population percent is 4%.

c. probability that the sample percent does equal the population percent is 4%.

d. difference between the sample percent and the population percent will exceed 4%.

5. In order to survey the opinions of its customers, a restaurant chain obtained a random sample of 30 customers from each restaurant in the chain. Each selected customer was asked to fill out a survey. Which one of the following sampling plans was used in this survey?

a. cluster sampling

b. stratified sampling

Lesson 7: Categorical Variables: Graphs and Relationships

Assignments

· See your Course Syllabus for the reading assignment.

· Work through the Lesson 7 online notes that follow.

· Complete the Practice Questions and Lesson 7 Assignment.

Learning Objectives

Chapters 9, 12, and 13

After successfully completing this lesson, you should be able to:

· Interpret graphs used with categorical data.

· Distinguish between a descriptive result and an inferential result.

· Apply what it means to be statistically significant.

· Distinguish between an actual (observed) count and an expected count.

· Distinguish between and interpret: chi-squared statistic, risk, relative risk, and increased risk.

Terms to Know

From Chapter 9

· categorical data (variables)

· pie chart

· bar graph

· cluster bar graph

From Chapters 12 and 13

· contingency table 

· 2 × 2 (contingency) table

· cell

· sample percent

· population percent

· conditional percent

· margin of error

· 95% confidence interval (C.I.)

· descriptive method

· inferential method

· statistically significant

· association (relationship)

· chi-squared statistic or chi-squared test

· actual (observed) counts

· expected counts

· p-value

· risk

· relative risk

· increased risk      

7.1 Overview Part I:

Section 7.1. Chapter 9 Section 9.2 in Textbook

We have learned that variables (observed characteristics) can be classified as either categorical variables or measurement variables. We have also learned different ways to describe measurement variables. We will now learn how to describe categorical variables.

Figure 7.1 Categorical Data Chart

Graphs: Displaying Categorical Variables

There are two graphs that can be used to describe categorical data. These graphs include:

1. Pie Chart

2. Bar Graph

These graphs are commonly found in the newspaper, so I suspect that you have seen them before. Categorical data must be numerically summarized in a table before it can be displayed on a graph. 

Example 7.1. Graphs (One Sample of One Categorical Variable)

Consider the following question that was asked on a STAT 100 Survey.

Survey Question: How would you describe your hometown?

Rural  Suburban  Small Town Big City

The results from this question are summarized in Table 7.1.

Table 7.1. Numerical Summary of Hometown Description

Hometown	Count	Proportion	Percent
Rural	75	75/555 = .14	.14 × 100% = 14%
Suburb	296	296/555 = .53	.53 × 100% = 53%
Small Town	139	139/555 = .25	.25 × 100% = 25%
Big City	45	45/555 = .08	.08 × 100% = 8%
Total	n = 555	555/555 = 1.0	1.0 x 100% = 100%

The percents from Table 7.1 are used to make the pie chart found in Figure 7.2.

Figure 7.2. Pie Chart of Hometown Description

As you can see in Figure 7.2, the majority of Penn State students who were enrolled in STAT 100 during this semester were from the suburbs. This data can also be displayed on a bar graph, as shown in Figure 7.3.

Figure 7.3. Bar Graph of Hometown Description

In this case, both graphs do an equally good job of displaying the data. In general, bar graphs are preferred over pie charts when the question (variable) has more than five categories (choices). Otherwise, it really does not matter which graph is used.

7.2 Example 7.2. Graphs (More than One Sample of One Categorical Variable)

Consider the following survey question that was asked of four different samples of Penn State students: 100  freshman (Fr), 100 sophomores (So), 100 juniors (Jr), and 100 seniors (Sr).

Question: Do you currently own at least one credit card?

Yes  No

The results to this question are found in Table 7.2.

Table 7.2. Responses to Credit Card Ownership by Year in School

Credit Card Response	Fr	So	Jr	Se
Yes	42	55	76	81
No	58	45	24	19
Total	100	100	100	100

Since there is more than one sample of categorical data, the bar graph is the only possibility. In this instance, the bar graph will be called a cluster bar graph, because there will be a cluster of bars for each sample. A cluster bar graph works best when the counts are converted to percents. Percents will allow us to compare the results from the four samples. Table 7.3 shows the conversion of counts to percents for this sample. Each of these percents are called conditional percents because each calculation is restricted to or contingent on the year in school.  In this case, it was not really necessary to convert the counts into percents because the sample size is the same for each sample. However, since this doesn’t always happen, the conversion to percents must be done so that a meaningful comparison can be made. Figure 7.4 is an example of a cluster bar graph that displays the conditional percents for the data found in Table 7.3.

Table 7.3. Conditional Percents for Data in Table 7.2

Credit Card Response	Fr	So	Jr	Se
Yes	42/100 = .42(42%)	55 (55%)	76 (76%)	81 (81%)
No	58/100 = .58(58%)	45 (45%)	24 (24%)	19 (19%)
Total	100/100 = 1.00(100%)	100 (100%)	100 (100%)	100 (100%)

Figure 7.4. Credit Card Ownership by Year in School

The graph in Figure 7.4 does suggest that there is a difference in the percent of Penn State students who own at least one credit card when considering year in school.  Specifically, as a Penn State student progresses from freshman to senior year, it is more likely that he or she will own at least one credit card.

You should also notice that there is redundant information on the graph because the question allows for only a "yes" or "no" response. As the percent who say "yes" increases from freshman to senior year, the percent who say "no" also decreases from freshman to senior year. This holds true because the data is summarized as percents within each school year.

7.3 Overview Part II:

Chapters 12 and 13 expand on the statistical methods that are possible with categorical variables. In order to fully appreciate what methods are now possible, we must first look at the overall picture of statistical methods.

Figure 7.5. Breakdown of Statistical Methods

Statistics is a collection of methods (procedures) for extracting information from data.  Overall, these procedures are classified as either descriptive methods or inferential methods.

· Descriptive methods are procedures that are used to describe a sample.  These procedures can either be graphs or numerical summaries. We have seen examples of descriptive methods. The choice of the graph and numerical summary varies based on the type and number of variables that are being described.

· Inferential methods are procedures that are used to make conclusions about a population. So far, we have only seen one example of an inferential method.  In Lesson 4, we learned about an interval that be can be used to estimate the population percent.  This (confidence) interval is a type of inferential method. In this lesson, we will learn more about inferential methods.

7.4 Example 7.3. Two Different Categorical Variables

Suppose a researcher conducted a study to determine if there is a gender effect when comparing individuals who frequently order a vegetarian meal when eating out. Table 7.4 numerically summarizes the results for 1180 people who were surveyed about this topic.

Table 7.4. Numerical Summary of Results from Survey

	Do you frequently order a vegetarian meal when eating out?
Gender	Yes	No	Total
Female	195	240	435
Male	260	485	745
Total	455	725	1180

1. What is your gender? Female Male

2. Do you frequently order a vegetarian meal when eating out? Yes No

In statistics, Table 7.4 is also called a contingency table because it summarizes the data for two variables. Specifically, this table is called a 2 × 2 contingency table or just a 2 × 2 tablebecause both variables (questions) have two choices.  Therefore, we have 2 rows and 2 columns in the table.  Chapter 13 explores issues about 2 × 2 tables.

Below are possible questions that could be answered using the contingency table found in Table 7.4.

1. What percent of the sample is female?

Answer: 435/1180 = .37 or 37%

2. Among males, what percent said "yes" to the question about frequently ordering a vegetarian meal when eating out? 

Answer: 260/745 = .35 or 35% (Note: An example of a conditional percent.)

Table 7.5. Conditional Percents by Gender on Data from Table 7.4

	Do you frequently order a vegetarian meal when eating out?
Gender	Yes	No	Total
Female	195/435 = .45(45%)	240/435 = .55(55%)	435/435 = 1.0 (100%)
Male	260/745 = .35(35%)	485/745 = .65(65%)	745/745 = 1.0 (100%)
Total	455	725	1180

The conditional percents are more valuable because they allow us to compare results of the two genders ignoring that fact that the two sample sizes are different. In this instance, the results suggest that females are more likely than males to frequently order a vegetarian meal when eating out, because the percent of females that said "yes" is 45% while the percent of males that said "yes" is 35%.

These conditional percents can also be displayed on a cluster bar graph as shown in Figure 7.6.

Figure 7.6. Likelihood of Frequently Ordering a Vegetarian Meal when Eating Out by Gender

However, even though this is an important finding, a comparison of these two sample percents is only a descriptive result and not an inferential conclusion about the two underlying populations. In order to make inferential conclusions about the two genders, we need to first calculate two 95% intervals, also known as 95% confidence intervals.

7.5 95% Confidence Intervals

Earlier, we learned how to calculate a 95% confidence interval to estimate the population percent:

95% Confidence Interval Formula: Sample Percent ± (Margin of Error)

Table 7.6 shows the calculation of two 95% confidence intervals that estimate the population percent who said "yes" about frequently ordering a vegetarian meal when eating out.

Table 7.6. The 95% Confidence Intervals to Estimate Population Percent who said "Yes"

Gender	Sample Percent that said "yes"	Sample Size (n)	Margin of Error (M.E.)  M.E. = 1/√n	95% Confidence Interval To Estimate Population Percent That Said "Yes"
Female	45%	435	M.E. = 5%	45% ± 5% = (40 to 50)%
Male	35%	745	M.E. = 4%	35% ± 4% = (31 to 39)%

As you examine the two calculated confidence intervals (C.I.s) found in Table 7.6, you should notice that these two confidence intervals have no common values.  Figure 7.7 shows that the two calculated confidence intervals do not overlap.  Because these two confidence intervals do not overlap or have any common values, we can conclude, at 95% confidence, that there is a difference in the two genders with regard to percent whom say "yes" about frequently ordering vegetarian meals when eating out.  

Figure 7.7.  95% Confidence Intervals from Table 7.6

Decision Rule used with two 95% Confidence Intervals to Make Conclusions

· If the two confidence intervals do not overlap, we can conclude that there is a difference in the two population percents at 95% confidence.

· If the two confidence intervals do overlap, we cannot conclude that there is a difference in two population percents, at 95% confidence.

7.6 Second Inferential Method (Assessing Statistical Significance)

Hopefully, while listening to the news, you have at least once heard someone report a finding that was "statistically significant." Chapter 13 will allow us to learn what is behind this statement of being "statistically significant" when considering data in 2 × 2 tables.

Example 7.3. (Continued)

Recall the data that was used in Example 7.3 and displayed in Table 7.4.

Table 7.7. Data from Table 7.4

	Do you frequently order a vegetarian meal when eating out?
Gender	Yes	No	Total
Female	195	240	435
Male	260	485	745
Total	455	725	1180

The research question of interest can be worded one of two ways: The two possible wordings include:

Wording 1: Is there a statistically significant difference between the percent who said "yes" when considering gender?

Wording 2: Is there a statistically significant relationship between gender and the likelihood of saying "yes"?

Because this data is summarized as a 2 × 2 table, each wording is equally acceptable.  Two categorical variables that are measured on the same individuals are related (associated) if some choices of one variable tend to occur more often with some choices of the second variable.  Both wordings include the phrase "statistically significant." Below is the proper definition of the term: "statistically significant".

A statistically significant relationship or difference is one that is large enough to be unlikely to have occurred in the sample if there is no relationship or difference in the population.

7.7 The Chi-Squared Statistic

Note: The term Chi-Square and Chi-Squared refer to the same statistic.  Both terms are used in textbooks. 

A "statistically significant" relationship between two categorical values is determined from a quantity called the chi-squared statistic. This chi-squared statistic is a single number that quantifies the amount of disparity between the actual (observed) counts that are found in the 2 × 2 table and the counts that would be expected if there were no relationship in the population. The first step in determining the chi-squared statistic is to calculate the expected count for each cell in the 2 × 2 table.  Below is the proper definition of an expected count.

An expected count is a hypothetical count that would occur if in fact there is no relationship between the two variables

Computer software was used to calculate both the expected counts and chi-squared statistic as shown in Figure 7.8.

Figure 7.8. Chi-Squared Results

Although you will not be expected to calculate an expected count or a chi-squared statistic, a more explicit idea of how expected counts are calculated may help you to understand what the chi-squared statistic is measuring.

In Figure 7.8, note that we have (455 / 1180) people who say "yes" to ordering a vegetarian meal when eating out. If there is no relationship between gender and ordering a vegetarian meal, then we would expect the same proportion of the 435 females to order vegetarian as in the overall sample. Therefore, we expect ((455 / 1180) × (435)) = 168 females to order a vegetarian meal if there is no relationship between the two categorical variables. We continue in this manner until we have calculated all of the expected counts. The computer results found in Figure 7.8show an expected count that is lower than the actual (observed) count in two of the four cells of the table.  The expected count is lower than the observed count for females who say "yes" and for males who say "no".   Therefore, more females than expected are ordering a vegetarian meal when eating out and fewer males than expected are ordering a vegetarian meal when eating out.

The following are interpretations of the numbers found in the first cell of the 2 × 2 table.

Interpretations

· The survey found that 195 females actually said "yes." (Note: this is an interpretation of an actual (observed) count).

· One would expect 168 females to say "yes" if there is no relationship between gender and likelihood of frequently ordering a vegetarian meal when eating out. (Note this is an interpretation of an expected count.)

As you compare the observed count of 195 with the expected count of 168, you do notice that there is a difference of 27 between the two counts. Because of this finding, we do have some support for a relationship between the two variables because the observed count is not consistent with the expected count that assumes the two variables are not related. The results from the other three cells show similar disparities between the observed and expected counts. The difference between the observed and expected counts must be large enough to suggest a relationship.  A difference of only a few counts would not be sufficient because such a small difference could result by chance alone.

The chi-squared statistic is a single number that quantifies the amount of disparity between the actual (observed) counts and the expected counts for all the cells of the table combined. With the chi-squared statistic the following holds true.

For any size contingency table:

· If the chi-squared statistic = 0, there is no relationship between the two variables. This means that for every cell in the table, the actual count will equal the expected count. This is the smallest value that a chi-squared statistic can assume.

For a 2 × 2 contingency table:

· If the chi-squared statistic ≥ 3.84 (a value called the "critical value"), there is support for a statistically significant relationship between the two variables. There is no upper boundary for the chi-squared statistic. For contingency tables larger than 2  rows and 2 columns, the "critical value" is larger than 3.84.

From Figure 7.8, we find that with our example, the chi-squared statistic is 11.43. Since 11.43 > 3.84 there is support for a statistically significant relationship between the two variables.

7.8 The P-value

With the advent of computer software, we now have another way to determine whether or not a relationship between two categorical variables is statistically significant. This is good news because most people, other than statisticians, have no clue why 3.84 is the magic boundary for a statistically significant relationship between the variables in a 2 × 2 contingency table. This is especially helpful for contingency tables that are larger than 2 × 2.   You do not have to determine the "critical value" for the chi-squared statistic for every size table. The p-value of the chi-squared statistic will give you all the information you need to determine statistical significance. The p-value is an inferential method.

Remember that a statistically significant relationship is one that is large enough to be unlikely to have occurred in the sample if there’s no relationship in the population. A p-value is a probability that measures how likely it is to observe the relationship or one even stronger if there’s really no relationship in the population. Two properties about a p-value are:

· possible values for the p-value are 0 to 1.0 because it is a probability

· calculation is based on the value of the chi-squared statistic

If you look at the computer output from Figure 7.8. you find that the p-value is .001. Remember this probability is based on the fact that our chi-squared statistic is 11.43.

· Interpretation of our p-value: The likelihood of getting our chi-squared statistic of 11.43 or any value more extreme, if in fact there is no relationship in the population, is .001.

Since it is highly unlikely (.001 = .1%) that we would get our chi-squared statistic of 11.43 or any chi-square statistic larger than 11.43, if there is really no relationship in the population, we can conclude that our results are inconsistent with the position that there is no relationship in the population.  So we can conclude that there is a statistically significant relationship in the population.

Since the p-value is confusing to some, we will revisit what the chi-squared statistic is measuring. The chi-squared statistic is a measure of the magnitude of the difference between what we observe (observed counts) and what we would expect to observe if there is no relationship between the variables (expected counts).  Statisticians have calculated the probability of having a chi-squared statistic of a certain value or larger value.   Statisticians call this the right-tail probability since values get larger as you move to the right on a number line.    The right-tail probability is what the p-value indicates when used with the chi-squared statistic.  Small p-values are associated with large chi-squared values and large chi-squared values mean that the difference between the observed and expected counts is too large to be by chance alone and that there must be a relationship between the two variables.

To further illustrate how unlikely a chi-squared value that is 11.43 or larger is, look at the histogram in Figure 7.9 below. The histogram shows the frequency (out of 10,000) of chi-squared values (for a 2 × 2 table) when there is no relationship in the population. Most of the chi-squared values are 0 and the histogram is right-skewed. Values above 11.43 are almost non-existent which is demonstrated by our p-value of  .001.

Figure 7.9 Histogram of the Chi-Squared Statistic for a 2 × 2 Table 

The histogram will change shape for tables larger than 2 rows and 2 columns, but the shape will still be right-skewed. So, larger chi-square values will always be highly unlikely and have lowp-values.   A histogram for the chi-squared statistic for a 2 × 8 table is shown in Figure 7.10 below.

Figure 7.10 Histogram of the Chi-Squared Statistic for a 2 × 8 Table 

Decision Rule used with P-Value to Make Conclusions

· If the p-value ≤ .05, we can conclude that there is a statistically significant relationship between the variables

· If the p-value > .05 we cannot conclude that there is a statistically significant relationship between the variables

Figure 7.11. Right-tail p-value probability for Chi-squared distribution 

Figure 7.11 shows the relationship between the chi-squared value (χ2) and the right-tail p-value probability.

Table 7.8 shows that we obtained the same conclusion with both inferential methods.

Table 7.8. Overall Inferential Conclusions with Example 7.3

Inferential Procedure	Inferential Result	Inferential Conclusion
95% Confidence Intervals (C.I.)	C.I.’s for Population Percent who said "yes" 95% C.I. for females: (40 to 50)% 95% C.I. for males: (31 to 39)%	Since the two C.I.’s do not overlap, we can conclude that there is a statistically significant difference in the two genders with regard to the percent who said "yes," at 95% confidence
Significance Test (P-value)	P-value = .001 from chi-squared statistic	Since the P-value ≤ .05 we can conclude that there is a statistically significant relationship between gender and likelihood of saying "yes"

Below is an overall summary of what has been so far discussed in the three examples.

 

Figure 7.12. Categorical Variables Breakdown

Sometimes one data set can be viewed either as (one sample of more than one categorical variable) or (two different categorical variables.) This is certainly possible with both Example 7.2and Example 7.3. To illustrate the point, let’s examine Example 7.3.

With Example 7.3, the first inferential procedure involved comparing two 95% Confidence Intervals. This seems to work best if you assume that there are really two samples (females and males) of the one categorical variable of interest: "whether they frequently order a vegetarian meal".   In contrast, the second inferential procedure involved using the p-value from the chi-squared statistic. In this instance, it makes more sense to assume that there are two different categorical variables because stated conclusions include the word "relationship". Either approach is acceptable. So it’s up to the researcher to decide which wording bests fits the proposed research question.

7.9 Other Numbers That Can Describe 2 × 2 Tables

Sometimes data that is collected in a 2 × 2 table has an outcome that is undesirable.  Because of this, measures other than the chi-squared statistic may be more informative.  These measures, which are found in Chapter 12, include the following:

1. Risk

2. Relative Risk

3. Increased Risk

Each of the measures is a number that is used to evaluate chance. Below are the formulas that are used to obtain these measures. (Note: you will not be asked to calculate any of these measures.)

Risk = (number with trait/total) Relative Risk = Risk1/Risk2 (Note: Always put smaller risk on the bottom) Increased Risk = (Relative Risk - 1.0) × 100%

Example 7.4. Risk, Relative Risk, and Increased Risk

A recent study examined the incidence of injuries for male and female high school athletes. The response variable was whether the athlete has experienced an injury during the school year or not.  Suppose the data were as found in Table 7.9.

Table 7.9. Experiencing Injury by Gender

	Experienced Injury?
Gender	Yes	No	Total
Female	150	350	500
Male	100	400	500
Total	250	750	1000

In this example, the undesirable trait (outcome) is experiencing injury. So the calculated risk of injury for each gender is:

For Females: Risk = (number with trait)/total = 150/500 = .3 (30%) For Males: Risk = (number with trait)/total = 100/500 = .2 (20%)

Risk is just another name for a probability or proportion. Risks can also be converted to percents. A risk is a type of conditional percent. In this example, we find that the risk for injury is higher for females than males. However, sometimes researchers prefer to report the two risks as a single quantity. Two possibilities are the relative risk and the increased risk.

Relative Risk = Riskfemales/Riskmales = .3/.2 = 30%/20% = 1.5

Relative Risk Interpretation: A female athlete is 1.5 times more likely to experience injury than a male athlete during the school year.

Increased Risk = (Relative Risk - 1.0) × 100% = (1.5 – 1.0) × 100% = 50%

Increased Risk Interpretation: The risk for injury during the school year is 50% higher for female athletes than for male athletes.

Relative risks and increased risks are reported in the news all the time. However, these measures are only descriptive and cannot be used to make inferential conclusions.

Example 7.5. Clarification of Risk, Relative Risk, and Increased Risk

Which of the following three choices show the largest magnitude of difference in the two risks when comparing "percent that quit smoking" for users and non-users of the patch? The results are based on sample of 50 smokers who used the patch and a sample of 50 smokers who did not use the patch.

Choice A: 50% of users quit whereas 25% of non-users quit Choice B: 10% of users quit whereas 5% of non-users quit Choice C: 2% of users quit whereas 1% of non-users quit

I suspect that most of you correctly selected Choice A. However, what is interesting is the fact that all three choices have the same relative risk and increased risk as shown in Table 7.10.

Table 7.10. Results for the Three Choices

Choice	Relative Risk	Increased Risk	Chi-Square Statistic & P-value
A	50%/25% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = 17.14 & P-value = .000*
B	10%/5% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = 1.81 & P-value = .179
C	2%/1% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = .338 & P-value = .561

*Statistically Significant

From Table 7.10, the following interpretations can be made.

· Relative Risk Interpretation: A person is 2 times more likely to quit smoking when using the patch.

· Increased Risk Interpretation: There is a 100% increase in the chance of quitting smoking when using the patch.

Seeing an increased risk of 100% does make one believe that there is a substantial difference in the two groups, which, in this case, suggests that using the nicotine patch does make a huge difference when trying to quit smoking. The smallest value that an increased risk can assume is 0%, which indicates that the risk is the same for the two groups.

In contrast, when compared to the increased risk, the relative risk may seem more subtle and less impressive. The smallest value that a relative risk can assume is 1.0,  which indicates that the risk is the same for the two groups. So while a relative risk of 2.0 certainly suggests that using the nicotine patch does make some difference, the finding is not as dramatic as the increased risk.

The critical point is that measures such as relative risk and increased risk are just one numerical result and by themselves cannot tell the entire story. You should also read Section 12.3 in your textbook to see how a missing baseline risk can also lead to misinterpreting the relative risk and the increased risk.

Choice A has the largest magnitude of effect that leads to not only the smallest p-value, but also the only p-value that is statistically significant. So with Choice A one can claim that there is a statistically significant relationship between quitting smoking and whether or not a person uses the patch. This finding supports that Choice A has the largest magnitude of difference in the two risks.

Lesson 7 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. Which of the following is not a graph that can be used to describe categorical data?

A. pie chart

B. histogram

C. bar graph

2. Which of the following is not a descriptive result?

A. p-value

B. increased risk

C. bar graph

D. relative risk

3. The statistical significance of the relationship between two categorical variables is examined using both the chi-square statistic and a p-value. Suppose the p-value of the test turns out to be .008. In this case, we should decide that:

A. There is a statistically significant relationship between the two categorical variables.

B. There is not a statistically significant relationship between the two categorical variables.

4. With a chi-squared statistic, an expected count is a hypothetical count that assumes there is a relationship between the two variables.

A. true

B. false

5. Which choice gives two variables for which a chi-square test can be used to analyze the relationship?

A. gender and GPA

B. opinion about the death penalty and opinion about gun control laws

C. heights of female students and heights of their mothers

Lesson 8: Measurement Variables: Graphs and Relationships

Assignments

· See your Course Syllabus for the reading assignment.

· Work through the Lesson 8 online notes that follow.

· Complete the Practice Questions and Lesson 8 Assignment.

Learning Objectives

Chapters 9, 10, and 11

After successfully completing this lesson you should be able to:

· Interpret a scatterplot, correlation, slope, and y-intercept.

· Explain the major features of a correlation.

· Identify the key features of a regression line.

· Apply what it means to be statistically significant.

· Find the predicted value of y for given choice of x on a regression equation plot.

Terms to Know

From Chapter 9

· measurement data (variables)

· scatterplots

· positive association (relationship)

· negative association (relationship)

· no association

From Chapters 10 and 11

· correlation

· straight line (linear relationship)

· positive correlation

· negative correlation

· statistically significant

· p-value

· causation

· sensitive measure

· outlier

· explanatory (predictor) variable (x)

· response variable (y)

· least squares

· regression equation (line)

· extrapolation

· descriptive methods

· inferential methods

8.1 Overview

Section 8.1.

Chapter 9 Section 9.3 in Textbook

We have previously learned how to display two different categorical variables. In this lesson we will now learn how to graph two different measurement variables. Remember that the overall goal with two different variables of the same type is to determine whether or not there is a relationship (association) between these two variables.

Figure 8.1. Variables Chart

Graphs: Displaying Two Different Measurement Variables

In a previous lesson, we learned about possible graphs to display measurement data.These graphs included: dotplots, stemplots, histograms and boxplots. These graphs, however, are only appropriate for one or more samples of one measurement variable. With two different measurement variables, the appropriate graph is the scatterplot.

Example 8.1. Graph of Two Measurement Variables

The following two questions were asked on a survey of 220 Stat 100 students:

1. What is your height (inches)?

2. What is your weight (lbs)?

Notice we have two different measurement variables.  It would be inappropriate to put these two variables on side-by-side boxplots because they do not have the same units of measurement. Comparing height to weight is like comparing apples to oranges.  However, we do want to put both of these variables on one graph so that we can determine if there is an association (relationship) between these two variables. The scatterplot of this data is found in Figure 8.2.

 

Figure 8.2. Scatterplot of Weight versus Height

In Figure 8.2, we notice that as height increases, weight also tends to increase. In statistics, we say that these two variables have a positive association because as the values of one measurement variable tend to increase, the values of the other measurement variable also increase. You should note that this holds true regardless of which variable is placed on the horizontal axis and which variable is placed on the vertical axis.

Example 8.2. Graph of Two Measurement Variables

The following two questions were asked on a survey of ten PSU students who live off-campus in unfurnished one-bedroom apartments.

1. How far do you live from campus (miles)?

2. How much is your monthly rent ($)?

The scatterplot of this data is found in Figure 8.3.

 

Figure 8.3. Scatterplot of Monthly Rent verses Distance

In Figure 8.3, we notice that the further an unfurnished one-bedroom apartment is away from campus, the less it costs to rent an unfurnished one-bedroom apartment.  In statistics, we say that two variables have a negative association when the values of one measurement variable tend to decrease as the values of the other measurement variable increase.

Example 8.3. Graph of Two Measurement Variables

The following two questions were asked on a survey of 220 Stat 100 students:

1. About how many hours do you typically study each week?

2. About how many hours do you typically exercise each week?

The scatterplot of this data is found in Figure 8.4.

Figure 8.4. Scatterplot of Study Hours versus Exercise Hours

In Figure 8.4, we notice that as the number of hours spent exercising each week increases there is really no increase or decrease in number of hours spent studying each week. Consequently, we say that that there is no association between the two variables.

8.2 Correlation

Section 8.2.   Chapters 10 and 11 in Textbook

Chapters 10 and 11 expand on the statistical methods that are possible with two different measurement variables.  Remember that overall statistical methods are one of two types:descriptive methods and inferential methods.

Correlation

Many relationships between two measurement variables tend to fall close to a straight line. In other words, the two variables exhibit a linear relationship. The graphs in Figure 8.2 andFigure 8.3 show a linear relationship between the two variables.

It is also helpful to have a single number that will measure the strength of the linear relationship between the two variables. This number is the correlation. Precisely, the correlation is a single number that indicates how close the values fall to a straight line.  In other words, the correlation quantifies both the strength and direction of the linear relationship between two measurement variables. Table 8.1 shows the correlations for data used in Examples 8.1-8.3. (Note: you will not be asked to actually calculate a correlation.)

Table 8.1. Correlations for Examples 8.1-8.3

Example	Variables	Correlation ( r )
Example 8.1	Height and Weight	r = .541
Example 8.2	Distance and Monthly Rent	r = -.903
Example 8.3	Study Hours and ExerciseHours	r = .109

Below are some features about the correlation.

· The correlation of a sample is represented by the letter r.

· The range of possible values for a correlation is between -1 to +1.

· A positive correlation indicates a positive linear association. The strength of the positive linear association increases as the correlation becomes closer to +1.

· A negative correlation indicates a negative linear association. The strength of the negative linear association increases as the correlation becomes closer to -1.

· A correlation of either +1 or –1 indicates a perfect linear relationship. This is hard to find with real data.

· A correlation of 0 indicates either that:

· there is no linear relationship between the two variables, and/or

· the best straight line through the data is horizontal.

· The correlation is independent of the original units of the two variables.

· The correlation is calculated using every observation in the data set.

· The correlation is a descriptive result.

As you compare the scatterplots of the data from the three examples with their actual correlations, you should notice that findings are consistent for each example.

· In Example 8.1, the scatterplot shows a positive association between weight and height.  However, because of the large number of observations, it is impossible for all of them to fall on one straight line. Consequently, a correlation of .541 is reasonable. It is common for a correlation to decrease as sample size increases.

· In Example 8.2, the scatterplot shows a negative association between monthly rent and distance from campus. Since the data points are very close to a straight line it is not surprising the correlation is -.903.

· In Example 8.3, the scatterplot does not show any strong association between exercise hours/week and study hours/week. This lack of association is supported by a correlation of .109.

Assessing Statistical Significance

Earlier we learned that a statistically significant relationship is one that is large enough to be unlikely to have occurred in the sample if there’s no relationship in the population. Moreover, thep-value is a probability that measures how likely it is to observe the relationship or one even stronger if there’s really no relationship in the population. A p-value can be calculated for a correlation. Table 8.2 displays the p-values for the three correlations found in Table 8.1.

Table 8.2. P-Values for Correlations from Table 8.1

Example	Sample Size	Correlation ( r )	p-value	StatisticallySignificant
Example 8.1	220	r = .541	.000	Yes
Example 8.2	10	r = -.903	.000	Yes
Example 8.3	220	r = .109	.104	No

Basically everything that we learned about the p-value in Lesson 7 still holds true. Using the results from Example 8.1:

· P-value Interpretation: The likelihood of getting our correlation of .541 or any correlation more extreme, if in fact there is no relationship in the population, is.000.

· Conclusion with p-value: Since the p-value ≤ .05, we can conclude that there is a statistically significant (linear) relationship between height and weight.

You should definitely spend some time reading Section 10.2 in your textbook. This section provides more background as to why the p-value is compared against .05.  Two additional points that need to be addressed about statistically significant relationships include:

· A relationship that is considered to be statistically significant has such convincing evidence that it is highly unlikely the results could have occurred by chance.

· Sample size does have an impact on whether or not a relationship is declared to be statistically significant. Even though we will explore this in more detail later, we have a glimpse of what may happen when comparing the results from Example 8.1 and Example 8.2. If you look at the results in Table 8.2, you find that both Example 8.1 and Example 8.2 are statistically significant with an identical p-value of .000, yet have different values for the correlation coefficient. It is not surprising that with Example 8.2 a correlation of -.903 leads to a p-value of 0. Yet, with Example 8.1, when the correlation is only .541 the p-value is also 0. In this instance, the large sample size allows a moderate correlation of .541 to be statistically significant.

8.3 Other Issues with Correlations

There are two other issues that we need to address in regard to correlation.

1. It is very hard to confirm causation with correlation.

2. Outliers can substantially inflate or deflate the correlation.

Correlation and Causation

It is often tempting to suggest that, when the correlation is statistically significant, the change in one variable causes the change in the other variable.  However, there are two issues that need to be addressed before making that claim.

1. A strong relationship between two variables may actually be the result of another variable that is lurking in the background.

2. It is always harder to claim causation when the data has come from an observational study. The best support for causation comes from randomized experiments.

Read Sections 11.2 and 11.4 in the textbook for further clarification about factors that affect whether or not a claim of causation can be made.

Example 8.4. Effect of Outliers on Correlation

Figure 8.5 is a scatterplot of two measurement variables: x and y.

Figure 8.5. Scatterplot of Two Measurement Variables

 Figure 8.6 and Figure 8.7 identify two observations that may be possible outliers for the data shown in Figure 8.5.

Figure 8.6. Scatterplot of Two Measurement Variables: Possible Outlier

Figure 8.7. Scatterplot of Two Measurement Variables: Possible Outlier

Table 8.3. Correlations for Data in Figures 8.5-8.7

Data in Figure	Correlation ( r )
Figure 8.5 (original data)	r = .370
Figure 8.6 (one possible outlier removed)	r = .089
Figure 8.7 (one possible outlier removed)	r = .749

In Table 8.3 we observe that the correlation can either increase or decrease depending on which point is removed from the data set.

· In Figure 8.6, when the “possible outlier” is removed, the remaining points show that the best line is now more horizontal. Consequently the correlation drops to .089. (In order to fully understand this change, put your hand over the“possible outlier” in Figure 8.6 and observe the change in the relationship.)

· In Figure 8.7, when the “possible outlier” is removed, the remaining points are closer to a line that is less horizontal. Consequently, the correlation increases to.749.

In general the correlation will either increase or decrease, based on where the outlier is relative to the other points remaining in the data set.

The reason that the correlation is affected by outliers is because the correlation is another example of a sensitive measure. Earlier we learned that a sensitive measure is one that when calculated, uses all observations. Consequently, correlation will react to unusual data points in the sample.

Despite saying all of this,  even though outliers may exist you should not just quickly remove these observations from the data set in order to change the value of the correlation. These data points may be telling you something very valuable about the relationship between the two variables. Moreover, statisticians have other measures that can be used to evaluate the strength of association between two measurement variables when outliers are found in the data set.

8.4 Regression

Regression is a descriptive method used with two different measurement variables to find the best straight line (equation) to fit the data points on the scatterplot. A key feature of the regression equation is that it can be used to make predictions. In order to do regression the two variables need to be designated as either the:

Explanatory or Predictor Variable = x (on horizontal axis)

Response or Outcome Variable = y (vertical axis)

The explanatory variable can be used to predict (estimate) a typical value for the response variable. (Note: It is not necessary to indicate which variable is the explanatory variable and which variable is the response with correlation.)

Equation of a Line

Hopefully in a previous math course, you learned the basics about the equation of a line.

Equation of a Line

y = a + bx    where:

a = y-intercept (the value of y when x = 0)

b = slope of the line (slope estimates the change (increase or decrease) in the variable (y) as the other variable (x) increases by one unit)  

 

Example 8.5 Example of Regression Equation

Consider the following two variables for a sample of ten STAT 100 students.  

x = quiz score y = exam score

Figure 8.8 displays the scatterplot of this data while Figure 8.9 displays the correlation and p-value for these two variables.

Figure 8.8. Scatterplot of Data

Figure 8.9. Correlation and P-Value for the Two Variables Found in Figure 8.8

We would like to take this example farther and find an equation of a line that will allow us to use the quiz score to predict the exam score. While we could just take a ruler and do a fairly good job at finding the best line to fit the data, for statistical purposes, we need a little more accuracy. The common method used to determine the best line to fit the data is called least squares.Least squares essentially finds the line that will be the closest to all the data points than any other possible line. Figure 8.10 displays the least squares regression for the data in Example 8.5.

Figure 8.10. Least Squares Regression Equation

As you look at the plot of the regression line in Figure 8.10, you find that some of the points lie above the line while other points lie below the line.  However, the overall goal was to find the regression equation that is the closest to all the data points than any other possible line.

The least squares regression equation used to plot the equation in Figure 8.10 is:  

y = 1.15 + 1.05  x      or      Exam = 1.15 + 1.05 Quiz

Interpretation of Y-Intercept

Y-Intercept = 1.15 points

Y-Intercept Interpretation:  If a student has a quiz score of 0 points, one would expect that he or she would score 1.15 points on the exam.

However, this y-intercept does not offer any logical interpretation in the context of this problem, because x = 0 is not in the sample. If you look at the graph, you will find the lowest quiz score is 56 points. So, while the y-intercept is a necessary part of the regression equation, by itself it provides no meaningful information about student performance on an exam when the quiz score is 0.

Interpretation of Slope

Slope = 1.05 = 1.05/1 = (change in exam score)/(1 unit change in quiz score)

Slope Interpretation:   For every increase in quiz score by 1 point, you can expect that a student will score 1.05 additional points on the exam.

In this example the slope is a positive number, which is not surprising because the correlation is also positive. A positive correlation always leads to a positive slope and a negative correlation always leads to a negative slope.

Remember that we can also use this equation for prediction.  So consider the following question:

Question: If a student has a quiz score of 85 points, what score would we expect the student to make on the exam?  We can use the regression equation to predict the exam score for the student.

Exam = 1.15 + 1.05 Quiz Exam = 1.15 + 1.05 (85) = 1.15 + 89.25 = 90.4 points

Figure 8.11 verifies that when a quiz score is 85 points, the predicted exam score is about 90 points.

Figure 8.11. Prediction of Exam Score at a Quiz Score of 85 Points

8.5 Two Warnings about Regression

· Avoid Extrapolation: Do not use the regression equation to predict values of the response variable (y) outside the range of the explanatory variable (x) found with the original data. WithExample 8.4 prediction is restricted to quiz scores that lie between 56 points and 94 points, as shown in Figures 8.8, 8.10, and 8.11.

· Logical Interpretation of the y-intercept in the context of a problem is restricted to when you have data where x = 0 is in the sample.  For example:  Suppose that you have data from a particular school district that was used to determine a regression equation relating salary (in $) to years of service (ranging from 0 years to 25 years).   The resulting regression equation is: 

Salary=$29,000+$1,500year×(Years of Service)

Even if you had not been told that "years of service (the x variable)" = 0 was in the sample, you would expect that there would be values with "years of service" = 0 since starting salaries would be in the data set.  Therefore, the y-intercept has a logical interpretation for this problem.   However, many samples do not contain x = 0 in the data set and we cannot logically interpret those y-intercepts.

Below is an overall summary of what has been covered in this lesson when linked to the different types of statistical methods.

Figure 8.12. Statistical Methods Chart

Lesson 8 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. Which of the following is not appropriate for studying the relationship (association) between two measurement variables?

a. Scatterplot

b. Bar Graph

c. Correlation

d. Regression

2. Which of the following is the range of possible values that a correlation can assume?

a. 0 to 1

b. -1 to 0

c. -1 to 1

d. 0 and above

3. The regression line for a set of points is given by y = -10 + 6 x. What is the slope of the line?

a. -10

b. 10

c. -6

d. 6

4. Describe the association found in the following graph.

a. positive linear association

b. negative linear association

c. neither positive nor negative linear association

5. Suppose a correlation -.65 is obtained from two measurement variables. Which of the following represent the slope that would be found if a regression equation were calculated?

a. 5.4

b. -2.8

c. 0

Lesson 9: Probability and Coincidences

Assignments

· See your Course Syllabus for the Reading Assignment.

· Work through the Lesson 9 online notes that follow.

· Complete the Practice Questions and Lesson 9 Assignment.

Learning Objectives

Chapters 16 and 18

After successfully completing this lesson, you should be able to:

· Identify and apply the relative frequency interpretation of probability.

· Apply the four basic rules of probability.

· Set up a calculation for and interpret an expected value.

· Explain and apply both the gambler's fallacy and the confusion of the inverse.

· Distinguish between mutual exclusiveness and independence.

Terms to Know

From Chapter 16

· probability or proportion

· relative frequency interpretation

· mutually exclusive events

· independent events

· subset

· expected value

From Chapter 18

· the gambler's fallacy

· confusion of the inverse

· base rate

· sensitivity

· specificity

· false positive

· false negative

Commentary

Section 9.1. Chapter 16 in Textbook

Overview:

When you hear the word probability, you may also think of words such as likelihood and chance. While this is certainly okay, in the discipline of statistics, the word probability has a more precise interpretation. In fact, there are two different ways to interpret probability. These include:

1. Relative Frequency Interpretation

2. Personal Probability Interpretation

In this course, we will only need to learn about and apply the relative frequency interpretation.

9.1 Relative Frequency Interpretation

The relative frequency interpretation is applied when it is possible to consider that a situation can be repeated many times. When doing this, the results are unpredictable in the short-run but have a regular and a predictable pattern in the long-run.

Example 9.1. Relative Frequency Interpretation

When you toss a coin there are only two possible outcomes, heads or tails. Figure 9.1 displays the results of tossing a coin 500 times and determining the proportion of heads that were observed.

Figure 9.1. Coin Toss

As you examine Figure 9.1 you should notice that since the first toss was a tail, the proportion of heads was 0.   As additional tosses were done, the proportion of heads changed but eventually settled down to a probability of .5. When you randomly toss a coin, the result from the individual toss cannot be predicted in advance. However, after many repetitions, a predictable pattern will emerge. In this case, the probability of a head settles down to a value of .5.

In statistics, randomness is not synonymous with "haphazard", but rather a kind of order that emerges only when a procedure is done repeatedly.  In life, we do encounter randomness on a daily basis.  However, since we rarely see enough repetitions of the same random process, we are unable to observe the long-term predictable pattern that will eventually emerge.

Determining the Relative Frequency Probability of an Outcome

There are two methods that can be used to determine a relative frequency probability.  These include:

Method 1: Make an assumption about the physical world

Method 2: Make a direct observation of how often something happens

Example 9.2. Determining the Relative Frequency Probability of an Outcome

Ask a person to pick a number from 1 to 10.

Question: What is the probability that a person will pick the number 7?

· If we use Method 1, we may suggest that the probability is 1/10 or .1 because people will just randomly pick numbers from 1 to 10 so each number has the same likelihood of happening.  This is an example of making an assumption about how the world will behave.

· If we use Method 2, we may find something different. Suppose I asked this question in one of my classes and found that out of 204 students, 61 picked the number 7.  The probability of picking the number 7 is 61/204 = .30. This is an example of making a direct observation of how often something happens when repeated many times.

As you can see, both methods did not come up with the same result. Even though both methods have merit,  Method 2 should always work as long as you use a large sample size.

The summary of the relative frequency interpretation of probability that is found in Section 16.2 of your text lists four important components. Please familiarize yourself with these four components.

9.2 Applying Some Basic Probability Rules

There are four basic rules of probability that can help us answer many questions. Even though these rules are rather elementary, it is still easy to make mistakes when applying these rules to everyday problems.

Probability Rule 1: If there are only two possible outcomes in an uncertain situation, then their probabilities must add to 1.0.

Example 9.3. Probability Rule 1

Consider a student that attends Penn State. Suppose the probability that the student is from Pennsylvania is .85 and the probability that the student is not from Pennsylvania is .15.

Question: What is the probability that a Penn State student is either from Pennsylvania or not from Pennsylvania?

Answer: .85 + .15 = 1.0. The answer must be 1.0 because there are only two outcomes when classifying a Penn State student as being either from or not from Pennsylvania.

 

Probability Rule 2: If the two events cannot happen simultaneously, they are said to be mutually exclusive.   With mutually exclusive events, there is no possibility of an element of one event being an element of the other event at the same time (cannot be a member of both events at the same time).  The probability of one or the other of the two mutually exclusive events happening is the sum of their individual probabilities.

Example 9.4. Probability Rule 2

Consider the following two events for the current semester at Penn State.

Event A: A Penn State student is enrolled in a Statistics 100 class that is held MWF at 12:20

Event B: A Penn State student is enrolled in an Accounting 214 class that is held MWF at 12:20

Event A and Event B are mutually exclusive because a student cannot be enrolled in two different classes that are held at the same time. Suppose:

Probability of Event A = .05  Probability of Event B = .09

Question: What is the probability that a Penn State student will be enrolled in either a Stat100 class or an Accounting 214 class that is held MWF at 12:20 this semester?

Answer: .05 + .09 = .14. The individual probabilities can be added together because the two events are mutually exclusive. However, this probability does not equal 1.0 because there are many other courses in addition to Statistics 100 and Accounting 214 that are offered on MWF at 12:20 PM.

Example 9.5. Probability Rule 2

Consider the following four events.

Event A: STAT 100 student smokes cigarettes Event B: STAT 100 student smokes marijuana Event C: STAT 100 student only smokes cigarettes Event D: STAT 100 student only smokes marijuana

Table 9.1 is a summary of a survey where STAT 100 students were asked the following two questions

1. Have you ever smoked cigarettes

2. How you ever smoked marijuana?

Table 9.1. Results from STAT 100 Survey

	Marijuana - No	Marijuana – Yes	Total
Cigarettes – No	140	50	190
Cigarettes – Yes	10	20	30
Total	150	70	220

Table 9.1 can be used to obtain the following probabilities

Probability of Event A = (10 + 20)/220 = 30/220 = .14 Probability of Event B = (50 + 20)/220 = 70/220 = .32 Probability of Event C = 10/220 = .045 Probability of Event D = 50/220 = .23

Question: What is the probability that a STAT 100 student has either smoked cigarettes or marijuana?  In other words, what is the probability of (Event A or Event B)?

Answer: In this case, Event A and Event B are not mutually exclusive because 20 students indicated that they have smoked both cigarettes and marijuana.   If we just added the two probabilities for Event A and Event B, we would actually count the 20 students twice.  In this course, we will not dwell on how to calculate this probability, but rather just appreciate that because the two events are not mutually exclusive, we cannot just add the probabilities for the two individual events.

 

Question: What is the probability that a STAT 100 student has either (only smoked cigarettes) or (only smoked marijuana)? In other words, what is the probability of (Event C or Event D)?

Answer: In this case, Event C and Event D are mutually exclusive, so the two probabilities can just be added. Probability = (.045 + .23) = .275.

 

Probability Rule 3: If two events do not influence each other, and if  knowledge about one event doesn't help with knowledge of the probability of the other event, the events are said to be independent of each other.  When events are independent, the occurrence of each event does not change the probability of the other event.   If two events areindependent of each other, the probability that they both happen is found by multiplying their individual probabilities.

Example 9.6. Probability Rule 3

How Did You Celebrate Your 21st Birthday? (Penn State Pulse, January 2001)

A Penn State Pulse Survey found that 82% of Penn State students who recently turned 21 had a celebration that involved alcohol. A random sample of two Penn State students who recently turned 21 was obtained from the Penn State population.

Event A: Penn State student 1 had a celebration that involved alcohol Event B: Penn State student 2 had a celebration that involved alcohol

 

Question: What is the probability that both of these students had a celebration that involved alcohol?

Since these two students were picked randomly, Event A and Event B should be independent. In other words, what happened at the first student's celebration should have no influence on what happened at the second student's celebration. Since the two events are independent, each event has the same probability, which in this case is based on the probability that was obtained from the Penn State Pulse Survey.

Probability of Event A = .82  Probability of Event B = .82

Answer:  .82 × .82 = .67, because the two events are independent.

 

Question: What is the probability that the first student had a celebration that involved alcohol and the second student had a celebration that did not involve alcohol?

Event A: Penn State student 1 had a celebration that involved alcohol  Event B: Penn State student 2 had a celebration that did not involve alcohol

Probability of Event A = .82 Probability of Event B = .18 (because 1 - .82 = .18)

Answer:  .82 × .18 = .15, because the two events are independent

 

Example 9.7. Probability Rule 3

Suppose 60% of State College residents voted for the democratic candidate in the last mayoral election.

Consider the following two events for a married couple that resides in State College and voted in the last mayoral election.

Event A: The husband voted for the democratic candidate  Event B: The wife voted for the democratic candidate

Question: What is the probability that both the husband and wife voted for the democratic candidate in the last mayoral election in State College?

Since the individuals are a married couple and not two randomly sampled individuals, the two events are not independent. Because of this, the answer to the question is not .60 × .60 = .36. In this course, we will not stress how to calculate probabilities for two events are that are not independent.  Instead we will just appreciate that these two events are not independent.

It is easy for a novice to confuse mutual exclusiveness and independence. Hopefully the following two points will help you distinguish between them.

· Two events are mutually exclusive if they cannot happen simultaneously. This implies that we are considering what can happen when one observation is made. 

· Two events are independent if the occurrence of one event does not affect the probability of the occurrence of the second event. Independence is generally associated with a random process that is based on repeated observations.

Probability Rule 4: If the ways in which Event A can occur are a subset of those for Event B, then the probability of the Event A cannot be higher than the probability of Event B.

Example 9.8. Probability Rule 4

Consider the following two events.

Event A: Penn State student is a female sociology major Event B: Penn State student is a sociology major

Question: Which of the two events would have the highest probability?

Answer: Event A is a subset of Event B because Event A only includes female Penn State sociology majors while Event B includes all the sociology majors at Penn State.   Consequently, Event B must have a probability larger than or equal to the probability of Event A.

Example 9.9. Probability Rule 4

Consider the following four events.

Event A: STAT 100 student smokes cigarettes Event B: STAT 100 student smokes marijuana Event C: STAT 100 student only smokes cigarettes Event D: STAT 100 student only smokes marijuana

Table 9.2. Data from Table 9.1
	Marijuana - No	Marijuana – Yes	Total
Cigarettes – No	140	50	190
Cigarettes – Yes	10	20	30
Total	150	70	220

Question: Is  Event C  a subset of  Event A?

Answer: Yes, because Event C contains the 10 STAT 100 students who said "yes" to cigarettes and "no" to marijuana, while Event A contains the (10 + 20) = 30 Stat 100 students who said "yes" to cigarettes and either "yes" or "no" to marijuana.  Event C lies inside Event A .  Stating this another way, we could say that Event C is a subset of Event A since ALL members of Event C are also members of Event A.  Note: Event A and Event C also cannot be mutually exclusive.

Question: Is  Event C  a subset of  Event D?

Answer: No, because Event C contains the 10 STAT 100 students who said "yes" to cigarettes and "no" to marijuana, while Event B contains the 50 STAT 100 students who said "no" to cigarettes and "yes" to marijuana.   Note: Event C and Event D also have nothing in common and therefore are mutually exclusive.

9.3 Expected Value

In statistics, the phrase expected value (EV) represents the average value of any variable in the long run.   Stated another way, the expected value is the mean value that would be obtained from an infinite number of observations. The expected value is found by multiplying each numerical outcome by its probability and then summing over all possible outcomes.

Example 9.10. Expected Value

Table 9.3 shows the distribution of number of classes taken by full-time Penn State students.

Table 9.3. Distribution of the Number of Classes Taken by Full-time PSU Students

Numerical Outcome: Number of Classes Taken	Probability
4	.2
5	.2
6	.4
7	.2
Total	1.0

Question: Calculate the expected number of classes taken by Penn State students.

Answer: Expected Value (EV) = 4 × .2 + 5 × .2 + 6 × .4 + 7 × .2 = 5.6 classes.

 

The answer is not an actual observation but rather lies somewhere between 5 and 6 classes. Remember this calculated expected value is what you would expect on the average from a very large sample of Penn State students.

Example 9.11. Expected Value

An insurance company will pay $400 to replace a stolen 32-inch television that was lost due to theft. The insurance company charges $25/year for this policy. The insurance company has determined that the probability of such television theft is .05.

Question: What is the expected gain per policy for the insurance company in any given year?

In order to solve this question, the information must first be presented in a table that will display the possible outcomes and the corresponding probabilities. Table 9.4 provides this information.

Table 9.4. Distribution of Gain per Policy for Any Given Year

Possible Outcome in Words	Numerical Outcome: Gain per Policy for Insurance Company	Probability
Television is not stolen	$25	1 - .05 = .95
Television is stolen	-$400 + $25 = $-375 (Note: company pays out $400 but still receives the $25 for the policy so the company loses $375 on this policy)	.05
Total		1.0

Answer: Expected Value (EV) = $25 × .95 + (-$375) × .05 = $23.75 - $18.75 = $5.00

What this answer is saying is that on the average, the insurance company expects to gain $5 per policy. As you might expect the number should be positive because the insurance company wants to make a profit. If the insurance company is not satisfied with this expected gain per policy, it will just raise the amount that it charges for each policy per year.

9.4 Coincidences and Probability

Section 9.2. Chapter 18 in Textbook

Overview:

Can coincidences be explained by probability? Suppose you:

· travel to a city across the country on a vacation and run into someone you know

· think about a person you haven't seen for a long time and suddenly you cross paths on campus

Events such as those listed above certainly could be called coincidences. However, statistically, are these coincidences improbable?  While we unfortunately cannot obtain the actual probabilities for the events found above, we will look at other situations that may change some of your thinking about coincidences.

Example 9.12.

Let's consider that there is a 2% chance that a Penn State Student will achieve a 4.0 during any given semester. Many students may be discouraged by such a low percent and not even try to achieve a 4.0. However, these students should also remember that the student population at University Park is about 40,000. Consequently, for any given semester, there should be about 800 students that will achieve a 4.0.   A 4.0 is certainly not a coincidence.   Students are often more motivated when seeing the number 800 because they are reminded that someone will achieve a 4.00 and just maybe they can be one of those 800 students. 

9.5 The Gambler's Fallacy

A sample of four STAT 100 students was asked to toss a coin six times and record each result as either heads (H) or tails (T).  Table 9.3 displays the results from these four students.

Table 9.3. Coin Toss Results

Student	Sequence of 6 H's and T's
1	H T T T H H
2	T T T T T T
3	H T H T H T
4	T T T H H H

Question: Do the results from each the four students all have the same probability of happening?

Answer: Many people think that the sequences from the four students should not all have the same likelihood of happening. In particular, most people think that the sequence from student 2 would have a smaller probability of happening than that found with any of the other three sequences. However, this is not true because of three false beliefs that are typically associated with sequences. Collectively, these three false beliefs are called the Gambler's Fallacy.

· False Belief 1: Random events should be self-correcting. People mistakenly believe that what happens in the long run should also happen in the short-run. In Chapter 16, we already have shown that a predictable pattern does not appear in the short-run.

 

· False Belief 2:  Law of Small Numbers should apply. People mistakenly believe that a small sample should always reflect what is found in the underlying population. In fact this is often not true. In contrast, the Law of Large Numbers correctly states that a large sample will reflect what is found in the underlying population.

 

· False Belief 3:  Independent chance events have a memory. People mistakenly believe that a random process has a memory from trial to trial. In Chapter 16, we learned that random events are independent. Consequently what happens in one trial has no impact on what will happen in future trials. Statisticians call this the "lack of memory" property.

Table 9.4 shows the probability for each of the four sequences found in Table 9.3. As you see each sequence has the same likelihood of happening. The reason being that on any given toss, the probability of obtaining a head is .5 and the probability of obtaining a tail is also .5.  Because each toss is independent of another, the individual probabilities can be multiplied together so that the final probability is .016.

Table 9.4. Probabilities for the Random Sequences found in Table 9.3

Student	Random Sequence of 6H's and T's	Probability
1	H T T T H H	(.5) × (.5) × (.5) × (.5) × (.5) × (.5) = .016
2	T T T T T T	(.5) × (.5) × (.5) × (.5) × (.5) × (.5) = .016
3	H T H T H T	(.5) × (.5) × (.5) × (.5) × (.5) × (.5) = .016
4	T T T H H H	(.5) × (.5) × (.5) × (.5) × (.5) × (.5) = .016

9.6 Confusion of the Inverse

Consider this hypothetical scenario:

Jane goes to her physician for a routine medical test. The physician calls her to tell her that she tested positive for a certain kind of cancer. Thinking it must be a mistake, she inquired about the accuracy of the test. The physician said the test is 90% accurate.

The question that Jane was really trying to ask was: Suppose I have a test that comes back positive, what is the probability that I really do have cancer? What we will try to figure out is whether or not the probability of 90% quoted by the physician is the correct answer for Jane's question.

Example 9.13. Confusion of the Inverse

Suppose a type of cancer has a base rate of 1%. In other words, the probability that this type of cancer will appear in the general population is .01. If the general population size is 100,000, then one would expect about 1000 people to have this type of cancer. This information is the starting point for creating the hypothetical information found in Table 9.5.

Table 9.5. Hypothetical Results for a Type of a Cancer

	Positive Test for Cancer	Negative Test for Cancer	Total
Have Cancer	900	100	1,000
Don't Have Cancer	9,900	89,100	99,000
Total	10,800	89,200	100,000

A lot of physicians confuse the following two probabilities

· Probability 1: Probability of a positive test given that the person has cancer =900/1000 = .90

· Probability 2: Probability that a person has cancer given that the test is positive =900/10,800 = .083

Probability 1 is what the physician quoted to Jane.  However, Probability 2 is really the one that Jane wanted to know.  Probability 1 only represents the accuracy of the test, not the likelihood that Jane has cancer given that her test was positive.  Obviously, Jane would feel much better knowing that the probability that she has cancer given that her test was positive is only .083.

Below are four important quantities that are regularly used by the medical community and can be obtained from Table 9.5. These include:

1. The sensitivity of the test is the proportion of people who correctly test positive when they actually have the disease.

Sensitivity = 900/1000 = .90

2. The specificity of the test is the proportion of people who correctly test negative when they don't have the disease.

Specificity = 89,100/99,000 = .90

3. The probability of a false positive is the proportion of people who test positive when they actually don't have the disease.

False Positive = 9,900/99,000 = .10

4. The probability of a false negative is the proportion of people who test negative when they actually do have the disease.

False Negative = 100/1,000 = .10

Table 9.6 shows the position of these four quantities when using the information from Table 9.5

Table 9.6. Four Important Quantities Based on Information from Table 9.5.

	Positive Test for Cancer	Negative Test for Cancer	Total
Have Cancer	900 (.90) Sensitivity	100 (.10) False Negative	1,000
Don't Have Cancer	9,900 (.10) False Positive	89,100 (.90) Specificity	99,000
Total	10,800	89,200	100,000

Table 9.6 makes two key points. In situations where the base rate for the disease is very low:

· As the sensitivity decreases, the probability of a false negative increases.

· As the specificity decreases, the probability of a false positive increases.

So overall it is desirable to have medical tests where the sensitivity and specificity are both around .98 or .99. Otherwise there will be a fairly high chance of either a false positive and/or a false negative.

Lesson 9 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. Which of the following is a legitimate value for a probability?

a. -.25

b. 1.3

c. .40

d. -2.2

2. Suppose a die is rolled once. Consider the following two events:

Event A: The die shows an even number. Event B: The die shows the number five.

Are Events A and B mutually exclusive?

a. yes

b. no

3. Consider the following two events:

Event A: It will rain tomorrow. Event B: Tomorrow's temperature will be in the single digits (0 to 9).

Are Events A and B independent?

a. yes

b. no

4. Suppose you flip a coin four times. Which of the following sequences are most likely to occur?

a. HHHH

b. HTHT

c. THHT

d. All three sequences are equally likely to occur.

e. The sequences found in b and c are more likely to occur than the sequence found in a.

5. Consider the following probability: the proportion of people who test positive when they actually don't have the disease. What is the correct name for this probability?

a. sensitivity

b. specificity

c. false positive

d. false negative

Lesson 12: Designing Studies and Statistical Inference (Confidence Intervals)

Assignments

· See your Course Syllabus for the reading assignment.

· Work through the Lesson 12 online notes that follow.

· Complete the Practice Questions and Lesson 12 Assignment.

Learning Objectives

Chapters 5, 19, 20, and 21

After successfully completing this lesson, you should be able to:

· Distinguish between randomized experiments and observational studies.

· Distinguish between two independent and two dependent (matched paired) samples.

· Apply basic terms associated with research studies.

· Set up a calculation for and interpret a confidence interval for a population value.

· Apply appropriate decision rules to determine whether or not there is a difference in two population values.

Terms to Know

From Chapter 5

· experimental unit

· explanatory variable

· treatment

· response (outcome) variable

· confounding variable

· observational studies

· randomized experiments

· cause and effect

· independent samples

· dependent samples

· matched pair design

· block design

· carryover effect

· placebo (placebo effect)

· case-control study

· retrospective study

· prospective study

· single blind

· double blind

From Chapter 19 (Section 19.4)

· descriptive methods

· inferential methods

· confidence interval

· hypothesis testing

From Chapter 20

· population value

· population proportion

· sample proportion

· categorical data

· 95% confidence interval

· multiplier (confidence level)

· standard deviation (S.D.)

· margin of error

From Chapter 21

· population mean

· sample mean

· standard error (S.E.)

· population mean difference

· sample mean difference

· measurement data

12.1 Important Terms Used in Research

Section 12.1. Chapter 5 in Textbook

Overview: We've learned some of the very basics about research studies that compare two or more samples of one variable.  Chapter 5 allows us to explore this topic in more detail.  We first need to learn a few terms. These include:

1. experimental unit 

2. explanatory variable

3. treatment

4. response (outcome) variable

5. confounding variable

The experimental unit  is the smallest basic object to which one can assign different conditions (treatments.)  In research studies, the experimental unit does not always have to be a person. In fact, the statistical terminology that is associated with research studies actually came from studies done in agriculture. Examples of an  experimental unit include:

· person

· animal

· plant

· set of twins

· married couple

· plot of land

· building

The explanatory variable is the variable used to form or define the different samples.  In randomized experiments, the explanatory variable is the variable that is used to explain differences in the groups. In this instance, the explanatory variable can also be called a treatment when each experimental unit is randomly assigned a certain condition. Examples of explanatory variables include:

· gender

· type of plant

· type of drug

· type of medical procedure

· teaching method

You should note that gender and type of plant cannot be called treatments because one cannot randomly assign gender or type of plant.

The response (outcome) variable is the outcome of the study that is either measured or counted. We have seen the response (outcome) variable in previous lessons. Examples ofresponse variables include:

· height

· weight

· temperature

· classification of whether a person is a vegetarian

· classification of symptom severity for an illness

A confounding variable is a variable that affects the response variable and is also related to the explanatory variable. The effect of a confounding variable on the response variable cannot be separated from the effect of the explanatory variable.  Therefore, we cannot clearly determine that the explanatory variable is solely responsible for any effect on the response or outcome variable when a confounding variable is present.   Confounding variables are problematic in observational studies.

12.2 Types of Research Studies

In statistics, there are two basic types of statistical research studies. These include:

· randomized experiments

· observational studies

With a randomized experiment, the researcher

· creates differences in the explanatory variable when randomly assigning treatments

· allows for possible "cause and effect" conclusions

· can minimize the effect of "confounding" variables

With an observational study, the researcher

· observes differences in the explanatory variable in natural settings/groupings (no variable is randomly assigned)

· strives for association conclusions since "cause and effect" conclusions are not possible

· must accept that confounding variables are potential problems

Example 12.1. Randomized Experiment (Two Independent Samples)

An educator wants to compare the effectiveness of computer software that teaches reading with a standard curriculum used to teach reading. The educator tests the reading ability of a group of 60 students and then randomly divides them into classes of 30 students each. One class uses the computer regularly while the other class uses a standard curriculum. At the end the semester, the educator retests the students and compares the mean increase in reading ability for the two groups.

This example is a randomized experiment because the students were randomly assigned to one of two methods to learn reading.  Also in this example:

· the experimental unit is the student

· the explanatory variable (treatment) is the method used to teach reading

· the response variable is the change found in reading ability at the end of the semester for each individual student.

The randomization that is used in this example cancels out other factors (confounding variables) that could also affect a change in reading ability. Specifically, the randomization will cancel out factors that may result from either self-selected or haphazardly-formed groups. With self-selection, students may base their decision on whether or not they like the computer or whether or not their friends will be in the class. This is no longer a problem when the groups are randomly formed.  Consequently, "cause and effect" statements can be used if statistical significance is found.

In statistics, we also say that the two samples in this study are independent. The label of independent samples is used when the results for the one sample have no impact on the results found with the second sample.  In this instance, each student provided a measurement for only one treatment.  The results from students in one group will not impact the results of students in the other group, so the results from the two samples are independent.

Example 12.2. Observational Study (Two Independent Samples)

A medical researcher conjectures that smoking can result in wrinkled skin around the eyes. The researcher obtained a sample of smokers and a sample of nonsmokers. Each person was classified as either having or not having prominent wrinkles. The study compared the percent of prominent wrinkles for the two groups.

This example cannot be a randomized experiment because it would be both unrealistic and unethical to randomly assign who would be the smoker and who would be the nonsmoker. Also in this example:

· the experimental unit is the person

· the explanatory variable is smoking status

· the response variable is whether or not each person has prominent wrinkles

Because this example is an observational study, it is possible that confounding variables may also be responsible for whether or not a person has prominent wrinkles. Possible confounding variables include (1) how much time the person spends outside, (2) whether or not the person wears sun screen, and (3) other variables that revolve around health and nutrition. Because we can't separate the impact that these variables may have on the response variable, "cause and effect" conclusions are never possible. The researcher would be limited to saying either that there is an association between smoking status and wrinkle status or that there is a difference in the two percents when comparing smokers to nonsmokers.

This is also an example where the two samples are independent. The individuals in this study were classified as being either smokers or nonsmokers. The results from the smoking group had no impact on the results from the nonsmoking group.

Example 12.3. Randomized Experiment (Two Dependent Samples or Matched Pairs)

Is the right hand stronger than the left hand for those who are right-handed? An instrument has been developed to measure the force exerted (in pounds) when squeezed by one hand. The subjects for this study include 10 right-handed people.  How can we best answer this question?

What would happen if we tried to implement what was done in Example 12.1?  This would mean that we would randomly assign five people to use their right and five people to use their left hand. The results from the two groups would then be compared. Hopefully you see that even though randomization is being used with this approach, the results may not be the best because it is possible that the one group could be comprised of strong people while the other group could be comprised of weak people.  If this happened, one could erroneously conclude that one hand is stronger for reasons other than that there is a difference in the two hands.

A better approach would be to have each person use both hands and then compare the results for the two hands. With this approach, the

· the experimental unit is the person

· the explanatory variable (treatment) is hand being used (right hand or left hand)

· the response variable is force exerted (in pounds) for each hand.

The design used in the example is called a block design because the results from each person form a block. Specifically, this block design is called a matched pairs (block) design because each person provides two data observations that can be paired together. Consequently, we can say that we have two dependent samples.  Table 12.1 shows a picture of the matched pairs design.

Table 12.1. Picture of Matched Pairs (Block) Design for Example 12.3

Person	Force from Right Hand	Force from Left Hand
1
2
3
"
10

In Table 12.1, one sees that the results from each person form a block. The reason that this design is used is so that unwanted or extraneous variation can be removed from the data.  In order to accomplish this goal, the data analysis is based on the differences rather than on the original data.    By using the differences, we are comparing the two data observations each personprovides to each other which distinguishes matched pairs from independent samples.     Table 12.2 shows some data that could have been collected in this study.

Table 12.2. Picture Data of Matched Pairs (Block) Design for Example 12.3

Person	Force from Right Hand (pounds)	Force from Left Hand (pounds)	Difference = (Right Hand Force) – (Left Hand Force)
1	47	38	47-38 = 9 pounds
2	20	15	20-15 = 5 pounds
3	33	26	33-26 = 7 pounds
"	"	"	"
10	28	27	28-27 = 1 pound

As you examine the results from Table 12.2, you should see that there are innate differences in strength when comparing the people who participated in the study. For example, Person 1 is much stronger than Person 2.  However, the variation from person to person is no longer a factor when the differences are used in the data analysis rather than using the original data.

Also, as you examine Table 12.2, you should see why we classify the two measurements for each experimental unit as dependent.   A higher value in one hand is usually followed by a higher value in the other hand.   The values are more similar for each pair of measurements for each experimental unit than the values are between experimental units.

Even though the matched paired design is critical in this example, this study would also benefit from randomization. Since each person is doing both things or providing two measurements, the randomization could be used to determine the order in which the treatments are done. Why would this enhance the study? Problems can exist with block designs, including matched pair designs, when what happens with the first measurement "carries over" to the second measurement. This "carryover" effect is a type of confounding that is found with block designs.

With this example, a "carryover" effect could possibly occur if complicated equipment was used to measure force exerted by a hand. If everyone used their right hand first, they might not do so well with the right hand because of not understanding the equipment, but do much better with their left hand because they learned how the equipment worked.  In statistics, this is called a training effect. The opposite, however, could also take place. Suppose everyone was asked to first exert force with their right hand for 15 minutes and then repeat this task with their left hand.  Participants might do okay with their right hand but become either bored or fatigued when asked to repeat with this task with the left hand. So again, what happened with the first measurement would "carryover" and affect the second measurement. One may conclude that one hand is stronger than the other, not because this is really true, but because the "carryover" effect allowed this to happen.

The overall conclusion is that if you randomly assign the order of treatment, some people will use their right hand first and other people will use their left hand first. This randomization should cancel out the possibility of a "carryover" effect.  In statistics, we call this a randomized block design, as shown in Table 12.3. Randomizing the order of treatment makes this a randomized experiment.

Table 12.3. Randomized Matched Pairs (Block) Design for Example 12.3

Person	Hand Used First	Hand Used Second
1	Right Hand	Left Hand
2	Left Hand	Right Hand
3	Right Hand	Left Hand
"
10	Left Hand	Right Hand

Example 12.4. Observational Study (Two Dependent Samples or Matched Pairs)

An owner of a theater wants to determine if the time of the showing affects attendance at a "scary" movie. In order to check this claim, a sample of five nights from all possible  nights over the past month was obtained. The attendance (total number of tickets sold) for both the 7:00 PM and the 9:30 PM showings was determined for each of the five nights.

In this example:

· the experimental unit is the night

· the explanatory variable is the showing time

· the response variable is the attendance at each showing.

This example also uses a matched pair (block) design because there are two measurements made on each night. A picture of this matched pair block design is found in Table 12.4.

Table 12.4. Matched Paired (Block) Design for Example 12.4

Night	Attendance at 7:00 PM Showing	Attendance at 9:30 PM Showing
1
2
3
4
5

Again, why is the matched pairs design preferred over two independent samples? In this example, our goal is determine whether or not time of showing affects attendance at the "scary" movie. We do not want any extraneous or other unwanted variation to explain the differences in attendance. In this example, the potential unwanted variation would be the variation that would exist from night to night. Some of the selected nights may fall on a weekend while other nights may fall on a weekday. This factor could affect attendance. However, this will no longer be a problem when both measurements are made on the same night.

This example, however, cannot be a randomized experiment because it would be impossible to randomly assign time of showing. The 7:00 PM showing will always take place before the 9:30 PM showing. Consequently, there is a possibility that what happens at the 7:00 PM showing may "carryover" and affect attendance at the 9:30 PM.  A possible "carryover" effect could be the fact there is a limited amount of parking near the theatre. If this were true, perhaps those at the 7:00 PM showing take all the available spots. Then people planning to attend that 9:30 PM showing may not attend because of not being able to find a parking spot. However, this problem may not exist if there is sufficient time between the two showings so that those who attended the 7:00 PM showing had time to leave before those who arrived for the 9:30 PM showing.  In any event, because this is an observational study, confounding variables are possible. "Cause and effect" conclusions may not be used if statistical significance is found.

Table 12.5. Summary of the Four Examples

Examples	Type of Study	Type of Samples	Randomization Used	Is Confounding Possible?
12.1	Experiment	Two Independent	Randomize type of treatment	No, randomization cancels out confounding
12.2	Observational	Two Independent	None	Yes
12.3	Experiment	Two Dependent (Matched Pairs)	Randomize order of treatment	No, randomization cancels out confounding
12.4	Observational	Two Dependent (Matched Pairs)	None	Yes

12.3 Confidence Intervals to Estimate a Population Proportion or Population Percent

Section 12.3. Chapter 20

Overview:

Chapter 20 covers confidence intervals to estimate a population proportion or population percent. Even though we learned about such confidence intervals in previous lessons, we actually did not learn the best way to calculate a confidence interval to estimate the population proportion. We will now learn what statisticians use to calculate a confidence interval to estimate the population proportion. Before doing so, first remember that proportions and percents are used to summarize categorical data.

Below is the general formula to estimate a population proportion with a 95% confidence interval. This formula is labeled "conservative" because the formula overestimates the margin of error resulting in a wider interval. We used this formula earlier.

Conservative 95% Confidence Interval Formula to Estimate a Population Proportion

Sample Proportion ± (Margin of Error)

where the margin of error =1n√

This margin of error formula is used with poll results so that people that have no background in statistics can try to at least partially understand what is behind the margin of error. Even though this formula tends to be conservative, it is based on statistical theory.

Below is the 95% confidence interval formula that is found in Chapter 20 of your textbook. You should notice that the margin of error formula is different from the one found in Chapter 4.

95% Confidence Interval Formula to Estimate a Population Proportion

Sample Proportion ± (Margin of Error)

where the margin of error = 2 × S.D.

2 is multiplier for a 95% Confidence Level S.D. = Standard Deviation for sample proportion (will always be provided)

This confidence interval formula is actually somewhat based on the results from the empirical rule that were discussed in Lesson 3 (Chapter 8). Remember that the empirical rule says that for any normal (bell-shaped) curve, approximately:

· 68% of the values (data) fall within 1 standard deviation

· 95% of the values (data) fall within 2 standard deviations

· 99.7% of the values (data) fall within 3 standard deviations

Table 12.6 shows commonly used multipliers with confidence intervals.

Table 12.6. Commonly Used Multipliers
Multiplier	Level of Confidence
3.0	99.7%
2.58 (2.576)	99%
2.0 (more precisely 1.96)	95%
1.645	90%
1.15	75%
1.0	68%

Because we want a 95% confidence interval, we use a multiplier of 2.  We also need a standard deviation (S.D.) that will quantify the amount of variation that exists when trying to estimate the population proportion. The standard deviation formula does incorporate the sample size (n) into the calculation. So what you have learned before about the impact of sample size on the margin of error still holds true. (Remember you will always be given the value of the standard deviation. These calculations will be based on the formula that is found in Chapter 20).

Example 12.5. Confidence Interval to Estimate the Population Proportion (Percent)

Suppose a national poll found that 70% of the respondents believe it is okay to use a handheld cell phone while driving. The poll was based on a sample size of 1020.

Calculation: Calculate a 95% confidence interval to estimate the population proportion who believe that it is okay to use a handheld cell phone while driving. Also include interpretations of both the margin of error and the confidence interval.

Answer: From the provided information we can find the following:

· Sample Percent = 70% (Sample Proportion = .70)

· Standard Deviation (S.D.) = .0143 (calculated using formula in Chapter 20 – this calculation incorporates the sample size of 1020)

· Margin of Error = 2 × .0143 = .0286 or 2.86% (Remember 2 is the multiplier for 95%)

Possible Margin of Error Interpretations (Recall Lesson 4)

· The difference between our sample proportion and the true population proportion will be within .0286, at least 95% of the time.

· The difference between our sample percent and the true population percent will be within 2.86%, at least 95% of the time.

95% Confidence Interval to Estimate Population Proportion

Sample Proportion ± (Margin of Error) .7 ± .0286 = (.6714 to .7286) or (67.14 to 72.86) %

 

Possible Confidence Interval Interpretations

· We are 95% confident that the population proportion who believe that it is okay to use a handheld cell phone while driving is between .6714 and .7286.

· We are 95% confident that the population percent who believe that it is okay to use a handheld cell phone while driving is between 67.14% and 72.86%

12.4 Remaining Topics in Chapter 5 of Textbook

While reading Chapter 5 you should take time to review the "Difficulties and Disasters in Experiments" that are listed in Section 5.3. You should also pay attention to the information found inSection 5.4. There are ways to design observational studies so that problems with confounding can be minimized. A case-control study is the best example of where confounding can be minimized. So please make sure you know both the characteristics of case-control studies and the advantages of case-control studies.

Section 12.2. Chapter 19 – Section 19.4

Overview: Previously we have learned that statistics is a collection of methods or procedures for extracting information from data.  Remember that these procedures can be a classified as either descriptive methods or inferential methods.

· Descriptive methods are procedures that are used to describe sample(s). These procedures can either be graphs or numerical summaries. These procedures have been covered in previous lessons.

· Inferential methods are procedures that are used to make conclusions about population(s). These procedures include both confidence intervals and hypothesis (significance) tests. These procedures have been partially covered in previous lessons.

A confidence interval (C.I.) is a procedure that is used to suggest possible values for an unknown population value such as a population mean, population proportion, or population percent. Specifically, the confidence interval uses information that is obtained from the sample to suggest possible values for an unknown population value.  We have already been introduced to confidence intervals in Lesson 7.   We will expand on this topic in the remaining parts of this lesson.

Hypothesis testing is a procedure that uses sample data to determine if it is possible to reject the idea that nothing is happening in the data. Specifically hypothesis testing is used to determine whether or not we have statistical significance.  Statistical significance is found when we determine that what we have found in our sample is unlikely to have occured if the finding is not present in the population.  We briefly explored this topic earlier and will examine this topic again in Lesson 13.

12.5 Confidence Intervals to Estimate the Population Proportion (Percent) for Two Independent Samples

Example 12.6. Confidence Intervals to Estimate the Population Proportion (Percent) for Two Independent Samples (Based on Example 12.2)

A medical researcher conjectures that smoking can result in wrinkled skin around the eyes. The researcher obtained 150 smokers and 250 nonsmokers where 95 of the smokers and 105of the nonsmokers were found to have prominent wrinkles around the eyes. Use two 95% confidence intervals to determine whether or not smokers have a higher percent with prominent wrinkles when compared against nonsmokers. These results are found in Table 12.7.

Table 12.7. Two 95% Confidence Intervals for Example 12.6
	Smokers	Nonsmokers
Sample Size	150	250
Sample Proportion with Prominent Wrinkles	95/150 = .63	105/250 = .42
Multiplier for 95%	2	2
S.D.	.04	.03
Margin of Error	2 × .04 = .08	2 × .03 = .06
95% Confidence Interval for Population Percent	.63 ± .08 = (.55 to .71) or (55 to 71)%	.42 ± .06 = (.36 to .48) or (36 to 48)%

Remember the decision rule that is used to compare two confidence intervals.

Decision Rule Used with Two Confidence Intervals to Make Conclusions

· If the two confidence intervals do not overlap, we can conclude that there is a difference in the two population values at the given level of confidence.

· If the two confidence intervals do overlap, we cannot conclude that there is a difference in the two population values, at the given level of confidence.

In this example, the two confidence intervals do not overlap, so we can conclude that there is a difference in the two population percents that have prominent wrinkles when comparing smokers to nonsmokers. Since the confidence interval for the smokers has values that are larger than the values found with the confidence interval for nonsmokers, we can also say the population percent with prominent wrinkles is greater for smokers than that found with nonsmokers, at 95% confidence.

Example 12.7. Modify Example 12.6

The information in Table 12.7 is again used to calculate two 95% confidence intervals, as shown in Table 12.8. The only change is that both sample sizes have been doubled.

Table 12.8. Two 95% Confidence Intervals with Sample Sizes Doubled
	Smokers	Nonsmokers
Sample Size	300	500
Sample Proportionwith Prominent Wrinkles	190/300 = .63	210/500 = .42
Multiplier for 95%	2	2
S.D.	.03	.02
Margin of Error	2 × .03 = .06	2 × .02 = .04
95% Confidence Interval for Population Percent	.63 ± .06 = (.57 to .69) or (57 to 69)%	.42 ± .04 = (.38 to .46) or (38 to 46)%

What you should now notice is that when the sample size increases, the standard deviation decreases. This leads to a smaller margin of error and confidence interval. The two confidence intervals found in Table 12.8 are smaller than those found in Table 12.7. The two intervals still do not overlap, so the overall conclusion does not change.

Example 12.8. Modify Example 12.6

The data in Table 12.7 is instead used to calculate two 99.7% confidence intervals, as shown in Table 12.9. Everything will stay the same except for the multiplier that is used with 99.7%.

Table 12.9. Two 99.7% Confidence Intervals (Data from Table 12.7)

	Smokers	Nonsmokers
Sample Size	150	250
Sample Proportion with Prominent Wrinkles	95/150 = .63	105/250 = .42
Multiplier for 99.7%	3	3
S.D.	.04	.03
Margin of Error	3 × .04 = .12	3 × .03 = .09
99.7% Confidence Interval for Population Percent	.63 ± .12 = (.51 to .75) or (51 to 75)%	.42 ± .09 = (.33 to .51) or (33 to 51)%

As you look at these results, you find that the two confidence intervals do overlap, even though just barely. However, it is still enough to not conclude a difference in the two population percents when comparing smokers to nonsmokers, at 99.7% confidence.

The following conclusions about confidence intervals can be made when collectively examining the results in Examples 12.6-12.8.

· At a given level of confidence: the width of the confidence interval or margin of error will decrease as the sample size increases. As you might expect, a larger sample will always lead to more precision in the estimate of the population value.

· At a given sample size: the width of the confidence interval or margin of error will increase as the level of confidence (multiplier) increases. This may seem counterintuitive. However, in order to be more confident that the interval contains the true value for the population value, the interval must become wider and suggest more possible values.  

There are three factors that affect the margin of error: standard deviation, sample size, and the level of confidence.  We only have control over the sample size and the level of confidence, so it is useful to know how these two factors affect the margin of error.

12.6 Confidence Intervals That Estimate the Population Mean

Section 12.4. Chapter 21 – Section 21.1

Overview:

Chapter 21 covers confidence intervals that estimate the population mean.What makes a confidence interval for the population mean different from a confidence interval for the population proportion or population percent is the type of data.

Proportions and percents are numerical summaries of categorical data while means are numerical summaries of measurement data.

Below is the 95% confidence interval formula that is found in Chapter 21 of your textbook.

95% Confidence Interval Formula to Estimate a Population Mean

Sample Mean ± (Margin of Error)

where the margin of error = 2 × S.E.

2 is the multiplier for a 95% confidence interval S.E. = is called a standard error and is a modified standard deviation that also incorporates the sample size into the calculation. This quantity will always be given to you.  The S.E. is used with means rather than the S.D.

Example 12.9. Confidence Interval to Estimate the Population Mean

How much credit card debt do students typically have when they graduate from Penn State? Suppose a sample of 20 recent Penn State graduates was obtained. Each of these recent graduates was asked to indicate the amount of credit card debt they had at the time of graduation. Below is a summary of the information that was obtained from this sample.

Sample Mean = $2430 S.D. = $2300  Standard Error (S.E) = $515 Sample Size (n) = 20

Calculation: Calculate a 95% confidence interval to estimate the population mean amount of credit card debt for all recent Penn State graduates. Also include an interpretation of both the margin of error and the confidence interval.

Margin of Error = 2 × S.E. = 2 × $515 = $1030

Margin of Error Interpretation: The difference between our sample mean and the true population mean will be within $1030, at least 95% of the time.

95% Confidence Interval to Estimate Population Mean

Sample Mean ± (Margin of Error) $2430 ± $1030 = ($1400 to $3460)

Confidence Interval Interpretation: We are 95% confident that the population mean amount of credit card debt for all recent Penn State graduates is between $1400 and $3460.

As you look at this confidence interval, the range of possible values for the population mean is quite large. This is because a sample size of 20 is rather small. Had a larger sample size been obtained, the confidence interval would have had a smaller margin of error.

Example 12.10. Confidence Intervals to Estimate the Population Mean for two Independent Samples (Based on Example 12.1)

An educator wants to compare the effectiveness of computer software that teaches reading with a standard curriculum used to teach reading. The educator tests the reading ability of a group of 60 students and then randomly divides them into classes of 30 students each. One class uses the computer regularly while the other class uses a standard curriculum. At the end of the semester, the educator retests the students and compares the mean increase in reading ability for the two groups. The increase in reading ability is measured as additional points achieved on a reading test that is given at the end of the semester. The data from this study is summarized in Table 12.10. Use two 95% confidence intervals to determine whether or not the mean increase in reading ability is greater for those who used computer software.

Table 12.10. Calculation of two 95% Confidence Intervals for Example 12.10

	Standard Method	Computer Software
Sample Size (n)	30 students	30 students
Sample Mean Increase in Reading Ability (points)	6.8 points	13.9 points
Standard Deviation (S.D.)	1.8 points	2.9 points
Standard Error (S.E.)	.3	.5
Margin of Error (M.E)	2 × .3 = .6	2 × .5 =1.0
95% Confidence Interval for Population Mean	6.8 ± .6 = (6.2 to 7.4) points	13.9 ± 1.0 = (12.9 to 14.9) points

As you look at the two 95% confidence intervals you can see that they do not overlap. Because of this we can say that there is a difference in the two population mean amounts of increase in reading ability when comparing those taught by the standard method to those who used computer software. The two confidence intervals also show that those who used computer software had a greater increase in reading ability at the end of the semester since the range of values of this confidence interval falls above the range of values found with the confidence interval for those who learned by the standard method.  Since this is a randomized experiment, we can also say that the method of reading is responsible for the mean amount that reading ability increased at the end of the semester.

Example 12.11. Confidence Intervals to Estimate the Population Mean Difference for Two Dependent Samples or Matched Pairs (Based on Example 12.4)

An owner of a theater wants to determine if the time of the showing affects attendance at a "scary" movie. In order to check this claim, a sample of five nights from all possible nights over the past month was obtained. The attendance (total number of tickets sold) for both the 7:00 PM and the 9:30 PM showings was determined for each of the five nights. Remember that since the data is paired or matched, the sample of interest is the differences. In this example, difference = (Attendance at 9:30 PM showing – Attendance at 7:00 PM showing). The actual data for this example is found in Table 12.11.

Table 12.11. Matched Pair (Block) Design for Example 12.4

Night	Attendance at 7:00 PM Showing	Attendance at 9:30 PM Showing	Difference = (9:30 PM – 7: PM Showing
1	157 people	165 people	165 – 157 = 8 people
2	195 people	212 people	212 – 195 = 17 people
3	98 people	101 people	101 – 98 = 3 people
4	168 people	181 people	181 – 168 = 13 people
5	143 people	157 people	157 – 143 = 14 people

Below is a summary of the information that was obtained from this sample of differences.

Sample Mean (Difference) = 11 people S.D. = 5.5 people Standard Error (S.E.) = 2.5 people Sample Size (n) = 5 nights

Calculation: Calculate a 95% confidence interval to estimate the population mean difference in attendance when comparing the attendance at the 9:30 PM showing with the attendance for the 7:00 PM showing.  Also include an interpretation of both the margin of error and the confidence interval.

Margin of Error = 2 × S.E. = 2 × 2.5 = 5.0 people

Margin of Error Interpretation: The difference between our (sample mean difference) and the true (population mean difference) will be within 5 people, at least 95% of the time.

95% Confidence Interval to Estimate Population Mean Difference

Sample Mean Difference ± (Margin of Error) 11 ± 5 = (6 to 16) people

Confidence Interval Interpretation: We are 95% confident that the population mean difference in attendance when comparing the 9:30 PM to the 7:00 PM showing is between 6 to 16 people.

Since there is only one confidence interval the following decision rule is used when the two samples are matched and the differences are used in the calculation.

Decision Rule Used with a Confidence Interval for the Population Mean Difference

·  If the CI contains 0, we may not conclude there is a difference in the two means for the two groups. In other words, the population mean difference likely equals 0.

· If the CI does not contain 0, there is a difference in the two means for the two groups. In other words, the population mean difference does not equal 0.

With this example we find that there are only positive numbers so there is a difference in the mean attendance when we compare the two times that the movie is shown.  Stated another way, the population mean difference does not equal 0.  Specifically, this 95% confidence interval states that on the average, about 6 to 16 more people attend the 9:30 showing when compared against the 7:00 PM showing.

12.7 Example 12.12

Example 12.12. Example 12.11 Analyzed Incorrectly as Two Independent Samples

Suppose someone incorrectly assumed the two samples in Example 12.11 were actually independent. Under this assumption, the two calculated confidence intervals found in Table 12.12would now be used.

Table 12.12. Two Confidence Intervals (Example 12.11)
	9:30 PM Showing	7:00 PM Showing
Sample Size (n)	5 nights	5 nights
Sample Mean Attendance	162 people	153 people
Standard Deviation (S.D.)	36 people	40 people
Standard Error (S.E.)	16 people	18 people
Margin of Error (M.E.)	2 × 16 = 32	2 × 18 = 36 people
95% Confidence Interval for Population Mean	162 ± 32 = (130 to 194) people	153 ± 36 = (117 to 189) people

When using the confidence interval found in Table 12.12 one would erroneously conclude that there is no difference in the population mean amount of attendance when comparing the 7:00 PM showing to the 9:30 PM showing because there is overlap. The reason that these two intervals overlap is because extraneous variation is now present. This extraneous variation is caused by the variability of attendance between the different nights. The extraneous variation leads to larger standard deviations that then lead to larger margin of errors. It is easier for the two confidence intervals to overlap. In this case we have allowed the between night variability to affect our ability to detect the between showings difference.

Lesson 12 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. Eighty individuals who wish to lose weight are randomly divided into two groups of 40. One group is given an exercise program to follow while the other group follows a special diet. After three months, the researcher compares mean weight losses in the two groups. What type of study is this?

a. randomized experiment

b. observational study

2. Which type of randomization plays an important role in experiments?

a. randomizing the type of treatment

b. randomizing the order of treatment

c. both a and b

d. neither a nor b

3. The goal of a confidence interval is to suggest possible values for a __________.

a. population value

b. sample estimate

4. Which of the following is the correct general formula for a confidence interval?

a. sample estimate ± (multiplier)

b. sample estimate ± (margin of error)

c. (margin of error) ± sample estimate

5. A researcher conducts a study to determine whether or not smoking causes lung cancer. What is the explanatory variable in this study?

a. whether or not someone smokes

b. whether or not someone has cancer

Lesson 13: Statistical Inference (Significance Tests) and Reading the News

Assignments

· See your Course Syllabus for the reading assignment.

· Work through the Lesson 13 online notes that follow.

· Complete the Practice Questions and Lesson 13 Assignment.

Learning Objectives

Chapters 2, 22, 23, and 24

After successfully completing this lesson, you should be able to:

· State appropriate null and alternative hypotheses.

· Identify the type 1 and the type 2 error in the context of the problem.

· Interpret a p-value in terms of the problem.

· State an appropriate conclusion for a hypothesis test after using the appropriate decision rule.

· Apply the seven critical components when evaluating an article in the news.

Terms to Know

From Chapters 22 and 23

· inferential methods

· hypothesis (significance) tests

· null hypothesis

· alternative hypothesis

· population value

· population mean

· population proportion (p)

· population percent

· sample estimate

· test statistic

· p-value

· type 1 error

· type 2 error

· statistical significance

· response variable

· explanatory (grouping) variable

From Chapters 2 and 24

· data

· seven critical components

· extraneous differences

· magnitude of effects or differences

· statistical significance

· practical significance

13.1 Hypothesis Testing

Section 13.1. Chapters 22-23 in Textbook

Overview

Remember that inferential methods are procedures that are used to make conclusions about population(s) and/or population values. These procedures include both confidence intervals and hypothesis (significance) tests. In Lesson 12, we learned the specifics about confidence intervals. In this lesson we will fill in the blanks about hypothesis testing.

Hypothesis testing is a procedure that uses sample data to determine if there is sufficient evidence to reject the idea that nothing is happening in the population. Specifically hypothesis testing is used to determine whether or not we have statistical significance.  Statistical significance is found when we determine that what we have found in our sample is unlikely to have occured if the finding was not present in the population.  When statistical significance has been found, we can rule out chance as a possible explanation for our finding. 

In order to properly conduct tests of statistical significance we first need to establish two hypotheses: the null hypothesis and the alternative hypothesis.

The null hypothesis is a statement that: 

· nothing is happening.

· there is no relationship.

· there is no difference.

· the status quo is correct.

· long-standing idea still holds true.

The alternative hypothesis is a statement that: 

· something is happening.

· there is a relationship.

· there is a difference.

· the status quo is incorrect.

· supports a challenging or new idea.

These hypotheses make statements about populations or population values. The language used in stating the hypotheses is affected by both the number and type of variables that are being considered, as shown in Table 13.1.

Table 13.1. Set-Up of Null Hypotheses Based on Type of Data

Type of  Data	Generic Set Up of Null Hypothesis
One Sample: One Categorical Variable	Population Proportion (p) = (a value)
One Sample: One Measurement Variable	Population Mean = (a value)
Two Samples: One Categorical Variable	No Difference in the Two Population Proportions or (p1 = p2)
Two Samples: One Measurement Variable	No Difference in the Two Population Means or (Population Mean1 = Population Mean2)
Two Different Categorical Variables	No relationship between the two variables
Two Different Measurement Variables	No (linear) relationship between the two variables

Types of Alternative Hypotheses:

The type of alternative hypothesis also determines the type of hypothesis test we use.

A One-Sided alternative hypothesis is when the alternative hypothesis states that a population proportion or population mean is either “greater than” or “less than” a set value or another group’s population proportion or population mean. The Hypothesis Test we use is then called a One-Sided Test or called a One-Tailed Test.

· In One-Sided Tests where the alternative hypothesis is “greater than”, we must only consider the possibility of obtaining a population proportion or population mean greater than the set value or greater than another group’s population proportion or population mean.

· In One-Sided Tests where the alternative hypothesis is “less than”, we must only consider the possibility of obtaining a population proportion or population mean less than the set value or less than another group’s population proportion or population mean.

A Two-Sided alternative hypothesis is when the alternative hypothesis states that a population proportion or population mean is “not equal to” a set value or another group’s population proportion or population mean. We then use a Two-Sided Hypothesis Test or Two-Tailed Test.

· In Two-Sided Tests where the alternative hypothesis is “not equal to”, we must consider both the possibility of obtaining a population proportion or population mean “greater than” and “less than” the set value or “greater than” and “less than” another group’s population proportion or population mean.   The Test is considered two-sided since we must consider both “greater than” and “less than” possibilities

13.2 Examples

Example 13.1. Hypotheses with One Sample of One Categorical Variable

About 10% of the human population is left-handed. Suppose a researcher at Penn State speculates that students in the College of Arts and Architecture are more likely to be left-handed than people found in the general population.   We only have one sample since we will be comparing a population proportion based on a sample value to a known population value.

Research Question: Are artists more likely to be left-handed than people found in the general population?

Response Variable: Classification of student as either right-handed or left handed

State Null and Alternative Hypotheses

· Null Hypothesis: Students in the College of Arts and Architecture are no more likely to be left-handed than people in the general population (population percent of left-handed students in the College of Art and Architecture = 10% or p= .10). 

· Alternative Hypothesis: Students in the College of Arts and Architecture are more likely to be left-handed than people in the general population (population percent of left-handed students in the College of Art and Architecture > 10% or p> .10).  This is a one-sided alternative hypothesis.

Example 13.2. Hypotheses with One Sample of One Measurement Variable

Previous studies have shown that, on the average, a Penn State student spends about two hours/day watching television.  However, anecdotal evidence suggests that this value may now be something other than 2.0 hours/day. Again, we have only one sample, since we will be comparing a population mean based on a sample value to a known population value.

Research Question: Does the data suggest that the population mean number of hours/day spent watching television differs from 2.0?

Response Variable: hours/day spent watching television

State Null and Alternative Hypotheses

· Null Hypothesis: On the average, a Penn State student spends about 2.0 hours/day watching television (population mean number of hours/day spent watching television = 2.0).

· Alternative Hypothesis: On the average, a Penn State student does not spend about 2.0 hours/day watching television (population mean number of hours/day spent watching television ≠ 2.0).  This is a two-sided alternative hypothesis.

Example 13.3. Hypotheses with Two Samples of One Categorical Variable

Many people are starting to prefer vegetarian meals on a regular basis. Specifically, a researcher believes that females are more likely than males to eat vegetarian meals on a regular basis.

Research Question: Does the data suggest that females are more likely than males to eat vegetarian meals on a regular basis? 

Response Variable: Classification of whether or not a person eats vegetarian meals on a regular basis

Explanatory (Grouping) Variable: Gender

State Null and Alternative Hypotheses

· Null Hypothesis: There is no gender effect regarding those who eat vegetarian meals on a regular basis (population percent of females who eat vegetarian meals on a regular basis = population percent of males who eat vegetarian meals on a regular basis or pfemales = pmales).

· Alternative Hypothesis: Females are more likely than males to eat vegetarian meals on a regular basis (population percent of females who eat vegetarian meals on a regular basis > population percent of males who eat vegetarian meals on a regular basis or pfemales > pmales).  This is a one-sided alternative hypothesis.

Example 13.4. Hypotheses with Two Samples of One Measurement Variable

Obesity is a major health problem today.  Research is starting to show that people may be able to lose more weight on a low carbohydrate diet  than on a low fat diet.

Research Question: Does the data suggest that, on the average, people are able to lose more weight on a low carbohydrate diet than on a low fat diet?

Response Variable: Weight loss (pounds)

Explanatory (Grouping) Variable: Type of diet

State Null and Alternative Hypotheses

· Null Hypothesis: There is no difference in the mean amount of weight loss when comparing a low carbohydrate diet with a low fat diet (population mean weight loss on a low carbohydrate diet = population mean weight loss on a low fat diet).

· Alternative Hypothesis: The mean weight loss should be greater for those on a low carbohydrate diet when compared with those on a low fat diet (population mean weight loss on a low carbohydrate diet > population mean weight loss on a low fat diet).   This is a one-sided alternative hypothesis.

Example 13.5. Hypotheses with Two Different Categorical Variables

Research Question: Is there an asssociation between cigarette use and ecstasy use?  (Are people who smoke cigarettes more likely to use ecstasy or vice versa?)

Variables: There are two different categorical variables. When looking for the existence of a relationship, one does not need to identify the explanatory and the response variable. The data analysis for this type of hypothesis test was covered in Lesson 7.

State Null and Alternative Hypotheses

· Null Hypothesis: There is no relationship between whether or not a person smokes and whether or not a person uses ecstasy.

· Alternative Hypothesis: There is a relationship between whether or not a person smokes and whether or not a person uses ecstasy.

 

Example 13.6. Hypotheses with Two Measurement Variables

Research Question: Is there a linear relationship between the amount of the bill ()atarestaurantandthetip() that was left?

Variables: There are two different measurement variables. When looking for the existence of a linear relationship, one does not need to identify the explanatory and the response variable. The data analysis for this type of  hypothesis test was covered earlier in Lesson 8.

State Null and Alternative Hypotheses.

· Null Hypothesis: There is no linear relationship between the amount of the bill ()atarestaurantandthetip() that was left.

· Alternative Hypothesis: There is a linear relationship between the amount of the bill ()atarestaurantandthetip() that was left.

13.3 Possible Decisions with Hypotheses

After the data is collected and analyzed there are two different possible decisions.

Decision 1:

· Data provides sufficient new evidence to go with the alternative hypothesis.

· The p-value is small enough to rule out chance (i.e. p-value ≤ .05). 

· Statistical significance has been found.

Decision 2:

· Data does not provide sufficient new evidence to go with the alternative hypothesis.

· The p-value is not small enough to rule out chance (i.e. p-value > .05). 

· Statistical significance has not been found.

The goal of the hypothesis test is to determine the strength of evidence against the null hypothesis. The hypothesis test is never trying to prove that the null hypothesis is true.

Possible Mistakes with Decisions

Even though there are two possible decisions, there is always a chance that a mistake can be made. The two types of errors are the type 1 and the type 2 errors.

· Type 1 error occurs when one concludes that the alternative hypothesis is true, when in fact the null hypothesis is actually true.

· Type 2 error occurs when one fails to conclude the alternative hypothesis, when in fact the alternative hypothesis is true.

 

Example 13.7. Example of  Type 1 and Type 2 Errors

Consider the following two hypotheses:

Null Hypothesis: Filing income tax forms electronically takes as much time as doing them by hand.

Alternative Hypothesis: Filing income tax forms electronically take less time than doing them by hand.

Question: Identify the type 1 and type 2 errors.

Type 1 Error: Claim filing income tax forms electronically takes less time than doing them by hand when really it does not take less time.

Type 2 Error: Fail to claim filling income tax forms electronically takes less time than doing them by hand, when it really does take less time.

13.4 Steps Used in a Hypothesis Test

1. State the null and alternative hypotheses. Also think about the type 1 and type 2 errors at this time.

2. Collect and summarize the data so that a test statistic can be calculated. A test statistic is a single value summary of the data that has been standardized so that a  p-value can be obtained. You will not be asked to calculate any test statistics in this course.

3. Use the test statistic to find the p-value.  The p-value represents the likelihood of getting our test statistic or any test statistic more extreme, if in fact the null hypothesis is true.  

· For a one-sided "greater than" alternative hypothesis, the "more extreme" part of the interpretation refers to test statistic values larger than the test statistic given.   

· For a one-sided "less than" alternative hypothesis, the "more extreme" part of the interpretation refers to test statistic values smaller than the test statistic given.   

· For a two-sided "not equal to" alternative hypothesis, the "more extreme" part of the interpretation refers to test statistic values that are "more extreme" than the given test statistic and are "more exteme" than the negative of the given test statistic (both "tails"). 

Even though you will not be asked to calculate any p-values in this course, you may be asked to interpret a p-value.   The p-value is interpreted the same way no matter what hypothesis test is done.

4. Make a decision using the p-value. Remember the decision rule that we used in previous lessons. State a conclusion in terms of the problem. If statistically significant, we can conclude the alternative hypothesis.

Decision Rule used with P-Values

· If the p-value ≤ .05, we can conclude that there is a statistically significant result.

· If the p-value > .05, we cannot conclude that there is a statistically significant result.

 

Example 13.8. Hypothesis Test based on Example 13.1

About 10% of the human population is left-handed. Suppose a researcher at Penn State speculates that students in the College of Arts and Architecture are more likely to be left-handed that people in the general population. A random sample of 100 students in the College of Arts and Architecture is obtained such that 18 were found to be left-handed.

Research Question: Are artists more likely to be left-handed than people in the general population?

Step 1: State Null and Alternative Hypotheses

· Null Hypothesis: Population proportion of left-handed students in the College of Art and Architecture = 0.10 (p = 0.10).

· Alternative Hypothesis: Population proportion of left-handed students in the College of Art and Architecture > 0.10 (p > 0.10).

Now that you know the null and alternative hypothesis, did you think about what the type 1 and type 2 errors are? It is important to note that Step 1 is before we even collect data. Identifying these errors helps to improve the design of your research study. Let’s write them out:

Type 1 error: Claim artists are more likely to be left-handed than people in the general population, when in truth they are not more likely.

Type 2 error: Fail to claim artists are more likely to be left-handed than people in the general population, when they are in fact more likely.

Step 2: Collect and summarize the data so that a test statistic can be calculated.

In the sample of 100 students listed above, the sample proportion is .18 (18/100). The hypothesis test will determine whether or not the sample proportion provides sufficient evidence to conclude that in the population of all students in the College of Art and Architecture, the proportion of left-handed students is greater than .10.

The data was used to obtain the computer output found in Figure 13.1. Even though you will not be expected to fully understand everything found on the output you should appreciate that this output provides all relevant information for this hypothesis test. The sample proportion of .18 is used to determine the Z-value of 2.67.  The Z-value is the test statistic for this hypothesis test.

Test and CI for One Proportion

 Test of p = 0.1 vs p > 0.1    

Sample   X       N      Sample p      95% Lower Bound       Z-Value       P-Value

1            18     100    0.180000          0.116807                    2.67            0.004

 Figure 13.1. Computer Output for Example 13.8   

Step 3: Use the test statistic to find the p-value.

The computer used the Z-value of 2.67 to calculate the p-value that equals 0.004.

Interpretation of the p-value. The likelihood of getting our test statistic of 2.67 or any value more extreme, if in fact the null hypothesis is true, is .004.  In this case, since the alternative hypothesis is that the population proportion of left-handed students is "greater than" .10, the p-value is the likelihood of getting our test statistic of 2.67 or any value LARGER, if in fact the null hypothesis is true.  When we have a "greater than" alternative hypothesis, we want the right-tail p-value since more extreme values of our test statistic will be those values of 2.67 or larger (those values to the right of 2.67).

Step 4: Make a decision using the p-value.

Since the p-value of .004 ≤ .05, we can conclude the population proportion of left-handed students in the College of Art and Architecture exceeds .10.

Now that we have made our decision, we are only at risk of making a type 1 error. It is not possible at this point to make a type 2 error because we rejected the null hypothesis.

 

Example 13.9. Hypothesis Test Based on Example 13.2

Previous studies have shown that, on the average, a Penn State student spends about  2.0 hours/day watching television.  However, anecdotal evidence suggests that this value may have changed. A sample of 16 Penn State students was obtained where the sample mean hours/day spent watching television was 2.6 hours.

Research Question: Does the data suggest that the population mean hours/day spent watching television by Penn State students differs from 2.0?

Step 1: State Null and Alternative Hypotheses.

· Null Hypothesis: Population mean number of hours/day spent watching television = 2.0

· Alternative Hypothesis: Population mean number of hours/day spent watching television ≠ 2.0.

Step 2: Collect and summarize the data so that a test statistic can be calculated.

The sample mean number of hours/day spent watching television is 2.6.  The hypothesis test will determine whether or not the sample mean provides sufficient evidence to say that the population mean number of hours/day spent watching television is no longer 2.0.

The sample data was used to obtain the computer output found in Figure 13.2.  The t-value of 1.21 is the test statistic for this hypothesis test. This test statistic is the standardized value of the sample mean.

Figure 13.2. Computer Output for Example 13.9

Step 3: Use the test statistic to find the p-value.

The computer used the t-value of 1.21 to calculate the p-value that equals 0.244.

Interpretation of the p-value. The likelihood of getting our test statistic of 1.21 or any value more extreme, if in fact the null hypothesis is true, is .244.  In this case, since the alternative hypothesis is that the population mean value "does not equal" 2.0, the p-value is the likelihood of getting our test statistic of 2.67 or any value LARGER as well as the likelihood of getting a test statistic of -2.67 or any value SMALLER, if in fact the null hypothesis is true.  When we have a "not equal" alternative hypothesis, we want the two-tailed p-value since we have a two-sided alternative hypothesis.   The alternative hypothesis is considered two-sided since we must consider both "less than" and "greater than" possibilities when the alternative hypothesis is "not equal". 

Step 4: Make a decision using the p-value.

Since the p-value of .244 > .05, we cannot conclude the population mean number of hours/day spent watching television by Penn State students differs from 2.0. There is insufficient evidence to go with the alternative hypothesis.  However, we have not proved the population mean = 2.0 hours/day.   In statistical terminology, we have failed to reject the null hypothesis.

13.5 Critically Evaluating Statistical Results

Section 13.2. Chapters 2 and 24

We are doing Chapter 2 last because one of the goals of the course is that by the end you will be able to critically evaluate statistical results that are reported in a newspaper or on the Web. We all know that we should never believe everything that is published or posted on the Web. From a statistical perspective, the primary reason that we should not believe all that we read is that in general we don't have access to the actual (raw) data but only summary information. The choice of summary information presented in an article can affect the conclusions that one will make from these statistical results.

In Chapter 2 you are given seven critical components that should be applied when evaluating results from statistical studies. These components include:

Component 1: The source of the research and funding Component 2: The researchers who had contact with the participants Component 3: The individuals or objects studied and how they were selected Component 4: The exact nature of the measurements made or the questions asked Component 5: The setting in which the measurements were taken Component 6: The extraneous differences between groups being compared Component 7: The magnitude of any claimed effects or differences

Overall, the textbook does a great job of explaining what contributes to each of the seven components. The only components that need some additional clarification are Components 6 and7.  Components 6 and 7 are the two that are most directly related to statistical principles.

The key phrase found with Component 6 is "extraneous differences." The phrase extraneous differences is just another way of saying that differences in groups may exist for reasons other than the explanatory variable. When a study is not a randomized experiment, it is also possible that confounding variables might be responsible for the differences in the two groups.  So remember to consider extraneous differences and stay away from cause and effect conclusions with observational studies.

Component 7 is based on one of the two warnings about statistical significance that were presented in Chapter 10. These two warnings include:

1. A large sample size will find a relationship or difference that is statistically significant.  However, this does not imply that a relationship or difference is practically significant.

2. A very strong relationship or difference won't necessarily achieve statistical significance if the sample size is very small.  However, it still could be practically significant.

Before we address these warnings we need to know the difference between statistical significance and practical significance.

Statistical Significance has been found when the appropriate statistical test has been done and the corresponding  p-value ≤ .05.  Chance has been ruled out as the explanation as to why there is support for the alternative hypothesis.

Practical Significance has been found when the results include findings that may have a meaningful affect for you or society.  Practical significance is not determined by a statistician. It can best be determined by evaluating results that are reported in the same units as found with the response variable. The larger the magnitude of effect, the more likely the results will also be practically significant.   We look at the values in the confidence interval(s) to assess practical significance.

 

Example 13.10. Based on Example 13.4 – Statistical versus Practical Significance

Obesity is a major health problem today. Research is starting to show that people may be able to lose more weight on a low carbohydrate diet rather than on a low fat diet.

Research Question: Does the data suggest that, on the average, people are able to lose more weight on a low carbohydrate diet than on a low fat diet?

State Null and Alternative Hypotheses

· Null Hypothesis: Population mean weight loss on a low carbohydrate diet = population mean weight loss on a low fat diet.

· Alternative Hypothesis: Population mean weight loss on a low carbohydrate diet > population mean weight loss on a low fat diet.

Consider a study that obtained a group of 40 people and randomly assigned half to go on a low carbohydrate diet and half to go on a low fat diet. After eight weeks, the amount of weight loss was determined for each person in the study. The results for the study are found in Figure 13.3.

Figure 13.3. Computer Output for Example 13.10

As you examine the computer output, you find that the p-value for this study is 0.000. Please note that the p-value does not equal 0; the output is a little lazy on this. It should read that the p-value is less than 0.001, rather than equal to 0.000. In this instance, we can claim population mean weight loss on a low carbohydrate diet > population mean weight loss on a low fat diet. The results are indeed statistically significant. However, we must look further to determine whether or not we have also found practical significance.

On the output one should see that the 95% confidence interval for the defined difference is (1.00 to 3.00) pounds. This interval suggests that on the average, one can expect to lose 1 to 3 more pounds on the low carbohydrate diet. This interval allows us to both examine the magnitude of the differences between the two diets and the issue of practical significance.  For most of us, we would probably say that the weight loss is not that much better on the low carbohydrate diet, when compared against a low fat diet, at least when the diet is followed for eight weeks.

 

Example 13.11. Based on Example 7.5

A study compares the "percent that quit smoking" for users and nonusers of the nicotine patch. The reported relative risk shows that a person is 2 times more likely to quit smoking when using the patch. The corresponding reported increased risk shows that there is a 100% increase in the chance of quitting smoking when using the patch.  However, these results by themselves do not allow one to determine the magnitude of effect.

Suppose the results are based on a sample of 50 smokers who used the patch and a sample of 50 smokers who did not use the patch. Table 13.2 provides three possible results where the relative risk of 2.0 and the increased risk is 100%. However, each choice does not have the same magnitude of effect.

Of the three choices, Choice A shows the largest magnitude of effect to the extent that it is also statistically significant. However, it would be up to each individual to decide if the difference between 50% and 25% when calculated from sample sizes of 50 is enough evidence to consider using the patch to stop smoking. One may also feel that practical significance may have been achieved with Choices B and/or C, even though both are not statistically significant. Remember statisticians do not make decisions about practical significance.

Table 13.2. Results for the Three Choices (Based on Sample Sizes of 50)

Choice	Relative Risk	Increased Risk	Chi-Square Statistic & P-value
A	50%/25% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = 17.14 & P-value = .000*
B	10%/5% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = 1.81 & P-value = .179
C	2%/1% = 2.0	(2.0-1) × 100% = 100%	Chi-Square = .338 & P-value = .561

*statistically significant

Lesson 13 Practice Questions

Answer the following Practice Questions to check your understanding of the material in this lesson.

Think About It!

Come up with an answer to these questions by yourself and then click the icon on the left to reveal the answer.

1. The goal of a hypothesis (significance) test is to find the strength of evidence against the:

a. null hypothesis

b. alternative hypothesis

2. Statistical hypotheses make statements about:

a. sample estimates

b. populations and/or population values

3. The null hypothesis includes statements that:

a. nothing is happening

b. something is happening

4. In hypothesis testing, a Type 1 error occurs when ____________.

a. The null hypothesis is rejected when in fact the null hypothesis is really true

b. The null hypothesis is not rejected when the alternative hypothesis is really true

5. A result that is statistically significant must also be practically significant.

a. True

b. False