During this assignment, you will familiarize yourself with research terminology as you use the terms to write your paper

statistical_reasoning_ch01.pdf

Home >Law homework help >During this assignment, you will familiarize yourself with research terminology as you use the terms to write your paper

ee eBook Collection

Speaking of Statistics

As a general rule, the most successful [person] in life is the [person] who has the best information.

—Benjamin Disraeli

IS YOUR DRINKING WATER SAFE? DO MOST PEOPLE

approve of the President’s tax plan? Are we getting good

value for our health care dollars? Questions like these can be

addressed only through statistical studies. In this first chapter,

we will discuss basic principles of statistical research and lay a

foundation for the more detailed study of statistics that fol-

lows in the rest of this book. Along the way, we will consider a

variety of examples that show how well-designed statistical

studies can provide guidance for social policy and personal

decisions, as well as a few cases in which statistics can be

misleading.

LEARNING GOALS

1.1 What Is/Are Statistics?

Understand the two meanings of the term statistics and the basic ideas behind any statistical study, including the relationships among the study’s population, sample, sample statistics, and population parameters.

1.2 Sampling

Understand the importance of choosing a representative sample and become familiar with several common methods of sampling.

1.3 Types of Statistical Studies

Understand the differences between observational studies and experiments; recognize key issues in experiments, including the selection of treatment and control groups, the placebo effect, and blinding.

1.4 Should You Believe a Statistical Study?

Be able to evaluate statistical studies that you hear about in the media, so that you can decide whether the results are meaningful.

IS B

N : 0

-5 36

-8 82

21 -5

Statistical Reasoning for Everyday Life, Third Edition, by Jeffrey O. Bennett, William L. Briggs, and Mario F. Triola. Published by Addison-Wesley. Copyright © 2009 by Pearson Education, Inc.joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka

joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka joka

2 Speaking of Statistics

Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.

—H. G. Wells

1.1 What Is/Are Statistics?

If you are like most students using this textbook, you are new to the study of statistics. Youmay not be sure why you are studying statistics, and you may be feeling anxious aboutwhat lies ahead. But as you begin reading, we hope you’ll put your concerns aside and prepare to be pleasantly surprised.

The subject of statistics is often stereotyped as dry or technical, but it touches on almost everything in modern society. Statistics can tell us whether a new drug is effective in treating cancer, it can help agricultural inspectors ensure that our food is safe, and it’s the key to all opinion polls. Businesses use statistics in market research and advertising. We even use statis- tics in sports, often as a way of ranking teams and players. Indeed, you’ll be hard-pressed to think of any topic that is not linked with statistics in some important way.

The primary goal of this book is to help you learn the core ideas behind statistical meth- ods. These basic ideas are not difficult to understand, although mastery of the details and theory behind them can require years of study. One of the great things about statistics is that even the small amount of theory covered in this book will give you the power to understand the statistics you encounter in the news, in your classes or workplace, and in your everyday life.

A good place to start is with the term statistics itself, which can be either singular or plural and has different meanings in the two cases. When it is singular, statistics is the science that helps us understand how to collect, organize, and interpret numbers or other information about some topic; we refer to the numbers or other pieces of information as data. When it is plural, statistics are the actual data that describe some characteristic. For example, if there are 30 students in your class and they range in age from 17 to 64, the numbers “30 students,” “17 years,” and “64 years” are all statistics that describe your class in some way.

By the Way .. . Statistics originated with the collection of census and tax data, which are affairs of state. That is why the word state is at the root of the word statistics.

Two Definitions of Statistics

• Statistics is the science of collecting, organizing, and interpreting data.

• Statistics are the data (numbers or other pieces of information) that describe or summarize something.

Note that although you’ll sometimes see the word data used as a singular synonym for information, technically it is plural: One piece of information is called a datum, and two or more pieces are called data.

How Statistics Works Did you watch the Super Bowl? Advertisers want to know, because the cost of commercial time during the big game is approaching $3 million for a 30-second spot. This advertising can be well worth the price if enough people are watching. For example, news reports stated that 93.2 million Americans watched the Indianapolis Colts win Super Bowl XLI. But you may wonder: Who counted all these people?

The answer is no one. The claim that 93.2 million people watched the Super Bowl came from statistical studies conducted by a company called Nielsen Media Research. This company publishes the results of its studies as the famous Nielsen ratings. Remarkably, Nielsen com- piles these ratings by monitoring the television viewing habits of people in only 5,000 homes.

If you are new to the study of statistics, Nielsen’s conclusion may seem like a stretch. How can anyone draw a conclusion about millions of people by studying just a few thousand? However, statistical science shows that this conclusion can be quite accurate, as long as the

IS B

N : 0-536-88221-5

1.1 What Is/Are Statistics? 3

statistical study is conducted properly. Let’s take the Nielsen ratings of the Super Bowl as an example, and ask a few key questions that will illustrate how statistics works in general.

What Is the Goal of the Research? Nielsen’s goal is to determine the total number of Americans who watched the Super Bowl. In the language of statistics, we say that Nielsen is interested in the population of all Americans. The number that Nielsen hopes to determine—the number of people who watched the Super Bowl—is a particular characteristic of the population. In statistics, characteristics of the popu- lation are called population parameters.

Although we usually think of a population as a group of people, a statistical population can be any kind of group—people, animals, or things. For example, in a study of automobile safety, the population might be all cars on the road. Similarly, the term population parameter can refer to any characteristic of a population. In the case of automobile safety, the population parameters might include the total number of cars on the road during a certain time period, the accident rate among cars on the road, or the range of weights of cars on the road.

By the Way .. . Arthur C. Nielsen founded his company and invented market research in 1923. He introduced the Nielsen Radio Index to rate radio programs in 1942 and extended his methods to television programming in the 1960s.

Definitions

The population in a statistical study is the complete set of people or things being studied.

Population parameters are specific characteristics of the population.

EXAMPLE 1 Populations and Population Parameters For each of the following situations, describe the population being studied and identify some of the population parameters that would be of interest.

a. You work for Farmers Insurance and you’ve been asked to determine the average amount paid to accident victims in cars without side-impact air bags.

b. You’ve been hired by McDonald’s to determine the weights of the potatoes delivered each week for french fries.

c. You are a business reporter covering Genentech Corporation and you are investigating whether their new treatment is effective against childhood leukemia.

Solution

a. The population consists of people who have received insurance payments for accidents in cars that lacked side-impact air bags. The relevant population parameter is the mean (aver- age) amount paid to these people. (See Chapter 4 for a more detailed discussion of the mean and other measures of “average.”)

b. The population consists of all the potatoes delivered each week for french fries. Relevant population parameters include the mean weight of the potatoes and the variation of the weights (for example, are most of them close to or far from the mean?).

c. The population consists of all children with leukemia. Important population parameters are the percentage of children who recover without the new treatment and the percentage of children who recover with the new treatment.

What Actually Gets Studied? If researchers at Nielsen were all-powerful they might determine the number of people watch- ing the Super Bowl by asking every individual American. But no one can do that, so instead they try to estimate the number of Americans watching by studying a relatively small group ofIS

B N

: 0 -5

36 -8

82 21

-5

4 Speaking of Statistics

people. In other words, Nielsen attempts to learn about the population of all Americans by carefully monitoring the viewing habits of a much smaller sample of Americans. More speci- fically, Nielsen has devices (called “people meters”) attached to televisions in about 5,000 homes, so the roughly 13,000 people who live in these homes are the sample of Americans that Nielsen studies. (To keep up with rapidly changing viewing habits that now include cable TV and Internet viewing, Nielsen has been increasing its number of people meters, but the 5,000 households given here is still representative of the general ratings process.)

The individual measurements that Nielsen collects from the people in the 5,000 homes con- stitute the raw data. Nielsen collects much raw data—for example, when and how long each TV in the household is on, what show it is tuned to, and who in the household is watching it. Nielsen then consolidates these raw data into a set of numbers that characterize the sample, such as the per- centage of viewers in the sample who watched each individual television show or the total number of people in the sample who watched the Super Bowl. These numbers are called sample statistics.

By the Way .. . By the Labor Department defini- tion, someone who is not working is not necessarily unemployed. For example, stay-at-home moms and dads are not counted among the unemployed unless they are actively trying to find a job, and people who tried to find work but gave up in frustration are not counted as unemployed.

Definitions

A sample is a subset of the population from which data are actually obtained.

The actual measurements or observations collected from the sample constitute the raw data.

Sample statistics are characteristics of the sample found by consolidating or summarizing the raw data.

EXAMPLE 2 Unemployment Survey The U.S. Labor Department defines the civilian labor force as all those people who are either employed or actively seeking employment. Each month, the Labor Department reports the un- employment rate, which is the percentage of people actively seeking employment within the entire civilian labor force. To determine the unemployment rate, the Labor Department surveys 60,000 households. For the unemployment reports, describe the

a. population b. sample c. raw data d. sample statistics e. population parameters

Solution

a. The population is the group that the Labor Department wants to learn about, which is all the people who make up the civilian labor force.

b. The sample consists of all the people among the 60,000 households surveyed.

c. The raw data consist of all the information collected in the survey.

d. The sample statistics summarize the raw data for the sample. In this case, the relevant sam- ple statistic is the percentage of people in the sample who are actively seeking employment. (The Labor Department also calculates similar sample statistics for subgroups in the popula- tion, such as the percentages of teenagers, men, women, and veterans who are unemployed.)

e. The population parameters are the characteristics of the entire population that correspond to the sample statistics. In this case, the relevant population parameter is the actual unem- ployment rate. Note that the Labor Department does not actually measure this population parameter, since data are collected only for the sample and then are used to estimate the population parameter.

How Do Sample Statistics Relate to Population Parameters? Suppose Nielsen finds that 31% of the people in the 5,000 homes in its sample watched the Super Bowl. This “31%” is a sample statistic, because it characterizes the sample. But what

IS B

N : 0-536-88221-5

1.1 What Is/Are Statistics? 5

Nielsen really wants to know is the corresponding population parameter, which is the percent- age of all Americans who watched the Super Bowl.

There is no way for Nielsen researchers to know the exact value of the population parameter, because they’ve studied only a sample. However, Nielsen researchers hope that they’ve done their work so that the sample statistic is a good estimate of the population parameter. In other words, they would like to conclude that because 31% of the sample watched the Super Bowl, approximately 31% of the population also watched the Super Bowl. One of the primary purposes of statistics is to help researchers assess the validity of this type of conclusion.

Suppose Nielsen concludes that 30% of Americans watched the Super Bowl. How many people does this represent? (The population of the United States is approximately 300 million.)

Statistical science provides methods that enable researchers to determine how well a sample statistic estimates a population parameter. For example, results from surveys or opinion polls are usually quoted along with something called the margin of error. By adding and subtracting the margin of error from the sample statistic, we find a range of values, or confidence interval, that is likely to contain the population parameter. In most cases, the margin of error is defined so that we can have 95% confidence that this range contains the population parameter. We’ll discuss the precise meaning of “likely” and “95% confidence” in Chapter 8, but for now you might find an explanation given by the New York Times useful (Figure 1.1). In the case of the Nielsen ratings, the margin of error is about 1 percentage point. Therefore, if 31% of the sample was watching the Super Bowl, then we can be 95% confident that the range 30% to 32% contains the actual percentage of the population watching the Super Bowl.

The latest New York Times/CBS News Poll of New York State is based on telephone interviews conducted Oct. 23 to Oct. 28 with 1,315 adults throughout the state. Of those, 1,026 said they were registered to vote. Interviews were conducted in either English or Spanish.

In theory, in 19 cases out of 20 the results based on such samples will differ by no more than three percentage points in either direction from what would have been obtained by seeking out all adult residents of New York State. For smaller subgroups, the potential sampling error is larger.

How the Poll Was Conducted

Figure 1.1 The margin of error in a survey or opinion poll usually describes a range that is likely (with 95% confidence, meaning in 19 out of 20 cases) to contain the population parameter. This excerpt from the New York Times explains a margin of error of 3 percentage points.

One of the most remarkable findings of statistical science is that it is possible to get meaningful results from surprisingly small samples. Nevertheless, larger sample sizes are better (when they are feasible), because the margin of error is generally smaller for larger sam- ples. For example, the margin of error for a 95% confidence interval in a well-conducted poll is typically about 5 percentage points for a sample size of 400, but drops to 3 percentage points for a sample size of 1,000 and to 1 percentage point for a sample of 10,000. (See Chapter 8 to understand how margins of error are calculated.)IS

B N

: 0 -5

36 -8

82 21

-5

6 Speaking of Statistics

EXAMPLE 3 People on Mars? The Pew Research Center for the People and the Press interviewed 1,546 adult Americans about their attitudes toward the future. Asked whether humans would land on Mars within the next 50 years, 76% of these 1,546 people said either definitely yes or probably yes. The margin of error for the poll was 3 percentage points. Describe the population and the sample for this sur- vey, and explain the meaning of the sample statistic of 76%. What can we conclude about the percentage of the population that thinks humans will land on Mars within the next 50 years?

Solution The population is all adult Americans and the sample consists of the 1,546 people who were interviewed. The sample statistic of 76% is the actual percentage of people in the sam- ple who answered that humans would definitely or probably land on Mars in the next 50 years. The 76% sample statistic and the margin of error of 3 percentage points tell us that the range of values

is likely (with 95% confidence) to contain the population parameter, which in this case is the true percentage of all adult Americans who think humans will definitely or probably land on Mars within the next 50 years.

EXAMPLE 4 Cloning Humans? The same Pew survey also asked people whether they believed that humans would be cloned within the next 50 years. On this question, 51% answered either definitely yes or probably yes. Again, the margin of error was 3 percentage points. Can we be confident that a majority of adult Americans think that humans will be cloned in the next 50 years?

Solution No. To find the range of values likely to contain the percentage of all adult Americans who think human cloning will definitely or probably occur, we add and subtract the margin of error of 3 percentage points from the sample statistic of 51%. This gives a range of values from 48% to 54%. Because this range includes values on both sides of 50%, we cannot be confident that the majority (that is, greater than 50%) of adult Americans think that humans will be cloned in the next 50 years.

Look for a report on an opinion poll in this week’s news. Does the report give a margin of error? What does it mean in this case?

to 76% + 3% = 79% from 76% - 3% = 73%

Definition

The margin of error in a statistical study is used to describe the range of values, or confidence interval, likely to contain the population parameter. We find this confidence interval by adding and subtracting the margin of error from the sample statistic obtained in the study. That is, the range of values likely to contain the population parameter is

The margin of error is usually defined to give a 95% confidence interval, meaning that 95% of samples of the size used in the study would contain the actual population parameter (and 5% would not).

to (sample statistic + margin of error) from (sample statistic - margin of error)

By the Way .. . A clone is an exact genetic copy of its parent. For example, a clone of you would be genetically identical to you, but would be born as a baby and have different life experi- ences than you. The first successful cloning of an adult mammal came in 1997, when Ian Wilmut and his colleagues in Scotland cloned a sheep. The clone, named Dolly, has no father, since all her genes came from the mother. Scientists have now cloned many other mammals.

IS B

N : 0-536-88221-5

1.1 What Is/Are Statistics? 7

Putting It All Together: The Process of a Statistical Study The process used by Nielsen Media Research is similar to that used in many statistical studies. Figure 1.2 and the box below summarize the basic steps in a statistical study. Keep in mind that these steps are somewhat idealized, and the actual steps may differ from one study to another. Moreover, the details hidden in the basic steps are critically important. For example, a poorly chosen sample in Step 2 can render the entire study meaningless, and great care must be taken in inferring conclusions about a population from results found for the much smaller sample of that population.

Basic Steps in a Statistical Study

Step 1. State the goal of your study precisely; that is, determine the population you want to study and exactly what you’d like to learn about it.

Step 2. Choose a sample from the population. (Be sure to use an appropriate sampling technique, as discussed in the next section.)

Step 3. Collect raw data from the sample and summarize these data by finding sample statistics of interest.

Step 4. Use the sample statistics to make inferences about the population.

Step 5. Draw conclusions; determine what you learned and whether you achieved your goal.

EXAMPLE 5 Identifying the Steps Consider the Pew Research Center survey described in Examples 3 and 4. Identify how researchers applied the five basic steps in a statistical study.

Solution The steps apply as follows.

1. The researchers had a goal of learning about specific attitudes of Americans toward the future. They chose adult Americans as the population, deliberately leaving out children.

2. They chose 1,546 adult Americans for their sample. Although we are not told how the sam- ple was drawn, we will assume that it was drawn so that the 1,546 adult Americans are typ- ical of the entire adult American population.

POPULATION SAMPLE

POPULATION PARAMETERS

SAMPLE STATISTICS

START

2. Draw from population.

1. Identify goals.

5. Draw conclusions. 3. Collect raw data and summarize.

4. Make inferences about population.

Figure 1.2 The process of a statistical study.

By the Way .. . Statisticians often divide their subject into two major branches: descriptive statistics, which deals with describing raw data in the form of graphics and sample statistics, and inferential statistics, which deals with inferring (or estimating) population parameters from sample data. In this book, Chapters 2 through 5 primarily cover descriptive statistics, while Chapters 6 through 10 focus on inferential statistics.

IS B

N : 0

-5 36

-8 82

21 -5

8 Speaking of Statistics

3. They collected the raw data by asking carefully chosen questions of the people in the sam- ple. The raw data are the individual responses to the questions. They summarized these data with sample statistics, such as the overall percentages of people in the sample who answered yes or no to each question.

4. Techniques of statistical science allowed the researchers to infer population characteristics. In this case, the inference consisted of estimating certain population parameters and calcu- lating the margins of error.

5. By making sure that the study was conducted properly and interpreting the estimates of the population parameters, the researchers drew overall conclusions about Americans’ attitudes toward the future.

Statistics: Decisions for an Uncertain World The Nielsen ratings and most of the examples we’ve discussed so far involve surveys or polls. However, the subject of statistics encompasses much more, including experiments designed to test new medical treatments, analyses of the dangers of global warming, and even assessments of the value of a college education. Indeed, it is fair to say that the primary purpose of statis- tics is to help us make good decisions whenever we are confronted with a variety of possible options.

CASE STUDY The Salk Polio Vaccine

If you had been a parent in the 1940s or 1950s, one of your greatest fears would have been the disease known as polio. Each year during this long polio epidemic, thousands of young children were paralyzed by the disease. In 1954, a large experiment was conducted to test the effectiveness of a new vaccine created by Dr. Jonas Salk (1914–1995). The experiment involved a sample of 400,000 children chosen from the population of all children in the United States. Half of these 400,000 children received an injection of the Salk vaccine. The other half received an injection that contained only salt water. (The salt water injection was a placebo; see Section 1.3.) Among the children receiving the Salk vaccine, only 33 contracted polio. In contrast, there were 115 cases of polio among the children who did not get the Salk vaccine. Using techniques of statistical science that we’ll study later, the researchers concluded that the vaccine was effective at preventing polio. They therefore decided to launch a major effort to improve the Salk vaccine and distribute it to the population of all children. As a result, children in the United States and other developed countries began to get the vaccine routinely, and the horror of polio is now largely a thing of the past.

By the Way .. . Polio quickly became rare in the United States after the development of the Salk vaccine, but it remained common in less-developed coun- tries. A global effort to vaccinate children against polio began in 1998 and has achieved great success, though it has not yet reached its goal of completely eradicating the disease.

The Purpose of Statistics

Statistics has many uses, but perhaps its most important purpose is to help us make good decisions about issues that involve uncertainty.

This purpose will be clear in most of the case studies and examples we consider in this book, but occasionally we’ll have to discuss a bit of theory that may seem somewhat abstract at first. If you keep the overall purpose of statistics in mind, you’ll be rewarded in the end when you see how the theory helps us understand our world. The following case study will give you a taste of what lies ahead. It involves several important theoretical ideas that led to one of the 20th century’s greatest accomplishments in public health.

The greatest reward for doing is the opportunity to do more.

—Jonas Salk

IS B

N : 0-536-88221-5

1.1 What Is/Are Statistics? 9

Section 1.1 Exercises

Statistical Literacy and Critical Thinking

1. Population and Sample. What is a population, what is a sample, and what is the difference between them?

2. Statistic and Statistics. Suppose that, in a discussion, one person refers to baseball statistics and another refers to the use of statistics in showing that a particular drug is an effective treatment. Do both uses of the term statistics have the same meaning? If not, how do they differ?

3. Statistics and Parameters. What is a sample statistic, what is a population parameter, and what is the difference between them?

4. Margin of Error. What is the margin of error in a statisti- cal study and why is it important?

Does It Make Sense? For Exercises 5–10, decide whether the statement makes sense (or is clearly true) or does not make sense (or is clearly false). Explain clearly; not all of these state- ments have definitive answers, so your explanation is more important than your chosen answer.

5. Statistics and Parameters. My professor conducted a statistical study in which he was unable to measure any sample statistics, but he succeeded in determining the population parameters with a very small margin of error.

6. Poor Poll. A poll conducted two weeks before the elec- tion found that Smith would get 70% of the vote, with a margin of error of 3%, but he ended up losing the election anyway.

7. Poll Certainty. There is no doubt that Johnson won the election, because an exit poll showed that she received 54% of the vote and the margin of error was only 3 percentage points.

8. Beating Nielsen. A new startup company intends to compete with Nielsen Media Research by providing data with a larger margin of error for the same price.

9. Depression Sample. The goal of my study is to learn about depression among people who have suffered through a family tragedy, so I plan to choose a sample from the population of everyone who has been sick in the past month.

10. New Product. Our market research department surveyed 1,000 consumers on their attitudes toward our new prod- uct. Because the people in this sample were so enthusias- tic about their desire to purchase the product, we decided to roll out a nationwide advertising campaign.

Concepts and Applications

Sample, Population, Statistic, and Parameter. Exercises 11–14 each describe a statistical study. In each case, identify the sample, the population, the sample statistic, and the popula- tion parameter.

11. Stem Cell Research. In a Newsweek poll conducted by Princeton Survey Research Associates International, 1,002 adults in the United States were asked whether they favor or oppose the use of “federal tax dollars to fund medical research using stem cells obtained from human embryos.” Among the respondents, 48% said that they were in favor.

12. Friday the 13th. In a Gallup poll of 1,236 adults, 9% said that bad luck happens on a Friday that is the 13th day of a month.

13. Galactic Distances. Astronomers determine the distance to a far-away galaxy by measuring the distances of just a few stars within it and taking the mean of these distance measurements.

14. Allergy Drug. Nasonex is a drug used to treat allergy symptoms. In a test of Nasonex, 374 children aged 3 to 11 years were treated with 100 micrograms of Nasonex, and 17% of them experienced headaches.

Identifying the Range of Values. In Exercises 15–18, use the given statistic and margin of error to identify the range of values (confidence interval) likely to contain the true value of the population parameter.

15. Stem Cell Research. In a Newsweek poll conducted by Princeton Survey Research Associates International, 1,002 adults in the United States were asked whether they favor or oppose the use of “federal tax dollars to fund medical research using stem cells obtained from human embryos.” Based on the poll results, it is estimated that 48% of adults are in favor, with a margin of error of 3 percentage points.

16. ANWR Oil. In a CBS News/New York Times poll, 1,241 adults in the United States were asked whether they approve or disapprove of drilling for oil and natural gas in Alaska’s Arctic National Wildlife Refuge (ANWR). Based on the poll results, it is estimated that 45% of adults disapprove, with a margin of error of 3 percentage points.

17. Body Mass Index. Forty adult men in the United States are randomly selected and measured for their body mass index (BMI). Based on that sample, it is estimated that theIS

B N

: 0 -5

36 -8

82 21

-5

10 Speaking of Statistics

average (mean) BMI for men is 26.0, with a margin of error of 3.4.

18. Body Temperatures. Researchers randomly selected 106 adults and measured their body temperatures. Based on that sample, it is estimated that the average (mean) body temperature is 98.2°F, with a margin of error of 0.1°F.

19. Interpreting Poll Results. A poll is conducted the day before a state election for Senator. There are only two can- didates running for this office. The poll results show that 58% of the voters favor the Republican candidate, with a margin of error of 3 percentage points. Should the Repub- lican expect to win? Why or why not?

20. Interpreting Taste Test Results. In a nationally televised live taste test, a sample of regular drinkers of Budweiser was given blind samples of Michelob and Schlitz. Based on the results, we estimate that 52% of such subjects had a prefer- ence for Michelob, with a margin of error of 10%. Can we conclude that the majority of Budweiser drinkers prefer Michelob when given tastes of Michelob and Schlitz? Why or why not?

21. Do People Lie About Voting? In a survey of 1,002 people, 701 (or 70%) said that they voted in a particular presidential election (based on data from ICR Research Group). The margin of error for this survey was 3 percent- age points. However, actual voting records show that only 61% of all eligible voters actually did vote. Does this imply that people lied when they responded in the survey? Explain.

22. Why the Discrepancy? In an Eagleton Institute poll, surveyed men were asked if they agreed with this state- ment: “Abortion is a private matter that should be left to the woman to decide without government intervention.” Among the men who were interviewed by women, 77% agreed with the statement. Among the men who were interviewed by men, 70% agreed with the statement. Assuming that the discrepancy is significant, how might that discrepancy be explained?

Interpreting Real Studies. For each of Exercises 23–26, do the following:

a. Based on the given information, state what you think was the goal of the study. Identify a possible population and the population parameter of interest.

b. Briefly describe the sample, raw data, and sample statistic for the study.

c. Based on the sample statistic and the margin of error, identify the range of values (confidence interval) likely to contain the population parameter of interest.

23. Changing Careers. In a Korn/Ferry International survey of 1,733 executives, it was found that 51% of them said that if they could start their careers again, they would choose a different field. The margin of error was 3 percentage points.

24. Superpowers. In a Caravan survey of 1,018 adults, it was found that 11% would choose the ability to be invisi- ble as the superpower that they would most prefer. The margin of error was 3.2 percentage points.

25. Unemployment Rate. Based on a recent survey of adults in 60,000 households, the U.S. Labor Department reported an unemployment rate of 4.6%. The margin of error was 0.2 percentage point.

26. For a Rainy Day. A Roper Organization survey of 2,000 adults in the United States showed that 64% of those surveyed kept money in regular savings accounts. The margin of error for the survey was 2.0 percentage points.

Five Steps in a Study. Describe how you would apply the five basic steps in a statistical study (as listed in the box on p. 7) to the issues in Exercises 27–30.

27. Cell Phones and Driving. You want to determine the percentage of licensed drivers who used a cell phone at least once while they were driving during the last week.

28. Credit Scores. FICO (Fair Isaac Corporation) scores are routinely used to rate the quality of consumer credit. You want to determine the average (mean) FICO score of all adults in the United States.

29. Passenger Weight. Recognizing that overloading commercial aircraft would lead to unsafe flights, you want to determine the average (mean) weight of airline passengers.

30. Pacemaker Batteries. Because the batteries used in heart pacemakers are so critically important, you want to determine the average (mean) length of time that such bat- teries last before failure.

Projects for the Internet and Beyond

For useful links, select “Links for Internet Projects” for Chapter 1 at www.aw.com/bbt.

31. Current Nielsen Ratings. Find the Nielsen ratings for the past week. What were the three most popular televi- sion shows? Explain the meaning of the “rating” and the “share” for each show.

32. Nielsen Methods. Nielsen Media Research frequently revises the details of its data collection methods. Visit its

IS B

N : 0-536-88221-5

1.2 Sampling 11

Web site and read about its current strategies for rating television shows. Summarize these methods in a bulleted list format.

33. Comparing Airlines. The U.S. Department of Trans- portation routinely publishes on-time performance, lost baggage rates, and other statistics for different airline companies. Find a recent example of such statistics. Based on what you find, is it fair to say that any particu- lar airline stands out as better or worse than others? Explain.

34. Labor Statistics. Use the Bureau of Labor Statistics Web site to find monthly unemployment rates over the past 12 months. If you assume that the monthly survey has a margin of error of about 0.2 percentage point, has there been a noticeable change in the unemployment rate over the past year? Explain.

35. Statistics and Safety. Identify a study that has been done (or should be done) to improve the safety of car dri- vers and passengers. Briefly describe the importance of statistics to the study.

36. Pew Research Center. The Pew Research Center for the People and the Press studies public attitudes toward the press, politics, and public policy issues. Go to its Web site and find the latest survey about attitudes. Select a particular recent survey, write a summary of what was surveyed, how the survey was conducted, and what was found.

37. Statistics in the News. Identify three stories from the past week that involve statistics in some way. In each case, write a brief statement describing the role of statis- tics in the story.

38. Statistics in Your Major. Write a brief description of some ways in which you think that the science of statistics can be used in your major field of study. (If you have not yet selected a major, answer the question for a major that you are considering.)

39. Statistics and Entertainment. The Nielsen ratings are well known for their role in gauging television viewing. Identify another way that statistics are used in the entertainment industry. Briefly describe the role of statistics in this application.

40. Statistics in Sports. Choose a sport and describe at least three different statistics commonly tracked by par- ticipants in or spectators of the sport. In each case, briefly describe the importance of the statistic to the sport.

41. Economic Statistics. The government regularly pub- lishes many different economic statistics, such as the un- employment rate, the inflation rate, and the surplus or deficit in the federal budget. Study recent newspapers and identify five important economic statistics. Briefly explain the purpose of each of these five statistics.

1.2 Sampling

The only way to know the true value of a population parameter is to observe every mem-ber of the population. For example, to learn the exact mean height of all students atyour school, you’d need to measure the height of every student. A collection of data from every member of a population is called a census. Unfortunately, conducting a census is often impractical. In some cases, the population is so large that it would be too expensive or time-consuming to collect data from every member. In other cases, a census may be ruled out because it would interfere with a study’s overall goals. For example, a study designed to test the quality of candy bars before shipping could not involve a census because that would mean testing a piece of every candy bar, leaving none intact to sell.

Definition

A census is the collection of data from every member of a population.

Fortunately, most statistical studies can be done without going to the trouble of conduct- ing a census. Instead of collecting data from every member of the population, we collect data

Not everything that can be counted counts, and not everything that counts can be counted.

—Albert Einstein

IS B

N : 0

-5 36

-8 82

21 -5

12 Speaking of Statistics

By the Way .. . Many medical studies are experi- ments designed to test whether a new drug is effective. In an article published in the Journal of the American Medical Association, the authors found that studies with positive results (the drug is effec- tive) are more likely to be pub- lished than studies with negative results (the drug is not effective). This “publication bias” tends to make new drugs, as a group, seem more effective than they really are.

from a sample and use the sample statistics to make inferences about the population. Of course, the inferences will be reasonable only if the members of the sample represent the population fairly, at least in terms of the characteristics under study. That is, we seek a representative sample of the population.

Definition

A representative sample is a sample in which the relevant characteristics of the sample members are generally the same as the characteristics of the population.

EXAMPLE 1 A Representative Sample for Heights Suppose you want to determine the mean height of all students at your school. Which is more likely to be a representative sample for this study: the men’s basketball team or the students in your statistics class?

Solution The men’s basketball team is not a representative sample for a study of height, both because it consists only of men and because basketball players tend to be taller than aver- age. The mean height of the students in your statistics class is much more likely to be close to the mean height of all students, so the members of your class make a more representative sam- ple than the members of the men’s basketball team.

Bias Imagine that, for the 5,000 homes in its sample, Nielsen chose only homes in which the pri- mary wage earners worked a late-night shift. Because late-night workers aren’t home to watch late-night television, Nielsen would find late-night shows to be unpopular among the homes in this sample. Clearly, this sample would not be representative of all American homes, and it would be wrong to conclude that late-night shows were unpopular among all Americans. We say that such a sample is biased because the homes in the sample differed in a specific way from “typical” American homes. (In reality, Nielsen takes great care to avoid such obvious bias in the sample selection.) More generally, the term bias refers to any problem in the design or conduct of a statistical study that tends to favor certain results. We cannot trust the conclusions of a biased study.

Definition

A statistical study suffers from bias if its design or conduct tends to favor certain results.

Bias can arise in many ways. For example:

• A sample is biased if the members of the sample differ in some specific way from the members of the general population. In that case, the results of the study will reflect the unusual characteristics of the sample rather than the actual characteristics of the population.

• A researcher is biased if he or she has a personal stake in a particular outcome. In that case, the researcher might intentionally or unintentionally distort the true meaning of the data.

• The data set itself is biased if its values were collected intentionally or unintentionally in a way that makes the data unrepresentative of the population.

• Even if a study is done well, it may be reported in a biased fashion. For example, a graph representing the data may tell only part of the story or depict the data in a misleading way (see Section 3.4).

IS B

N : 0-536-88221-5

1.2 Sampling 13

Preventing bias is one of the greatest challenges in statistical research. Looking for bias is therefore one of the most important steps in evaluating a statistical study or media reports about a statistical study.

EXAMPLE 2 Why Use Nielsen? Nielsen Media Research earns money by charging television stations and networks for its ser- vices. For example, NBC pays Nielsen to provide ratings for its television shows. Why doesn’t NBC simply do its own ratings, instead of paying a company like Nielsen to do them?

Solution The cost of advertising on a television show depends on the show’s ratings. The higher the ratings, the more the network can charge for advertising—which means NBC would have a clear bias if it conducted its own ratings. Advertisers therefore would not trust ratings that NBC produced on its own. By hiring an independent source, such as Nielsen, NBC can provide information that advertisers are more likely to believe.

The fact that NBC pays Nielsen for its services might seem to give Nielsen a financial incentive to make NBC look good in the ratings. How does the fact that Nielsen also provides ratings for other networks help prevent it from being biased toward NBC?

Sampling Methods A good statistical study must have a representative sample. Otherwise the sample is biased and conclusions from the study are not trustworthy. Let’s examine a few common sampling meth- ods that, at least in principle, can provide a representative sample.

Simple Random Samples In most cases, the best way to obtain a representative sample is by choosing randomly from the population. A random sample is one in which every member of the population has an equal chance of being selected to be part of the sample. For example, you could obtain a random sam- ple by having everyone in a population roll a die and choosing those people who roll a 6. In con- trast, the sample would not be random if you chose everyone taller than 6 feet, because not everyone would have an equal chance of being selected.

In statistics, we usually decide in advance the sample size that is needed. With simple random sampling, every possible sample of a particular size has an equal chance of being selected. For example, to choose a simple random sample of 100 students from all the students in your school, you could assign a number to each student in your school and choose the sam- ple by drawing 100 of these numbers from a hat. As long as each student’s number is in the hat only once, every sample of 100 students has an equal chance of being selected. As a faster alternative to using a hat, you might choose the student numbers with the aid of a computer or calculator that has a built-in random number generator.

Look for the random number key on a calculator. (Nearly all scientific calculators have one.) What happens when you push it? How could you use the random number key to select a sample of 100 students?

Because simple random sampling gives every sample of a particular size the same chance of being chosen, it is likely to provide a representative sample, as long as it is large enough. The larger the simple random sample, the more likely it is to be representative of the population.IS

B N

: 0 -5

36 -8

82 21

-5

14 Speaking of Statistics

EXAMPLE 3 Telephone Book Sampling You want to conduct an opinion poll in which the population is all the residents in a town. Could you choose a simple random sample by randomly selecting names from the local tele- phone book?

Solution A sample drawn from a telephone book is not a simple random sample of the town population because phone books invariably are missing a lot of names, and anyone whose name is missing has no chance of being selected. For example, the phone book will be missing names when two or more people share the same phone number but have only one listing, when people choose to have an unlisted phone number or to rely exclusively on a cell phone, or when people (such as the homeless) don’t have a telephone.

Systematic Sampling Simple random sampling is effective, but in many cases we can get equally good results with a simpler technique. Suppose you are testing the quality of microchips produced by Intel. As the chips roll off the assembly line, you might decide to test every 50th chip. This ought to give a representative sample because there’s no reason to believe that every 50th chip has any special characteristics compared to other chips. This type of sampling, in which we use a system such as choosing every 50th member of a population, is called systematic sampling.

EXAMPLE 4 Museum Assessment When the National Air and Space Museum wanted to test possible ideas for a new solar system exhibit, a staff member interviewed a sample of visitors selected by systematic sampling. She interviewed a visitor exactly every 15 minutes, choosing whoever happened to enter the current solar system exhibit at that time. Why do you think she chose systematic sampling rather than simple random sampling? Was systematic sampling likely to produce a representative sample in this case?

Solution Simple random sampling might occasionally have selected two visitors so soon after each other that the staff member would not have had time to interview each of them. The systematic process of choosing a visitor every 15 minutes prevented this problem from arising. Because there’s no reason to think that the people entering at a particular moment are any dif- ferent from those who enter a few minutes earlier or later, this process is likely to give a repre- sentative sample of the population of visitors during the time of the sampling.

EXAMPLE 5 When Systematic Sampling Fails You are conducting a survey of students in a co-ed dormitory in which males are assigned to odd-numbered rooms and females are assigned to even-numbered rooms. Can you obtain a representative sample when you choose every 10th room?

Solution No. If you start with an odd-numbered room, every 10th room will also be odd- numbered (such as room numbers 3, 13, 23, . . .). Similarly, if you start with an even-numbered room, every 10th room will also be even-numbered. You will therefore obtain a sample consist- ing of either all males or all females, neither of which is representative of the co-ed population.

Suppose you chose every 5th room, rather than every 10th room, in Example 5. Would the sample then be representative?

By the Way .. . The sampling described in Example 4 was undertaken prior to the construction of the Voyage Scale Model Solar System, a per- manent scale model exhibit that stretches along the National Mall from the National Air and Space Museum to the Smithsonian Castle. The photo below shows the son of one of the exhibit creators (also an author of this textbook) touching the scale model Sun.

IS B

N : 0-536-88221-5

1.2 Sampling 15

Convenience Samples Systematic sampling is easier than simple random sampling, but may also be impractical in many cases. For example, suppose you want to know the proportion of left-handed students at your school. It would take great effort to select a simple random sample or a systematic sample, because both require drawing from all the students in the school. In contrast, it would be easy to use the students in your statistics class as your sample—you could just ask the left-handed students to raise their hands. This type of sample is called a convenience sample because it is chosen for convenience rather than by a more sophisticated procedure. For trying to find the proportion of left-handed people, the convenience sample of your statistics class is probably fine; there is no reason to think that there would be a different proportion of left-handed students in a statistics class than anywhere else. But if you were trying to determine the proportions of students with different majors, this sample would be biased because some majors require a statistics course and others do not. In general, conve- nience sampling tends to be more prone to bias than most other forms of sampling.

EXAMPLE 6 Salsa Taste Test A supermarket wants to decide whether to carry a new brand of salsa, so it offers free tastes at a stand in the store and asks people what they think. What type of sampling is being used? Is the sample likely to be representative of the population of all shoppers?

Solution The sample of shoppers stopping for a taste of the salsa is a convenience sample because these people happen to be in the store and are willing to try the new product. (This type of convenience sample, in which people choose whether or not to be part of the sample, is also called a self-selected sample. We will study self-selected samples further in Section 1.4.) This sample is unlikely to be representative of the population of all shoppers, because different types of people may shop at different times (for example, stay-at-home parents are more likely to shop at mid-day than are working parents) and only people who like salsa are likely to par- ticipate. The data may be still be useful, however, because the opinions of people who like salsa are probably the most important ones in this case.

Cluster Samples Cluster sampling involves the selection of all members in randomly selected groups, or clusters. Imagine that you work for the Department of Agriculture and wish to determine the percentage of farmers who use organic farming techniques. It would be difficult and costly to collect a simple random sample or a systematic sample because either would require visit- ing many individual farms that are located far from one another. A convenience sample of farmers in a single county would be biased because farming practices vary from region to region. You might therefore decide to select a few dozen counties at random from across the United States and survey every farmer in each of those counties. We say that each county contains a cluster of farmers, and the sample consists of every farmer within the randomly selected clusters.

EXAMPLE 7 Gasoline Prices You want to know the mean price of gasoline at gas stations located within a mile of rental car locations at airports. Explain how you might use cluster sampling in this case.

Solution You could randomly select a few airports around the country. For these airports, you would check the gasoline price at every gas station within a mile of the rental car location.

IS B

N : 0

-5 36

-8 82

21 -5

16 Speaking of Statistics

Stratified Samples Suppose you are conducting a poll to predict the outcome of the next U.S. presidential election. The population under study is all likely voters, so you might choose a simple random sample from this population. However, because presidential elections are decided by electoral votes cast on a state-by-state basis, you’ll get a better prediction if you determine voter preferences within each state. Your overall sample should therefore consist of separate random samples from each of the 50 states. In statistical terminology, the populations of the 50 states represent subgroups, or strata, of the total population. Because your overall sample consists of randomly selected members from each stratum, you’ve used stratified sampling.

EXAMPLE 8 Unemployment Data The U.S. Labor Department surveys 60,000 households each month to compile its unemploy- ment report (see Example 2 in Section 1.1). To select these households, the Department first groups cities and counties into about 2,000 geographic areas. It then randomly selects house- holds to survey within these geographic areas. How is this an example of stratified sampling? What are the strata? Why is stratified sampling important in this case?

Solution The unemployment survey is an example of stratified sampling because it first breaks the population into subgroups. The subgroups, or strata, are the people in the 2,000 geographic regions. Stratified sampling is important in this case because unemployment rates are likely to differ in different geographic regions. For example, unemployment rates in rural Kansas may be very different from those in Silicon Valley. By using stratified sampling, the Labor Department ensures that its sample fairly represents all geographic regions.

Summary of Sampling Methods The following box and Figure 1.3 summarize the five sampling methods we have discussed. No single method is “best,” as each one has its uses. (Some studies even combine two or more types of sampling.) But regardless of how a sample is chosen, keep in mind the following three key ideas:

• A study can be successful only if the sample is representative of the population. • A biased sample is unlikely to be a representative sample. • Even a well-chosen sample may still turn out to be unrepresentative just because of bad

luck in the actual drawing of the sample.

By the Way .. . As mandated by the U.S. Constitu- tion, voting for the President is actually done by a small group of people called electors. Each state may select as many electors as it has members of Congress (count- ing both Senators and Representa- tives). When you cast a ballot for President, you actually cast a vote for your state’s electors, each of whom has promised to vote for a particular presidential candidate. The electors cast their votes a few weeks after the general election.

Common Sampling Methods

• Simple random sampling: We choose a sample of items in such a way that every sample of the same size has an equal chance of being selected.

• Systematic sampling: We use a simple system to choose the sample, such as selecting every 10th or every 50th member of the population.

• Convenience sampling: We use a sample that happens to be convenient to select.

• Cluster sampling: We first divide the population into groups, or clusters, and select some of these clusters at random. We then obtain the sample by choosing all the members within each of the selected clusters.

• Stratified sampling: We use this method when we are concerned about differences among subgroups, or strata, within a population. We first identify the strata and then draw a random sample within each stratum. The total sample consists of all the samples from the individual strata.

IS B

N : 0-536-88221-5

1.2 Sampling 17

Every sample of the same size has an equal chance of being selected. Computers are often used to generate random numbers.

Simple Random Sampling:

Partition the population into at least two strata, then draw a sample from each.

Stratified Sampling:

Systematic Sampling:

Use results that are readily available.

Convenience Sampling:

Hey! Do you support

the death penalty?

Select every kth member.

Divide the population into clusters, randomly select some of those clusters, then choose all members of the selected clusters.

Cluster Sampling:

Election precincts in Carson County

21 3

4 6 7 8 9

10 11 12 13 14 15 17

18 19 20 21 22 23 24 25

26 27 29 30 31 32

Interview all voters in shaded precincts.

Figure 1.3

IS B

N : 0

-5 36

-8 82

21 -5

18 Speaking of Statistics

EXAMPLE 9 Sampling Methods Identify the type of sampling used in each of the following cases.

a. The apple harvest from an orchard is collected in 1,200 baskets. An agricultural inspector randomly selects 25 baskets and then checks every apple in each of these baskets for worms.

b. An educational researcher wants to know whether, at a particular college, men or women tend to ask more questions in class. Of the 10,000 students at the college, she interviews 50 randomly selected men and 50 randomly selected women.

c. In trying to learn whether planetary systems are common, astronomers conduct a survey by looking for planets among 100 of the nearest stars.

d. To determine who will win autographed footballs, a computer program randomly selects the ticket numbers of 11 people in a stadium filled with people.

Solution

a. The apple inspection is an example of cluster sampling because the inspector begins with a randomly selected set of clusters (baskets) and then checks every apple in the selected clusters.

b. The groups of men and women represent two different strata for this study, so this is an example of stratified sampling.

c. The astronomers focus on nearby stars because they are easier to study, so this is an exam- ple of convenience sampling.

d. Because the computer selects the 11 ticket numbers at random, every ticket number has an equal chance of being chosen. This is an example of simple random sampling.

Section 1.2 Exercises

Statistical Literacy and Critical Thinking

1. Census and Sample. What is a census, what is a sample, and what is the difference between them?

2. Biased Sample. In a pre-election survey, a sample is drawn randomly from a list of registered Democrats. Is there anything wrong with this sampling method?

3. Cluster and Stratified Sampling. Cluster sampling and stratified sampling both involve selecting subjects in subgroups of the population. What is the difference between these two types of sampling?

4. Sample of Students. One of the authors once con- ducted a study by recording whether each student in his class was right-handed. The objective was to form a con- clusion about the proportion of college students who are right-handed. What type of sample was obtained? Is this sample likely to be biased? Why or why not?

Does It Make Sense? For Exercises 5–8, decide whether the statement makes sense (or is clearly true) or does not make sense (or is clearly false). Explain clearly; not all of these state- ments have definitive answers, so your explanation is more important than your chosen answer.

5. Graduation Age. For a statistics class project, I con- ducted a census to determine the mean age of students when they earn their bachelor’s degree.

6. Convenience Sample. For a statistics class project, I used a convenience sample, but the results may still be meaningful.

7. Biased Sample. The study must have been biased, because it concluded that 75% of Americans are more than 6 feet tall.

8. Death Row. There are currently 3,366 convicts on death row (based on 2007 data from the Bureau of Justice Statis- tics). We obtained a simple random sample of those con- victs by compiling a numbered list, then using a computer to randomly generate 20 numbers between 1 and 3,366, and then selecting the convicts who corresponded to the generated numbers.

Concepts and Applications

Census. In Exercises 9–12, determine whether a census is practical in the situation described. Explain your reasoning.

9. Laker Heights. You want to determine the mean height of all basketball players on the LA Lakers team.

IS B

N : 0-536-88221-5

1.2 Sampling 19

10. High School Heights. You want to determine the mean height of all high school basketball players in the United States.

11. IQ Scores. You want to determine the mean IQ score of all statistics instructors in the United States.

12. Instructor Ages. You want to determine the mean age of all statistics instructors at the University of Colorado.

Representative Samples? In Exercises 13–16, identify the sample, population, and sampling method. Then comment on whether you think it is likely that the sample is representative of the population.

13. Senate Terms. A political scientist randomly selects 4 of the 100 Senators currently serving in Congress and then finds the lengths of time that they have served.

14. Super Bowl. During the Super Bowl game, Nielsen Media Research conducts a survey of 5,108 randomly selected households and finds that 44% of them have tele- vision sets tuned to the Super Bowl.

15. Cloning. In a Gallup poll of 1,012 randomly selected adult Americans, 89% said that cloning of humans should not be allowed.

16. Mail Survey. A graduate student at the University of Newport conducts a research project about how adult Americans communicate. She begins with a survey mailed to 500 of the adults she knows. She asks them to mail back a response to this question: “Do you prefer to use e-mail or snail mail (the U.S. Postal Service)?” She gets back 65 responses, with 42 of them indicating a preference for snail mail.

Evaluating Sample Choices. Exercises 17 and 18 each des- cribe the goal of a study and then offer you four possible sam- ples. In each case, decide which sample is most likely to be a representative sample, and explain why. Then explain why each of the other choices is not likely to be a representative sample for the study.

17. Credit Card Debt. You want to determine the mean amount of credit card debt owed by adult consumers in Florida.

a. The Florida drivers who own and have registered Land Rover vehicles

b. The first 1,000 Florida residents listed in the Fort Lauderdale phone book

c. The first 1,000 Florida residents in a complete list of all Florida telephone numbers

d. The Florida residents who mail back a survey printed in the Miami Herald

18. California Voters. You want to conduct a survey to determine the proportion of eligible voters in California likely to vote for the Democratic presidential candidate in the next election.

a. All eligible voters in San Diego County

b. All eligible voters in the city of Sonoma

c. All eligible voters who respond to an America OnLine (AOL) Internet survey

d. Every 1000th person on a complete list of all eligible voters in California

Bias. Are there sources of bias in the situations described in Exercises 19–22? Explain.

19. Movie Critic. A film critic for ABC News gives her opin- ion of the latest movie from Disney, which also happens to own ABC.

20. Car Reviews. Consumer Reports magazine, which does not accept free products or advertising from anyone, prints a review of new cars.

21. GMO Soybeans. Monsanto hires independent univer- sity scientists to determine whether its new, geneti- cally engineered soybean poses any threat to the environment.

22. Drug Study Funding. The Journal of the American Medical Association prints an article evaluating a drug, and some of the physicians who wrote the article received funding from the pharmaceutical company that produces the drug.

Sampling Methods. In Exercises 23–38, identify each sample as a simple random sample, systematic sample, con- venience sample, stratified sample, or cluster sample. In each case, state whether you think the sampling method is likely to yield a representative sample or a biased sample, and explain why.

23. Clinical Trial. In phase II testing of a new drug designed to increase the red blood cell count, a researcher obtains envelopes with the names and addresses of all treated subjects. She wants to increase the dosage in a sub–sample of 12 subjects, so she thor- oughly mixes all of the envelopes in a bin and then pulls 12 of those envelopes to identify the subjects to be given the increased dosage.

24. Sobriety Checkpoint. One of the authors was an observer at a police sobriety checkpoint at which every 5th driver was stopped and interviewed.

25. Exit Polls. On the day of the presidential election, the news media organize an exit poll in which specific pollingIS

B N

: 0 -5

36 -8

82 21

-5

20 Speaking of Statistics

stations are randomly selected and all voters are surveyed as they leave the premises.

26. Education and Sports. A researcher for the Spalding athletic equipment company is studying the relationship between level of education and participation in any sport. He conducts a survey of 40 randomly selected golfers, 40 randomly selected tennis players, and 40 randomly selected swimmers.

27. Ergonomics. An engineering student measures the strength of fingers used to push buttons by testing family members.

28. Tax Cheating. An Internal Revenue Service researcher investigates cheating on income tax reports by survey- ing all waiters and waitresses at 20 randomly selected restaurants.

29. MTV Survey. A marketing expert for MTV is planning a survey in which 500 people will be randomly selected from each of the age groups 10–19, 20–29, and so on.

30. Credit Card Data. One of the authors surveyed all of his students to obtain sample data consisting of the number of credit cards students possess.

31. Fund Raising. Fundraisers for the College of Newport test a new telemarketing campaign by obtaining an alphabetical list of all alumni and calling every 100th name on that list.

32. Telephone Poll. In a Gallup poll of 1,059 adults, the inter- view subjects were selected by using a computer to ran- domly generate telephone numbers that were then called.

33. Market Research. A market researcher has partitioned all California residents into categories of unemployed, employed full-time, and employed part-time. She is sur- veying 50 people from each category.

34. Student Drinking. Motivated by the death of a student from binge drinking, the College of Newport conducts a study of student drinking by randomly selecting 10 differ- ent classes and interviewing all of the students in each of those classes.

35. Magazine Survey. People magazine chooses its “best- dressed celebrities” by compiling responses from readers who mail in a survey printed in the magazine.

36. Heart Transplants. A medical researcher at Johns Hopkins University obtains a numbered list of all patients waiting for a heart transplant, uses a computer to ran- domly generate 50 numbers, and then select the patients corresponding to the 50 numbers.

37. Quality Control. A sample of manufactured CDs is ob- tained by using a computer to randomly generate a num- ber between 1 and 1,000 for each CD and then selecting the CD if the generated number is 1,000.

38. Seat Belts. Every 500th seat belt is tested by stressing it until it fails.

Choosing a Sampling Method. For each of Exercises 39–42, suggest a sampling method that is likely to produce a representative sample. Explain why you chose this method over other methods.

39. Student Election. You want to predict the winner of an upcoming election for student body president.

40. Blood Type. You want to determine the percentage of people in this country in each of the four major blood groups (A, B, AB, and O).

41. Heart Deaths. You want to determine the percentage of deaths due to heart disease each year.

42. Mercury in Tuna. You want to determine the average mercury content of the tuna fish consumed by U.S. residents.

Projects for the Internet and Beyond

For useful links, select “Links for Internet Projects” for Chapter 1 at www.aw.com/bbt.

43. Public Opinion Poll. Use information available on the Web site of a polling organization, such as Gallup, Harris, Pew, or Yankelovich, to answer the following questions.

a. How exactly is a sample of subjects selected?

b. Based on what you have learned, do you think the poll results are reliable? If so, why? If not, why not?

44. Unemployment Sample. Use the Bureau of Labor Sta- tistics Web page to find details on how the Bureau chooses the sample of households in its monthly survey. Write a short summary of the procedure and why it is likely to yield a representative sample.

45. Selective Voting. The Academy Awards, the Heisman Trophy, and the New York Times “Bestseller List” are just three examples of cases in which selections are deter- mined by the votes of specially selected individuals. Pick one of these selection processes and describe who votes and how those people are chosen. Discuss sources of bias in the process.

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 21

46. Sampling in the News. Find a recent news report about a statistical study that you find interesting. Write a short summary of how the sample for the study was chosen, and briefly discuss whether you think the sam- ple was representative of the population under study.

47. Opinion Poll Sample. Find a recent news report about an opinion poll carried out by a news organiza- tion (such as Gallup, Harris, USA Today, New York Times, or CNN). Briefly describe the sample and how it was chosen. Was the sample chosen in a way that was likely to introduce bias? Explain.

48. Political Polls. Find results from a recent poll con- ducted by a political organization (such as the Re- publican or Democratic party or an organization that seeks to influence Congress on some particular issue). Briefly describe the sample and how it was chosen. Was the sample chosen in a way that was likely to introduce bias? Should you be more concerned about bias in such a poll than you would be in a poll conducted by a news organization? Explain.

1.3 Types of Statistical Studies

Statistical studies are conducted in many different ways. In all cases, the people, animals (orother living things), or objects chosen for the sample are called the subjects of the study. Ifthe subjects are people, it is common to refer to them as participants in the study.

You can observe a lot by just watching.

—Yogi Berra

Definition

The subjects of a study are the people, animals (or other living things), or objects chosen for the sample; if the subjects are people, they may also be called the participants in the study.

There are two basic types of statistical study: observational studies and experiments. In an observational study, we observe or measure specific characteristics while trying to be careful to avoid influencing or modifying the characteristics we are observing. The Nielsen ratings are an example of an observational study, because Nielsen uses its “people meters” to observe what the subjects are watching on TV, but does not try to influence what they watch.

Note that an observational study may involve activities that go beyond the usual defini- tion of observing. Measuring people’s weights requires interacting with them, as in asking them to stand on a scale. But in statistics, we consider these measurements to be observations because the interactions do not change people’s weights. Similarly, an opinion poll in which researchers conduct in-depth interviews is considered observational as long as the researchers attempt only to learn people’s opinions, not to change them.

In contrast, consider a medical study designed to test whether large daily doses of vita- min C help prevent colds. To conduct this study, the researchers must ask some people in the sample to take large doses of vitamin C every day. This type of statistical study is called an experiment. The purpose of an experiment is to study the effects of some treatment—in this case, large daily doses of vitamin C.

IS B

N : 0

-5 36

-8 82

21 -5

22 Speaking of Statistics

EXAMPLE 1 Type of Study Identify the study as an observational study or an experiment.

a. The Salk polio vaccine study (see the Case Study on page 8)

b. A poll in which people are asked for whom they plan to vote in the next election

Solution

a. The Salk vaccine study was an experiment because researchers tested a treatment—in this case, the vaccine—to see whether it reduced the incidence of polio.

b. The poll is an observational study because it attempts to determine voting preference but does not try to sway votes.

Identifying the Variables Statistical studies—whether observations or experiments—generally are attempts to measure what we call variables of interest. The term variable simply refers to an item or quantity that can vary or take on different values, and variables of interest are those we seek to learn about. For example, variables of interest in the Nielsen studies of television viewing habits include show being watched and number of viewers. The variable show being watched can take on dif- ferent values such as “Super Bowl,” “60 Minutes,” or “Lost.” The variable number of viewers depends on the popularity of a particular show. In essence, the raw data in any statistical study are the different values of the variables of interest.

In cases where we think cause and effect may involved, we sometimes subdivide the variables of interest into two categories. For example, each person in the study of vitamin C and colds may take a different daily dose of vitamin C and may end up with a different number of colds over some period of time. Because we are trying to learn if vitamin C causes a lower number of colds, we say that daily dose of vitamin C is an explanatory variable—it may explain or cause a change in the number of colds. Similarly, we say that number of colds is a response variable, because we expect it to respond to changes in the explanatory variable (the dose of vitamin C).

Two Basic Types of Statistical Study

There are two basic types of statistical study:

1. In an observational study, researchers observe or measure characteristics of the subjects, but do not attempt to influence or modify these characteristics.

2. In an experiment, researchers apply some treatment and observe its effects on the subjects of the experiment.

Definitions

A variable is any item or quantity that can vary or take on different values.

The variables of interest in a statistical study are the items or quantities that the study seeks to measure.

When cause and effect may be involved, an explanatory variable is a variable that may explain or cause the effect, while a response variable is a variable that responds to changes in the explanatory variable.

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 23

EXAMPLE 2 Identify the Variables Identify the variables of interest for each study.

a. The Salk polio vaccine study

b. A poll in which people are asked for whom they plan to vote in the next election

Solution

a. The two variables of interest in the Salk vaccine study are vaccine and polio. They are vari- ables because they can take on two different values: A child either did or did not get the vaccine and either did or did not contract polio. In this case, because the study seeks to determine whether the vaccine prevents polio, we say that vaccine is the explanatory vari- able (it may explain a change in the incidence of polio) and polio the response variable (it is supposed to change in response to the vaccine).

b. The variables of interest in the pre-election poll might be called candidate’s name, since people must choose one of the candidates as the one they plan to vote for, and proportion of supporters, meaning the fraction of the people polled who plan to vote for a particular can- didate. (There is no cause and effect involved in this study, so we do not need to decide whether the variable is explanatory or response.)

Observational Studies The observational studies we have discussed up to this point, such as Nielsen ratings, opinion polls, and determining the mean height of students, are studies in which the data are all gener- ally collected around the same time. Sometimes, however, observational studies look at past data or are designed to look at future data over a long period of time.

A retrospective study (sometimes called a case-control study) is an observational study that uses data from the past—such as official records or past interviews—to learn about some issue of concern. Retrospective studies are especially valuable in cases where it may be impractical or unethical to perform an experiment. For example, suppose we want to learn how alcohol consumed during pregnancy affects newborn babies. Because it is already known that consuming alcohol during pregnancy can be harmful, it would be unethical to ask pregnant mothers to test the “treatment” of consuming alcohol. However, because many mothers con- sumed alcohol in past pregnancies (either before the dangers were known or choosing to ignore the dangers), we can do a retrospective study in which we compare children born to those mothers to children born to mothers who did not consume alcohol.

Sometimes, the data we need to reach clear conclusions are not available in past records. In those cases, researchers may set up a prospective study (sometimes called a longitudinal study) designed to collect observations in the future from groups that share common factors. A classic example of a prospective study is the Harvard Nurses’ Health Study, which was started in 1976 in order to collect data about how different lifestyles affect women’s health (see the “Focus on Pub- lic Health” section at the end of this chapter). The study, still ongoing today, has followed thou- sands of nurses over more than three decades, collecting data about their lifestyles and health.

Variations on Observational Studies

The most familiar observational studies are those in which data are collected all at once (or as close to that as possible). Two variations on observational studies are also common:

1. A retrospective (or case-control) study uses data from the past, such as official records or past interviews.

2. A prospective (or longitudinal) study is set up to collect data in the future from groups that share common factors.IS

B N

: 0 -5

36 -8

82 21

-5

24 Speaking of Statistics

EXAMPLE 3 Observational Study You want to know whether children born prematurely do as well in elementary school as chil- dren born at full term. What type of study should you do?

Solution An observational, retrospective study is the only real option in this case. You would collect data on past births and compare the elementary school performance of those born prematurely to that of those born at full term.

Experiments Because experiments require active intervention, such as applying a treatment, we must take special care to ensure that they are designed in ways that will provide the information we seek. Let’s examine a few of the issues that arise in the design of experiments.

The Need for Controls Consider an experiment that gives some participants in a study vitamin C to determine its effect on colds. Suppose the people taking vitamin C daily get an average of 1.5 colds in a three-month period. How can the researchers know whether the subjects would have gotten more colds without the vitamin C? To answer this type of question, the researchers must conduct their experiment with two (or more) groups of subjects: One group takes large doses of vitamin C daily and another group does not. As we’ll discuss shortly, in most cases it is important that participants be randomly assigned to the two groups.

The group of people who are randomly assigned to take vitamin C is called the treatment group because its members receive the treatment being tested (vitamin C). The group of people who do not take vitamin C is called the control group. The researchers can be confident that vitamin C is an effective treatment only if the people in the treatment group get significantly fewer colds than the people in the control group. The control group gets its name from the fact that it helps control the way we interpret experi- mental results.

Treatment and Control Groups

The treatment group in an experiment is the group of subjects who receive the treatment being tested.

The control group in an experiment is the group of subjects who do not receive the treatment being tested.

In most cases, it is important to choose the members of the two groups by random selection from the available pool of subjects.

EXAMPLE 4 Treatment and Control Look again at the Salk polio vaccine Case Study (page 8). What was the treatment? Which group of children constituted the treatment group? Which constituted the control group?

Solution The treatment was the Salk vaccine. The treatment group consisted of the chil- dren who received the Salk vaccine. The control group consisted of the children who did not get the Salk vaccine and instead got an injection of salt water.

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 25

EXAMPLE 5 Mozart Treatment A study divided college students into two groups. One group listened to Mozart or other classical music before being assigned a specific task and the other group simply was assigned the task. Researchers found that those listening to the classical music performed the task slightly better, but only if they did the task within a few minutes of listening to the music (the two groups per- formed equally on tasks given later). Identify the treatment and the control and treatment groups.

Solution The treatment was the classical music. The treatment group consisted of the stu- dents who listened to the music. The control group consisted of the students who did not listen to the music.

Confounding Variables Using control groups helps to ensure that we account for known variables that could affect a study’s results. However, researchers may be unaware of or be unable to account for other important variables. Consider an experiment in which a statistics teacher seeks to determine whether students who study collaboratively (in study groups with other students) earn higher grades than students who study independently. The teacher chooses five students who will study collaboratively (the treatment group) and five others who will study independently (the control group). To ensure that the students all have similar abilities and will study diligently, the teacher chooses only students with high grade-point averages. At the end of the semester, the teacher finds that the students who studied collaboratively earned higher grades.

The variables of interest for this study are collaborative study (whether they do so or not) and final grade. But suppose that, unbeknownst to the teacher, the collaborative students all lived in a dormitory where a curfew ensured that they got plenty of sleep. This fact introduces a new variable—which we might call amount of sleep—that might partially explain the results. In other words, the experiment’s conclusion may seem to support the benefits of collaborative study, but this conclusion is not justified because the teacher did not account for how much students slept.

In statistical terminology, this study suffers from confounding. The higher grades may be due either to the variable of interest (collaborative study) or to the differing amounts of sleep or to a combination of both. Because the teacher did not account for differences in the amount of sleep, we say that amount of sleep is a confounding variable for this study. You can probably think of other potentially confounding variables that could affect a study like this one.

By the Way .. . The so-called Mozart effect holds that listening to Mozart can make babies smarter. The supposed effect spawned an entire industry of Mozart products for children. The state of Georgia even began passing out Mozart CDs to new mothers. However, more recent studies of the Mozart effect have been unable to substantiate the claimed effect.

Definition

A study suffers from confounding if the effects of different variables are mixed so we cannot determine the specific effects of the variables of interest. The variables that lead to the confusion are called confounding variables.

CASE STUDY Confounding Drug Results

Pfizer Corporation developed a new drug (called fluconazole) designed to prevent fungal infec- tions in hospital patients. Several studies found the new drug to be more effective than an older drug (called amphotericin B). However, a subsequent analysis by other researchers found that the older drug had been administered orally when it was supposed to be given by injection. This introduced a source of confounding into the studies: The original researchers thought the results showed the new drug to be more effective than the old drug, but they had not taken into account the confounding effects of how the old drug was administered. Once the new researchers took this effect into account (by comparing only cases in which both drugs were administered properly), they found that the new drug was no more effective than the old drug.IS

B N

: 0 -5

36 -8

82 21

-5

26 Speaking of Statistics

Assigning Treatment and Control Groups As the collaborative study experiment illustrates, results are almost sure to suffer from con- founding if the treatment and control groups differ in important ways (other than receiving or not receiving the treatment). Researchers generally employ two strategies to prevent such dif- ferences and thereby ensure that the treatment and control groups can be compared fairly. First, they assign participants to the treatment and control groups at random, meaning that they use a technique designed to ensure that each participant has an equal chance of being assigned to either group. When the participants are randomly assigned, it is less likely that the people in the treatment and control groups will differ in some way that will affect the study results.

Second, researchers try to ensure that the treatment and control groups are sufficiently large. For example, in the collaborative study experiment, including 50 students in each group rather than 5 would have made it much less likely that all the students in one group would live in a special dormitory.

A strong imagination brings on the [placebo effect]. . . . Everyone feels its impact, but some are knocked over by it. . . . [Doctors] know that there are men on whom the mere sight of medicine is operative.

—French philosopher Michel de Montaigne (1533–1592)

Strategies for Selecting Treatment and Control Groups

1. Select groups at random. Make sure that the subjects of the experiment are assigned to the treatment or control group at random, meaning that each subject has an equal chance of being assigned to either group.

2. Use sufficiently large groups. Make sure that the treatment and control groups are both sufficiently large that they are unlikely to differ in a significant way (aside from the fact that one group gets the treatment and the other does not).

EXAMPLE 6 Salk Study Groups Briefly explain how the two strategies for selecting treatment and control groups were used in the Salk polio vaccine study.

Solution A total of about 400,000 children participated in the study, with half receiving an injection of the Salk vaccine (the treatment group) and the other half receiving an injection of salt water (the control group). The first strategy was implemented by choosing children for the two groups randomly from among all the children. The second strategy was implemented by using a large number of participants (200,000 in each group) so that the two groups were unlikely to differ by accident.

The Placebo Effect When an experiment involves people, effects can occur simply because people know they are part of the experiment. For example, suppose you are testing the effectiveness of a new anti-depression drug. You find 500 people who suffer from depression and randomly divide them into a treatment group that receives the new drug and a control group that does not. A few weeks later, interviews with the patients show that people in the treatment group tend to be feeling much better than peo- ple in the control group. Can you conclude that the new drug works?

Unfortunately, it’s quite possible that the mood of people receiving the drug improved simply because they were glad to be getting some kind of treatment, which means you cannot be sure that the drug really helped. This type of effect, in which people improve because they believe that they are receiving a useful treatment, is called the placebo effect. (The word placebo comes from the Latin “to please.”)

To distinguish between results caused by a placebo effect and results that are truly due to the treatment, researchers try to make sure that the participants do not know whether they are part of the treatment or control group. To accomplish this, the researchers give the people in the control group a placebo: something that looks or feels just like the treatment being tested, but

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 27

lacks its active ingredients. For example, in a test of a drug that comes in pill form, the placebo might be a pill of the same shape and size that contains sugar instead of the real drug. In a test of an injected vaccine, the placebo might be an injection that contains only a saline solution (salt water) instead of the real vaccine. In a recent test of the effectiveness of acupuncture, the placebo consisted of treatment with needles as in real acupuncture, except the needles were not put in the special places that acupuncturists claim to be important.

As long as the participants do not know whether they received the real treatment or a placebo, the placebo effect ought to affect the treatment and control groups equally. If the results for the two groups are significantly different, the differences can be attributed to the treatment. For example, in the study of the anti-depression drug, we would conclude that the drug was effective only if the control group received a placebo and members of the treatment group improved much more than members of the control group. For even better control, some experi- ments use three groups: a treatment group, a placebo group, and a control group. The placebo group is given a placebo while the control group is given nothing.

By the Way .. . The placebo effect can be remark- ably powerful. In some studies, up to 75% of the participants receiving the placebo actually improve. For some patients, the effect is so powerful that they plead to continue their treatment even after being told they were given a placebo rather than the real treatment. Nevertheless, different researchers disagree about the strength of the placebo effect, and some even question the reality of the effect.

Definitions

A placebo lacks the active ingredients of a treatment being tested in a study, but looks or feels like the treatment so that participants cannot distinguish whether they are receiving the placebo or the real treatment.

The placebo effect refers to the situation in which patients improve simply because they believe they are receiving a useful treatment.

Note: Although participants should not know whether they belong to the treatment or con- trol group, for ethical reasons it is very important that participants be told that some of them will be given a placebo, rather than the real treatment.

In decades past, researchers often told all the participants in a study that they were receiving the real treatment, but actually gave a placebo to half the participants. Discuss a few reasons why this would now be considered unethical. Should researchers be allowed to use results of past studies that do not meet today’s ethical criteria? Defend your opinion.

EXAMPLE 7 Vaccine Placebo What was the placebo in the Salk polio vaccine study? Why did researchers use a placebo in this experiment?

Solution The placebo was the salt water injection given to the children in the control group. To understand why the researchers used a placebo for the control group, suppose that a placebo had not been used. When improvements were observed in the treatment group, it would have been impossible to know whether the improvements were due to the vaccine or to the placebo effect. In order to remove this confounding, all participants had to believe that they were being treated in the same way. This ensured that any placebo effect would occur in both groups equally, so that researchers could attribute any remaining dif- ferences to the vaccine.

Experimenter Effects Even if the study subjects don’t know whether they received the real treatment or a placebo, the experimenters may still have an effect. In testing an anti-depression drug, for example, experi- menters will probably interview patients to find out whether they are feeling better. But if the experimenters know who received the real drug and who received the placebo, they may

By the Way .. . A related effect, known as the Hawthorne effect, occurs when treated subjects somehow respond differently simply because they are part of an experiment—regardless of the particular way in which they are treated. The Hawthorne effect gets it name from the fact that it was first observed in a study of factory workers at Western Electric’s Hawthorne plant.

IS B

N : 0

-5 36

-8 82

21 -5

28 Speaking of Statistics

inadvertently smile more at the people in the treatment group. Their smiles might improve those participants’ moods, making it seem as if the treatment worked when in fact the improve- ment was caused by the experimenter. This type of confounding, in which the experimenter somehow influences the results, is called an experimenter effect (or a Rosenthal effect). The only way to avoid experimenter effects is to make sure that the experimenters don’t know which subjects are in which group.

By the Way .. . Many similar cases of supposedly widespread child abuse at daycare centers and preschools are being re-examined to see if experimenter effects (by those who interviewed the children) may have led to wrongful convictions. Similar claims of experimenter effects have been made in cases involving repressed memory, in which counseling supposedly helped people retrieve lost memories of traumatic events.

Definition

An experimenter effect occurs when a researcher or experimenter somehow influences subjects through such factors as facial expression, tone of voice, or attitude.

EXAMPLE 8 Child Abuse? In a famous case, two couples from Bakersfield, California, were convicted of molesting dozens of preschool-age children at their daycare center. The evidence for the abuse came primarily from interviews with the children. However, the conviction was overturned—after one man had served 14 years in prison—when a judge re-examined the interviews and concluded that the children had given answers that they thought the interviewers wanted to hear. If we think of the interviewers as experimenters, this is an example of an experimenter effect because the inter- viewers influenced the children’s answers through the tone and style of their questioning.

Blinding In statistical terminology, the practice of keeping people in the dark about who is in the treat- ment group and who is in the control group is called blinding. A single-blind experiment is one in which the participants don’t know which group they belong to, but the experimenters do know. If neither the participants nor the experimenters know who belongs to each group, the study is said to be double-blind. Of course, someone has to keep track of the two groups in order to evaluate the results at the end. In a double-blind experiment, the researchers conduct- ing the study typically hire experimenters to make any necessary contact with the participants. The researchers thereby avoid any contact with the participants, ensuring that they cannot in- fluence them in any way. The Salk polio vaccine study was double-blind because neither the participants (the children) nor the experimenters (the doctors and nurses giving the injections and diagnosing polio) knew who got the real vaccine and who got the placebo.

Blinding in Experiments

An experiment is single-blind if the participants do not know whether they are members of the treatment group or members of the control group, but the experimenters do know.

An experiment is double-blind if neither the participants nor any experimenters know who belongs to the treatment group and who belongs to the control group.

EXAMPLE 9 What’s Wrong with This Experiment? For each of the experiments described below, identify any problems and explain how the prob- lems could have been avoided.

a. A new drug for attention deficit disorder (ADD) is supposed to make affected children more polite. Randomly selected children suffering from ADD are divided into treatment and control groups. The experiment is single-blind. Experimenters evaluate how polite the children are during one-on-one interviews.

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 29

b. Educational researchers wonder if listening to classical music when studying improves learning. They give two groups of students an identical 2-hour lesson and then allow time to study for a short exam. One group, made up of 50 students who told the researchers that they like classical music, listens to classical music while they study. The other group, made up of 50 other students who told the researchers that they don’t like classical music, studies in silence. The results show that the students who listened to classical music did better on the test.

c. Researchers wonder if the effects of a rare degenerative disease can be slowed by exercise. They identify 6 people suffering from the disease and randomly assign 3 to a treatment group that exercises every day and 3 to a control group that avoids exercise. After six months, they compare the amounts of degeneration in each group.

d. A chiropractor performs adjustments on 25 patients with back pain. Afterward, 18 of the patients say they feel better. He concludes that the adjustments are an effective treatment.

Solution

a. The experimenters assess politeness in interviews, but because they know which children received the real drug, they may inadvertently speak differently to these children during the interviews. Or, they might interpret the children’s behavior differently since they know which subjects received the real drug. These are experimenter effects that can confound the study results. The experiment should have been double-blind.

b. The problem with this study is that students were not assigned to the two groups at random. By placing students who like classical music in one group and students who don’t like it in the other, the researchers created a situation in which the two groups do not share the same general characteristics.

c. The results of this study will be difficult to interpret because the sample sizes are not suffi- ciently large. Of course, with a rare disease, finding people to participate in an experiment may be difficult.

d. The 25 patients who receive adjustments represent a treatment group, but this study lacks a control group. The patients may be feeling better because of a placebo effect rather than any real effect of the adjustments. The chiropractor might have improved his study by hiring an actor to do a fake adjustment (one that feels similar, but doesn’t actually conform to chiropractic guidelines) on a control group. Then he could have compared the results in the two groups to see whether a placebo effect was involved.

EXAMPLE 10 Identifying the Study Type For each of the following questions, what type of statistical study is most likely to lead to an answer? Be as specific as possible.

a. What is the mean income of stock brokers?

b. Do seat belts save lives?

c. Can lifting weights improve runners’ times in a 10-kilometer (10 K) race?

d. Does skin contact with a particular glue cause a rash?

e. Can a new herbal remedy reduce the severity of colds?

f. Do supplements of resveratrol (an extract from red grapes) increase life span?

Solution

a. An observational study can tell us the mean income of stock brokers. We need only survey the brokers, and the survey itself will not change their incomes.

With proper treatment, a cold can be cured in a week. Left to itself, it may linger for seven days.

—Medical Folk Saying

IS B

N : 0

-5 36

-8 82

21 -5

30 Speaking of Statistics

b. It would be unethical to do an experiment in which some people were told to wear seat belts and others were told not to wear them. A study to determine whether seat belts save lives must therefore be observational. Because some people choose to wear seat belts and others choose not to, we can conduct a retrospective study. By comparing the death rates in accidents between those who do and do not wear seat belts, we can learn whether seat belts save lives. (They do.)

c. We need an experiment to determine whether lifting weights can improve runners’ 10 K times. We select randomly from a group of runners to create a treatment group of runners who are put on a weight-lifting program and a control group that is asked to stay away from weights. We must try to ensure that all other aspects of their training are similar. Then we can see whether the runners in the lifting group improve their times more than those in the control group. We cannot use blinding in this experiment because there is no way to prevent participants from knowing whether they are lifting weights.

d. An experiment can help us determine whether skin contact with the glue causes a rash. In this case, it’s best to use a single-blind experiment in which we apply the real glue to participants in one group and apply a placebo that looks the same, but lacks the active ingredient, to members of the control group. There is no need for a double-blind experi- ment because it seems unlikely that the experimenters could influence whether a person gets a rash. (However, if the question of whether the subject has a rash is subject to interpretation, the experimenter’s knowledge of who got the real treatment could affect this interpretation.)

e. We should use a double-blind experiment to determine whether a new herbal remedy can reduce the severity of colds. Some participants get the actual remedy, while others get a placebo. We need the double-blind conditions because the severity of a cold may be affected by mood or other factors that researchers might inadvertently influence. In the double-blind experiment, the researchers do not know which participants belong to which group and thus cannot treat the two groups differently.

f. Resveratrol has been identified and made available in supplement form only recently; we will need many years of data to determine whether it has an effect on life span. We there- fore should use a prospective study designed to monitor participants over many years. The participants could keep written records regarding whether and how much of the supplement they take, and eventually researchers could analyze the data to see if resveratrol has an effect on life span.

Meta-Analysis All individual statistical studies are either observational studies or experiments. In recent years, however, statisticians have found it useful to “mine” groups of past studies to see if we can learn something that we were unable to learn from the individual studies. For example, hundreds of studies have considered the possible effects of vitamin C on colds, so researchers might decide to review the data from many of these studies as a group. This type of study, in which researchers review many past studies as a group, is called a meta-analysis.

By the Way .. . Experiments with mice show that the mice live longer and have greater endurance when taking large doses of resveratrol, an extract from the skin of red grapes. However, it is not yet known whether high doses of resveratrol would have the same effects for humans, nor is it known whether resveratrol is safe for humans. Moreover, the doses given to the mice are so large that it may be impractical to give equivalent doses to humans, with our larger body sizes.

Definition

In a meta-analysis, researchers review many past studies. The meta-analysis considers these studies as a combined group, with the aim of finding trends that were not evident in the individual studies.

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 31

Section 1.3 Exercises

Statistical Literacy and Critical Thinking

1. Placebo. What is a placebo, and why is it important in an experiment to test the effectiveness of a drug?

2. Blinding. What is blinding and why is it important in an experiment to test the effectiveness of a drug?

3. Confounding. In testing the effectiveness of a new vaccine, suppose that researchers used males for the treatment group and females for the placebo group. What is confounding, and how would it affect such an experiment?

4. Withholding Treatment. In one infamous study, 399 African American men with syphilis were not given a treatment that could have cured them. Others were given treatments. The intent was to learn about the effects of syphilis on African American men. Is this an experiment or an observational study? Why is this study morally and criminally wrong?

5. Clothing Color. A researcher plans to investigate the belief that people are more comfortable in the summer sun when they wear clothing with light colors instead of clothing with dark colors. Does it make sense to use a double-blind

CASE STUDY Drugs to Fight Depression: A Meta-Analysis

Government researchers estimate that 1 in 5 Americans suffers from depression at some time in their lives and that the annual cost to the economy in lost productivity is at least $40 billion. Until the 1990s, the only drugs available to combat depression belonged to a class known as tricyclic antidepressants. But in the past decade, several new drugs have come into common use. The most famous is Prozac, which has been prescribed so widely that some people claim we now live in a “Prozac nation.” But do Prozac and the other new drugs work? To answer this question, researchers from the Agency for Health Care Policy and Research conducted a meta-analysis.

The researchers began by searching medical literature for studies about treatment of depression. They found more than 8,000 studies reported over a nine-year period. By looking for studies that met certain special criteria, such as observing patients for at least six weeks and comparing new and old types of anti-depression drugs, they narrowed the list to about 300 studies. They also considered about 600 studies dealing with side effects of the drugs. Their meta-analysis consisted of analyzing the results of all these studies as a group so as to look for trends that might not have been evident in individual studies.

They found that, overall, the new drugs are more effective than placebos for severely depressed people. About 50% of such people responded to the new drugs, compared to about 32% who responded to a placebo. However, the old tricyclic antidepressants were equally effective— and were often available at lower prices. The old and new drugs had different side effects, but the degrees of severity of these side effects were roughly equal. While these results can be interpreted as both good and bad news for the makers and users of the new drugs, the meta-analysis probably was more valuable for what it could not say. The researchers found that the data were inadequate to determine whether the new drugs were effective for mild depression or for children, despite the fact that they are commonly prescribed for both. They also found the data inadequate to evaluate herbal treatments, such as kava and St. John’s wort. Thus, the meta-analysis pointed out the direc- tions in which new research is needed.

Look again at the percentage responses to the new drugs (50%) and to the placebo (32%) in the meta-analysis of anti-depression drugs. Does the relatively large response to the placebo suggest that patients should be given a placebo before being given a real drug with potential side effects? Why or why not? If you were a psychologist, what would you suggest to someone who was part of the 50% who did not respond to the real drug?

IS B

N : 0

-5 36

-8 82

21 -5

32 Speaking of Statistics

experiment in this case? Is it easy to implement blinding in this case? Explain.

6. Lawn Treatment. A researcher plans to test the effec- tiveness of a new fertilizer on grass growth. Does it make sense to use a double-blind experiment in this case?

7. Measuring Pain. Many studies involve methods for measuring pain after some treatment. Describe how an experimenter effect might adversely affect results, and describe how the effect can be avoided.

8. Treating Baldness. In one actual study of a treatment for baldness, it was claimed that the placebo group had better results in growing new hair than the group treated with a drug. Are such results possible? What would such results suggest about the treatment?

Concepts and Applications

Type of Study. For Exercises 9–20, state the type of study and identify the variables of interest. Be as specific as possible.

9. Touch Therapy. Nine-year-old Emily Rosa became an author of an article in the Journal of the American Medical Association after she tested professional touch therapists (see the “Focus on Education” section in Chapter 10). Using a cardboard partition, she held her hand above one of the therapist’s hands, and the therapist was asked to identify the hand that Emily chose.

10. Quality Control. The Federal Drug Administration randomly selects a sample of Bayer aspirin tablets. The amount of aspirin in each tablet is measured for accuracy.

11. Magnetic Bracelets. Some cruise ship passengers are given magnetic bracelets, which they agree to wear in an attempt to eliminate or diminish the effects of motion sickness. Others are given similar bracelets that have no magnetism.

12. Gender Selection. In a study of the YSORT gender selection method developed by the Genetics & IVF Insti- tute, 152 couples had 127 baby boys and 25 baby girls.

13. Twins. In a study of hundreds of Swedish twins, it was determined that the level of mental skills was more similar in identical twins (twins coming from a single egg) than in fraternal twins (twins coming from two separate eggs) (Science).

14. Heart Disease. A European study of 1,500 men and women with exceptionally high levels of the amino acid homocysteine found that these individuals had double the risk of heart disease. However, the risk was substantially lower for those in the study who took vitamin B supple- ments (Journal of the American Medical Association).

15. Prostate Cancer. An analysis of 11 individual studies attempted to determine if there is a conclusive link between vasectomies and the incidence of prostate cancer (Chance).

16. AOL Poll. An America OnLine (AOL) poll resulted in 38,410 responses to the question “How much stock do you put in long range weather forecasts?” Among those respondents, 47% said “very little” or “none.”

17. GMO Corn. Researchers at New York University found that the genetically modified corn known as Bt corn releases an insecticide through its roots into the soil (Nature).

18. Stem Cell Research. In a Newsweek poll conducted by Princeton Survey Research Associates, 48% of the 1,002 adult respondents said that they were in favor of using federal tax money to fund medical research using stem cells obtained from embryos.

19. Magnet Treatment. In a study of the effects of magnets on back pain, some subjects were treated with magnets while others were given nonmagnetic devices with a simi- lar appearance. The magnets did not appear to be effective in treating back pain (Journal of the American Medical Association).

20. Power Lines and Cancer. Hundreds of scientific and statistical studies have been done to determine whether high-voltage overhead power lines increase the incidence of cancer among those living nearby. A summary study based on many previous studies concluded that there is no significant link between power lines and cancer (Journal of the American Medical Association).

What’s Wrong with This Experiment? For each of the studies described in Exercises 21–28, identify any problems that are likely to cause confounding and explain how the prob- lems could be avoided. Discuss any other problems that might affect the results.

21. Poplar Tree Growth. An experiment is designed to eval- uate the effectiveness of irrigation and fertilizer on poplar tree growth. Fertilizer is used with one group of poplar trees in a moist region, and irrigation is used with poplar trees in a dry region.

22. Internet Shopping. Two hundred volunteers are recruited through an ad in PC Magazine. They are to be used for a study to compare the total amounts spent by those who use the Internet and those who do not. Each person is allowed to choose whether to be in the Internet user group or the group that does not use the Internet. After one month, the amounts of money spent by the two groups are compared.

23. Octane Rating. In a comparison of gasoline with differ- ent octane ratings, 24 vans are driven with 87-octane gasoline, while 28 sport utility vehicles are driven with

IS B

N : 0-536-88221-5

1.3 Types of Statistical Studies 33

91-octane gasoline. After a vehicle has been driven for 250 miles, the amount of gasoline consumed is measured.

24. Aspirin Trial. In phase I of a clinical trial designed to test the effectiveness of aspirin in preventing myocardial infarctions, aspirin is given to 3 people and a placebo to 7 other people.

25. Hypertension Trial. In a clinical trial of the effective- ness of a drug to treat hypertension, subjects are told whether they are getting the actual drug or a placebo.

26. Athlete’s Foot. In a clinical trial of the effectiveness of a lotion used to treat tinea pedis (athlete’s foot), the physicians who evaluate the results know which subjects were given the treatment and which were given a placebo.

27. Weight Lifting. In a test of the effects of lifting heavy weights on blood pressure, one group undergoes a treat- ment consisting of a weight-lifting program while another group lifts tennis balls.

28. Paint Mixtures. In durability tests of two different paint mixtures, the researchers who evaluate the results know which samples are from each of the two different mixtures.

Analyzing Experiments. Exercises 29–32 present questions that might be addressed in an experiment. If you were to design the experiment, how would you choose the treatment and con- trol groups? Should the experiment be single-blind, double- blind, or neither? Explain your reasoning.

29. Beethoven and Intelligence. Does listening to Beethoven make infants more intelligent?

30. Lipitor and Cholesterol. Does the drug Lipitor result in lower cholesterol levels?

31. Ethanol and Mileage. Does an ethanol additive in gaso- line result in reduced mileage?

32. Home Siding. Does aluminum siding last longer on a home than wood siding?

Projects for the Internet and Beyond

For useful links, select “Links for Internet Projects” for Chapter 1 at www.aw.com/bbt.

33. Experimenter Effects in Repressed Memory Cases. Search the Web for articles and information about the controversy regarding recovering repressed memories. Briefly summarize one or two of the most interesting cases and, based on what you read, express your own opinion as to whether the allegedly recovered memories are being influenced by experimenter effects.

34. Ethics in Experiments. Ethical standards change from era to era. One notoriously unethical case was a study of syphilis conducted in Tuskegee, Alabama, from 1932 to 1972. (See Exercise 4.) In this study, African American males were told that they were receiving treatment for syphilis, but in fact they were not. The researchers’ hidden goal was to study the long-term effects of the disease. Use the Web to learn about the history of the Tuskegee syphilis study. Hold a class dis- cussion about the ethical issues involved in this case, or write a short essay summarizing the case and its ethical lessons.

35. Debate: Should We Use Data from Unethical Experiments? Past research often did not conform to today’s ethical standards. In extreme cases, such as research conducted by doctors in Nazi Germany, the researchers sometimes killed the subjects of their exper- iments. While this past unethical research clearly vio- lated the human rights of the experimental subjects, in some cases it led to insights that could help people today. Is it ethical to use the results of unethical research?

36. Study Stopped Early. It sometimes happens that a study is stopped before its completion. Use the Internet to find an example of such a study. Why was the study stopped? Should it have been stopped, or would it have been better to complete the study?

37. Observational Studies. Look through newspapers from the past few weeks and find an example of a sta- tistical study that was observational. Briefly describe the study and summarize its conclusions.

38. Experimental Studies. Look through newspapers for the past few weeks and find an example of a statisti- cal study that involved an experiment. Briefly describe the study and summarize its conclusions.

39. Retrospective Studies. Look through newspapers for the past few weeks and find an example of an observational, retrospective study. Briefly describe the study and summarize its conclusions.

40. Meta-Analysis. Look through newspapers for the past few weeks and find an example of a meta- analysis. Briefly describe the study and summarize its conclusions.

IS B

N : 0

-5 36

-8 82

21 -5

34 Speaking of Statistics

If I have ever made any valuable discoveries, it has been owing more to patient attention, than to any other talent.

—Isaac Newton

1.4 Should You Believe a Statistical Study?

Much of the rest of this book is devoted to helping you build a deeper understandingof the concepts and definitions we’ve studied up to this point. But already you knowenough to achieve one of the major goals of this book: being able to answer the question “Should you believe a statistical study?”

Most researchers conduct their statistical studies with honesty and integrity, and most statistical research is carried out with diligence and care. Nevertheless, statistical research is sufficiently complex that bias can arise in many different ways, making it very important that we always examine reports of statistical research carefully. There is no definitive way to answer the question “Should I believe a statistical study?” However, in this section we’ll look at eight guidelines that can be helpful. Along the way, we’ll also introduce a few more defini- tions and concepts that will prepare you for discussions to come later.

Eight Guidelines for Critically Evaluating a Statistical Study

1. Identify the goal of the study, the population considered, and the type of study.

2. Consider the source, particularly with regard to whether the researchers may be biased.

3. Examine the sampling method to decide whether it is likely to produce a representative sample.

4. Look for problems in defining or measuring the variables of interest, which can make it difficult to interpret any reported results.

5. Watch out for confounding variables that can invalidate the conclusions of a study.

6. Consider the setting and wording in surveys or polls, looking for anything that might tend to produce inaccurate or dishonest responses.

7. Check that results are fairly represented in graphics and concluding statements, because both researchers and media often create misleading graphics or jump to conclusions that the results do not support.

8. Stand back and consider the conclusions. Did the study achieve its goals? Do the conclusions make sense? Do the results have any practical significance?

News reports do not always provide enough information for you to apply all of the above guidelines, but you can usually find additional information on the Web. Look for clues such as “reported by NASA” or “published in the New England Journal of Medicine” to help you track down original sources or other relevant information.

Guideline 1: Identify the Goal, Population, and Type of Study The first step in evaluating a statistical study is to understand the goal and approach of the study. Based on what you hear or read about a study, try to answer these basic questions:

• What was the study designed to determine? • What was the population under study? Was the population clearly and appropriately defined? • Was the study an observational study, an experiment, or a meta-analysis? If it was an obser-

vational study, was it retrospective? If it was an experiment, was it single- or double-blind, and were the treatment and control groups properly randomized? Given the goal, was the type of study appropriate?

IS B

N : 0-536-88221-5

1.4 Should You Believe a Statistical Study? 35

By the Way .. . Surveys show that nearly half of Americans believe their horoscopes. However, in controlled experiments, the predictions of horoscopes come true no more often than would be expected by chance.

EXAMPLE 1 Appropriate Type of Study? Imagine the following (hypothetical) newspaper report: “Researchers gave 100 participants their individual astrological horoscopes and asked whether the horoscopes were accurate. 85% of the participants said their horoscopes were accurate. The researchers concluded that horo- scopes are valid most of the time.” Analyze this study according to Guideline 1.

Solution The goal of the study was to determine the validity of horoscopes. Based on the news report, it appears that the study was observational: The researchers simply asked the par- ticipants about the accuracy of the horoscopes. However, because the accuracy of a horoscope is somewhat subjective, this study should have been a controlled experiment in which some people were given their actual horoscope and others were given a fake horoscope. Then the researchers could have looked for differences between the two groups. Moreover, because researchers could easily influence the results by how they questioned the participants, the experiment should have been double-blind. In summary, the type of study was inappropriate to the goal and its results are meaningless.

Try your own test of horoscopes. Find yesterday’s horoscope for each of the 12 signs and put each one on a separate piece of paper, without anything identifying the sign. Shuffle the pieces of paper randomly, and ask a few people to guess which one was supposed to be their personal horoscope. How many people choose the right one? Discuss your results.

EXAMPLE 2 Does Aspirin Prevent Heart Attacks? A study reported in the New England Journal of Medicine (Vol. 318, No. 4) sought to deter- mine whether aspirin is effective in preventing heart attacks. It involved 22,000 male physi- cians considered to be at risk for heart attacks. The men were divided into a treatment group that took aspirin and a control group that did not. The results were so convincing in favor of the benefits of aspirin that the experiment was stopped for ethical reasons before it was com- pleted, and the subjects were informed of the results. Many news reports led with the head- line that taking aspirin can help prevent heart attacks. Analyze this headline according to Guideline 1.

Solution The study was an experiment, which is appropriate, and its results appear con- vincing. However, the fact that the sample consisted only of men means that the results should be considered to apply only to the population of men. Because results of medical tests on men do not necessarily apply to women, the headlines misstated the results when they did not qual- ify the population.

Guideline 2: Consider the Source Statistical studies are supposed to be objective, but the people who carry them out and fund them may be biased. It is therefore important to consider the source of a study and evaluate the potential for biases that might invalidate the study’s conclusions.

Bias may be obvious in cases where a statistical study is carried out for marketing, pro- motional, or other commercial purposes. For example, a toothpaste advertisement that claims “4 out of 5 dentists prefer our brand” appears to be statistically based, but we are given no details about how the survey was conducted. Because the advertisers obviously want to say good things about their brand, it’s difficult to take the statistical claim seriously without much more information about how the result was obtained.

By the Way .. . Many recent studies show substan- tial differences in the ways men and women respond to the same medical treatments. For example, aspirin is more effective at thinning blood in men than in women (thin- ning is thought to help prevent heart attacks in some people). Morphine controls pain better in women, but ibuprofen is more effective for men. And women reject heart transplants more often than men do. These differences may stem from interactions with hormones that differ in men and women or from differences in the rates at which men and women metabolize different drugs.

IS B

N : 0

-5 36

-8 82

21 -5

36 Speaking of Statistics

Other cases of bias may be more subtle. For example, suppose that a carefully conducted study concludes that a new drug helps cure cancer. On the surface, the study might seem quite believable. But what if the study was funded by a drug company that stands to gain billions of dollars in sales if the drug is proven effective? The researchers may well have carried out their work with great integrity despite the source of funding, but it might be worth a bit of extra in- vestigation to be sure.

Major statistical studies are usually evaluated by unbiased experts. For example, the process by which scientists examine each other’s research is called peer review (because the scientists who do the evaluation are peers of those who conducted the research). Reputable sci- entific journals require all research reports to be peer reviewed before the research is accepted for publication. Peer review does not guarantee that a study is valid, but it lends credibility because it implies that other experts agree that the study was carried out properly.

By the Way .. . After decades of arguing to the contrary, in October 1999 the Philip Morris Company—the world’s largest seller of tobacco products— publicly acknowledged that smoking causes lung cancer, heart disease, emphysema, and other serious diseases. Shortly thereafter, Philip Morris changed its name to Altria.

Definition

Peer review is a process in which several experts in a field evaluate a research report before the report is published.

EXAMPLE 3 Is Smoking Healthy? By 1963, research had so clearly shown the health dangers of smoking that the Surgeon General of the United States publicly announced that smoking is bad for health. Research done since that time built further support for this claim. However, while the vast majority of studies showed that smoking is unhealthy, a few studies found no dangers from smoking and perhaps even health benefits. These studies generally were carried out by the Tobacco Research Insti- tute, funded by the tobacco companies. Analyze these studies according to Guideline 2.

Solution Even in a case like this, it can be difficult to decide whom to believe. However, the studies showing smoking to be unhealthy came primarily from peer-reviewed research. In contrast, the studies carried out at the Tobacco Research Institute had a clear potential for bias. The potential for bias does not mean the research was biased, but the fact that it contradicts virtually all other research on the subject should be cause for concern.

EXAMPLE 4 Press Conference Science Suppose the nightly TV news shows scientists at a press conference announcing that they’ve discovered evidence that a newly developed chemical can stop the aging process. The work has not yet gone through the peer review process. Analyze this study according to Guideline 2.

Solution Scientists often announce the results of their research at a press conference so that the public may hear about their work as soon as possible. However, a great deal of exper- tise may be required to evaluate their study for possible biases or other errors—which is the goal of the peer review process. Until the work is peer reviewed and published in a reputable journal, any findings should be considered preliminary—especially about an astonishing claim such as being able to stop the aging process.

Guideline 3: Examine the Sampling Method A statistical study cannot be valid unless the sample is representative of the population under study. Poor sampling methods almost guarantee a biased sample that makes the study results useless.

IS B

N : 0-536-88221-5

1.4 Should You Believe a Statistical Study? 37

Biased samples can arise in many ways, but two closely related problems are particularly common. The first problem, called selection bias (or a selection effect), occurs whenever researchers select their sample in a way that tends to make it unrepresentative of the popula- tion. For example, a pre-election poll that surveys only registered Republicans has selection bias because it is unlikely to reflect the opinions of voters of all parties (and independents).

The second problem, called participation bias, can arise when people choose to be part of a study—that is, when the participants are volunteers. The most common form of participa- tion bias occurs in self-selected surveys (or voluntary response surveys)—surveys or polls in which people decide for themselves whether to participate. In such cases, people who feel strongly about an issue are more likely to participate, and their opinions may not represent the opinions of the larger population that has less emotional attachment to the issue.

By the Way .. . A young pollster named George Gallup conducted his own survey prior to the 1936 election. Sending postcards to only 3,000 randomly selected people, he correctly predicted not only the outcome of the election, but also the outcome of the Literary Digest poll, to within 1%. Gallup went on to establish a very successful polling organization.

Definitions

Selection bias (or a selection effect) occurs whenever researchers select their sample in a biased way.

Participation bias occurs any time participation in a study is voluntary.

A self-selected survey (or voluntary response survey) is one in which people decide for themselves whether to be included in the survey.

CASE STUDY The 1936 Literary Digest Poll

The Literary Digest, a popular magazine of the 1930s, successfully predicted the outcomes of several elections using large polls. In 1936, editors of the Literary Digest conducted a particularly large poll in advance of the presidential election. They randomly chose a sample of 10 million people from various lists, including names in telephone books and rosters of country clubs. They mailed a postcard “ballot” to each of these 10 million people. About 2.4 million people returned the postcard ballots. Based on the returned ballots, the editors of the Literary Digest predicted that Alf Landon would win the presidency by a margin of 57% to 43% over Franklin Roosevelt. Instead, Roosevelt won with 62% of the popular vote. How did such a large survey go so wrong?

The sample suffered from both selection bias and participation bias. The selection bias arose because the Literary Digest chose its 10 million names in ways that favored affluent people. For example, selecting names from telephone books meant choosing only from those who could afford telephones back in 1936. Similarly, country club members are usually quite wealthy. The selection bias favored the Republican Landon because affluent voters of the 1930s tended to vote for Republican candidates.

The participation bias arose because return of the postcard ballots was voluntary, so peo- ple who felt strongly about the election were more likely to be among those who returned their ballots. This bias also tended to favor Landon because he was the challenger—people who did not like President Roosevelt could express their desire for change by returning the postcards. Together, the two forms of bias made the sample results useless, despite the large number of people surveyed.

EXAMPLE 5 Self-Selected Poll The television show Nightline conducted a poll in which viewers were asked whether the United Nations headquarters should be kept in the United States. Viewers could respond to the poll by paying 50 cents to call a “900” phone number with their opinions. The poll drew 186,000 responses, of which 67% favored moving the United Nations out of the United States. Around the same time, a poll using simple random sampling of 500 people found that 72%

By the Way .. . More than a third of all Americans routinely shut the door or hang up the phone when contacted for a survey, thereby making self- selection a problem for legitimate pollsters. One reason people hang up may be the proliferation of selling under the guise of market research (often called “sugging”), in which a telemarketer pretends you are part of a survey in order to try to get you to buy something.

IS B

N : 0

-5 36

-8 82

21 -5

38 Speaking of Statistics

wanted the United Nations to stay in the United States. Which poll is more likely to be repre- sentative of the general opinions of Americans?

Solution The Nightline sample was severely biased. It had selection bias because its sam- ple was drawn only from the show’s viewers, rather than from all Americans. The poll itself was a self-selected survey in which viewers not only chose whether to respond, but also had to pay 50 cents to participate. This cost made it even more likely that respondents would be those who felt a need for change. Despite its large number of respondents, the Nightline survey was therefore unlikely to give meaningful results. In contrast, the simple random sample of 500 people is quite likely to be representative, so the finding of this small survey has a better chance of representing the true opinions of all Americans.

EXAMPLE 6 Planets Around Other Stars Until the mid-1990s, astronomers had never found conclusive evidence for planets outside our own solar system. But improving technology made it possible to begin finding such planets, and more than 200 had been discovered by early 2007. The existing technology makes it easier to find large planets than small ones and easier to find planets that orbit close to their stars than planets that orbit far from their stars. According to the leading theory of solar system forma- tion, large planets that orbit close to their stars should be very rare. But they are fairly common among the first 200 discoveries. Does this mean there is something wrong with the leading theory of solar system formation?

Solution Although the theory suggests that large planets in close orbits should be rare, the technology makes these rare cases the easiest ones to find. The finding of many of these planets may therefore represent a selection effect that biases the sample (of discovered planets) toward a rare type. In fact, this seems almost certainly to be the case, because many “normal” planets are now being discovered, and astronomers have found that with slight modifications the existing theory can account for the rare types found in early discoveries.

Guideline 4: Look for Problems in Defining or Measuring the Variables of Interest Results of a statistical study may be difficult to interpret if the variables under study are difficult to define or measure. For example, imagine trying to conduct a study of how exercise affects resting heart rates. The variables of interest would be amount of exercise and resting heart rate. Both variables are difficult to define and measure. In the case of amount of exercise, it’s not clear what the definition covers—does it include walking to class? Even if we specify the defin- ition, how can we measure amount of exercise given that some forms of exercise are more vigor- ous than others?

How would you measure your resting heart rate? Describe some difficulties in defining and measuring resting heart rate.

EXAMPLE 7 Can Money Buy Love? A Roper poll reported in USA Today involved a survey of the wealthiest 1% of Americans. The survey found that these people would pay an average of $487,000 for true love, $407,000 for great intellect, $285,000 for talent, and $259,000 for eternal youth. Analyze this result accord- ing to Guideline 4.

By the Way .. . NASA’s Kepler mission, scheduled for launch in late 2008, is an orbiting telescope that should be able to detect planets as small as Earth around other stars. Kepler will look for slight dimming of a star’s light each time an orbiting planet passes in front of it (a “transit”), which means it will be able to detect planets only for the small fraction of stars that happen to have their planetary systems aligned with our line of sight.

IS B

N : 0-536-88221-5

1.4 Should You Believe a Statistical Study? 39

Solution The variables in this study are very difficult to define. How, for example, do you define true love? And does it mean true love for a day, a lifetime, or something else? Similarly, does the ability to balance a spoon on your nose constitute talent? Because the variables are so poorly defined, it’s likely that different people interpreted them differently, making the results very difficult to interpret.

EXAMPLE 8 Illegal Drug Supply A commonly quoted statistic is that law enforcement authorities succeed in stopping only about 10% to 20% of the illegal drugs entering the United States. Should you believe this statistic?

Solution There are essentially two variables in a study of illegal drug interception: quantity of illegal drugs intercepted and quantity of illegal drugs NOT intercepted. It should be relatively easy to measure the quantity of illegal drugs that law enforcement offi- cials intercept. However, because the drugs are illegal, it’s unlikely that anyone is reporting the quantity of drugs that are not intercepted. How, then, can anyone know that the inter- cepted drugs are 10% to 20% of the total? In a New York Times analysis, a police officer was quoted as saying that his colleagues refer to this type of statistic as “PFA” for “pulled from the air.”

Guideline 5: Watch Out for Confounding Variables Often, variables that are not intended to be part of the study can make it difficult to interpret results properly. As discussed in Section 1.3, it’s not always easy to discover these confounding variables. Sometimes they are discovered only years after a study is completed, and other times they are not discovered at all, in which case a study’s conclusion may be accepted even though it’s not correct. Fortunately, confounding variables are sometimes more obvious and can be discovered simply by thinking hard about factors that may have influenced a study’s results.

EXAMPLE 9 Radon and Lung Cancer Radon is a radioactive gas produced by natural processes (the decay of uranium) in the ground. The gas can leach into buildings through the foundation and can accumulate to relatively high concentrations if doors and windows are closed. Imagine a (hypothetical) study that seeks to determine whether radon gas causes lung cancer by comparing the lung cancer rate in Colorado, where radon gas is fairly common, with the lung cancer rate in Hong Kong, where radon gas is less common. Suppose the study finds that the lung cancer rates are nearly the same. Would it be reasonable to conclude that radon is not a significant cause of lung cancer?

Solution The variables of interest are amount of radon (an explanatory variable in this case) and lung cancer rate (a response variable) However, radon gas is not the only possible cause of lung cancer. For example, smoking can cause lung cancer, so smoking rate may be a confounding variable in this study—especially because the smoking rate in Hong Kong is much higher than the smoking rate in Colorado. As a result, we cannot draw any conclusions about radon and lung cancer without taking the smoking rate into account (and perhaps other variables as well). In fact, careful studies have shown that radon gas can cause lung cancer, and the U.S. Environmental Protection Agency (EPA) recommends taking steps to prevent radon from building up indoors.

By the Way .. . Many hardware stores sell simple kits that you can use to test whether radon gas is accumulating in your home. If it is, the problem can be eliminated by installing an appropriate “radon mitigation” system, which usually consists of a fan that blows the radon out from under the house before it can get into the house.

IS B

N : 0

-5 36

-8 82

21 -5

40 Speaking of Statistics

Guideline 6: Consider the Setting and Wording in Surveys Even when a survey is conducted with proper sampling and with clearly defined terms and questions, you should watch for problems in the setting or wording that might produce inaccu- rate or dishonest responses. Dishonest responses are particularly likely when the survey con- cerns sensitive subjects, such as personal habits or income. For example, the question “Do you cheat on your income taxes?” is unlikely to elicit honest answers from those who cheat, unless they are assured of complete confidentiality (and perhaps not even then).

In other cases, even honest answers may not really be accurate if the wording of ques- tions invites bias. Sometimes just the order of words in a question can affect the outcome. A poll conducted in Germany asked the following two questions.

• Would you say that traffic contributes more or less to air pollution than industry? • Would you say that industry contributes more or less to air pollution than traffic?

The only difference is the order of the words traffic and industry, but this difference dramati- cally changed the results: With the first question, 45% answered traffic and 32% answered industry. With the second question, only 24% answered traffic while 57% answered industry.

EXAMPLE 10 Do You Want a Tax Cut? At a time when the U.S. government was running annual budget surpluses, Republicans in Congress proposed a tax cut and the Republican National Committee commissioned a poll to find out whether Americans supported the proposal. Asked “Do you favor a tax cut?,” 67% of respondents answered yes. Should we conclude that Americans supported the proposal?

Solution A question like “Do you favor a tax cut?” is biased because it does not give other options. In fact, an independent poll conducted at the same time gave respondents a list of options for using surplus revenues. This poll found that 31% wanted the money devoted to Social Security, 26% wanted it used to reduce the national debt, and only 18% favored using it for a tax cut. (The remaining 25% of respondents chose a variety of other options.)

EXAMPLE 11 Sensitive Survey Two surveys asked Catholics in the Boston area whether contraceptives should be made avail- able to unmarried women. The first survey involved in-person interviews, and 44% of the respondents answered yes. The second survey was conducted by mail and telephone, and 75% of the respondents answered yes. Which survey was more likely to be accurate?

Solution Contraceptives are a sensitive topic, particularly among Catholics (because the Catholic Church officially opposes contraceptives). The first survey, with in-person interviews, may have encouraged dishonest responses. The second survey made responses seem more private and therefore was more likely to reflect the respondents’ true opinions.

Guideline 7: Check That Results Are Fairly Represented in Graphics or Concluding Statements Even when a statistical study is done well, it may be misrepresented in graphics or concluding statements. Researchers occasionally misinterpret the results of their own studies or jump to conclusions that are not supported by the results, particularly when they have personal biases. News reporters may misinterpret a survey or jump to unwarranted conclusions that make a story seem more spectacular. Misleading graphics are especially common (we will devote

By the Way .. . People are more likely to choose the item that comes first in a survey because of what psychologists call the availability error—the tendency to make judgments based on what is available in the mind. Professional polling organizations take great care to avoid this problem; for example, they may pose a question with two choices in one order to half the people in the sample and in the opposite order to the other half.

IS B

N : 0-536-88221-5

1.4 Should You Believe a Statistical Study? 41

much of Chapter 3 to this topic). You should always look for inconsistencies between the inter- pretation of a study (in pictures and in words) and any actual data given along with it.

EXAMPLE 12 Does the School Board Need a Statistics Lesson? The school board in Boulder, Colorado, created a hubbub when it announced that 28% of Boulder school children were reading “below grade level” and hence concluded that methods of teaching reading needed to be changed. The announcement was based on reading tests on which 28% of Boulder school children scored below the national average for their grade. Do these data support the board’s conclusion?

Solution The fact that 28% of Boulder children scored below the national average for their grade implies that 72% scored at or above the national average. Thus, the school board’s ominous statement about students reading “below grade level” makes sense only if “grade level” means the national average score for a particular grade. This interpretation of “grade level” is curious because it would imply that half the students in the nation are always below grade level—no matter how high the scores. It may still be the case that teaching methods needed to be improved, but these data did not justify that conclusion.

Guideline 8: Stand Back and Consider the Conclusions Finally, even if a study seems reasonable according to all the previous guidelines, you should stand back and consider the conclusions. Ask yourself questions such at these:

• Did the study achieve its goals? • Do the conclusions make sense? • Can you rule out alternative explanations for the results? • If the conclusions make sense, do they have any practical significance?

EXAMPLE 13 Extraordinary Claims Suppose a (hypothetical) study concludes that wearing a gold chain increases your chances of surviving a car accident by 10%. The claim is based on a statistical analysis of data about survival rates and what people were wearing. Careful analysis of the research shows that it was conducted properly and carefully. Should you start wearing a gold chain whenever you drive a car?

Solution Despite the care that went into the study, the claim that a gold chain can save your life in a collision is difficult to believe. After all, how could a thin chain help in a high- speed collision? It’s certainly possible that some unknown effect of gold chains makes the con- clusion correct, but it seems far more likely that the results were either a fluke or due to an unidentified confounding variable (for example, perhaps those who wear gold chains are wealthier and drive newer cars with more advanced safety features, lowering their fatality rate).

EXAMPLE 14 Practical Significance An experiment is conducted in which the weight losses of people who try a new “Fast Diet Sup- plement” are compared to the weight losses of a control group of people who try to lose weight in other ways. After eight weeks, the results show that the treatment group lost an average of one-half pound more than the control group. Assuming that it has no dangerous side effects, does this study suggest that the Fast Diet Supplement is a good treatment for people wanting to lose weight?

Solution Compared to the average person’s body weight, a weight loss of one-half pound hardly matters at all. So while loss results may be interesting, they don’t seem to have much practical significance.

Extraordinary claims require extraordinary evidence.

—Carl Sagan, astronomer and Pulitzer Prize winner

IS B

N : 0

-5 36

-8 82

21 -5

42 Speaking of Statistics

Section 1.4 Exercises

Statistical Literacy and Critical Thinking

1. Peer Review. What is peer review? How is it useful?

2. Selection Bias and Participation Bias. Describe and contrast selection bias and participation bias in sampling.

3. Self-Selected Surveys. Why are self-selected surveys always prone to participation bias?

4. Confounding Variables. What are confounding vari- ables, and what problems can they cause?

5. Survey Validity. The TV survey got more than 1 million phone-in responses, so it is clearly more valid than the sur- vey by the professional pollsters, which involved inter- views with only a few hundred people.

6. Survey Location. The survey of religious beliefs suf- fered from selection bias because the questionnaires were handed out only at Catholic churches.

7. Vitamin C and Colds. My experiment proved beyond a doubt that vitamin C can reduce the severity of colds, because I controlled the experiment carefully for every possible confounding variable.

8. Faster Jogging. Everyone who jogs for exercise should try the new training regimen, because careful studies sug- gest it can increase your speed by 1%.

Concepts and Applications

Applying Guidelines. In Exercises 9–16, determine which of the eight guidelines for evaluating statistical studies appears to be most relevant. Explain your reasoning.

9. Quality Control. The public relations department of Telektronics conducted a study of the defect rate of the circuit breakers manufactured by the company.

10. Gender Selection. After a test of a gender selection method designed to increase the likelihood of a baby girl, it was found that 23 babies were girls and 21 were boys, so a company spokesperson stated that the method is ef- fective most of the time.

11. Ethics. In a study of 250 attorneys, each was asked whether he or she has good ethics.

12. Agriculture. Researchers conclude that an irrigation sys- tem used to grow tomatoes in California is more effective than a competing system used in Arizona.

13. Prisoner Survey. Prisoners attending classes at the college of one of the authors were mailed surveys about recidivism, and 10% of them responded.

14. Election Poll. Under the headline “Turner predicted to win in a landslide,” it was reported that Turner received 55% of the votes in a pre-election poll, compared with 45% for her opponent.

15. Nuclear Energy Poll. Randomly selected adults were asked this question: “Do you agree or disagree with increasing the production of nuclear energy that could potentially kill thousands of innocent people?”

16. Counterfeit Goods. A consortium of manufacturers plans a study designed to compare the value of counterfeit goods produced in the United States in the year 2000 to the value of those produced in the current year.

Bias. In Exercises 17–20, identify and explain at least one source of bias in each study described. Then suggest how the bias might have been avoided.

17. Healthy Chocolate. The New York Times published an article that included these statements: “At long last, choco- late moves toward its rightful place in the food pyramid, somewhere in the high-tone neighborhood of red wine, fruits and vegetables, and green tea. Several studies, re- ported in the Journal of Nutrition, showed that after eating chocolate, test subjects had increased levels of antioxi- dants in their blood. Chocolate contains flavonoids, antioxidants that have been associated with decreased risk of heart disease and stroke. Mars Inc., the candy company, and the Chocolate Manufacturers Association financed much of the research.”

18. Famous Book. When author Shere Hite wrote Women and Love: A Cultural Revolution in Progress, she based conclusions about the general population of all women on 4,500 replies that she received after mailing 100,000 ques- tionnaires to various women’s groups.

19. Napster. Newsweek magazine ran a survey about the Napster Web site for downloading music. Readers could register their responses on Newsweek’s Web site.

20. Survey Method. You plan to conduct a survey to find the percentage of people in your state who can name the Lieutenant Governor (who plans to run for the United States Senate). You obtain addresses from a list of property

IS B

N : 0-536-88221-5

1.4 Should You Believe a Statistical Study? 43

owners in the state and you mail a survey to 850 randomly selected people from the list.

21. It’s All in the Wording. Princeton Survey Research Asso- ciates did a study for Newsweek magazine illustrating the effects of wording in a survey. Two questions were asked:

• Do you personally believe that abortion is wrong?

• Whatever your own personal view of abortion, do you favor or oppose a woman in this country having the choice to have an abortion with the advice of her doctor?

To the first question, 57% of the respondents replied yes, while 36% responded no. In response to the second ques- tion, 69% of the respondents favored the choice, while 24% opposed the choice. Discuss why the two questions pro- duced seemingly contradictory results. How could the re- sults of the questions be used selectively by various groups?

22. Tax or Spend? A Gallup poll asked the following two questions:

• Do you favor a tax cut or “increased spending on other government programs”? Result: 75% for the tax cut.

• Do you favor a tax cut or “spending to fund new retire- ment savings accounts, as well as increased spending on education, defense, Medicare and other programs”? Result: 60% for the spending.

Discuss why the two questions produced seemingly con- tradictory results. How could the results of the questions be used selectively by various groups?

What Do You Want to Know? Exercises 23–26 pose two re- lated questions that might form the basis of a statistical study. Briefly discuss how the two questions differ and how these differ- ences would affect the goal of a study and the design of the study.

23. Internet Dating

First question: What percentage of Internet dates lead to marriage?

Second question: What percentage of marriages begin with Internet dates?

24. Full-Time Faculty

First question: What percentage of introductory classes on campus are taught by full-time faculty members?

Second question: What percentage of full-time faculty members teach introductory classes?

25. Binge Drinking

First question: How often do college students binge drink?

Second question: How often is binge drinking done by college students?

26. Statistics Courses

First question: What is the proportion of college graduates who have taken a statistics course?

Second question: What proportion of all statistics courses are taken by college students?

Accurate Headlines? Exercises 27 and 28 give a headline and a brief description of the statistical news story that accom- panied the headline. In each case, discuss whether the headline accurately represents the story.

27. Headline: “Drugs shown in 98 percent of movies”

Story summary: A “government study” claims that drug use, drinking, or smoking was depicted in 98% of the top movie rentals (Associated Press).

28. Headline: “Sex more important than jobs”

Story summary: A survey found that 82% of 500 people interviewed by phone ranked a satisfying sex life as important or very important, while 79% ranked job satisfaction as important or very important (Associated Press).

Stat-Bytes. Politicians commonly believe that they must make their political statements (often called sound-bytes) very short because the attention span of listeners is so short. A similar effect occurs in reporting statistical news. Major statistical studies are often reduced to one or two sentences. The summaries of statistical reports in Exercises 29–32 are taken from various news sources. Describe what crucial information is missing in the given statement and what more you would want to know before you acted on the report.

29. Confidence in Military. USA Today reports on a Harris poll claiming that the percentage of adults with a “great deal of confidence” in military leaders stands at 54% (up from 37% in 1997).

30. Top Restaurants. CNN reports on a Zagat Survey of America’s Top Restaurants which found that “only nine restaurants achieved a rare 29 out of a possible 30 rating and none of those restaurants are in the Big Apple.”

31. Forecasting Weather. A USA Today headline reported that “More companies try to bet on forecasting weather.” The article gave examples of companies believing that long-range forecasts are reliable, and four companies were cited.

32. Births in China. A USA Today headline reported that “China thrown off balance as boys outnumber girls,” and the accompanying graph showed that for every 100 girls born in China, 116.9 boys are born.IS

B N

: 0 -5

36 -8

82 21

-5

44 Speaking of Statistics

Projects for the Web and Beyond

For useful links, select “Links for Internet Projects” for Chapter 1 at www.aw.com/bbt.

33. Analyzing a Statistical Study. Find a detailed report on some recent statistical study of interest to you. Write a short report applying each of the eight guidelines given in this section. (Some of the guidelines may not apply to the particular study you are analyzing; in that case, explain why the guideline is not applicable.)

34. Harper’s Index. Go to the Web site for Harper’s Index and examine a few of the recently quoted statistics. If pos- sible, obtain the sources for the statistics. Choose three statistics that you find particularly interesting and discuss whether you believe them, based on the guidelines given in this section.

35. Twin Studies. Researchers doing statistical studies in biology, psychology, and sociology are grateful for the existence of twins. Twins can be used to study whether certain traits are inherited from parents (nature) or acquired from one’s surroundings during upbringing (nur- ture). Identical twins are formed from the same egg in the mother and have the same genetic material. Fraternal twins are formed from two separate eggs and share roughly half of the same genetic material. Find a pub- lished report of a twin study. Discuss how identical and fraternal twins are used to form case and control groups. Apply Guidelines 1–8 to the study and comment on whether you find the conclusions of the report convincing.

36. American Demographics. Consult an issue of American Demographics, a nontechnical magazine that

specializes in summarizing statistical studies involving Americans and their lifestyles. Select one specific article and use the ideas of this section to summarize and evalu- ate the study. Be sure to cite information that you believe is missing and should be provided for you to make a com- plete analysis.

37. Applying the Guidelines. Find a recent newspaper article or television report about a statistical study on a topic that you find interesting. Write a short report applying each of the eight guidelines given in this sec- tion. (Some of the guidelines may not apply to the par- ticular study you are analyzing; in that case, explain why the guideline is not applicable.)

38. Believable Results. Find a recent news report about a statistical study whose results you believe are mean- ingful and important. In one page or less, summarize the study and explain why you find it believable.

39. Unbelievable Results. Find a recent news report about a statistical study whose results you don’t believe are meaningful and important. In one page or less, sum- marize the study and explain why you don’t believe its claims.

40. Self-Selected Survey. Find an example of a recent survey in which the sample was self-selected. Describe the makeup of the sample and how you think the self- selection affected the results of the survey.

IS B

N : 0-536-88221-5

Chapter Review Exercises

Chapter Review Exercises 45

1. Guns in Homes. A Gallup poll involved a survey of adults selected in the United States. They were asked this question: “Do you have a gun in your home?” Among the 1,012 responses received, 38% were yes. The margin of error was reported as 3 percentage points.

a. Interpret the margin of error by identifying the range of values likely to contain the proportion of all house- holds with guns.

b. Identify the population.

c. Is this study an experiment or an observational study? Explain. Identify the variable of interest.

d. Is the reported value of 38% a population parameter or a sample statistic? Why?

e. If you learned that survey subjects responded to a mag- azine article asking readers to phone in their responses, would you consider the survey results to be valid? Why or why not?

f. Describe a procedure for selecting the survey subjects using a simple random sample.

g. Describe a procedure for selecting similar survey subjects using stratified sampling.

h. Describe a procedure for selecting similar survey sub- jects using cluster sampling.

i. Describe a procedure for selecting similar survey subjects using systematic sampling.

j. Describe a procedure for selecting similar survey sub- jects using a convenience sample.

2. Simple Random Sample. An important element of this chapter is the concept of a simple random sample.

a. What is a simple random sample?

b. When the Bureau of Labor Statistics conducts a survey, it begins by partitioning the United States adult population into 2,007 groups called primary sampling units. Assume that these primary sampling units all contain the same number of adults. If you randomly select one adult from each primary sam- pling unit, is the result a simple random sample? Why or why not?

c. Refer to the primary sampling units described in part b and describe a sampling plan that results in a simple random sample.

3. Testing Zocor. In clinical trials of the drug Zocor, used to treat high cholesterol levels, 1,583 Zocor users were observed for adverse reactions. It was found that 3.5% of the Zocor users experienced headaches.

a. Based on the given information, can you conclude that in some cases Zocor causes headaches? Why or why not?

b. While 1,583 subjects were treated with Zocor, another 157 subjects were given a placebo, and 5.1% of the placebo group experienced headaches. What does this additional information suggest about headaches as an adverse reaction to the use of Zocor?

c. In this clinical trial, what is blinding and why is it important in testing the effects of Zocor?

d. Is this clinical trial an observational study or an experi- ment? Explain.

e. What is an experimenter effect, and how might this effect be minimized?

4. Wording of a Survey Question. In The Superpollsters, David W. Moore describes an experiment in which differ- ent subjects were asked if they agree with the following statements:

• Too little money is being spent on welfare.

• Too little money is being spent on assistance to the poor.

Even though it is the poor who receive welfare, only 19% agreed when the word welfare was used, but 63% agreed with assistance to the poor.

a. Which of the two questions should be used in a sur- vey? Why?

b. If you were working on a campaign for a conservative candidate for Congress and you wanted to emphasize opposition to the use of federal funds for assistance to the poor, which of the two questions would you use? Why?

c. Is it ethical to deliberately word a survey question so as to influence responses? Why or why not?

IS B

N : 0

-5 36

-8 82

21 -5

Chapter Quiz

46 Speaking of Statistics

Choose the best answer to each of the following questions. Explain your reasoning with one or more complete sentences.

1. You conduct a poll in which you randomly select 1,000 reg- istered voters from Texas and ask if they approve of the job their governor is doing. The population for this study is

a. all registered voters in the state of Texas.

b. the 1,000 people that you interview.

c. the Governor of Texas.

2. For the poll described in Exercise 1, your results would most likely suffer from bias if you chose the participants from

a. all registered voters in Texas.

b. all people with a Texas drivers license.

c. people who donated money to the Governor’s campaign.

3. When we say that a sample is representative of the popu- lation, we mean that

a. the results found for the sample are similar to those we would find for the entire population.

b. the sample is very large.

c. the sample was chosen in the best possible way.

4. Consider an experiment designed to see whether cash incentives can improve school attendance. The researcher chooses two groups of 100 high school students. She offers one group $10 for every week of perfect attendance. She tells the other group that they are part of an experi- ment but does not give them any incentive. The students who do not receive an incentive represent

a. the treatment group.

b. the control group.

c. the observation group.

5. The experiment described in Exercise 4 is

a. single-blind.

b. double-blind.

c. not blind.

6. The purpose of a placebo is

a. to prevent participants from knowing whether they belong to the treatment group or the control group.

b. to distinguish between the cases and the controls in a case-control study.

c. to determine whether diseases can be cured without any treatment.

7. If we see a placebo effect in an experiment to test a new treatment designed to cure warts,

a. the experiment was not properly double-blind.

b. the experimental groups were too small.

c. warts were cured among members of the control group.

8. An experiment is single-blind if

a. it lacks a treatment group.

b. it lacks a control group.

c. the participants do not know whether they belong to the treatment or the control group.

9. Poll X predicts that Powell will receive 49% of the vote, while Poll Y predicts that she will receive 53% of the vote. Both polls have a margin of error of 3 percentage points. What can you conclude?

a. One of the two polls must have been conducted poorly.

b. The two polls are consistent with each another.

c. Powell will receive 51% of the vote.

10. A survey reveals that 12% of Americans believe Elvis is still alive, with a margin of error of 4 percentage points. The confidence interval for this poll is

a. from 10% to 14%.

b. from 8% to 16%.

c. from 4% to 20%.

11. A study conducted by Exxon Mobil shows that there was no lasting damage from a large oil spill in Alaska. This conclusion

a. is definitely invalid, because the study was biased.

b. may be correct, but the potential for bias means you should look very closely at how the conclusion was reached.

c. could be correct if it falls within the confidence inter- val of the study.

IS B

N : 0-536-88221-5

Chapter Quiz 47

12. The television show American Idol selects winners from votes cast by anyone who wants to vote. This means the winner

a. is the person most Americans want to win.

b. may or may not be the person most Americans want to win, because the voting is subject to participation bias.

c. may or may not be the person most Americans want to win, because the voting should have been double- blind.

13. Consider an experiment in which you measure the weights of 6-year-olds. The variable of interest in this study is

a. the size of the sample.

b. the weights of 6-year-olds.

c. the ages of the children under study.

14. Imagine a survey of randomly selected people in which it is found that people who use sunscreen were more likely to have been sunburned in the past year. Which explana- tion for this result seems most likely?

a. Sunscreen is useless.

b. The people in this study all used sunscreen that had passed its expiration date.

c. People who use sunscreen are more likely to spend time in the sun.

15. If a statistical study is carefully conducted in every possi- ble way, then

a. its results must be correct.

b. we can have confidence in its results, but it is still pos- sible that they are not correct.

c. we say that the study is perfectly biased.

IS B

N : 0

-5 36

-8 82

21 -5

Does Daycare Breed Bullies? Decades ago, it was almost a given that mothers would stay at home to take care of their young children. But with more women in the work- force, more children are spending more of their waking hours in some type of daycare. In 1975, fewer than 40% of women with children under age 6 worked full- or part-time outside the home. Today, more than 60% of such mothers do. This dramatic increase in the number of chil- dren in daycare led psychologists and sociologists to study the effects of daycare. In 1991, the National Institute of Child Health and Human Development (NICHD) started the Early Child Care study, which tracked more than 1,300 children enrolled in daycare at 10 different re- search sites around the United States and continued to track them through age 15. Notice that this set-up makes it an observational, prospective study (see Section 1.3).

The study has been conducted carefully by outstanding researchers, but some results have still provoked controversy. As an example, consider this result reported in 2001: Young children who spend more time in daycare are significantly more likely to be aggressive bullies as they get older. Working mothers everywhere felt pangs of guilt as they worried about the effects of daycare on their children, and the news media reported widely that daycare breeds bullies. But should we believe this claim? There are many ways to evaluate the claim, but for practice, let’s use the eight guidelines given in Section 1.4 (p. 34).

Guideline 1: The goal of the study seems clear (to learn about effects of daycare), as does the population under study (all children in daycare). However, the type of study raises a poten- tial problem. The study is observational, rather than a randomized experiment, because it would be unethical to tell parents whether or not to use daycare. Unfortunately, while an obser- vational study can help establish a link (or correlation) between variables, it cannot by itself establish cause and effect (see Chapter 7). In this case, the study provides evidence of a link between daycare and bullying, but does not tell us whether one is the cause of the other. The problem is that, because parents chose whether their children were in daycare or at home, we cannot be sure that the sample and control groups are truly comparable. For example, it may be that mothers who must juggle children and work tend to be more stressed than nonworking mothers, which in turn might make daycare children more stressed than their stay-at-home counterparts. Or, it could be that mothers with rambunctious children are more likely to put their children in daycare. Thus, regardless of the validity of the link between daycare and bul- lying, the observational nature of the study makes it impossible to support a claim that daycare causes the bullying.

Guideline 2: The study has been conducted with care, and we have no reason to suspect any particular bias on the part of the researchers. However, daycare is an emotionally charged issue and researchers may have strong opinions about it, raising the potential for bias. Indeed, different researchers have interpreted the study results differently, which suggests that bias may play a role in the interpretations, if not in the study itself.

Guideline 3: Researchers used a type of stratified sampling to recruit participants for the study, because they wanted to match national demographics for factors such as socioeconomic background, race, and family structure. This type of sampling seems appropriate to the study. However, the families that were recruited had a choice about whether to participate in the

FOCUS ON SOCIOLOGY

IS B

N : 0-536-88221-5

study. This self-selection automatically raises the potential for participation bias, which may affect the study’s results.

Guideline 4: The claim about daycare and bullying essentially involves two variables: the amount of time a child spends in daycare and the child’s level of aggressiveness. Both are very difficult to define. For example, the Early Child Care study defines daycare as care pro- vided by anyone other than the mother, which means that care by fathers or grandparents counts as daycare. Many people object to this definition. Defining aggression is also diffi- cult; for example, it may be difficult to distinguish between active play and true aggression.

Guideline 5: There are many potentially confounding variables in the Early Child Care study. As noted earlier, the study does not provide data about whether mothers of children in daycare are subject to more stress than stay-at-home moms, which could confound the results. Perhaps of greater concern, there is no accepted way to measure or control for the quality of daycare. As a result, even some of the researchers involved in the study have suggested that the observed bullying is a result of poor or mediocre daycare rather than of daycare in general.

Guideline 6: The researchers used interviews to determine the type and amount of daycare and the level of aggression for each child. But because aggressiveness is difficult to define and may be thought of differently by different people, there is no clear way to ensure that all interviewers ask questions in precisely the same way (not only in wording, but also in facial expression and intonation) and that all respondents interpret the ques- tions similarly.

Guideline 7: The reports produced by the Early Child Care researchers acknowledge potential flaws in the study and emphasize that the study shows only a possible link between daycare and bullying. So where does the claim that daycare breeds bullies come from? It comes from the way the findings are presented in the media. Because the actual results do not support this interpretation, the news reports clearly reflect either bias or misunderstanding. For example, journalists who have not studied statistics may not be aware of the difference between finding a link and finding a cause, and their misunderstanding is then reflected in their news reports. Alternatively, the media might deliberately sensationalize the results in order to attract more attention. And in some cases, people who believe that women should stay home with their children might misrepresent the results so as to claim that the study lends scientific support to their views.

Guideline 8: The question of practical significance is very important with this study. On an individual level, many working women don’t believe they have a choice about daycare; they must work for either financial or professional reasons. They might therefore use daycare even if there were proof that home care was better. On the societal level, the findings might be practically significant if they led to new policies that could improve child care generally. But as long as the results do not yet prove cause and effect, they cannot give clear guidance to policy makers.

In summary, we’ve found that a provocative headline about daycare and bullying falls apart upon close scrutiny. The research might still be quite valuable, especially because the researchers themselves are aware of potential flaws in the study. The results may lead to future studies with more conclusive results—information that would surely be appreciated by working parents everywhere. But until then, we need to be careful about how we interpret the Early Child Care study results, and even more careful about how we react to any sensational headlines the study may produce.

QUESTIONS FOR DISCUSSION 1. A third potential confounding variable is amount of time children spend watching televi-

sion and playing video games, which has also been linked to aggressive behavior. Do you think this variable might affect the Early Child Care study results and, if so, how? What other confounding variables might affect its results? Explain your answers clearly.

Focus on Sociology 49 IS

B N

: 0 -5

36 -8

82 21

-5

2. Suppose you were designing a follow-up study intended to give more conclusive results. Based on what has been learned from the Early Child Care study, what would you do dif- ferently?

3. What is your personal opinion about whether it is better for a parent to stay at home or to work? Would your opinion be swayed if the Early Child Care study produced any defini- tive results about daycare? Why or why not?

4. Overall, do you think the Early Child Care study was a worthwhile effort in terms of the time and expense it required? Defend your opinion.

SUGGESTED READING Read more about the Study of Early Child Care (SECC) on its Web site at http://secc.rti.org.

Carey, Benedict. “Study Finds Rise in Behavior Problems After Significant Time in Day Care.” New York Times, March 26, 2007.

Stolberg, Sheryl Gay. “Science, Studies, and Motherhood.” New York Times, April 22, 2001.

Sweeney, Jennifer Foote. “The Day-Care Scare, Again.” Salon.com, April 20, 2001.

50 Speaking of Statistics IS

B N

: 0-536-88221-5

Is Your Lifestyle Healthy? Consider the following findings from statistical studies:

• Smoking increases the risk of heart disease. • Eating margarine can increase the risk of heart disease. • One glass of wine per day can protect against heart disease, but

increases the risk of breast cancer.

You are probably familiar with some of these findings, and perhaps you’ve even altered your lifestyle as a result of them. But where do they come from? Remarkably, these and hundreds of other important findings on public health come from a huge prospective study that has provided data for hundreds of smaller statistical studies. Known as the Harvard Nurses’ Health Study, it is the most enduring public health study ever undertaken. If it has not already changed the way you live, it almost certainly will in the future.

The Harvard Nurses’ Health Study began in 1976 when Dr. Frank E. Speizer, a professor at Harvard Medical School, decided to study the long-term effects of oral contraceptives. He mailed questionnaires to approximately 370,000 registered nurses and received more than 120,000 responses. He chose to survey nurses because he believed that their medical training would make their responses more reliable than those of the general public.

As Dr. Speizer and his colleagues sifted through the data in the returned questionnaires, they realized that the study could be expanded to include more than just the effects of contra- ceptives. Today, this research team continues to follow some 90% of the original 120,000 respondents. Hundreds of other researchers also use the data to study public health.

Annual questionnaires are still a vital part of the study, allowing researchers to gather data about what the nurses eat; what medicines and vitamins they take; whether and how much they exercise, drink, and smoke; and what illnesses they have contracted. Some of the nurses also provide blood samples, which are used to measure such things as choles- terol level, hormone levels, genetic variations, and residues from pesticides and environ- mental pollutants. Dr. Speizer’s faith in nurses has proven justified, as they reliably complete surveys and almost always provide properly drawn and labeled blood samples upon request.

After more than 30 years of correspondence, both the researchers and the nurses say they feel a sense of closeness. Many of the nurses look forward to hearing from the researchers and say that the study has helped them to pay more attention to how they live their lives. Researchers feel deep sorrow when they must record the death of one of the nurses.

The sorrow of death will play an increasing role in the study, as many of the nurses are now entering their 70s and 80s. But this sorrow will also yield a wealth of valuable data about factors that influence longevity and health in old age. Researchers hope that the data will point the way toward definitive understanding about what constitutes a healthy diet, how pollution and chemical exposure influence health, and how we might prevent debilitating diseases like osteoporosis and Alzheimer’s. In death, the 120,000 women of the Harvard Nurses’ Health Study may give the gift of a better life to future generations.

FOCUS ON PUBLIC HEALTH

IS B

N : 0

-5 36

-8 82

21 -5

QUESTIONS FOR DISCUSSION 1. Consider some of the results that are likely to come from the Harvard Nurses’ Health

Study over the next 10–20 years. What types of results do you think will be most impor- tant? Do you think the findings will alter the way you live your life?

2. Explain why the Harvard Nurses’ Health Study is an observational study. Critics some- times say that the results would be more valid if obtained by experiments rather than observations. Discuss whether it would be possible to gather similar data by carrying out experiments in a practical and ethical way.

3. In principle, the Harvard Nurses’ Health Study is subject to participation bias because only 120,000 of the original 370,000 questionnaires were returned. Should the researchers be concerned about this bias? Why or why not?

4. Another potential pitfall comes from the fact that the questionnaires often deal with sensi- tive issues of personal health, and researchers have no way to confirm that the nurses an- swer honestly. Do you think that dishonesty could be leading researchers to incorrect conclusions? Defend your opinion.

5. All of the participants in the Harvard Nurses’ Health Study are women. Do you think that the results also are of use to men? Why or why not?

6. Do a Web search for news articles that discuss results from the Harvard Nurses’ Health Study. Choose one recent result that interests you, and discuss what it means and how it may affect public health or your own health in the future.

SUGGESTED READING Read more about the Harvard Nurses’ Health Study (headquartered at Harvard’s Channing Labora- tory) on its Web site at http://www.nurseshealthstudy.org.

Brophy, Beth. “Doing It for Science.” U.S. News & World Report, Vol. 126, March 21, 1999, p. 67.

Conwell, Vikki. “Forever Friends: Ties Can Last a Lifetime and Sustain Body and Soul.” Atlanta Journal-Constitution, January 2, 2007.

Parker-Pope, Tara. “Drinking Has Hidden Health Risks for Women.” Wall Street Journal, Decem- ber 26, 2006.

Yoon, Carol Kaesuk. “In Nurses’ Lives, a Treasure Trove of Health Data.” New York Times, September 15, 1998.

52 Speaking of Statistics IS

B N

: 0-536-88221-5

boom:

help: If you are experiencing problems unlocking this document or you have questions regarding Protectedpdf files please contact a Technical Support representative: In the United States: 1-877-832-4867 In Canada: 1-800-859-3682 Outside the U.S. and Canada: 1-602-387-2222 Email: [email protected].
protectedpdf_version:
abstract:
logo:
unlocked:

rememberme:
rememberme_label: Remember my user name and password.

title: Statistical Reasoning Ch01
version: 2012-12-07
form:

neversend:

pwd:

response:

docid: 1000-0001-6FE7-000FD42A
username:

username_label: User Name:
pwd_label: Password:
login:

subtitle:
text:

instructions: This is a Protected PDF document. Please enter your user name and password to unlock the text.

info: COSString{Please Login to View}

boom.info_publisher: This document is published by: Apollo Group, Inc.