ACADEMIC GIANT Statistic Research Methods

Statistical equivalence: Seeing Is Believing

14.1 Take an ordinary deck of 52 playing cards, shuffle the cards well, and deal them into two piles of 26. Notice that you have randomly assigned the cards to two groups. Now, count the number of red cards in each group. Is it exactly 13, or a little less or more? Reshuffle the deck and repeat (or have different groups or individuals in class doing this at the same time). Average the results together.

a. How does this card game illustrate the idea of statistical equivalence?

b. Why isn’t the results exactly 13 red cards after each shuffle? How does this shed light on the use of significance test in a randomized experiment?

Random Assignment versus Random Sampling

14.2 Below are several examples of research. For each one, decide if it involves random assignment or random sampling.

a. A law center selects 900 civil court records at random from the archive to investigate the effect of having, versus not having, legal representation (a lawyer) on the outcome of the case.

b. A clinic recruits drug users and divides them, randomly, into two groups to test the effectiveness of a new prevention therapy.

c. A government statistical agency takes a probability sample of the nation’s public school students and then randomly administers two different versions of a reading test to compare their difficulty.

Treatment for Alcoholism

14.3 A large randomized field experiment of treatment for alcoholics was conducted with 10,000 subjects. They were randomized to either an intensive short-term inpatient treatment program or a less intensive long-term outpatient treatment program with equal total costs. The main outcome measure is long-term (5 year) sobriety. Results showed that 35% of those in the inpatient program remained sober for 5 years, while 55% of those in the outpatient program remained sober for 5 years.

a. The p Value associated with a test of the difference between the programs was less than .001. What is the practical significance of these results?

b. What is the practical significance of these results?

c. The study was performed according to standard ethical principles of informed consent: Subjects were fully informed about what the study would entail and voluntarily chose to participate. What kinds of alcoholics would agree to participate in the experiment? What kinds would not?

d. Someone reading this impressive study concludes that long-term sobriety rates of all alcoholics would be 20 percentage points higher if all alcoholics received long-term outpatient treatment than if all alcoholics received short-term inpatient treatment. Do you think this conclusion is valid? Explain.

e. The study as described does not shed light on why the outpatient program had better results—the mechanism. Describe how the use of one qualitative method (e.g., open-ended interviews, structured interviews, focus groups, observation) during the experiment could have shed light on this issue. Briefly describe which method you think would have been best and why.

Chapter 15

Family Income and Child Mental Health

15.1 Think of a variety of different casual pathways that could explain the correlation between the family income of a child and the child’s mental health. Include examples of mechanisms (interviewing variables) for causation, common causes, and reverse causation.

a. Draw path diagrams to illustrate.

b. What counterfactual question would you like to answer to address the casual question of the effect of poverty on mental health?

c. Describe a randomized experiment that could be used to answer this question and that meets ethical standards.

d. Compare the validity of the causal conclusions of the casino natural experience with that of the randomized standards.

e. Compare the generalizability of the casino natural experiment with that of the randomized experiment.

Noise, Student Learning, and Diligent Teachers

15.2 Recall the Bronzaft and McCarthy (1975) study of how train noise affected student learning. How might a conscientious teacher respond to the noise? Might such a teacher aggressively pursue a classroom on the less noisy side of the building? If so, how might that affect your conclusions from this study? Explain

HMOs and Medical Care Usage

13.3 Suppose two manufacturing companies merge and one company switches to the more restrictive health insurance options (only closed-panel health maintenance organizations, or HMOs) of the other company.

a. Explain why this provides a natural experiment that could be used to evaluate the effects of closed-panel HMOs on medical care usage.

b. Describe the date that you would want to collect for such a study.

c. Describe how you would analyze the date.

d. Discuss the validity of the causal conclusions. What weaknesses are there?

e. Discuss the generalizability of the study.