6-8 probability and statistic math problem

AN INTRODUCTION TO MATHEMATICAL STATISTICS

AND ITS APPLICATIONS Sixth Edition

Richard J. Larsen Vanderbilt University

Morris L. Marx University of West Florida

Director, Portfolio Management: Deirdre Lynch Courseware Portfolio Manager: Patrick Barbera Courseware Portfolio Assistant: Justin Billing Content Producer: Kathleen A. Manley Managing Producer: Karen Wernholm Media Producer: Audra Walsh Product Marketing Manager: Yvonne Vannatta Product Marketing Assistant: Jennifer Myers Senior Author Support/Technology Specialist: Joe Vetere Cover and Text Design, Production Coordination, Composition, and Illustrations:

iEnergizer Aptara®, Ltd. Cover Image: Glow Green Circle Arcs © Shutterstock/R.T. Wohlstadter

Copyright © 2018, 2012, 2006 Pearson Education, Inc. All Rights Reserved. Printed in the United States of America. This publication is protected by copyright, and permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise. For information regarding permissions, request forms and the appropriate contacts within the Pearson Education Global Rights & Permissions department, please visit www.pearsoned.com/permissions/.

PEARSON, ALWAYS LEARNING, and MyStatLab are exclusive trademarks owned by Pearson Education, Inc. or its affiliates in the U.S. and/or other countries.

Unless otherwise indicated herein, any third-party trademarks that may appear in this work are the property of their respective owners and any references to third-party trademarks, logos or other trade dress are for demonstrative or descriptive purposes only. Such references are not intended to imply any sponsorship, endorsement, authorization, or promotion of Pearson’s products by the owners of such marks, or any relationship between the owner and Pearson Education, Inc. or its affiliates, authors, licensees or distributors.

Library of Congress Cataloging-in-Publication Data

Names: Larsen, Richard J. | Marx, Morris L. Title: An introduction to mathematical statistics and its applications /

Richard J. Larsen, Vanderbilt University, Morris L. Marx, University of West Florida.

Description: Sixth edition. | Boston : Pearson, [2018] | Includes bibliographical references and index.

Identifiers: LCCN 2016035117 | ISBN 9780134114217 Subjects: LCSH: Mathematical statistics—Textbooks. Classification: LCC QA276 .L314 2018 | DDC 519.5—dc23 LC record available at https://lccn.loc.gov/2016035117

1 16

ISBN 13: 978-0-13-411421-7 ISBN 10: 0-13-411421-3

Contents Preface viii

1 Introduction 1 1.1 An Overview 1

1.2 Some Examples 2

1.3 A Brief History 6

1.4 A Chapter Summary 14

2 Probability 15 2.1 Introduction 15

2.2 Sample Spaces and the Algebra of Sets 17

2.3 The Probability Function 26

2.4 Conditional Probability 31

2.5 Independence 50

2.6 Combinatorics 65

2.7 Combinatorial Probability 89

2.8 Taking a Second Look at Statistics (Monte Carlo Techniques) 99

3 Random Variables 102 3.1 Introduction 102

3.2 Binomial and Hypergeometric Probabilities 103

3.3 Discrete Random Variables 116

3.4 Continuous Random Variables 127

3.5 Expected Values 137

3.6 The Variance 153

3.7 Joint Densities 160

3.8 Transforming and Combining Random Variables 174

3.9 Further Properties of the Mean and Variance 182

3.10 Order Statistics 192

3.11 Conditional Densities 199

3.12 Moment-Generating Functions 206

3.13 Taking a Second Look at Statistics (Interpreting Means) 215 iii

iv Contents

4 Special Distributions 218 4.1 Introduction 218

4.2 The Poisson Distribution 219

4.3 The Normal Distribution 235

4.4 The Geometric Distribution 257

4.5 The Negative Binomial Distribution 259

4.6 The Gamma Distribution 267

4.7 Taking a Second Look at Statistics (Monte Carlo Simulations) 271

Appendix 4.A.1 Properties of Frequently-Used pdfs 274

Appendix 4.A.2 A Proof of the Central Limit Theorem 276

5 Estimation 278 5.1 Introduction 278

5.2 Estimating Parameters: The Method of Maximum Likelihood and the Method of Moments 280

5.3 Interval Estimation 293

5.4 Properties of Estimators 308

5.5 Minimum-Variance Estimators: The Cramér-Rao Lower Bound 316

5.6 Sufficient Estimators 319

5.7 Consistency 326

5.8 Bayesian Estimation 329

5.9 Taking a Second Look at Statistics (Beyond Classical Estimation) 341

6 Hypothesis Testing 343 6.1 Introduction 343

6.2 The Decision Rule 344

6.3 Testing Binomial Data—H0: p = po 353 6.4 Type I and Type II Errors 359

6.5 A Notion of Optimality: The Generalized Likelihood Ratio 375

6.6 Taking a Second Look at Hypothesis Testing (Statistical Significance versus “Practical” Significance) 378

7 Inferences Based on the Normal Distribution 380 7.1 Introduction 380

7.2 Comparing Y−μ σ/

√ n

and Y−μ S/

√ n

381

7.3 Deriving the Distribution of Y−μ S/

√ n

383

Contents v

7.4 Drawing Inferences About μ 389

7.5 Drawing Inferences About σ2 404

7.6 Taking a Second Look at Statistics (Type II Error) 412

Appendix 7.A.1 Some Distribution Results for Y and S2 414

Appendix 7.A.2 A Proof That the One-Sample t Test Is a GLRT 416

Appendix 7.A.3 A Proof of Theorem 7.5.2 418

8 Types of Data: A Brief Overview 421 8.1 Introduction 421

8.2 Classifying Data 427

8.3 Taking a Second Look at Statistics (Why Samples Are Not “Valid”!) 448

9 Two-Sample Inferences 450 9.1 Introduction 450

9.2 Testing H0: μX = μY 451 9.3 Testing H0: σ2X = σ2Y—The F Test 463 9.4 Binomial Data: Testing H0: pX = pY 468 9.5 Confidence Intervals for the Two-Sample Problem 473

9.6 Taking a Second Look at Statistics (Choosing Samples) 478

Appendix 9.A.1 A Derivation of the Two-Sample t Test (A Proof of Theorem 9.2.2) 480

10 Goodness-of-Fit Tests 483 10.1 Introduction 483

10.2 The Multinomial Distribution 484

10.3 Goodness-of-Fit Tests: All Parameters Known 488

10.4 Goodness-of-Fit Tests: Parameters Unknown 498

10.5 Contingency Tables 507

10.6 Taking a Second Look at Statistics (Outliers) 517

11 Regression 520 11.1 Introduction 520

11.2 The Method of Least Squares 520

11.3 The Linear Model 543

11.4 Covariance and Correlation 563

vi Contents

11.5 The Bivariate Normal Distribution 570

11.6 Taking a Second Look at Statistics (How Not to Interpret the Sample Correlation Coefficient) 576

Appendix 11.A.1 A Proof of Theorem 11.3.3 577

12 The Analysis of Variance 580 12.1 Introduction 580

12.2 The F Test 582

12.3 Multiple Comparisons: Tukey’s Method 592

12.4 Testing Subhypotheses with Contrasts 596

12.5 Data Transformations 604

12.6 Taking a Second Look at Statistics (Putting the Subject of Statistics Together—The Contributions of Ronald A. Fisher) 606

Appendix 12.A.1 A Proof of Theorem 12.2.2 608

Appendix 12.A.2 The Distribution of SSTR/(k−1)SSE/(n−k) When H1 Is True 608

13 Randomized Block Designs 613 13.1 Introduction 613

13.2 The F Test for a Randomized Block Design 614

13.3 The Paired t Test 628

13.4 Taking a Second Look at Statistics (Choosing between a Two-Sample t Test and a Paired t Test) 634

14 Nonparametric Statistics 638 14.1 Introduction 638

14.2 The Sign Test 639

14.3 Wilcoxon Tests 645

14.4 The Kruskal-Wallis Test 658

14.5 The Friedman Test 662

14.6 Testing for Randomness 665

14.7 Taking a Second Look at Statistics (Comparing Parametric and Nonparametric Procedures) 669

Contents vii

15 Factorial Data (Available Online) 15-1 15.1 Introduction 15-1

15.2 The Two-Factor Factorial 15-4

15.3 Sums of Squares for Two-Factor Factorials 15-16

15.4 Expected Mean Squares 15-26

15.5 Examples 15-30

15.6 The Three-Factor Factorial Design 15-40

15.7 2n Designs 15-51

15.8 Fractional Factorials 15-72

Appendix A: Statistical Tables 674

Answers to Selected Odd-Numbered Questions 701

Bibliography 725

Index 737

Preface

John Tukey (1915–2000) was one of the most prominent (and quotable) statisticians of the last half of the twentieth century. He was once asked what he especially en- joyed about the profession he chose for his life’s work. “The best thing about being a statistician,” he said without hesitation, “is that you get to play in everyone’s back- yard.” That sentiment says much about Tukey’s well known eclectic interests; it also speaks to what this book is all about.

Our hope is that this text satisfies two objectives: 1) It introduces the basic tech- niques of probability and mathematical statistics in a comprehensive and interesting way at a level appropriate for students who have completed three semesters of cal- culus, and 2) it provides students with the skills and insights necessary to apply those principles. In our opinion, satisfying (1) but not (2) would be an inadequate “take- away” for a student’s two-semesters worth of time and effort.

It may seem that completing Objective 1 automatically confers on students the wherewithal to meet Objective 2. Not so. Mathematical statistics deals primarily with the nature of individual measurements or simple properties calculated from samples of measurements—means, variances, distributions, relationships to other measure- ments, and so on. Analyzing data, though, requires an additional knowledge of the experimental design to which an entire set of measurements belong. To borrow some terminology from economists, mathematical statistics is much like the micro aspect of the subject; experimental design is the macro aspect. There is enough time in a two-semester course to do justice to both.

Experimental designs come in many variations, but eight are especially impor- tant in terms of the frequency of their occurrence and their relationship to the math- ematical statistics covered in a first course. The initial step in teaching someone how to analyze data is helping them learn how to recognize those eight “data models”: One-sample data, Two-sample data, k-sample data, Paired data, Randomized block data, Regression/Correlation data, Categorical data, and Factorial data. We believe that mentioning them in passing is not sufficient. They need to be compared and de- scribed, altogether in one chapter, side-by-side, and illustrated with real-world data.

Identifying data models, of course, is not a difficult skill to acquire. Anyone in- volved in analyzing data learns it quickly. But for students taking their first course in statistics, ignoring the topic leaves them without a sense of where the subject is going and why. Fully addressing the issue before students encounter all the Z tests, t tests, χ2 tests, and F tests that come in rapid succession provides a very helpful framework for putting all that material in context.

The final step in dealing with Objective 2 is to show the application of math- ematical statistics and the methodologies it created to real-world data. Made-up or contrived data will not suffice. They do not provide the detail or complexity necessary to point out, for example, why one particular design was used rather than another or what to make of certain anomalies that appear to have occurred or what follow-up studies seem to be warranted. For their help and obvious expertise, we are deeply in- debted to all the researchers who have graciously allowed us to use portions of their data to base the more than 80 Case Studies scattered throughout the text. We hope these are as informative and helpful as the “backyards” that Professor Tukey found so enjoyable.

viii

Preface ix

New to This Edition

• Chapter 15, Factorial Data, is a new, downloadable chapter describing the the- ory and practice of the analysis of variance as it applies to factorial data. It covers two-factor factorials, three-factor factorials, 2n designs, and fractional factorials, all at the same mathematical level as the book’s other two treat- ments of the analysis of variance, Chapter 12 and Chapter 13. This is the most important of all the multifactor experimental designs.

• Chapter 2 contains ten new examples, including a repeated-independent-trials analysis of the often-quoted “Caesar’s last breath” problem.

• Overall, the Sixth Edition contains 18 new Case Studies for additional concept application.

• An Appendix has been added at the end of Chapter 4 summarizing all the important properties of the most frequently used pdfs.

• Much of Section 5.2 dealing with parameter estimation has been rewritten, and the margin of error portion of Section 5.3 has been completely redone.

• Discussions of the different data models in Chapter 8 have been expanded, and an eighth model (factorial data) has been added. The chapter includes seven new Case Studies.

• In Chapter 11, the section on nonlinear models has been thoroughly revised with an emphasis put on their relationship to different laws of growth.

• Because of space and cost considerations, journals and technical reports often display only summaries of an experiment’s results. A section has been added to Chapter 12 showing how the entire ANOVA table for a set of k-sample data can be “reconstructed” without knowing any of the individual measurements.

• The text companion website www.pearsonhighered.com/mathstatsresources/ has the online, downloadable Chapter 15 and data sets analyzed in the text, in generic form to copy for input into statistical software. The site also has ad- ditional resources to help students and instructors.

Acknowledgments

We would like to thank all of the following reviewers of the present and previous editions of this text for their many helpful comments and suggestions. Every edition benefited from your insight and your expertise.

Reviewers for this edition: Adam Bowers, University of California, San Diego Bree Ettinger, Emory University Eugene D. Gallagher, University of Massachusetts, Boston Mohammad Kazemi, University of North Carolina, Charlotte Ralph Russo, University of Iowa Neslihan Uler, University of Michigan, Ann Arbor Bin Wang, University of Southern Alabama

Reviewers of previous editions: Abera Abay, Rowan University Kyle Siegrist, University of Alabama in Huntsville Ditlev Monrad, University of Illinois at Urbana-Champaign

x Preface

Vidhu S. Prasad, University of Massachusetts, Lowell Wen-Qing Xu, California State University, Long Beach Katherine St. Clair, Colby College Yimin Xiao, Michigan State University Nicolas Christou, University of California, Los Angeles Daming Xu, University of Oregon Maria Rizzo, Ohio University Dimitris Politis, University of California, San Diego

We would also like to express our deepest gratitude to all those on the Pearson team who guided this project to completion. Your professionalism and friendliness made working on this endeavor a pleasure. Thanks to all of you!

What has not changed in this edition is our sincere hope that the reader will find mathematical statistics challenging, informative, interesting, and—maybe in a few places here and there when least expected—almost fun.

Richard J. Larsen Vanderbilt University

Morris L. Marx University of West Florida

Chapte r

1Introduction CHAPTER OUTLINE

1.1 An Overview 1.2 Some Examples 1.3 A Brief History 1.4 A Chapter Summary

“Until the phenomena of any branch of knowledge have been submitted to measurement and number it cannot assume the status and dignity of a science.”

—Francis Galton

1.1 AN OVERVIEW Sir Francis Galton was a preeminent biologist of the nineteenth century. A passionate advocate for the theory of evolution (his nickname was “Darwin’s bulldog”), Galton was also an early crusader for the study of statistics and believed the subject would play a key role in the advancement of science:

Some people hate the very name of statistics, but I find them full of beauty and inter- est. Whenever they are not brutalized, but delicately handled by the higher methods, and are warily interpreted, their power of dealing with complicated phenomena is extraordinary. They are the only tools by which an opening can be cut through the formidable thicket of difficulties that bars the path of those who pursue the Science of man.

Did Galton’s prediction come to pass? Absolutely—try reading a biology journal or the analysis of a psychology experiment before taking your first statistics course. Science and statistics have become inseparable, two peas in the same pod. What the good gentleman from London failed to anticipate, though, is the extent to which all of us—not just scientists—have become enamored (some would say obsessed) with numerical information. The stock market is awash in averages, indicators, trends, and exchange rates; federal education initiatives have taken standardized testing to new levels of specificity; Hollywood uses sophisticated demographics to see who’s watching what, and why; and pollsters regularly tally and track our every opinion, regardless of how irrelevant or uninformed. In short, we have come to expect every- thing to be measured, evaluated, compared, scaled, ranked, and rated—and if the results are deemed unacceptable for whatever reason, we demand that someone or something be held accountable (in some appropriately quantifiable way).

To be sure, many of these efforts are carefully carried out and make perfectly good sense; unfortunately, others are seriously flawed, and some are just plain nonsense. What they all speak to, though, is the clear and compelling need to know something about the subject of statistics, its uses and its misuses.

This book addresses two broad topics—the mathematics of statistics and the prac- tice of statistics. The two are quite different. The former refers to the probability the- ory that supports and justifies the various methods used to analyze data. For the most

1

2 Chapter 1 Introduction

part, this background material is covered in Chapters 2 through 7. The key result is the central limit theorem, which is one of the most elegant and far-reaching results in all of mathematics. (Galton believed the ancient Greeks would have personified and deified the central limit theorem had they known of its existence.) Also included in these chapters is a thorough introduction to combinatorics, the mathematics of sys- tematic counting. Historically, this was the very topic that launched the development of probability in the first place, back in the seventeenth century. In addition to its con- nection to a variety of statistical procedures, combinatorics is also the basis for every state lottery and every game of chance played with a roulette wheel, a pair of dice, or a deck of cards.

The practice of statistics refers to all the issues (and there are many!) that arise in the design, analysis, and interpretation of data. Discussions of these topics appear in several different formats. Included in most of the case studies throughout the text is a feature entitled ‘‘About the Data.’’ These are additional comments about either the particular data in the case study or some related topic suggested by those data. Then near the end of most chapters is a Taking a Second Look at Statistics section. Several of these deal with the misuses of statistics—specifically, inferences drawn incorrectly and terminology used inappropriately. The most comprehensive data-related discussion comes in Chapter 8, which is devoted entirely to the critical problem of knowing how to start a statistical analysis—that is, knowing which procedure should be used, and why.

More than a century ago, Galton described what he thought a knowledge of statistics should entail. Understanding “the higher methods,” he said, was the key to ensuring that data would be “delicately handled” and “warily interpreted.” The goal of this book is to help make that happen.

1.2 Some Examples Statistical methods are often grouped into two broad categories—descriptive statis- tics and inferential statistics. The former refers to all the various techniques for summarizing and displaying data. These are the familiar bar graphs, pie charts, scat- terplots, means, medians, and the like, that we see so often in the print media. The much more mathematical inferential statistics are procedures that make generaliza- tions and draw conclusions of various kinds based on the information contained in a set of data; moreover, they calculate the probability of the generalizations being correct.

Described in this section are three case studies. The first illustrates a very ef- fective use of several descriptive techniques. The latter two illustrate the sorts of questions that inferential procedures can help answer.

CASE STUDY 1.2.1

Pictured at the top of Figure 1.2.1 is the kind of information routinely recorded by a seismograph—listed chronologically are the occurrence times and Richter magnitudes for a series of earthquakes. As raw data, the numbers are largely meaningless: No patterns are evident, nor is there any obvious connection between the frequencies of tremors and their severities.

Shown at the bottom of the figure is the result of applying several descrip- tive techniques to an actual set of seismograph data recorded over a period of several years in southern California (73). Plotted above the Richter (R) value of 4.0, for example, is the average number (N) of earthquakes occurring per year in that region having magnitudes in the range 3.75 to 4.25. Similar points are

Section 1.2 Some Examples 3

30

20

10

0 4 5

Magnitude on Richter scale, R

A ve

ra ge

n um

be r

of s

ho ck

s pe

r ye

ar , N

6 7

N = 80,338.16e

Episode number Date Time Severity (Richter scale)

217 6/19 4:53 P .M. 2.7 218 7/2 6:07 A.M. 3.1 219 7/4 8:19 A.M. 2.0 220 8/7 1:10 A.M. 4.1 221 8/7 10:46 P .M. 3.6

– 1.981R

Figure 1.2.1

included for R-values centered at 4.5, 5.0, 5.5, 6.0, 6.5, and 7.0. Now we can see that earthquake frequencies and severities are clearly related: Describing the (N, R)’s exceptionally well is the equation

N = 80,338.16e−1.981R (1.2.1) which is found using a procedure described in Chapter 11. (Note: Geologists have shown that the model N = β0eβ1R describes the (N, R) relationship all over the world. All that changes from region to region are the numerical values for β0 and β1.)

Notice that Equation 1.2.1 is more than just an elegant summary of the ob- served (N, R) relationship. It also allows us to estimate the likelihood of future earthquake catastrophes having values of R that have never been observed. On the minds of all Californians, of course, is the Big One, the dreaded rip-roaring 10.5-Richter-scale monster megaquake that turns buildings into piles of rubble, sends busloads of tourists careening into the San Francisco Bay, and moves the intersection of Hollywood and Vine to somewhere in downtown El Segundo. How often might an earthquake of that magnitude be expected?

Letting R = 10.5 in Equation 1.2.1 gives N = 80,338.16e−1.981(10.5)

= 0.0000752 earthquakes/year (Continued on next page)

4 Chapter 1 Introduction

(Case Study 1.2.1 continued)

so the predicted frequency would be once every 13,300 years (= 1/0.0000752). On the one hand, the rarity of such a disaster has to be reassuring; on the other hand, not knowing when the last such earthquake occurred is a bit unsettling. Are we on borrowed time? What if the most recent megaquake occurred forty thousand years ago?

Comment The megaquake prediction prompted by Equation 1.2.1 raises an obviousquestion: Whyis thecalculationthat ledto themodel N = 80,338.16e−1.981R not considered an example of inferential statistics even though it did yield a prediction for R = 10.5? The answer is that Equation 1.2.1—by itself—does not tell us anything about the ‘‘error’’ associated with its predictions. In Chapter 11, a more elaborate probability method based on Equation 1.2.1 is described that does yield error estimates and qualifies as a bona fide inference procedure.

About the Data For the record, the strongest earthquake ever recorded in California occurred January 9, 1857 along the San Andreas fault near Fort Tejon, a sparsely populated settlement about seventy miles north of Los Angeles. Its magnitude was estimated to be between 7.9 and 8.0. The state’s most famous, deadliest, and costliest earthquake, though, occurred in San Francisco on April 18, 1906. It had a magnitude of 7.8, killed three thousand people, and destroyed 80% of the city. Over the years, a number of Hollywood movies featured that particular earthquake as part of their storylines, the best known being San Fran- cisco, a 1936 production starring Clark Gable and Spencer Tracy.

CASE STUDY 1.2.2

In folklore, the full moon is often portrayed as something sinister, a kind of evil force possessing the power to control our behavior. Over the centuries, many prominent writers and philosophers have shared this belief. Milton, in Paradise Lost, refers to

Demoniac frenzy, moping melancholy And moon-struck madness.

And Othello, after the murder of Desdemona, laments

It is the very error of the moon, She comes more near the earth than she was wont And makes men mad.

On a more scholarly level, Sir William Blackstone, the renowned eighteenth-century English barrister, defined a “lunatic” as

one who hath . . . lost the use of his reason and who hath lucid intervals, some- times enjoying his senses and sometimes not, and that frequently depending upon changes of the moon.

The possibility of lunar phases influencing human affairs is a theory not with- out supporters among the scientific community. Studies by reputable medical researchers have attempted to link the “Transylvania effect,” as it has come to be known, with higher suicide rates, pyromania, and epilepsy (not to mention the periodic unseemly behavior of werewolves . . .).

The possible relationship between lunar cycles and mental breakdowns has also been studied. Table 1.2.1 shows one year’s admission rates to the emergency room of a Virginia mental health clinic before, during, and after its twelve full moons (13).

Section 1.2 Some Examples 5

Table 1.2.1 Admission Rates (Patients/Day)

Month Before Full Moon During Full Moon After Full Moon

Aug. 6.4 5.0 5.8 Sept. 7.1 13.0 9.2 Oct. 6.5 14.0 7.9 Nov. 8.6 12.0 7.7 Dec. 8.1 6.0 11.0 Jan. 10.4 9.0 12.9 Feb. 11.5 13.0 13.5 Mar. 13.8 16.0 13.1 Apr. 15.4 25.0 15.8 May 15.7 13.0 13.3 June 11.7 14.0 12.8 July 15.8 20.0 14.5

Averages: 10.9 13.3 11.5

Notice for these data that the average admission rate “during” the full moon is, in fact, higher than the “before” and “after” admission rates: 13.3 as opposed to 10.9 and 11.5. Can it be inferred, then, from these averages that the data sup- port the existence of a Transylvania effect? No. Another explanation for the averages is possible—namely, that there is no such thing as a Transylvania effect and the observed differences among the three averages are due solely to chance.

Questions of this sort—that is, choosing between two conflicting explana- tions for a set of data—are resolved using a variety of techniques known as hy- pothesis testing. The insights that hypothesis tests provide are without question the most important contribution that the subject of statistics makes to the ad- vancement of science.

The particular hypothesis test appropriate for the data in Table 1.2.1 is known as a randomized block analysis of variance, which will be covered at length in Chapter 13. As we will see, the conclusion reached in this case is both unexpected and a bit disconcerting.

CASE STUDY 1.2.3

It may not be made into a movie anytime soon, but the way that statistical infer- ence was used to spy on the Nazis in World War II is a pretty good tale. And it certainly did have a surprise ending!

The story began in the early 1940s. Fighting in the European theatre was in- tensifying, and Allied commanders were amassing a sizeable collection of aban- doned and surrendered German weapons. When they inspected those weapons, the Allies noticed that each one bore a different number. Aware of the Nazis’ reputation for detailed record keeping, the Allies surmised that each number represented the chronological order in which the piece had been manufactured. But if that was true, might it be possible to use the “captured” serial numbers to estimate the total number of weapons the Germans had produced?

That was precisely the question posed to a group of government statisticians working out of Washington, D.C. Wanting to estimate an adversary’s manufac- turing capability was, of course, nothing new. Up to that point, though, the only sources of that information had been spies and traitors; using serial numbers was an approach entirely different.

(Continued on next page)

6 Chapter 1 Introduction

(Case Study 1.2.3 continued)

The answer turned out to be a fairly straightforward application of the prin- ciples that will be introduced in Chapter 5. If n is the total number of captured serial numbers and xmax is the largest captured serial number, then the estimate for the total number of items produced is given by the formula

estimated output = [(n + 1)/n]xmax − 1 (1.2.2)

Suppose, for example, that n = 5 tanks were captured and they bore the serial numbers 92, 14, 28, 300, and 146, respectively. Then xmax = 300 and the esti- mated total number of tanks manufactured is three hundred fifty-nine:

estimated output = [(5 + 1)/5]300 − 1 = 359

Did Equation 1.2.2 work? Better than anyone could have expected (prob- ably even the statisticians). When the war ended and the Third Reich’s “true” production figures were revealed, it was found that serial number estimates were far more accurate in every instance than all the information gleaned from tradi- tional espionage operations, spies, and informants. The serial number estimate for German tank production in 1942, for example, was 3400, a figure very close to the actual output. The “official” estimate, on the other hand, based on intel- ligence gathered in the usual ways, was a grossly inflated 18,000.

About the Data Large discrepancies, like 3400 versus 18,000 for the tank esti- mates, were not uncommon. The espionage-based estimates were consistently erring on the high side because of the sophisticated Nazi propaganda ma- chine that deliberately exaggerated the country’s industrial prowess. On spies and would-be adversaries, the Third Reich’s carefully orchestrated dissembling worked exactly as planned; on Equation 1.2.2, though, it had no effect whatso- ever! (69)

1.3 A Brief History For those interested in how we managed to get to where we are, Section 1.3 offers a brief history of probability and statistics. The two subjects were not mathematical littermates—they began at different times in different places by different people for different reasons. How and why they eventually came together makes for an inter- esting story, reacquaints us with some towering figures from the past, and introduces several others whose names will probably not be familiar but whose contributions were critically important in demonstrating the linkage between science and statistics.

PROBABILITY: THE EARLY YEARS

No one knows where or when the notion of chance first arose; it fades into our pre- history. Nevertheless, evidence linking early humans with devices for generating ran- dom events is plentiful: Archaeological digs, for example, throughout the ancient world consistently turn up a curious overabundance of astragali, the heel bones of sheep and other vertebrates. Why should the frequencies of these bones be so dis- proportionately high? One could hypothesize that our forebears were fanatical foot

Section 1.3 A Brief History 7

fetishists, but two other explanations seem more plausible: The bones were used for religious ceremonies and for gambling.

Astragali have six sides but are not symmetrical (see Figure 1.3.1). Those found in excavations typically have their sides numbered or engraved. For many ancient civilizations, astragali were the primary mechanism through which oracles solicited the opinions of their gods. In Asia Minor, for example, it was customary in divination rites to roll, or cast, five astragali. Each possible configuration was associated with the name of a god and carried with it the sought-after advice. An outcome of (1, 3, 3, 4, 4), for instance, was said to be the throw of the savior Zeus, and its appearance was taken as a sign of encouragement (37):

One one, two threes, two fours The deed which thou meditatest, go do it boldly. Put thy hand to it. The gods have given thee

favorable omens Shrink not from it in thy mind, for no evil

shall befall thee.

Sheep astragalus

Figure 1.3.1

A (4, 4, 4, 6, 6), on the other hand, the throw of the child-eating Cronos, would send everyone scurrying for cover:

Three fours and two sixes. God speaks as follows. Abide in thy house, nor go elsewhere, Lest a ravening and destroying beast come nigh thee. For I see not that this business is safe. But bide

thy time.

Gradually, over thousands of years, astragali were replaced by dice, and the latter became the most common means for generating random events. Pottery dice have been found in Egyptian tombs built before 2000 b.c.; by the time the Greek civiliza- tion was in full flower, dice were everywhere. (Loaded dice have also been found. Mastering the mathematics of probability would prove to be a formidable task for our ancestors, but they seemed quite adept at learning how to cheat!)

The lack of historical records blurs the distinction initially drawn between div- ination ceremonies and recreational gaming. Among more recent societies, though, gambling emerged as a distinct entity, and its popularity was irrefutable. The Greeks and Romans were consummate gamblers, as were the early Christians (99).

Rules for many of the Greek and Roman games have been lost, but we can recog- nize the lineage of certain modern diversions in what was played during the Middle Ages. The most popular dice game of that period was called hazard, the name de- riving from the Arabic al zhar, which means “a die.” Hazard is thought to have been brought to Europe by soldiers returning from the Crusades; its rules are much like those of our modern-day craps. Cards were first introduced in the fourteenth cen- tury and immediately gave rise to a game known as Primero, an early form of poker. Board games such as backgammon were also popular during this period.

8 Chapter 1 Introduction

Given this rich tapestry of games and the obsession with gambling that char- acterized so much of the Western world, it may seem more than a little puzzling that a formal study of probability was not undertaken sooner than it was. As we will see shortly, the first instance of anyone conceptualizing probability in terms of a mathematical model occurred in the sixteenth century. That means that more than two thousand years of dice games, card games, and board games passed by before someone finally had the insight to write down even the simplest of probabilistic abstractions.

Historians generally agree that, as a subject, probability got off to a rocky start because of its incompatibility with two of the most dominant forces in the evolution of our Western culture, Greek philosophy and early Christian theology. The Greeks were comfortable with the notion of chance (something the Christians were not), but it went against their nature to suppose that random events could be quantified in any useful fashion. They believed that any attempt to reconcile mathematically what did happen with what should have happened was, in their phraseology, an improper juxtaposition of the “earthly plane” with the “heavenly plane.”

Making matters worse was the antiempiricism that permeated Greek thinking. Knowledge, to them, was not something that should be derived by experimentation. It was better to reason out a question logically than to search for its explanation in a set of numerical observations. Together, these two attitudes had a deadening effect: The Greeks had no motivation to think about probability in any abstract sense, nor were they faced with the problems of interpreting data that might have pointed them in the direction of a probability calculus.

If the prospects for the study of probability were dim under the Greeks, they became even worse when Christianity broadened its sphere of influence. The Greeks and Romans at least accepted the existence of chance. However, they believed their gods to be either unable or unwilling to get involved in matters so mundane as the outcome of the roll of a die. Cicero writes:

Nothing is so uncertain as a cast of dice, and yet there is no one who plays often who does not make a Venus-throw1 and occasionally twice and thrice in succession. Then are we, like fools, to prefer to say that it happened by the direction of Venus rather than by chance?

For the early Christians, though, there was no such thing as chance: Every event that happened, no matter how trivial, was perceived to be a direct manifestation of God’s deliberate intervention. In the words of St. Augustine:

Nos eas causas quae dicuntur fortuitae . . . non dicimus nullas, sed latentes; easque tribuimus vel veri Dei . . .

(We say that those causes that are said to be by chance are not non-existent but are hidden, and we attribute them to the will of the true God . . .)

Taking Augustine’s position makes the study of probability moot, and it makes a probabilist a heretic. Not surprisingly, nothing of significance was accomplished in the subject for the next fifteen hundred years.

It was in the sixteenth century that probability, like a mathematical Lazarus, arose from the dead. Orchestrating its resurrection was one of the most eccen- tric figures in the entire history of mathematics, Gerolamo Cardano. By his own admission, Cardano personified the best and the worst—the Jekyll and the Hyde— of the Renaissance man. He was born in 1501 in Pavia. Facts about his personal life

1 When rolling four astragali, each of which is numbered on four sides, a Venus-throw was having each of the four numbers appear.

Section 1.3 A Brief History 9

are difficult to verify. He wrote an autobiography, but his penchant for lying raises doubts about much of what he says. Whether true or not, though, his “one-sentence” self-assessment paints an interesting portrait (135):

Nature has made me capable in all manual work, it has given me the spirit of a philosopher and ability in the sciences, taste and good manners, voluptuousness, gai- ety, it has made me pious, faithful, fond of wisdom, meditative, inventive, courageous, fond of learning and teaching, eager to equal the best, to discover new things and make independent progress, of modest character, a student of medicine, interested in curiosities and discoveries, cunning, crafty, sarcastic, an initiate in the mysterious lore, industrious, diligent, ingenious, living only from day to day, impertinent, con- temptuous of religion, grudging, envious, sad, treacherous, magician and sorcerer, miserable, hateful, lascivious, obscene, lying, obsequious, fond of the prattle of old men, changeable, irresolute, indecent, fond of women, quarrelsome, and because of the conflicts between my nature and soul I am not understood even by those with whom I associate most frequently.

Formally trained in medicine, Cardano’s interest in probability derived from his addiction to gambling. His love of dice and cards was so all-consuming that he is said to have once sold all his wife’s possessions just to get table stakes! Fortunately, something positive came out of Cardano’s obsession. He began looking for a math- ematical model that would describe, in some abstract way, the outcome of a random event. What he eventually formalized is now called the classical definition of prob- ability: If the total number of possible outcomes, all equally likely, associated with some action is n, and if m of those n result in the occurrence of some given event, then the probability of that event is m/n. If a fair die is rolled, there are n = 6 possible outcomes. If the event “outcome is greater than or equal to 5” is the one in which we are interested, then m = 2 (the outcomes 5 and 6) and the probability of the event is 26 , or

1 3 (see Figure 1.3.2).

1

3

5

2

4

6

Possible outcomes

Outcomes greater than or equal to 5; probability = 2/6

Figure 1.3.2

Cardano had tapped into the most basic principle in probability. The model he discovered may seem trivial in retrospect, but it represented a giant step forward: His was the first recorded instance of anyone computing a theoretical, as opposed to an empirical, probability. Still, the actual impact of Cardano’s work was minimal. He wrote a book in 1525, but its publication was delayed until 1663. By then, the focus of the Renaissance, as well as interest in probability, had shifted from Italy to France.

The date cited by many historians (those who are not Cardano supporters) as the “beginning” of probability is 1654. In Paris, a well-to-do gambler the Chevalier de Méré asked several prominent mathematicians, including Blaise Pascal, a series of questions, the best known of which is the problem of points:

Two people, A and B, agree to play a series of fair games until one person has won six games. They each have wagered the same amount of money, the intention being that the winner will be awarded the entire pot. But suppose, for whatever reason, the series is prematurely terminated, at which point A has won five games and B three. How should the stakes be divided?

10 Chapter 1 Introduction

(The correct answer is that A should receive seven-eighths of the total amount wa- gered. Hint: Suppose the contest were resumed. What scenarios would lead to A’s being the first person to win six games?)

Pascal was intrigued by de Méré’s questions and shared his thoughts with Pierre Fermat, a Toulouse civil servant and probably the most brilliant mathematician in Europe. Fermat graciously replied, and from the now-famous Pascal-Fermat corre- spondence came not only the solution to the problem of points but the foundation for more general results. More significantly, news of what Pascal and Fermat were working on spread quickly. Others got involved, of whom the best known was the Dutch scientist and mathematician Christiaan Huygens. The delays and the indiffer- ence that had plagued Cardano a century earlier were not going to happen again.

Best remembered for his work in optics and astronomy, Huygens, early in his career, was intrigued by the problem of points. In 1657 he published De Ratiociniis in Aleae Ludo (Calculations in Games of Chance), a very significant work, far more comprehensive than anything Pascal and Fermat had done. For almost fifty years it was the standard “textbook” in the theory of probability. Not surprisingly, Huygens has supporters who feel that he should be credited as the founder of probability.

Almost all the mathematics of probability was still waiting to be discovered. What Huygens wrote was only the humblest of beginnings, a set of fourteen propo- sitions bearing little resemblance to the topics we teach today. But the foundation was there. The mathematics of probability was finally on firm footing.

STATISTICS: FROM ARISTOTLE TO QUETELET

Historians generally agree that the basic principles of statistical reasoning began to coalesce in the middle of the nineteenth century. What triggered this emergence was the union of three different “sciences,” each of which had been developing along more or less independent lines (206).

The first of these sciences, what the Germans called Staatenkunde, involved the collection of comparative information on the history, resources, and military prowess of nations. Although efforts in this direction peaked in the seventeenth and eigh- teenth centuries, the concept was hardly new: Aristotle had done something similar in the fourth century b.c. Of the three movements, this one had the least influence on the development of modern statistics, but it did contribute some terminology: The word statistics, itself, first arose in connection with studies of this type.

The second movement, known as political arithmetic, was defined by one of its early proponents as “the art of reasoning by figures, upon things relating to government.” Of more recent vintage than Staatenkunde, political arithmetic’s roots were in seventeenth-century England. Making population estimates and construct- ing mortality tables were two of the problems it frequently dealt with. In spirit, political arithmetic was similar to what is now called demography.

The third component was the development of a calculus of probability. As we saw earlier, this was a movement that essentially started in seventeenth-century France in response to certain gambling questions, but it quickly became the “engine” for analyzing all kinds of data.

STAATENKUNDE: THE COMPARATIVE DESCRIPTION OF STATES

The need for gathering information on the customs and resources of nations has been obvious since antiquity. Aristotle is credited with the first major effort toward that objective: His Politeiai, written in the fourth century b.c., contained detailed descriptions of some one hundred fifty-eight different city-states. Unfortunately, the

Section 1.3 A Brief History 11

thirst for knowledge that led to the Politeiai fell victim to the intellectual drought of the Dark Ages, and almost two thousand years elapsed before any similar projects of like magnitude were undertaken.

The subject resurfaced during the Renaissance, and the Germans showed the most interest. They not only gave it a name, Staatenkunde, meaning “the compara- tive description of states,” but they were also the first (in 1660) to incorporate the subject into a university curriculum. A leading figure in the German movement was Gottfried Achenwall, who taught at the University of Göttingen during the middle of the eighteenth century. Among Achenwall’s claims to fame is that he was the first to use the word statistics in print. It appeared in the preface of his 1749 book Abriss der Statswissenschaft der heutigen vornehmsten europaishen Reiche und Republiken. (The word statistics comes from the Italian root stato, meaning “state,” implying that a statistician is someone concerned with government affairs.) As terminology, it seems to have been well-received: For almost one hundred years the word statistics contin- ued to be associated with the comparative description of states. In the middle of the nineteenth century, though, the term was redefined, and statistics became the new name for what had previously been called political arithmetic.

How important was the work of Achenwall and his predecessors to the devel- opment of statistics? That would be difficult to say. To be sure, their contributions were more indirect than direct. They left no methodology and no general theory. But they did point out the need for collecting accurate data and, perhaps more impor- tantly, reinforced the notion that something complex—even as complex as an entire nation—can be effectively studied by gathering information on its component parts. Thus, they were lending important support to the then-growing belief that induction, rather than deduction, was a more sure-footed path to scientific truth.

POLITICAL ARITHMETIC

In the sixteenth century the English government began to compile records, called bills of mortality, on a parish-to-parish basis, showing numbers of deaths and their underlying causes. Their motivation largely stemmed from the plague epidemics that had periodically ravaged Europe in the not-too-distant past and were threatening to become a problem in England. Certain government officials, including the very in- fluential Thomas Cromwell, felt that these bills would prove invaluable in helping to control the spread of an epidemic. At first, the bills were published only occasionally, but by the early seventeenth century they had become a weekly institution.2

Figure 1.3.3 shows a portion of a bill that appeared in London in 1665. The grav- ity of the plague epidemic is strikingly apparent when we look at the numbers at the top: Out of 97,306 deaths, 68,596 (over 70%) were caused by the plague. The breakdown of certain other afflictions, though they caused fewer deaths, raises some interesting questions. What happened, for example, to the twenty-three people who were “frighted” or to the three hundred ninety-seven who suffered from “rising of the lights”?

Among the faithful subscribers to the bills was John Graunt, a London merchant. Graunt not only read the bills, he studied them intently. He looked for patterns, com- puted death rates, devised ways of estimating population sizes, and even set up a primitive life table. His results were published in the 1662 treatise Natural and Polit- ical Observations upon the Bills of Mortality. This work was a landmark: Graunt had

2 An interesting account of the bills of mortality is given in Daniel Defoe’s A Journal of the Plague Year, which purportedly chronicles the London plague outbreak of 1665.

12 Chapter 1 Introduction

The bill for the year—A General Bill for this present year, ending the 19 of De- cember, 1665, according to the Report made to the King’s most excellent Majesty, by the Co. of Parish Clerks of Lond., & c.—gives the following summary of the results; the details of the several parishes we omit, they being made as in 1625, except that the out-parishes were now 12:—

Buried in the 27 Parishes within the walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15,207 Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9,887 Buried in the 16 Parishes without the walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41,351 Whereof of the plague. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28,838 At the Pesthouse, total buried . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Of the plague. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Buried in the 12 out-Parishes in Middlesex and surrey . . . . . . . . . . . . . . . . . . 18,554 Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21,420 Buried in the 5 Parishes in the City and Liberties of Westminster . . . . . . . . 12,194 Whereof the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8,403 The total of all the christenings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9,967 The total of all the burials this year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97,306 Whereof of the plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,596

Abortive and Stillborne . . . . . . . . . . 617 Griping in the Guts . . . . . . . . . . . . . . . . 1,288 Palsie . . . . . . . . . . . . . . . . . . . . . . 30 Aged . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1,545 Hang’d & made away themselved . . 7 Plague . . . . . . . . . . . . . . . . . . . . . 68,596 Ague & Feaver . . . . . . . . . . . . . . . . . . 5,257 Headmould shot and mould fallen. . 14 Plannet . . . . . . . . . . . . . . . . . . . . 6 Appolex and Suddenly . . . . . . . . . . . 116 Jaundice . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Plurisie . . . . . . . . . . . . . . . . . . . . 15 Bedrid . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Impostume . . . . . . . . . . . . . . . . . . . . . . . . 227 Poysoned . . . . . . . . . . . . . . . . . . 1 Blasted . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Kill by several accidents . . . . . . . . . . . . 46 Quinsie . . . . . . . . . . . . . . . . . . . . 35 Bleeding . . . . . . . . . . . . . . . . . . . . . . . . . 16 King’s Evill . . . . . . . . . . . . . . . . . . . . . . . . 86 Rickets . . . . . . . . . . . . . . . . . . . . 535 Cold & Cough . . . . . . . . . . . . . . . . . . . 68 Leprosie . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Rising of the Lights . . . . . . . . 397 Collick & Winde . . . . . . . . . . . . . . . . . 134 Lethargy . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Rupture. . . . . . . . . . . . . . . . . . . . 34 Comsumption & Tissick . . . . . . . . . . 4,808 Livergrown . . . . . . . . . . . . . . . . . . . . . . . . 20 Scurry . . . . . . . . . . . . . . . . . . . . . 105 Convulsion & Mother . . . . . . . . . . . . 2,036 Bloody Flux, Scowring & Flux . . . . . 18 Shingles & Swine Pox . . . . . . 2 Distracted . . . . . . . . . . . . . . . . . . . . . . . 5 Burnt and Scalded . . . . . . . . . . . . . . . . . 8 Sores, Ulcers, Broken and Dropsie & Timpany . . . . . . . . . . . . . . 1,478 Calenture . . . . . . . . . . . . . . . . . . . . . . . . . 3 Bruised Limbs . . . . . . . . . . . . . 82 Drowned . . . . . . . . . . . . . . . . . . . . . . . . 50 Cancer, Cangrene & Fistula . . . . . . . . 56 Spleen . . . . . . . . . . . . . . . . . . . . . 14 Executed . . . . . . . . . . . . . . . . . . . . . . . . 21 Canker and Thrush . . . . . . . . . . . . . . . . 111 Spotted Feaver & Purples . . 1,929 Flox & Smallpox . . . . . . . . . . . . . . . . . 655 Childbed . . . . . . . . . . . . . . . . . . . . . . . . . . 625 Stopping of the Stomach . . . 332 Found Dead in streets, fields, &c. . 20 Chrisomes and Infants . . . . . . . . . . . . . 1,258 Stone and Stranguary . . . . . . 98 French Pox. . . . . . . . . . . . . . . . . . . . . . . 86 Meagrom and Headach . . . . . . . . . . . . 12 Surfe . . . . . . . . . . . . . . . . . . . . . . 1,251 Frighted . . . . . . . . . . . . . . . . . . . . . . . . . 23 Measles . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Teeth & Worms . . . . . . . . . . . . 2,614 Gout & Sciatica . . . . . . . . . . . . . . . . . . 27 Murthered & Shot . . . . . . . . . . . . . . . . . 9 Vomiting . . . . . . . . . . . . . . . . . . . 51 Grief . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Overlaid & Starved . . . . . . . . . . . . . . . . 45 Wenn . . . . . . . . . . . . . . . . . . . . . . 8

Christened-Males . . . . . . . . . . . . . . . . 5,114 Females . . . . . . . . . . . . . . . . . . . . . . . . . . . 4,853 In all . . . . . . . . . . . . . . . . . . . . . . . 9,967 Buried-Males . . . . . . . . . . . . . . . . . . . . 58,569 Females . . . . . . . . . . . . . . . . . . . . . . . . . . . 48,737 In all . . . . . . . . . . . . . . . . . . . . . . . 97,306 Of the Plague . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,596 Increase in the Burials in the 130 Parishes and the Pesthouse this year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79,009 Increase of the Plague in the 130 Parishes and the Pesthouse this year . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68,590

Figure 1.3.3

launched the twin sciences of vital statistics and demography, and, although the name came later, it also signaled the beginning of political arithmetic. (Graunt did not have to wait long for accolades; in the year his book was published, he was elected to the prestigious Royal Society of London.)

High on the list of innovations that made Graunt’s work unique were his objec- tives. Not content simply to describe a situation, although he was adept at doing so, Graunt often sought to go beyond his data and make generalizations (or, in current statistical terminology, draw inferences). Having been blessed with this particular turn of mind, he almost certainly qualifies as the world’s first statistician. All Graunt really lacked was the probability theory that would have enabled him to frame his

Section 1.3 A Brief History 13

inferences more mathematically. That theory, though, was just beginning to unfold several hundred miles away in France (160).

Other seventeenth-century writers were quick to follow through on Graunt’s ideas. William Petty’s Political Arithmetick was published in 1690, although it had probably been written some fifteen years earlier. (It was Petty who gave the move- ment its name.) Perhaps even more significant were the contributions of Edmund Halley (of “Halley’s comet” fame). Principally an astronomer, he also dabbled in political arithmetic, and in 1693 wrote An Estimate of the Degrees of the Mortal- ity of Mankind, drawn from Curious Tables of the Births and Funerals at the city of Breslaw; with an attempt to ascertain the Price of Annuities upon Lives. (Book titles were longer then!) Halley shored up, mathematically, the efforts of Graunt and oth- ers to construct an accurate mortality table. In doing so, he laid the foundation for the important theory of annuities. Today, all life insurance companies base their pre- mium schedules on methods similar to Halley’s. (The first company to follow his lead was The Equitable, founded in 1765.)

For all its initial flurry of activity, political arithmetic did not fare particularly well in the eighteenth century, at least in terms of having its methodology fine-tuned. Still, the second half of the century did see some notable achievements in improving the quality of the databases: Several countries, including the United States in 1790, es- tablished a periodic census. To some extent, answers to the questions that interested Graunt and his followers had to be deferred until the theory of probability could develop just a little bit more.

QUETELET: THE CATALYST

With political arithmetic furnishing the data and many of the questions, and the the- ory of probability holding out the promise of rigorous answers, the birth of statistics was at hand. All that was needed was a catalyst—someone to bring the two together. Several individuals served with distinction in that capacity. Carl Friedrich Gauss, the superb German mathematician and astronomer, was especially helpful in showing how statistical concepts could be useful in the physical sciences. Similar efforts in France were made by Laplace. But the man who perhaps best deserves the title of “matchmaker” was a Belgian, Adolphe Quetelet.

Quetelet was a mathematician, astronomer, physicist, sociologist, anthropolo- gist, and poet. One of his passions was collecting data, and he was fascinated by the regularity of social phenomena. In commenting on the nature of criminal tendencies, he once wrote (76):

Thus we pass from one year to another with the sad perspective of seeing the same crimes reproduced in the same order and calling down the same punishments in the same proportions. Sad condition of humanity! . . . We might enumerate in advance how many individuals will stain their hands in the blood of their fellows, how many will be forgers, how many will be poisoners, almost we can enumerate in advance the births and deaths that should occur. There is a budget which we pay with a frightful regularity; it is that of prisons, chains and the scaffold.

Given such an orientation, it was not surprising that Quetelet would see in prob- ability theory an elegant means for expressing human behavior. For much of the nineteenth century he vigorously championed the cause of statistics, and as a mem- ber of more than one hundred learned societies, his influence was enormous. When he died in 1874, statistics had been brought to the brink of its modern era.

14 Chapter 1 Introduction

1.4 A Chapter Summary The concepts of probability lie at the very heart of all statistical problems. Acknowl- edging that fact, the next two chapters take a close look at some of those concepts. Chapter 2 states the axioms of probability and investigates their consequences. It also covers the basic skills for algebraically manipulating probabilities and gives an introduction to combinatorics, the mathematics of counting. Chapter 3 reformulates much of the material in Chapter 2 in terms of random variables, the latter being a concept of great convenience in applying probability to statistics. Over the years, particular measures of probability have emerged as being especially useful: The most prominent of these are profiled in Chapter 4.

Our study of statistics proper begins with Chapter 5, which is a first look at the theory of parameter estimation. Chapter 6 introduces the notion of hypothesis test- ing, a procedure that, in one form or another, commands a major share of the remain- der of the book. From a conceptual standpoint, these are very important chapters: Most formal applications of statistical methodology will involve either parameter estimation or hypothesis testing, or both.

Among the probability functions featured in Chapter 4, the normal distribu- tion—more familiarly known as the bell-shaped curve—is sufficiently important to merit even further scrutiny. Chapter 7 derives in some detail many of the properties and applications of the normal distribution as well as those of several related prob- ability functions. Much of the theory that supports the methodology appearing in Chapters 9 through 13 comes from Chapter 7.

Chapter 8 describes some of the basic principles of experimental “design.” Its purpose is to provide a framework for comparing and contrasting the various statis- tical procedures profiled in Chapters 9 through 15.

Chapters 9, 12, and 13 continue the work of Chapter 7, but with the emphasis on the comparison of several populations, similar to what was done in Case Study 1.2.2. Chapter 10 looks at the important problem of assessing the level of agreement between a set of data and the values predicted by the probability model from which those data presumably came. Linear relationships are examined in Chapter 11.

Chapter 14 is an introduction to nonparametric statistics. The objective there is to develop procedures for answering some of the same sorts of questions raised in Chapters 7, 9, 12, and 13, but with fewer initial assumptions.

Chapter 15 (available as a download) is new to the sixth edition. It introduces a broad class of multifactor experiments known as factorial designs. These are rooted in the mathematics developed in Chapters 12 and 13, but their interpretation is con- siderably more nuanced because of the “interactions” that may be present between some or all of the factors.

As a general format, each chapter contains numerous examples and case studies, the latter including actual experimental data taken from a variety of sources, primar- ily newspapers, magazines, and technical journals. We hope that these applications will make it abundantly clear that, while the general orientation of this text is theoret- ical, the consequences of that theory are never too far from having direct relevance to the “real world.”

Chapte r

2Probability CHAPTER OUTLINE

2.1 Introduction 2.2 Sample Spaces and

the Algebra of Sets 2.3 The Probability

Function 2.4 Conditional

Probability 2.5 Independence 2.6 Combinatorics 2.7 Combinatorial

Probability 2.8 Taking a Second Look

at Statistics (Monte Carlo Techniques)

One of the most influential of seventeenth-century mathematicians, Fermat earned his living as a lawyer and administrator in Toulouse. He shares credit with Descartes for the invention of analytic geometry, but his most important work may have been in number theory. Fermat did not write for publication, preferring instead to send letters and papers to friends. His correspondence with Pascal was the starting point for the development of a mathematical theory of probability.

—Pierre de Fermat (1601–1665)

Pascal was the son of a nobleman. A prodigy of sorts, he had already published a treatise on conic sections by the age of sixteen. He also invented one of the early calculating machines to help his father with accounting work. Pascal’s contributions to probability were stimulated by his correspondence, in 1654, with Fermat. Later that year he retired to a life of religious meditation.

—Blaise Pascal (1623–1662)

2.1 INTRODUCTION Experts have estimated that the likelihood of any given UFO sighting being genuine is on the order of one in one hundred thousand. Since the early 1950s, some ten thou- sand sightings have been reported to civil authorities. What is the probability that at least one of those objects was, in fact, an alien spacecraft? In 1978, Pete Rose of the Cincinnati Reds set a National League record by batting safely in forty-four consec- utive games. How unlikely was that event, given that Rose was a lifetime .303 hitter? By definition, the mean free path is the average distance a molecule in a gas travels before colliding with another molecule. How likely is it that the distance a molecule travels between collisions will be at least twice its mean free path? Suppose a boy’s mother and father both have genetic markers for sickle cell anemia, but neither par- ent exhibits any of the disease’s symptoms. What are the chances that their son will also be asymptomatic? What are the odds that a poker player is dealt a full house or that a craps-shooter makes his “point”? If a woman has lived to age seventy, how likely is it that she will die before her ninetieth birthday? In 1994, Tom Foley was Speaker of the House and running for re-election. The day after the election, his race had still not been “called” by any of the networks: he trailed his Republican challenger by 2174 votes, but 14,000 absentee ballots remained to be counted. Foley, however, conceded. Should he have waited for the absentee ballots to be counted, or was his defeat at that point a virtual certainty?

As the nature and variety of these questions would suggest, probability is a sub- ject with an extraordinary range of real-world, everyday applications. What began as an exercise in understanding games of chance has proven to be useful everywhere. Maybe even more remarkable is the fact that the solutions to all of these diverse

15

16 Chapter 2 Probability

questions are rooted in just a handful of definitions and theorems. Those results, together with the problem-solving techniques they empower, are the sum and sub- stance of Chapter 2. We begin, though, with a bit of history.

THE EVOLUTION OF THE DEFINITION OF PROBABILITY

Over the years, the definition of probability has undergone several revisions. There is nothing contradictory in the multiple definitions—the changes primarily reflected the need for greater generality and more mathematical rigor. The first formulation (often referred to as the classical definition of probability) is credited to Gerolamo Cardano (recall Section 1.3). It applies only to situations where (1) the number of possible outcomes is finite and (2) all outcomes are equally likely. Under those con- ditions, the probability of an event comprised of m outcomes is the ratio m/n, where n is the total number of (equally likely) outcomes. Tossing a fair, six-sided die, for example, gives m/n = 36 as the probability of rolling an even number (that is, either 2, 4, or 6).

While Cardano’s model was well-suited to gambling scenarios (for which it was intended), it was obviously inadequate for more general problems, where outcomes are not equally likely and/or the number of outcomes is not finite. Richard von Mises, a twentieth-century German mathematician, is often credited with avoid- ing the weaknesses in Cardano’s model by defining “empirical” probabilities. In the von Mises approach, we imagine an experiment being repeated over and over again under presumably identical conditions. Theoretically, a running tally could be kept of the number of times (m) the outcome belongs to a given event di- vided by n, the total number of times the experiment is performed. According to von Mises, the probability of the given event is the limit (as n goes to infinity) of the ratio m/n. Figure 2.1.1 illustrates the empirical probability of getting a head by tossing a fair coin: As the number of tosses continues to increase, the ratio m/n converges to 12 .

1

0 1

n = numbers of trials

2 3 4 5

1 2

m n

lim m/n n ̀

Figure 2.1.1

The von Mises approach definitely shores up some of the inadequacies seen in the Cardano model, but it is not without shortcomings of its own. There is some con- ceptual inconsistency, for example, in extolling the limit of m/n as a way of defining a probability empirically, when the very act of repeating an experiment under identical conditions an infinite number of times is physically impossible. And left unanswered is the question of how large n must be in order for m/n to be a good approximation for lim m/n.

Andrei Kolmogorov, the great Russian probabilist, took a different approach. Aware that many twentieth-century mathematicians were having success developing subjects axiomatically, Kolmogorov wondered whether probability might similarly be defined operationally, rather than as a ratio (like the Cardano model) or as a limit (like the von Mises model). His efforts culminated in a masterpiece of mathematical

Section 2.2 Sample Spaces and the Algebra of Sets 17

elegance when he published Grundbegriffe der Wahrscheinlichkeitsrechnung (Foundations of the Theory of Probability) in 1933. In essence, Kolmogorov was able to show that a maximum of four simple axioms is necessary and sufficient to define the way any and all probabilities must behave. (These will be our starting point in Section 2.3.)

We begin Chapter 2 with some basic (and, presumably, familiar) definitions from set theory. These are important because probability will eventually be defined as a set function—that is, a mapping from a set to a number. Then, with the help of Kolmogorov’s axioms in Section 2.3, we will learn how to calculate and manipu- late probabilities. The chapter concludes with an introduction to combinatorics—the mathematics of systematic counting—and its application to probability.

2.2 Sample Spaces and the Algebra of Sets The starting point for studying probability is the definition of four key terms: experiment, sample outcome, sample space, and event. The latter three, all carryovers from classical set theory, give us a familiar mathematical framework within which to work; the former is what provides the conceptual mechanism for casting real-world phenomena into probabilistic terms.

By an experiment we will mean any procedure that (1) can be repeated, theo- retically, an infinite number of times and (2) has a well-defined set of possible out- comes. Thus, rolling a pair of dice qualifies as an experiment and so does measuring a hypertensive’s blood pressure or doing a spectrographic analysis to determine the carbon content of moon rocks. Asking a would-be psychic to draw a picture of an image presumably transmitted by another would-be psychic does not qualify as an experiment, because the set of possible outcomes cannot be listed, characterized, or otherwise defined.

Each of the potential eventualities of an experiment is referred to as a sample outcome, s, and their totality is called the sample space, S. To signify the membership of s in S, we write s ∈ S. Any designated collection of sample outcomes, including individual outcomes, the entire sample space, and the null set, constitutes an event. The latter is said to occur if the outcome of the experiment is one of the members of the event.

Example 2.2.1

Consider the experiment of flipping a coin three times. What is the sample space? Which sample outcomes make up the event A: Majority of coins show heads?

Think of each sample outcome here as an ordered triple, its components repre- senting the outcomes of the first, second, and third tosses, respectively. Altogether, there are eight different triples, so those eight comprise the sample space:

S = {HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}

By inspection, we see that four of the sample outcomes in S constitute the event A:

A = {HHH, HHT, HTH, THH}

Example 2.2.2

Imagine rolling two dice, the first one red, the second one green. Each sample out- come is an ordered pair (face showing on red die, face showing on green die), and the entire sample space can be represented as a 6 × 6 matrix (see Figure 2.2.1).

18 Chapter 2 Probability

A

Face showing on green die

F ac

e sh

ow in

g on

r ed

d ie

1 2 3 4 5 6

1 (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)

2 (2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)

3 (3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6)

4 (4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6)

5 (5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6)

6 (6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)

Figure 2.2.1

Gamblers are often interested in the event A that the sum of the faces showing is a 7. Notice in Figure 2.2.1 that the sample outcomes contained in A are the six diagonal entries, (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1).

Example 2.2.3

A local TV station advertises two newscasting positions. If three women (W1,W2,W3) and two men (M1, M2) apply, the “experiment” of hiring two coanchors generates a sample space of ten outcomes:

S = {(W1,W2), (W1,W3), (W2,W3), (W1, M1), (W1, M2), (W2, M1), (W2, M2), (W3, M1), (W3, M2), (M1, M2)}

Does it matter here that the two positions being filled are equivalent? Yes. If the station were seeking to hire, say, a sports announcer and a weather forecaster, the number of possible outcomes would be twenty: (W2, M1), for example, would repre- sent a different staffing assignment than (M1,W2).

Example 2.2.4

The number of sample outcomes associated with an experiment need not be fi- nite. Suppose that a coin is tossed until the first tail appears. If the first toss is itself a tail, the outcome of the experiment is T; if the first tail occurs on the second toss, the outcome is HT; and so on. Theoretically, of course, the first tail may never occur, and the infinite nature of S is readily apparent:

S = {T, HT, HHT, HHHT, . . .}

Example 2.2.5

There are three ways to indicate an experiment’s sample space. If the number of pos- sible outcomes is small, we can simply list them, as we did in Examples 2.2.1 through 2.2.3. In some cases it may be possible to characterize a sample space by showing the structure its outcomes necessarily possess. This is what we did in Example 2.2.4. A third option is to state a mathematical formula that the sample outcomes must satisfy.

A computer programmer is running a subroutine that solves a general quadratic equation, ax2 + bx + c = 0. Her “experiment” consists of choosing val- ues for the three coefficients a, b, and c. Define (1) S and (2) the event A: Equation has two equal roots.

First, we must determine the sample space. Since presumably no combinations of finite a, b, and c are inadmissible, we can characterize S by writing a series of inequalities:

S = {(a, b, c) : −∞ < a < ∞,−∞ < b < ∞,−∞ < c < ∞}

Section 2.2 Sample Spaces and the Algebra of Sets 19

Defining A requires the well-known result from algebra that a quadratic equation has equal roots if and only if its discriminant, b2 − 4ac, vanishes. Membership in A, then, is contingent on a, b, and c satisfying an equation:

A = {(a, b, c) : b2 − 4ac = 0}

Questions

2.2.1. A graduating engineer has signed up for three job interviews. She intends to categorize each one as being either a ‘‘success’’ or a ‘‘failure’’ depending on whether it leads to a plant trip. Write out the appropriate sample space. What outcomes are in the event A: Second success occurs on third interview? In B: First success never occurs? (Hint: Notice the similarity between this situation and the coin-tossing experiment described in Example 2.2.1.)

2.2.2. Three dice are tossed, one red, one blue, and one green. What outcomes make up the event A that the sum of the three faces showing equals 5?

2.2.3. An urn contains six chips numbered 1 through 6. Three are drawn out. What outcomes are in the event “Second smallest chip is a 3”? Assume that the order of the chips is irrelevant.

2.2.4. Suppose that two cards are dealt from a standard 52-card poker deck. Let A be the event that the sum of the two cards is 8 (assume that aces have a numerical value of 1). How many outcomes are in A?

2.2.5. In the lingo of craps-shooters (where two dice are tossed and the underlying sample space is the matrix pic- tured in Figure 2.2.1) is the phrase “making a hard eight.” What might that mean?

2.2.6. A poker deck consists of fifty-two cards, represent- ing thirteen denominations (2 through ace) and four suits (diamonds, hearts, clubs, and spades). A five-card hand is called a flush if all five cards are in the same suit but not all five denominations are consecutive. Pictured below is a flush in hearts. Let N be the set of five cards in hearts that are not flushes. How many outcomes are in N? [Note: In poker, the denominations (A, 2, 3, 4, 5) are considered to be consecutive (in addition to sequences such as (8, 9, 10, J, Q)).]

Denominations

2 3 4 5 6 7 8 9 10 J Q K A

D

H X X X X X Suits C

S

2.2.7. Let P be the set of right triangles with a 5′′ hy- potenuse and whose height and length are a and b, respec- tively. Characterize the outcomes in P.

2.2.8. Suppose a baseball player steps to the plate with the intention of trying to “coax” a base on balls by never swinging at a pitch. The umpire, of course, will necessar- ily call each pitch either a ball (B) or a strike (S). What outcomes make up the event A, that a batter walks on the sixth pitch? (Note: A batter “walks” if the fourth ball is called before the third strike.)

2.2.9. A telemarketer is planning to set up a phone bank to bilk widows with a Ponzi scheme. His past experience (prior to his most recent incarceration) suggests that each phone will be in use half the time. For a given phone at a given time, let 0 indicate that the phone is avail- able and let 1 indicate that a caller is on the line. Sup- pose that the telemarketer’s “bank” is comprised of four telephones. (a) Write out the outcomes in the sample space. (b) What outcomes would make up the event that exactly two phones are being used? (c) Suppose the telemarketer had k phones. How many outcomes would allow for the possibility that at most one more call could be received? (Hint: How many lines would have to be busy?)

2.2.10. Two darts are thrown at the following target:

2 4

1

(a) Let (u, v) denote the outcome that the first dart lands in region u and the second dart in region v. List the sample space of (u, v)’s. (b) List the outcomes in the sample space of sums, u + v. 2.2.11. A woman has her purse snatched by two teenagers. She is subsequently shown a police lineup consisting of five suspects, including the two perpetrators. What is the sample space associated with the experiment “Woman picks two suspects out of lineup”? Which outcomes are in the event A: She makes at least one incorrect identification?

2.2.12. Consider the experiment of choosing coefficients for the quadratic equation ax2 + bx + c = 0. Characterize the values of a, b, and c associated with the event A: Equa- tion has complex roots.

2.2.13. In the game of craps, the person rolling the dice (the shooter) wins outright if his first toss is a 7 or an 11. If his first toss is a 2, 3, or 12, he loses outright. If his first

20 Chapter 2 Probability

roll is something else, say a 9, that number becomes his “point” and he keeps rolling the dice until he either rolls another 9, in which case he wins, or a 7, in which case he loses. Characterize the sample outcomes contained in the event “Shooter wins with a point of 9.”

2.2.14. A probability-minded despot offers a convicted murderer a final chance to gain his release. The prisoner is given twenty chips, ten white and ten black. All twenty are to be placed into two urns, according to any allocation scheme the prisoner wishes, with the one proviso being that each urn contain at least one chip. The executioner will then pick one of the two urns at random and from that urn, one chip at random. If the chip selected is white, the

prisoner will be set free; if it is black, he “buys the farm.” Characterize the sample space describing the prisoner’s possible allocation options. (Intuitively, which allocation affords the prisoner the greatest chance of survival?)

2.2.15. Suppose that ten chips, numbered 1 through 10, are put into an urn at one minute to midnight, and chip number 1 is quickly removed. At one-half minute to mid- night, chips numbered 11 through 20 are added to the urn, and chip number 2 is quickly removed. Then at one-fourth minute to midnight, chips numbered 21 to 30 are added to the urn, and chip number 3 is quickly removed. If that pro- cedure for adding chips to the urn continues, how many chips will be in the urn at midnight (157)?

UNIONS, INTERSECTIONS, AND COMPLEMENTS

Associated with events defined on a sample space are several operations collectively referred to as the algebra of sets. These are the rules that govern the ways in which one event can be combined with another. Consider, for example, the game of craps described in Question 2.2.13. The shooter wins on his initial roll if he throws either a 7 or an 11. In the language of the algebra of sets, the event “Shooter rolls a 7 or an 11” is the union of two simpler events, “Shooter rolls a 7” and “Shooter rolls an 11.” If E denotes the union and if A and B denote the two events making up the union, we write E = A∪B. The next several definitions and examples illustrate those portions of the algebra of sets that we will find particularly useful in the chapters ahead.

Definition 2.2.1 Let A and B be any two events defined over the same sample space S. Then

a. The intersection of A and B, written A ∩ B, is the event whose outcomes belong to both A and B.

b. The union of A and B, written A ∪ B, is the event whose outcomes belong to either A or B or both.

Example 2.2.6

A single card is drawn from a poker deck. Let A be the event that an ace is selected:

A = {ace of hearts, ace of diamonds, ace of clubs, ace of spades} Let B be the event “Heart is drawn”:

B = {2 of hearts, 3 of hearts, . . . , ace of hearts} Then

A ∩ B = {ace of hearts} and

A ∪ B = {2 of hearts, 3 of hearts, . . . , ace of hearts, ace of diamonds, ace of clubs, ace of spades}

(Let C be the event “Club is drawn.” Which cards are in B ∪ C? In B ∩ C?)

Section 2.2 Sample Spaces and the Algebra of Sets 21

Example 2.2.7

Let A be the set of x’s for which x2 + 2x = 8; let B be the set for which x2 + x = 6. Find A ∩ B and A ∪ B.

Since the first equation factors into (x + 4)(x − 2) = 0, its solution set is A = {−4, 2}. Similarly, the second equation can be written (x + 3)(x − 2) = 0, making B = {−3, 2}. Therefore,

A ∩ B = {2} and

A ∪ B = {−4,−3, 2}

Example 2.2.8

Consider the electrical circuit pictured in Figure 2.2.2. Let Ai denote the event that switch i fails to close, i = 1, 2, 3, 4. Let A be the event “Circuit is not completed.” Express A in terms of the Ai’s.

1 3

2 4

Figure 2.2.2

Call the ① and ② switches line a; call the ③ and ④ switches line b. By inspection, the circuit fails only if both line a and line b fail. But line a fails only if either ① or ② (or both) fail. That is, the event that line a fails is the union A1 ∪ A2. Similarly, the failure of line b is the union A3 ∪ A4. The event that the circuit fails, then, is an intersection:

A = (A1 ∪ A2) ∩ (A3 ∪ A4)

Definition 2.2.2 Events A and B defined over the same sample space are said to be mutually exclusive if they have no outcomes in common—that is, if A ∩ B = ∅, where ∅ is the null set.

Example 2.2.9

Consider a single throw of two dice. Define A to be the event that the sum of the faces showing is odd. Let B be the event that the two faces themselves are odd. Then clearly, the intersection is empty, the sum of two odd numbers necessarily being even. In symbols, A ∩ B = ∅. (Recall the event B ∩ C asked for in Example 2.2.6.)

Definition 2.2.3 Let A be any event defined on a sample space S. The complement of A, written AC, is the event consisting of all the outcomes in S other than those contained in A.

Example 2.2.10

Let A be the set of (x, y)’s for which x2 + y2 < 1. Sketch the region in the xy-plane corresponding to AC.

From analytic geometry, we recognize that x2 + y2 < 1 describes the interior of a circle of radius 1 centered at the origin. Figure 2.2.3 shows the complement—the points on the circumference of the circle and the points outside the circle.

22 Chapter 2 Probability

AC : x2 +y2 $ 1

y

A x

Figure 2.2.3

The notions of union and intersection can easily be extended to more than two events. For example, the expression A1 ∪ A2 ∪ · · · ∪ Ak defines the set of out- comes belonging to any of the Ai’s (or to any combination of the Ai’s). Similarly, A1 ∩ A2 ∩ · · · ∩ Ak is the set of outcomes belonging to all of the Ai’s.

Example 2.2.11

Suppose the events A1, A2, . . . , Ak are intervals of real numbers such that

Ai = {x : 0 ≤ x < 1/i}, i = 1, 2, . . . , k Describe the sets A1 ∪ A2 ∪ · · · ∪ Ak = ∪ki=1Ai and A1 ∩ A2 ∩ · · · ∩ Ak = ∩ki=1Ai.

Notice that the Ai’s are telescoping sets. That is, A1 is the interval 0 ≤ x < 1, A2 is the interval 0 ≤ x < 12 , and so on. It follows, then, that the union of the k Ai’s is simply A1 while the intersection of the Ai’s (that is, their overlap) is Ak.

Questions

2.2.16. Sketch the regions in the xy-plane corresponding to A ∪ B and A ∩ B if

A = {(x, y): 0 < x < 3, 0 < y < 3} and

B = {(x, y): 2 < x < 4, 2 < y < 4} 2.2.17. Referring to Example 2.2.7, find A∩B and A∪B if the two equations were replaced by inequalities: x2 +2x ≤ 8 and x2 + x ≤ 6. 2.2.18. Find A ∩ B ∩ C if A = {x: 0 ≤ x ≤ 4}, B = {x: 2 ≤ x ≤ 6}, and C = {x: x = 0, 1, 2, . . .}. 2.2.19. An electronic system has four components divided into two pairs. The two components of each pair are wired in parallel; the two pairs are wired in series. Let Ai j denote the event “ith component in jth pair fails,” i = 1, 2; j = 1, 2. Let A be the event “System fails.” Write A in terms of the Ai j’s.

j = 1 j = 2

2.2.20. Define A = {x : 0 ≤ x ≤ 1}, B = {x : 0 ≤ x ≤ 3}, and C = {x : −1 ≤ x ≤ 2}. Draw diagrams showing each of the following sets of points: (a) AC ∩ B ∩ C (b) AC ∪ (B ∩ C) (c) A ∩ B ∩ CC (d) [(A ∪ B) ∩ CC]C 2.2.21. Let A be the set of five-card hands dealt from a fifty-two-card poker deck, where the denominations of the five cards are all consecutive—for example, (7 of hearts, 8 of spades, 9 of spades, 10 of hearts, jack of dia- monds). Let B be the set of five-card hands where the suits of the five cards are all the same. How many outcomes are in the event A ∩ B? 2.2.22. Suppose that each of the twelve letters in the word

T E S S E L L A T I O N

is written on a chip. Define the events F, R, and C as follows:

F : letters in first half of alphabet R: letters that are repeated V : letters that are vowels

Which chips make up the following events? (a) F ∩ R ∩ V (b) FC ∩ R ∩ VC (c) F ∩ RC ∩ V

Section 2.2 Sample Spaces and the Algebra of Sets 23

2.2.23. Let A, B, and C be any three events defined on a sample space S. Show that (a) the outcomes in A ∪ (B ∩ C) are the same as the out- comes in (A ∪ B) ∩ (A ∪ C). (b) the outcomes in A ∩ (B ∪ C) are the same as the out- comes in (A ∩ B) ∪ (A ∩ C). 2.2.24. Let A1, A2, . . . , Ak be any set of events defined on a sample space S. What outcomes belong to the event

(A1 ∪ A2 ∪ · · · ∪ Ak) ∪ ( AC1 ∩ AC2 ∩ · · · ∩ ACk

) 2.2.25. Let A, B, and C be any three events defined on a sample space S. Show that the operations of union and intersection are associative by proving that (a) A ∪ (B ∪ C) = (A ∪ B) ∪ C = A ∪ B ∪ C (b) A ∩ (B ∩ C) = (A ∩ B) ∩ C = A ∩ B ∩ C 2.2.26. Suppose that three events—A, B, and C—are de- fined on a sample space S. Use the union, intersection, and complement operations to represent each of the following events: (a) none of the three events occurs (b) all three of the events occur (c) only event A occurs (d) exactly one event occurs (e) exactly two events occur

2.2.27. What must be true of events A and B if (a) A ∪ B = B (b) A ∩ B = A 2.2.28. Let events A and B and sample space S be defined as the following intervals:

S = {x : 0 ≤ x ≤ 10} A = {x : 0 < x < 5} B = {x : 3 ≤ x ≤ 7}

Characterize the following events: (a) AC

(b) A ∩ B (c) A ∪ B (d) A ∩ BC (e) AC ∪ B (f) AC ∩ BC 2.2.29. A coin is tossed four times and the resulting se- quence of heads and/or tails is recorded. Define the events A, B, and C as follows:

A: exactly two heads appear B: heads and tails alternate C: first two tosses are heads

(a) Which events, if any, are mutually exclusive? (b) Which events, if any, are subsets of other sets?

2.2.30. Pictured below are two organizational charts de- scribing the way upper management vets new proposals. For both models, three vice presidents—1, 2, and 3—each voice an opinion.

1

1

2

2

3

3

(a)

(b)

For (a), all three must concur if the proposal is to pass; if any one of the three favors the proposal in (b), it passes. Let Ai denote the event that vice president i favors the proposal, i = 1, 2, 3, and let A denote the event that the proposal passes. Express A in terms of the Ai’s for the two office protocols. Under what sorts of situations might one system be preferable to the other?

EXPRESSING EVENTS GRAPHICALLY: VENN DIAGRAMS

Relationships based on two or more events can sometimes be difficult to express us- ing only equations or verbal descriptions. An alternative approach that can be highly effective is to represent the underlying events graphically in a format known as a Venn diagram. Figure 2.2.4 shows Venn diagrams for an intersection, a union, a com- plement, and two events that are mutually exclusive. In each case, the shaded interior of a region corresponds to the desired event.

Example 2.2.12

When two events A and B are defined on a sample space, we will frequently need to consider

a. the event that exactly one (of the two) occurs.

b. the event that at most one (of the two) occurs.

Getting expressions for each of these is easy if we visualize the corresponding Venn diagrams.

24 Chapter 2 Probability

Venn diagrams

A A > B

B S

A A < B

B S

S

A

A > B = ø

B S

AC

A

Figure 2.2.4

The shaded area in Figure 2.2.5 represents the event E that either A or B, but not both, occurs (that is, exactly one occurs).

S

A B

Figure 2.2.5

Just by looking at the diagram we can formulate an expression for E. The portion of A, for example, included in E is A ∩ BC. Similarly, the portion of B included in E is B ∩ AC. It follows that E can be written as a union:

E = (A ∩ BC) ∪ (B ∩ AC) (Convince yourself that an equivalent expression for E is (A ∩ B)C ∩ (A ∪ B).)

Figure 2.2.6 shows the event F that at most one (of the two events) occurs. Since the latter includes every outcome except those belonging to both A and B, we can write

F = (A ∩ B)C

A B

S

Figure 2.2.6

The final example in this section shows two ways of “verifying” identities involv- ing events. The first is nonrigorous and uses Venn diagrams; the second is a formal approach in which every outcome in the left-hand side of the presumed identity is shown to belong to the right-hand side, and vice versa. The particular identity being established is a very useful distributive property for intersections.

Example 2.2.13

Let A, B, and C be any three events defined over the same sample space S. Show that

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) (2.2.1) Figure 2.2.7 pictures the set formed by intersecting A with the union of B and C. Simi- larly, Figure 2.2.8 shows the union of the intersection of A and B with the intersection

Section 2.2 Sample Spaces and the Algebra of Sets 25

of A and C. It would appear at this point that the identity is true: The rightmost diagrams in Figures 2.2.7 and 2.2.8 do show the same shaded region. Still, this is not a proof. How the events were initially drawn may not do justice to the problem in general. Can we be certain, for example, that Equation 2.2.1 remains true if C, for example, is mutually exclusive of A or B? Or if B is a proper subset of A?

A

S S

B

B < C A > (B < C)

C

A B

C

Figure 2.2.7

A

S S

B

A > C

A > B (A > B) < (A > C)

C

A B

C

Figure 2.2.8

For a rigorous solution, we need to take the algebraic approach of showing that

(1) A ∩ (B ∪ C) is contained in (A ∩ B) ∪ (A ∩ C) and

(2) (A ∩ B) ∪ (A ∩ C) is contained in A ∩ (B ∪ C). To that end, let s ∈ A∩ (B ∪ C). Then, s ∈ A and s ∈ B∪C. But if s ∈ B∪C, then

either s ∈ B or s ∈ C. If s ∈ B, then s ∈ A ∩ B and s ∈ (A ∩ B) ∪ (A ∩ C). Likewise, if s ∈ C, it follows that s ∈ (A ∩ B) ∪ (A ∩ C). Therefore, every sample outcome in A ∩ (B ∪ C) is also contained in (A ∩ B) ∪ (A ∩ C). Going the other way, assume that s ∈ (A ∩ B) ∪ (A ∩ C). Therefore, either s ∈ (A ∩ B) or s ∈ (A ∩ C) (or both). Suppose s ∈ A ∩ B. Then s ∈ A and s ∈ B, in which case s ∈ A ∩ (B ∪ C). The same conclusion holds if s ∈ (A ∩ C). Thus, every sample outcome in (A ∩ B) ∪ (A ∩ C) is in A ∩ (B ∪ C). It follows that A ∩ (B ∪ C) and (A ∩ B) ∪ (A ∩ C) are identical.

Questions

2.2.31. During orientation week, the latest Spiderman movie was shown twice at State University. Among the entering class of 6000 freshmen, 850 went to see it the first time, 690 the second time, while 4700 failed to see it either time. How many saw it twice?

2.2.32. Let A and B be any two events. Use Venn dia- grams to show that (a) the complement of their intersection is the union of their complements:

(A ∩ B)C = AC ∪ BC

(b) the complement of their union is the intersection of their complements:

(A ∪ B)C = AC ∩ BC

(These two results are known as DeMorgan’s laws.)

2.2.33. Let A, B, and C be any three events. Use Venn diagrams to show that

(a) A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) (b) A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) 2.2.34. Let A, B, and C be any three events. Use Venn diagrams to show that (a) A ∪ (B ∪ C) = (A ∪ B) ∪ C (b) A ∩ (B ∩ C) = (A ∩ B) ∩ C 2.2.35. Let A and B be any two events defined on a sam- ple space S. Which of the following sets are necessarily subsets of which other sets?

A B A ∪ B A ∩ B AC ∩ B A ∩ BC (AC ∪ BC)C

2.2.36. Use Venn diagrams to suggest an equivalent way of representing the following events: (a) (A ∩ BC)C (b) B ∪ (A ∪ B)C (c) A ∩ (A ∩ B)C

26 Chapter 2 Probability

2.2.37. A total of twelve hundred graduates of State Tech have gotten into medical school in the past several years. Of that number, one thousand earned scores of twenty-seven or higher on the MCAT and four hundred had GPAs that were 3.5 or higher. Moreover, three hundred had MCATs that were twenty-seven or higher and GPAs that were 3.5 or higher. What proportion of those twelve hundred graduates got into medical school with an MCAT lower than twenty-seven and a GPA below 3.5?

2.2.38. Let A, B, and C be any three events defined on a sample space S. Let N(A), N(B), N(C), N(A ∩ B), N(A ∩ C), N(B ∩ C), and N(A ∩ B ∩ C) denote the numbers of outcomes in all the different intersections in which A, B, and C are involved. Use a Venn diagram to suggest a formula for N(A ∪ B ∪ C). [Hint: Start with the sum N(A) + N(B) + N(C) and use the Venn diagram to identify the “adjustments” that need to be made to that sum before it can equal N(A ∪ B ∪ C).] As a precedent,

note that N(A ∪ B) = N(A) + N(B) − N(A ∩ B). There, in the case of two events, subtracting N(A ∩ B) is the “adjustment.”

2.2.39. A poll conducted by a potential presidential candidate asked two questions: (1) Do you support the candidate’s position on taxes? and (2) Do you support the candidate’s position on homeland security? A total of twelve hundred responses were received; six hundred said “yes” to the first question and four hundred said “yes” to the second. If three hundred respondents said “no” to the taxes question and “yes” to the homeland security ques- tion, how many said “yes” to the taxes question but “no” to the homeland security question?

2.2.40. For two events A and B defined on a sample space S, N(A ∩ BC) = 15, N(AC ∩ B) = 50, and N(A ∩ B) = 2. Given that N(S) = 120, how many outcomes belong to neither A nor B?

2.3 The Probability Function Having introduced in Section 2.2 the twin concepts of ‘‘experiment’’ and ‘‘sample space,’’ we are now ready to pursue in a formal way the all-important problem of assigning a probability to an experiment’s outcome—and, more generally, to an event. Specifically, if A is any event defined on a sample space S, the symbol P(A) will denote the probability of A, and we will refer to P as the probability function. It is, in effect, a mapping from a set (i.e., an event) to a number. The backdrop for our discussion will be the unions, intersections, and complements of set theory; the starting point will be the axioms referred to in Section 2.1 that were originally set forth by Kolmogorov.

If S has a finite number of members, Kolmogorov showed that as few as three axioms are necessary and sufficient for characterizing the probability function P:

Axiom 1. Let A be any event defined over S. Then P(A) ≥ 0. Axiom 2. P(S) = 1. Axiom 3. Let A and B be any two mutually exclusive events defined over S. Then

P(A ∪ B) = P(A) + P(B) When S has an infinite number of members, a fourth axiom is needed:

Axiom 4. Let A1, A2, . . . , be events defined over S. If Ai ∩Aj = ∅ for each i = j, then

P

( ∞⋃ i=1

Ai

) =

∞∑ i=1

P(Ai)

From these simple statements come the general rules for manipulating probability functions—rules that apply no matter what specific mathematical forms the functions may happen to take.

SOME BASIC PROPERTIES OF P

Some of the immediate consequences of Kolmogorov’s axioms are the results given in Theorems 2.3.1 through 2.3.6. Despite their simplicity, these properties prove to be extraordinarily useful in solving all sorts of problems.

Section 2.3 The Probability Function 27

Theorem 2.3.1

P(AC) = 1 − P(A). Proof By Axiom 2 and Definition 2.2.3,

P(S) = 1 = P(A ∪ AC) But A and AC are mutually exclusive, so

P(A ∪ AC) = P(A) + P(AC) and the result follows.

Theorem 2.3.2

P(∅) = 0. Proof Since ∅ = SC, P(∅) = P(SC) = 1 − P(S) = 0.

Theorem 2.3.3

If A ⊂ B, then P(A) ≤ P(B). Proof Note that the event B may be written in the form

B = A ∪ (B ∩ AC) where A and (B ∩ AC) are mutually exclusive. Therefore,

P(B) = P(A) + P(B ∩ AC) which implies that P(B) ≥ P(A) since P(B ∩ AC) ≥ 0.

Theorem 2.3.4

For any event A, P(A) ≤ 1. Proof The proof follows immediately from Theorem 2.3.3 because A ⊂ S and P(S) = 1.

Theorem 2.3.5

Let A1, A2, . . . , An be events defined over S. If Ai ∩ Aj = ∅ for i = j, then

P

( n⋃

i=1 Ai

) =

n∑ i=1

P(Ai)

Proof The proof is a straightforward induction argument with Axiom 3 being the starting point.

Theorem 2.3.6

P(A ∪ B) = P(A) + P(B) − P(A ∩ B). Proof The Venn diagram for A ∪ B certainly suggests that the statement of the theorem is true (recall Figure 2.2.4). More formally, we have from Axiom 3 that

P(A) = P(A ∩ BC) + P(A ∩ B) and

P(B) = P(B ∩ AC) + P(A ∩ B) Adding these two equations gives

P(A) + P(B) = [P(A ∩ BC) + P(B ∩ AC) + P(A ∩ B)] + P(A ∩ B) By Theorem 2.3.5, the sum in the brackets is P(A ∪ B). If we subtract P(A ∩ B) from both sides of the equation, the result follows.

28 Chapter 2 Probability

The next result is a generalization of Theorem 2.3.6 that considers the probability of the union of n events. We have elected to retain the two-event case, P (A ∪ B), as a separate theorem simply for pedagogical reasons.

Theorem 2.3.7

Let A1, A2, . . . An be any n events defined on S. Then

P

( n⋃

i=1 Ai

) =

n∑ i=1

P (Ai) − ∑ i< j

P ( Ai ∩ Aj

) +

∑ i< j<k

P ( Ai ∩ Aj ∩ Ak

) − ∑ i< j<k<l

P ( Ai ∩ Aj ∩ Ak ∩ Al

) + · · · + (−1)n+1 · P (A1 ∩ A2 ∩ · · · ∩ An)

Proof The proof of Theorem 2.3.7 is basically an exercise in bookkeeping. By definition, A1 ∪ A2 ∪ · · · ∪ An is the set of outcomes belonging to any of the Ai’s individually or to any intersections of the Ai’s. The right-hand side of Theo- rem 2.3.7 is a counting scheme that alternately includes and excludes subsets of outcomes in such a way that each outcome in A1 ∪ A2 ∪ · · · ∪ An is included once and only once in the calculation of P (A1 ∪ A2 ∪ · · · ∪ An).

Consider, for example, the union of three events, A1, A2, and A3 as shown in Figure 2.3.1.

A1 A3

A2

Figure 2.3.1

Notice that simply adding P(A1), P(A2), and P(A3) results in adding P (A1 ∩ A2) twice, P (A1 ∩ A3) twice, and P (A2 ∩ A3) twice. It also results in P (A1 ∩ A2 ∩ A3) being added three times. The obvious “correction” is to subtract the probabilities associated with A1 ∩ A2, A1 ∩ A3, and A2 ∩ A3 and to add back the probability associated with A1 ∩ A2 ∩ A3 since those latter outcomes have pre- viously been added three times and subtracted three times. Therefore,

P (A1 ∪ A2 ∪ A3) = 3∑

i=1 P (Ai) −

∑ i< j

P ( Ai ∩ Aj

) + (−1)3+1 P (A1 ∩ A2 ∩ A3) = P (A1) + P (A2) + P (A3) − P (A1 ∩ A2) − P (A1 ∩ A3)

− P (A2 ∩ A3) + P (A1 ∩ A2 ∩ A3) . A formal proof of the theorem requires a knowledge of combinatorics and will be deferred until Section 2.6.

Example 2.3.1

Let A and B be two events defined on a sample space S such that P(A) = 0.3, P(B) = 0.5, and P(A ∪ B) = 0.7. Find (a) P(A ∩ B), (b) P(AC ∪ BC), and (c) P(AC ∩ B).

a. Transposing the terms in Theorem 2.3.6 yields a general formula for the proba- bility of an intersection:

P(A ∩ B) = P(A) + P(B) − P(A ∪ B)

Section 2.3 The Probability Function 29

Here

P(A ∩ B) = 0.3 + 0.5 − 0.7 = 0.1

b. The two cross-hatched regions in Figure 2.3.2 correspond to AC and BC. The union of AC and BC consists of those regions that have cross-hatching in either or both directions. By inspection, the only portion of S not included in AC ∪ BC is the intersection, A ∩ B. By Theorem 2.3.1, then

P(AC ∪ BC) = 1 − P(A ∩ B) = 1 − 0.1 = 0.9

S

AC

BC

A

B

Figure 2.3.2

S

AC

B

A

B

Figure 2.3.3

c. The event AC ∩ B corresponds to the region in Figure 2.3.3 where the cross- hatching extends in both directions—that is, everywhere in B except the inter- section with A. Therefore,

P(AC ∩ B) = P(B) − P(A ∩ B) = 0.5 − 0.1 = 0.4

Example 2.3.2

Show that

P(A ∩ B) ≥ 1 − P(AC) − P(BC) for any two events A and B defined on a sample space S.

From Example 2.3.1a and Theorem 2.3.1,

P(A ∩ B) = P(A) + P(B) − P(A ∪ B) = 1 − P(AC) + 1 − P(BC) − P(A ∪ B)

But P(A ∪ B) ≤ 1 from Theorem 2.3.4, so P(A ∩ B) ≥ 1 − P(AC) − P(BC)

30 Chapter 2 Probability

Example 2.3.3

Two cards are drawn from a poker deck without replacement. What is the probability that the second is higher in rank than the first?

Let A1, A2, and A3 be the events “First card is lower in rank,” “First card is higher in rank,” and “Both cards have same rank,” respectively. Clearly, the three Ai’s are mutually exclusive and they account for all possible outcomes, so from Theorem 2.3.5,

P(A1 ∪ A2 ∪ A3) = P(A1) + P(A2) + P(A3) = P(S) = 1 Once the first card is drawn, there are three choices for the second that would have the same rank—that is, P(A3) = 351 . Moreover, symmetry demands that P(A1) = P(A2), so

2P(A2) + 351 = 1

implying that P(A2) = 817 .

Example 2.3.4

Having endured (and survived) the mental trauma that comes from taking two years of chemistry, a year of physics, and a year of biology, Biff decides to test the medi- cal school waters and sends his MCATs to two colleges, X and Y . Based on how his friends have fared, he estimates that his probability of being accepted at X is 0.7, and at Y it is 0.4. He also suspects there is a 75% chance that at least one of his applica- tions will be rejected. What is the probability that he gets at least one acceptance?

Let A be the event “School X accepts him” and B the event “School Y accepts him.” We are given that P(A) = 0.7, P(B) = 0.4, and P(AC ∪ BC) = 0.75. The question is asking for P(A ∪ B).

From Theorem 2.3.6,

P(A ∪ B) = P(A) + P(B) − P(A ∩ B) Recall from Question 2.2.32 that AC ∪ BC = (A ∩ B)C, so

P(A ∩ B) = 1 − P[(A ∩ B)C] = 1 − 0.75 = 0.25 It follows that Biff’s prospects are not all that bleak—he has an 85% chance of getting in somewhere:

P(A ∪ B) = 0.7 + 0.4 − 0.25 = 0.85

Comment Notice that P(A ∪ B) varies directly with P(AC ∪ BC): P(A ∪ B) = P(A) + P(B) − [1 − P(AC ∪ BC)]

= P(A) + P(B) − 1 + P(AC ∪ BC) If P(A) and P(B), then, are fixed, we get the curious result that Biff’s chances of getting at least one acceptance increase if his chances of at least one rejection increase.

Questions

2.3.1. According to a family-oriented lobbying group, there is too much crude language and violence on tele- vision. Forty-two percent of the programs they screened had language they found offensive, 27% were too violent, and 10% were considered excessive in both language and

violence. What percentage of programs did comply with the group’s standards?

2.3.2. Let A and B be any two events defined on S. Sup- pose that P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.1. What is the probability that A or B but not both occur?

Section 2.4 Conditional Probability 31

2.3.3. Express the following probabilities in terms of P(A), P(B), and P(A ∩ B). (a) P(AC ∪ BC) (b) P(AC ∩ (A ∪ B)) 2.3.4. Let A and B be two events defined on S. If the prob- ability that at least one of them occurs is 0.3 and the prob- ability that A occurs but B does not occur is 0.1, what is P(B)?

2.3.5. Suppose that three fair dice are tossed. Let Ai be the event that a 6 shows on the ith die, i = 1, 2, 3. Does P(A1 ∪ A2 ∪ A3) = 12 ? Explain. 2.3.6. Events A and B are defined on a sample space S such that P((A ∪ B)C) = 0.5 and P(A ∩ B) = 0.2. What is the probability that either A or B but not both will occur?

2.3.7. Let A1, A2, . . . , An be a series of events for which Ai ∩ Aj = ∅ if i = j and A1 ∪ A2 ∪ · · · ∪ An = S. Let B be any event defined on S. Express B as a union of intersections.

2.3.8. Draw the Venn diagrams that would correspond to the equations (a) P(A ∩ B) = P(B) and (b) P(A ∪ B) = P(B).

2.3.9. In the game of “odd man out” each player tosses a fair coin. If all the coins turn up the same except for one, the player tossing the different coin is declared the odd man out and is eliminated from the contest. Suppose that three people are playing. What is the probability that someone will be eliminated on the first toss? (Hint: Use Theorem 2.3.1.)

2.3.10. An urn contains twenty-four chips, numbered 1 through 24. One is drawn at random. Let A be the event that the number is divisible by 2 and let B be the event that the number is divisible by 3. Find P(A ∪ B). 2.3.11. If State’s football team has a 10% chance of win- ning Saturday’s game, a 30% chance of winning two weeks from now, and a 65% chance of losing both games, what are their chances of winning exactly once?

2.3.12. Events A1 and A2 are such that A1 ∪ A2 = S and A1 ∩ A2 = ∅. Find p2 if P(A1) = p1, P(A2) = p2, and 3p1 − p2 = 12 .

2.3.13. Consolidated Industries has come under consid- erable pressure to eliminate its seemingly discriminatory hiring practices. Company officials have agreed that dur- ing the next five years, 60% of their new employees will be females and 30% will be minorities. One out of four new employees, though, will be a white male. What percentage of their new hires will be minority females?

2.3.14. Three events—A, B, and C—are defined on a sam- ple space, S. Given that P(A) = 0.2, P(B) = 0.1, and P(C) = 0.3, what is the smallest possible value for P[(A ∪ B ∪ C)C]? 2.3.15. A coin is to be tossed four times. Define events X and Y such that

X : first and last coins have opposite faces Y : exactly two heads appear

Assume that each of the sixteen head/tail sequences has the same probability. Evaluate (a) P(XC ∩ Y) (b) P(X ∩ YC) 2.3.16. Two dice are tossed. Assume that each possible outcome has a 136 probability. Let A be the event that the sum of the faces showing is 6, and let B be the event that the face showing on one die is twice the face showing on the other. Calculate P(A ∩ BC). 2.3.17. Let A, B, and C be three events defined on a sam- ple space, S. Arrange the probabilities of the following events from smallest to largest: (a) A ∪ B (b) A ∩ B (c) A (d) S (e) (A ∩ B) ∪ (A ∩ C) 2.3.18. Lucy is currently running two dot-com scams out of a bogus chatroom. She estimates that the chances of the first one leading to her arrest are one in ten; the “risk” associated with the second is more on the order of one in thirty. She considers the likelihood that she gets busted for both to be 0.0025. What are Lucy’s chances of avoiding incarceration?

2.4 Conditional Probability In Section 2.3, we calculated probabilities of certain events by manipulating other probabilities whose values we were given. Knowing P(A), P(B), and P(A ∩ B), for example, allows us to calculate P(A∪B) (recall Theorem 2.3.6). For many real-world situations, though, the “given” in a probability problem goes beyond simply knowing a set of other probabilities. Sometimes, we know for a fact that certain events have already occurred, and those occurrences may have a bearing on the probability we are trying to find. In short, the probability of an event A may have to be “adjusted” if

32 Chapter 2 Probability

we know for certain that some related event B has already occurred. Any probability that is revised to take into account the (known) occurrence of other events is said to be a conditional probability.

Consider a fair die being tossed, with A defined as the event “6 appears.” Clearly, P(A) = 16 . But suppose that the die has already been tossed—by someone who re- fuses to tell us whether or not A occurred but does enlighten us to the extent of confirming that B occurred, where B is the event “Even number appears.” What are the chances of A now? Here, common sense can help us: There are three equally likely even numbers making up the event B—one of which satisfies the event A, so the “updated” probability is 13 .

Notice that the effect of additional information, such as the knowledge that B has occurred, is to revise—indeed, to shrink—the original sample space S to a new set of outcomes S′. In this example, the original S contained six outcomes, the conditional sample space, three (see Figure 2.4.1).

1 B

P (6, relative to S) = 1/6

3

5

6 4

2

S S' (= B)

P (6, relative to S') = 1/3

6 4

2

Figure 2.4.1

The symbol P(A|B)—read “the probability of A given B”—is used to denote a conditional probability. Specifically, P(A|B) refers to the probability that A will occur given that B has already occurred.

It will be convenient to have a formula for P(A|B) that can be evaluated in terms of the original S, rather than the revised S′. Suppose that S is a finite sample space with n outcomes, all equally likely. Assume that A and B are two events contain- ing a and b outcomes, respectively, and let c denote the number of outcomes in the intersection of A and B (see Figure 2.4.2). Based on the argument suggested in Fig- ure 2.4.1, the conditional probability of A given B is the ratio of c to b. But c/b can be written as the quotient of two other ratios,

S BA

ca b

Figure 2.4.2

c b

= c/n b/n

so, for this particular case,

P(A|B) = P(A ∩ B) P(B)

(2.4.1)

The same underlying reasoning that leads to Equation 2.4.1, though, holds true even when the outcomes are not equally likely or when S is uncountably infinite.

Section 2.4 Conditional Probability 33

Definition 2.4.1 Let A and B be any two events defined on S such that P(B) > 0. The conditional probability of A, assuming that B has already occurred, is written P(A|B) and is given by

P(A|B) = P(A ∩ B) P(B)

Comment Definition 2.4.1 can be cross multiplied to give a frequently useful ex- pression for the probability of an intersection. If P(A|B) = P(A ∩ B)/P(B), then

P(A ∩ B) = P(A|B)P(B) (2.4.2) Example

2.4.1 A card is drawn from a poker deck. What is the probability that the card is a club, given that the card is a king?

Intuitively, the answer is 14 : The king is equally likely to be a heart, diamond, club, or spade. More formally, let C be the event “Card is a club”; let K be the event “Card is a king.” By Definition 2.4.1,

P(C|K) = P(C ∩ K) P(K)

But P(K) = 452 and P(C ∩ K) = P(Card is a king of clubs) = 152 . Therefore, confirm- ing our intuition,

P(C|K) = 1/52 4/52

= 1 4

[Notice in this example that the conditional probability P(C|K) is numerically the same as the unconditional probability P(C)—they both equal 14 . This means that our knowledge that K has occurred gives us no additional insight about the chances of C occurring. Two events having this property are said to be independent. We will examine the notion of independence and its consequences in detail in Section 2.5.]

Example 2.4.2

Our intuitions can often be fooled by probability problems, even ones that appear to be simple and straightforward. The “two boys” problem described here is an often- cited case in point.

Consider the set of families having two children. Assume that the four possible birth sequences—(younger child is a boy, older child is a boy), (younger child is a boy, older child is a girl), and so on—are equally likely. What is the probability that both children are boys given that at least one is a boy?

The answer is not 12 . The correct answer can be deduced from Definition 2.4.1. By assumption, each of the four possible birth sequences—(b, b), (b, g), (g, b), and (g, g)—has a 14 probability of occurring. Let A be the event that both children are boys, and let B be the event that at least one child is a boy. Then

P(A|B) = P(A ∩ B)/P(B) = P(A)/P(B) since A is a subset of B (so the overlap between A and B is just A). But A has one outcome {(b, b)} and B has three outcomes {(b, g), (g, b), (b, b)}. Applying Defini- tion 2.4.1, then, gives

P(A|B) = (1/4)/(3/4) = 1 3

Another correct approach is to go back to the sample space and deduce the value of P(A|B) from first principles. Figure 2.4.3 shows events A and B defined on the four

34 Chapter 2 Probability

family types that comprise the sample space S. Knowing that B has occurred rede- fines the sample space to include three outcomes, each now having a 13 probability. Of those three possible outcomes, one—namely, (b, b)—satisfies the event A. It follows that P(A|B) = 13 .

A

B

(b, b)

(g, b)

(b, g)

(g, g)

S = sample space of two-child families [outcomes written as (first born, second born)]

Figure 2.4.3

Example 2.4.3

Two events A and B are defined such that (1) the probability that A occurs but B does not occur is 0.2, (2) the probability that B occurs but A does not occur is 0.1, and (3) the probability that neither occurs is 0.6. What is P(A|B)?

The three events whose probabilities are given are indicated on the Venn dia- gram shown in Figure 2.4.4. Since

P(Neither occurs) = 0.6 = P((A ∪ B)C) it follows that

P(A ∪ B) = 1 − 0.6 = 0.4 = P(A ∩ BC) + P(A ∩ B) + P(B ∩ AC) so

P(A ∩ B) = 0.4 − 0.2 − 0.1 = 0.1

S

A

B

Neither A nor B

B AC A BC

> >

Figure 2.4.4

From Definition 2.4.1, then,

P(A|B) = P(A ∩ B) P(B)

= P(A ∩ B) P(A ∩ B) + P(B ∩ AC)

= 0.1 0.1 + 0.1

= 0.5

Section 2.4 Conditional Probability 35

Example 2.4.4

The possibility of importing liquified natural gas (LNG) from Algeria has been sug- gested as one way of coping with a future energy crunch. Complicating matters, though, is the fact that LNG is highly volatile and poses an enormous safety haz- ard. Any major spill occurring near a U.S. port could result in a fire of catastrophic proportions. The question, therefore, of the likelihood of a spill becomes critical in- put for future policymakers who may have to decide whether or not to implement the proposal.

Two numbers need to be taken into account: (1) the probability that a tanker will have an accident near a port, and (2) the probability that a major spill will develop given that an accident has happened. Although no significant spills of LNG have yet occurred anywhere in the world, these probabilities can be approximated from records kept on similar tankers transporting less dangerous cargo. On the basis of such data, it has been estimated (47) that the probability is 8/50,000 that an LNG tanker will have an accident on any one trip. Given that an accident has occurred, it is suspected that only three times in fifteen thousand will the damage be sufficiently severe that a major spill would develop. What are the chances that a given LNG shipment would precipitate a catastrophic disaster?

Let A denote the event “Spill develops” and let B denote the event “Accident occurs.” Past experience is suggesting that P(B) = 8/50,000 and P(A|B) = 3/15,000. Of primary concern is the probability that an accident will occur and a spill will ensue—that is, P(A ∩ B). Using Equation 2.4.2, we find that the chances of a catas- trophic accident are on the order of three in one hundred million:

P(Accident occurs and spill develops) = P(A ∩ B) = P(A|B)P(B)

= 3 15,000

· 8 50,000

= 0.000000032

Example 2.4.5

Max and Muffy are two myopic deer hunters who shoot simultaneously at a nearby sheepdog that they have mistaken for a 10-point buck. Based on years of well- documented ineptitude, it can be assumed that Max has a 20% chance of hitting a stationary target at close range, Muffy has a 30% chance, and the probability is 0.06 that they will both be on target. Suppose that the sheepdog is hit and killed by exactly one bullet. What is the probability that Muffy fired the fatal shot?

Let A be the event that Max hit the dog, and let B be the event that Muffy hit the dog. Then P(A) = 0.2, P(B) = 0.3, and P(A ∩ B) = 0.06. We are trying to find

P(B|(AC ∩ B) ∪ (A ∩ BC)) where the event (AC ∩ B) ∪ (A ∩ BC) is the union of A and B minus the intersection—that is, it represents the event that either A or B but not both occur (recall Figure 2.4.4).

Notice, also, from Figure 2.4.4 that the intersection of B and (AC ∩ B) ∪ (A ∩ BC) is the event AC ∩ B. Therefore, from Definition 2.4.1,

P(B|(AC ∩ B) ∪ (A ∩ BC)) = [P(AC ∩ B)]/[P{(AC ∩ B) ∪ (A ∩ BC)}] = [P(B) − P(A ∩ B)]/[P(A ∪ B) − P(A ∩ B)] = [0.3 − 0.06]/[0.2 + 0.3 − 0.06 − 0.06] = 0.63

36 Chapter 2 Probability

CASE STUDY 2.4.1

Several years ago, a television program (inadvertently) spawned a conditional probability problem that led to more than a few heated discussions, even in the national media. The show was Let’s Make a Deal, and the question involved the strategy that contestants should take to maximize their chances of winning prizes.

On the program, a contestant would be presented with three doors, behind one of which was the prize. After the contestant had selected a door, the host, Monty Hall, would open one of the other two doors, showing that the prize was not there. Then he would give the contestant a choice—either stay with the door initially selected or switch to the “third” door, which had not been opened.

For many viewers, common sense seemed to suggest that switching doors would make no difference. By assumption, the prize had a one-third chance of being behind each of the doors when the game began. Once a door was opened, it was argued that each of the remaining doors now had a one-half probability of hiding the prize, so contestants gained nothing by switching their bets.

Not so. An application of Definition 2.4.1 shows that it did make a difference—contestants, in fact, doubled their chances of winning by switching doors. To see why, consider a specific (but typical) case: The contestant has bet on Door #2 and Monty Hall has opened Door #3. Given that sequence of events, we need to calculate and compare the conditional probability of the prize being behind Door #1 and Door #2, respectively. If the former is larger (and we will prove that it is), the contestant should switch doors.

Table 2.4.1 shows the sample space associated with the scenario just de- scribed. If the prize is actually behind Door #1, the host has no choice but to open Door #3; similarly, if the prize is behind Door #3, the host has no choice but to open Door #1. In the event that the prize is behind Door #2, though, the host would (theoretically) open Door #1 half the time and Door #3 half the time.

Table 2.4.1

(Prize Location, Door Opened) Probability

(1, 3) 1/3

(2, 1) 1/6

(2, 3) 1/6

(3, 1) 1/3

Notice that the four outcomes in S are not equally likely. There is neces- sarily a one-third probability that the prize is behind each of the three doors. However, the two choices that the host has when the prize is behind Door #2 necessitate that the two outcomes (2, 1) and (2, 3) share the one-third probabil- ity that represents the chances of the prize being behind Door #2. Each, then, has the one-sixth probability listed in Table 2.4.1.

Section 2.4 Conditional Probability 37

Let A be the event that the prize is behind Door #2, and let B be the event that the host opened Door #3. Then

P(A|B) = P(Contestant wins by not switching) = [P(A ∩ B)]/P(B) = [ 16 ] /[ 13 + 16 ] = 13

Now, let A∗ be the event that the prize is behind Door #1, and let B (as before) be the event that the host opens Door #3. In this case,

P(A∗|B) = P(Contestant wins by switching) = [P(A∗ ∩ B)]/P(B) = [ 13 ] /[ 13 + 16 ] = 23

Common sense would have led us astray again! If given the choice, contestants should have always switched doors. Doing so upped their chances of winning from one-third to two-thirds.

Questions

2.4.1. Suppose that two fair dice are tossed. What is the probability that the sum equals 10 given that it exceeds 8?

2.4.2. Find P(A ∩ B) if P(A) = 0.2, P(B) = 0.4, and P(A|B) + P(B|A) = 0.75. 2.4.3. If P(A|B) < P(A), show that P(B|A) < P(B). 2.4.4. Let A and B be two events such that P((A ∪ B)C) = 0.6 and P(A ∩ B) = 0.1. Let E be the event that either A or B but not both will occur. Find P(E|A ∪ B). 2.4.5. Suppose that in Example 2.4.2 we ignored the ages of the children and distinguished only three family types: (boy, boy), (girl, boy), and (girl, girl). Would the condi- tional probability of both children being boys given that at least one is a boy be different from the answer found on pp. 33–34? Explain.

2.4.6. Two events, A and B, are defined on a sample space S such that P(A|B) = 0.6, P(At least one of the events oc- curs) = 0.8, and P(Exactly one of the events occurs) = 0.6. Find P(A) and P(B).

2.4.7. An urn contains one red chip and one white chip. One chip is drawn at random. If the chip selected is red, that chip together with two additional red chips are put back into the urn. If a white chip is drawn, the chip is re- turned to the urn. Then a second chip is drawn. What is the probability that both selections are red?

2.4.8. Given that P(A) = a and P(B) = b, show that

P(A|B) ≥ a + b − 1 b

2.4.9. An urn contains one white chip and a second chip that is equally likely to be white or black. A chip is drawn

at random and returned to the urn. Then a second chip is drawn. What is the probability that a white appears on the second draw given that a white appeared on the first draw? (Hint: Let Wi be the event that a white chip is se- lected on the ith draw, i = 1, 2. Then P(W2|W1) = P(W1∩W2)P(W1) . If both chips in the urn are white, P(W1) = 1; otherwise, P(W1) = 12 .) 2.4.10. Suppose events A and B are such that P(A ∩ B) = 0.1 and P((A ∪ B)C) = 0.3. If P(A) = 0.2, what does P[(A ∩ B)|(A ∪ B)C] equal? (Hint: Draw the Venn diagram.)

2.4.11. One hundred voters were asked their opinions of two candidates, A and B, running for mayor. Their re- sponses to three questions are summarized below:

Number Saying “Yes”

Do you like A? 65 Do you like B? 55 Do you like both? 25

(a) What is the probability that someone likes neither? (b) What is the probability that someone likes exactly one? (c) What is the probability that someone likes at least one? (d) What is the probability that someone likes at most one? (e) What is the probability that someone likes exactly one given that he or she likes at least one?

38 Chapter 2 Probability

(f) Of those who like at least one, what proportion like both? (g) Of those who do not like A, what proportion like B?

2.4.12. A fair coin is tossed three times. What is the prob- ability that at least two heads will occur given that at most two heads have occurred?

2.4.13. Two fair dice are rolled. What is the probability that the number on the first die was at least as large as 4 given that the sum of the two dice was 8?

2.4.14. Four cards are dealt from a standard fifty-two-card poker deck. What is the probability that all four are aces given that at least three are aces? (Note: There are 270,725 different sets of four cards that can be dealt. Assume that the probability associated with each of those hands is 1/270,725.)

2.4.15. Given that P(A ∩ BC) = 0.3, P((A ∪ B)C) = 0.2, and P(A ∩ B) = 0.1, find P(A|B). 2.4.16. Given that P(A) + P(B) = 0.9, P(A|B) = 0.5, and P(B|A) = 0.4, find P(A). 2.4.17. Let A and B be two events defined on a sample space S such that P(A ∩ BC) = 0.1, P(AC ∩ B) = 0.3, and P((A ∪ B)C) = 0.2. Find the probability that at least one of the two events occurs given that at most one occurs.

2.4.18. Suppose two dice are rolled. Assume that each possible outcome has probability 1/36. Let A be the event that the sum of the two dice is greater than or equal to 8, and let B be the event that at least one of the dice shows a 5. Find P(A|B).

2.4.19. According to your neighborhood bookie, five horses are scheduled to run in the third race at the local track, and handicappers have assigned them the following probabilities of winning:

Horse Probability of Winning

Scorpion 0.10 Starry Avenger 0.25 Australian Doll 0.15 Dusty Stake 0.30 Outandout 0.20

Suppose that Australian Doll and Dusty Stake are scratched from the race at the last minute. What are the chances that Outandout will prevail over the reduced field?

2.4.20. Andy, Bob, and Charley have all been serving time for grand theft auto. According to prison scuttlebutt, the warden plans to release two of the three next week. They all have identical records, so the two to be released will be chosen at random, meaning that each has a two-thirds probability of being included in the two to be set free. Andy, however, is friends with a guard who will know ahead of time which two will leave. He offers to tell Andy the name of one prisoner other than himself who will be released. Andy, however, declines the offer, believing that if he learns the name of one prisoner scheduled to be re- leased, then his chances of being the other person set free will drop to one-half (since only two prisoners will be left at that point). Is his concern justified?

APPLYING CONDITIONAL PROBABILITY TO HIGHER-ORDER INTERSECTIONS

We have seen that conditional probabilities can be useful in evaluating intersection probabilities—that is, P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A). A similar result holds for higher-order intersections. Consider P(A ∩ B ∩C). By thinking of A ∩ B as a single event—say, D—we can write

P(A ∩ B ∩ C) = P(D ∩ C) = P(C|D)P(D) = P(C|A ∩ B)P(A ∩ B) = P(C|A ∩ B)P(B|A)P(A)

Repeating this same argument for n events, A1, A2, . . . , An, gives a formula for the general case:

P(A1 ∩ A2 ∩ · · · ∩ An) = P(An|A1 ∩ A2 ∩ · · · ∩ An−1) · P(An−1|A1 ∩ A2 ∩ · · · ∩ An−2) · · · P(A2|A1) · P(A1)

(2.4.3)

Section 2.4 Conditional Probability 39

Example 2.4.6

An urn contains five white chips, four black chips, and three red chips. Four chips are drawn sequentially and without replacement. What is the probability of obtaining the sequence (white, red, white, black)?

Figure 2.4.5. shows the evolution of the urn’s composition as the desired sequence is assembled. Define the following four events:

W5 W

4 B

3 R

R4 W

4 B

3 R

W4 W

4 B

2 R

3 W

3 B

2 R

B3 W

4 B

2 R

Figure 2.4.5

A: white chip is drawn on first selection B: red chip is drawn on second selection C: white chip is drawn on third selection D: black chip is drawn on fourth selection

Our objective is to find P(A ∩ B ∩ C ∩ D). From Equation 2.4.3,

P(A ∩ B ∩ C ∩ D) = P(D|A ∩ B ∩ C) · P(C|A ∩ B) · P(B|A) · P(A) Each of the probabilities on the right-hand side of the equation here can be gotten by just looking at the urns pictured in Figure 2.4.5: P(D|A ∩ B∩ C) = 49 , P(C|A ∩ B) = 4

10 , P(B|A) = 311 , and P(A) = 512 . Therefore, the probability of drawing a (white, red, white, black) sequence is 0.02:

P(A ∩ B ∩ C ∩ D) = 4 9

· 4 10

· 3 11

· 5 12

= 240 11,880

= 0.02

CASE STUDY 2.4.2

Since the late 1940s, tens of thousands of eyewitness accounts of strange lights in the skies, unidentified flying objects, and even alleged abductions by little green men have made headlines. None of these incidents, though, has produced any hard evidence, any irrefutable proof that Earth has been visited by a race of extraterrestrials. Still, the haunting question remains—are we alone in the uni- verse? Or are there other civilizations, more advanced than ours, making the occasional flyby?

Until, or unless, a flying saucer plops down on the White House lawn and a strange-looking creature emerges with the proverbial “Take me to your leader” demand, we may never know whether we have any cosmic neighbors. Equa- tion 2.4.3, though, can help us speculate on the probability of our not being alone.

Recent discoveries suggest that planetary systems much like our own may be quite common. If so, there are likely to be many planets whose chemical make- ups, temperatures, pressures, and so on, are suitable for life. Let those planets be the points in our sample space. Relative to them, we can define three events:

A: life arises B: technical civilization arises (one capable of interstellar communication) C: technical civilization is flourishing now

(Continued on next page)

40 Chapter 2 Probability

(Case Study 2.4.2 continued)

In terms of A, B, and C, the probability that a habitable planet is presently sup- porting a technical civilization is the probability of an intersection—specifically, P(A ∩ B ∩ C). Associating a number with P(A ∩ B ∩ C) is highly problematic, but the task is simplified considerably if we work instead with the equivalent conditional formula, P(C|B ∩ A) · P(B|A) · P(A).

Scientists speculate (163) that life of some kind may arise on one-third of all planets having a suitable environment and that life on maybe 1% of all those planets will evolve into a technical civilization. In our notation, P(A) = 13 and P(B|A) = 1100 .

More difficult to estimate is P(C|A ∩ B). On Earth, we have had the ca- pability of interstellar communication (that is, radio astronomy) for only a few decades, so P(C|A ∩ B), empirically, is on the order of 1 × 10−8. But that may be an overly pessimistic estimate of a technical civilization’s ability to endure. It may be true that if a civilization can avoid annihilating itself when it first devel- ops nuclear weapons, its prospects for longevity are fairly good. If that were the case, P(C|A ∩ B) might be as large as 1 × 10−2.

Putting these estimates into the computing formula for P(A ∩ B ∩C) yields a range for the probability of a habitable planet currently supporting a technical civilization. The chances may be as small as 3.3×10−11 or as “large” as 3.3×10−5:

(1 × 10−8) (

1 100

) ( 1 3

) < P(A ∩ B ∩ C) < (1 × 10−2)

( 1

100

) ( 1 3

)

or

0.000000000033 < P(A ∩ B ∩ C) < 0.000033

A better way to put these figures in some kind of perspective is to think in terms of numbers rather than probabilities. Astronomers estimate there are 3 × 1011 habitable planets in our Milky Way galaxy. Multiplying that total by the two limits for P(A ∩ B ∩ C) gives an indication of how many cosmic neigh- bors we are likely to have. Specifically, 3 × 1011 · 0.000000000033 .= 10, while 3 × 1011 · 0.000033 .= 10,000,000. So, on the one hand, we may be a galactic rar- ity. At the same time, the probabilities do not preclude the very real possibility that the Milky Way is abuzz with activity and that our neighbors number in the millions.

About the Data In 2009, NASA launched the spacecraft Kepler, whose sole mission was to find other solar systems in the Milky Way galaxy. And find them it did! By 2015 it had documented over one thousand “exoplanets,” in- cluding more than a dozen belonging to so-called Goldilocks zones, mean- ing the planet’s size and configuration of its orbit would allow for liquid water to remain on its surface, thus making it a likely home for living organ- isms. Of particular interest is the planet Kepler 186f located in the constel- lation Cygnus, some five hundred light years away. Its size, orbit, and nature of its sun make it remarkably similar to Earth—maybe not quite a twin, as- tronomers say, but certainly a cousin. (So, maybe UFOs are real and maybe a few have landed, and maybe that weird guy your sister is dating isn’t really from Cleveland. . .)

Section 2.4 Conditional Probability 41

Questions

2.4.21. An urn contains six white chips, four black chips, and five red chips. Five chips are drawn out, one at a time and without replacement. What is the probability of getting the sequence (black, black, red, white, white)? Suppose that the chips are numbered 1 through 15. What is the probability of getting a specific sequence—say, (2, 6, 4, 9, 13)?

2.4.22. A man has n keys on a key ring, one of which opens the door to his apartment. Having celebrated a bit too much one evening, he returns home only to find himself unable to distinguish one key from another. Resourceful, he works out a fiendishly clever plan: He will choose a key at random and try it. If it fails to open the door, he will dis- card it and choose at random one of the remaining n − 1 keys, and so on. Clearly, the probability that he gains en- trance with the first key he selects is 1/n. Show that the probability the door opens with the third key he tries is also 1/n. (Hint: What has to happen before he even gets to the third key?)

2.4.23. Your favorite college football team has had a good season so far but they need to win at least two of their

last four games to qualify for a New Year’s Day bowl bid. Oddsmakers estimate the team’s probabilities of winning each of their last four games to be 0.60, 0.50, 0.40, and 0.70, respectively.

(a) What are the chances that you will get to watch your team play on Jan. 1? (b) Is the probability that your team wins all four games given that they have won at least three games equal to the probability that they win the fourth game? Explain. (c) Is the probability that your team wins all four games given that they won the first three games equal to the probability that they win the fourth game?

2.4.24. One chip is drawn at random from an urn that con- tains one white chip and one black chip. If the white chip is selected, we simply return it to the urn; if the black chip is drawn, that chip—together with another black—are re- turned to the urn. Then a second chip is drawn, with the same rules for returning it to the urn. Calculate the prob- ability of drawing two whites followed by three blacks.

CALCULATING “UNCONDITIONAL” AND “INVERSE” PROBABILITIES

We conclude this section with two very useful theorems that apply to partitioned sample spaces. By definition, a set of events A1, A2, . . . , An “partition” S if every outcome in the sample space belongs to one and only one of the Ai’s—that is, the Ai’s are mutually exclusive and their union is S (see Figure 2.4.6).

S

A2

A1

An

B

Figure 2.4.6

Let B, as pictured, denote any event defined on S. The first result, Theorem 2.4.1, gives a formula for the “unconditional” probability of B (in terms of the Ai’s). Then Theorem 2.4.2 calculates the set of conditional probabilities, P(Aj|B), j = 1, 2, . . . , n.

Theorem 2.4.1

Let {Ai}ni=1 be a set of events defined over S such that S = ⋃n

i=1 Ai, Ai ∩ Aj = ∅ for i = j, and P(Ai) > 0 for i = 1, 2, . . . , n. For any event B,

P(B) = n∑

i=1 P(B|Ai)P(Ai)

Proof By the conditions imposed on the Ai’s,

B = (B ∩ A1) ∪ (B ∩ A2) ∪ · · · ∪ (B ∩ An)

42 Chapter 2 Probability

and

P(B) = P(B ∩ A1) + P(B ∩ A2) + · · · + P(B ∩ An) But each P(B ∩ Ai) can be written as the product P(B|Ai)P(Ai), and the result follows.

Example 2.4.7

Urn I contains two red chips and four white chips; urn II, three red and one white. A chip is drawn at random from urn I and transferred to urn II. Then a chip is drawn from urn II. What is the probability that the chip drawn from urn II is red?

Let B be the event “Chip drawn from urn II is red”; let A1 and A2 be the events “Chip transferred from urn I is red” and “Chip transferred from urn I is white,” respectively. By inspection (see Figure 2.4.7), we can deduce all the probabilities appearing in the right-hand side of the formula in Theorem 2.4.1:

Red

Transfer one

Urn I Urn II

White Draw one

Figure 2.4.7

P(B|A1) = 45 P(B|A2) = 3 5

P(A1) = 26 P(A2) = 4 6

Putting all this information together, we see that the chances are two out of three that a red chip will be drawn from urn II:

P(B) = P(B|A1)P(A1) + P(B|A2)P(A2)

= 4 5

· 2 6

+ 3 5

· 4 6

= 2 3

Example 2.4.8

A standard poker deck is shuffled, and the card on top is removed. What is the prob- ability that the second card is an ace?

Define the following events:

B: second card is an ace A1: top card was an ace A2: top card was not an ace

Then P(B|A1) = 351 , P(B|A2) = 451 , P(A1) = 452 , and P(A2) = 4852 . Since the Ai’s partition the sample space of two-card selections, Theorem 2.4.1 applies. Substituting into the expression for P(B) shows that 452 is the probability that the second card is an ace:

P(B) = P(B|A1)P(A1) + P(B|A2)P(A2)

= 3 51

· 4 52

+ 4 51

· 48 52

= 4 52

Section 2.4 Conditional Probability 43

Comment Notice that P(B) = P(second card is an ace) is numerically the same as P(A1) = P(first card is an ace). The analysis in Example 2.4.8 illustrates a basic principle in probability that says, in effect, “What you don’t know, doesn’t matter.” Here, removal of the top card is irrelevant to any subsequent probability calculations if the identity of that card remains unknown.

Example 2.4.9

Ashley is hoping to land a summer internship with a public relations firm. If her interview goes well, she has a 70% chance of getting an offer. If the interview is a bust, though, her chances of getting the position drop to 20%. Unfortunately, Ashley tends to babble incoherently when she is under stress, so the likelihood of the interview going well is only 0.10. What is the probability that Ashley gets the internship?

Let B be the event “Ashley is offered internship,” let A1 be the event “Interview goes well,” and let A2 be the event “Interview does not go well.” By assumption,

P(B|A1) = 0.70 P(B|A2) = 0.20 P(A1) = 0.10 P(A2) = 1 − P(A1) = 1 − 0.10 = 0.90

According to Theorem 2.4.1, Ashley has a 25% chance of landing the internship:

P(B) = P(B|A1)P(A1) + P(B|A2)P(A2) = (0.70)(0.10) + (0.20)(0.90) = 0.25

Example 2.4.10

Three chips are placed in an urn. One is red on both sides, a second is blue on both sides, and the third is red on one side and blue on the other. One chip is selected at random and placed on a table. Suppose that the color showing on that chip is red. What is the probability that the color underneath is also red (see Figure 2.4.8)?

R e d

R e d

R e d

B l u e

B l u e

B l u e

Draw one

Red

Figure 2.4.8

At first glance, it may seem that the answer is one-half: We know that the blue/blue chip has not been drawn, and only one of the remaining two—the red/red chip—satisfies the event that the color underneath is red. If this game were played over and over, though, and records were kept of the outcomes, it would be found that the proportion of times that a red top has a red bottom is two-thirds, not the one-half that our intuition might suggest. The correct answer follows from an application of Theorem 2.4.1.

Define the following events:

A: bottom side of chip drawn is red B: top side of chip drawn is red A1: red/red chip is drawn A2: blue/blue chip is drawn A3: red/blue chip is drawn

From the definition of conditional probability,

P(A|B) = P(A ∩ B) P(B)

44 Chapter 2 Probability

But P(A ∩ B) = P(Both sides are red) = P (red/red chip) = 13 , and Theorem 2.4.1 can be used to find the denominator, P(B):

P(B) = P(B|A1)P(A1) + P(B|A2)P(A2) + P(B|A3)P(A3)

= 1 · 1 3

+ 0 · 1 3

+ 1 2

· 1 3

= 1 2

Therefore,

P(A|B) = 1/3 1/2

= 2 3

Comment The question posed in Example 2.4.10 gives rise to a simple but effective con game. The trick is to convince a “mark” that the initial analysis given earlier is correct, meaning that the bottom has a fifty-fifty chance of being the same color as the top. Under that incorrect presumption that the game is “fair,” both participants put up the same amount of money, but the gambler (knowing the correct analysis) always bets that the bottom is the same color as the top. In the long run, then, the con artist will be winning an even-money bet two-thirds of the time!

Questions

2.4.25. A toy manufacturer buys ball bearings from three different suppliers—50% of her total order comes from supplier 1, 30% from supplier 2, and the rest from supplier 3. Past experience has shown that the quality-control standards of the three suppliers are not all the same. Two percent of the ball bearings produced by supplier 1 are defective, while suppliers 2 and 3 produce defective bearings 3% and 4% of the time, respectively. What proportion of the ball bearings in the toy manufacturer’s inventory are defective?

2.4.26. A fair coin is tossed. If a head turns up, a fair die is tossed; if a tail turns up, two fair dice are tossed. What is the probability that the face (or the sum of the faces) showing on the die (or the dice) is equal to 6?

2.4.27. Foreign policy experts estimate that the probability is 0.65 that war will break out next year between two Middle East countries if either side significantly escalates its terrorist activities. Otherwise, the likelihood of war is estimated to be 0.05. Based on what has happened this year, the chances of terrorism reaching a critical level in the next twelve months are thought to be three in ten. What is the probability that the two countries will go to war?

2.4.28. A telephone solicitor is responsible for canvassing three suburbs. In the past, 60% of the completed calls to Belle Meade have resulted in contributions, compared to 55% for Oak Hill and 35% for Antioch. Her list of tele- phone numbers includes one thousand households from Belle Meade, one thousand from Oak Hill, and two thou- sand from Antioch. Suppose that she picks a number at random from the list and places the call. What is the prob- ability that she gets a donation?

2.4.29. If men constitute 47% of the population and tell the truth 78% of the time, while women tell the truth 63% of the time, what is the probability that a person selected at random will answer a question truthfully?

2.4.30. Urn I contains three red chips and one white chip. Urn II contains two red chips and two white chips. One chip is drawn from each urn and transferred to the other urn. Then a chip is drawn from the first urn. What is the probability that the chip ultimately drawn from urn I is red?

2.4.31. Medical records show that 0.01%of the general adult population not belonging to a high-risk group (for example, intravenous drug users) are HIV-positive. Blood tests for the virus are 99.9% accurate when given to someone infected and 99.99% accurate when given to someone not infected. What is the probability that a random adult not in a high- risk group will test positive for the HIV virus?

2.4.32. Recall the “survival” lottery described in Ques- tion 2.2.14. What is the probability of release associated with the prisoner’s optimal strategy?

2.4.33. In an upstate congressional race, the incumbent Republican (R) is running against a field of three Democrats (D1, D2, and D3) seeking the nomination. Political pundits estimate that the probabilities of D1, D2, or D3 winning the primary are 0.35, 0.40, and 0.25, respectively. Furthermore, results from a variety of polls are suggesting that R would have a 40% chance of defeating D1 in the general election, a 35% chance of defeating D2, and a 60% chance of defeating D3. Assuming all these estimates to be accurate, what are the chances that the Republican will retain his seat?

Section 2.4 Conditional Probability 45

2.4.34. An urn contains forty red chips and sixty white chips. Six chips are drawn out and discarded, and a sev- enth chip is drawn. What is the probability that the sev- enth chip is red?

2.4.35. A study has shown that seven out of ten people will say “heads” if asked to call a coin toss. Given that the coin is fair, though, a head occurs, on the average, only five times out of ten. Does it follow that you have the ad- vantage if you let the other person call the toss? Explain.

2.4.36. Based on pretrial speculation, the probability that a jury returns a guilty verdict in a certain high-profile mur- der case is thought to be 15% if the defense can discredit the police department and 80% if they cannot. Veteran court observers believe that the skilled defense attorneys have a 70% chance of convincing the jury that the po- lice either contaminated or planted some of the key evi- dence. What is the probability that the jury returns a guilty verdict?

2.4.37. As an incoming freshman, Marcus believes that he has a 25% chance of earning a GPA in the 3.5 to 4.0 range, a 35% chance of graduating with a 3.0 to 3.5 GPA, and a 40% chance of finishing with a GPA less than 3.0. From what the pre-med advisor has told him, Marcus has an eight in ten chance of getting into medical school if his GPA is above 3.5, a five in ten chance if his GPA is in the 3.0 to 3.5 range, and only a one in ten chance if his GPA

falls below 3.0. Based on those estimates, what is the prob- ability that Marcus gets into medical school?

2.4.38. The governor of a certain state has decided to come out strongly for prison reform and is preparing a new early release program. Its guidelines are simple: pris- oners related to members of the governor’s staff would have a 90% chance of being released early; the probability of early release for inmates not related to the governor’s staff would be 0.01. Suppose that 40% of all inmates are related to someone on the governor’s staff. What is the probability that a prisoner selected at random would be eligible for early release?

2.4.39. Following are the percentages of students of State College enrolled in each of the school’s main divisions. Also listed are the proportions of students in each divi- sion who are women.

Division % % Women

Humanities 40 60 Natural science 10 15 History 30 45 Social science 20 75

100

Suppose the registrar selects one person at random. What is the probability that the student selected will be a male?

BAYES’ THEOREM

The second result in this section that is set against the backdrop of a partitioned sam- ple space has a curious history. The first explicit statement of Theorem 2.4.2, coming in 1812, was due to Laplace, but it was named after the Reverend Thomas Bayes, whose 1763 paper (published posthumously) had already outlined the result. On one level, the theorem is a relatively minor extension of the definition of conditional probability. When viewed from a loftier perspective, though, it takes on some rather profound philosophical implications. The latter, in fact, have precipitated a schism among practicing statisticians: “Bayesians” analyze data one way; “non-Bayesians” often take a fundamentally different approach (see Section 5.8).

Our use of the result here will have nothing to do with its statistical interpreta- tion. We will apply it simply as the Reverend Bayes originally intended, as a formula for evaluating a certain kind of “inverse” probability. If we know P(B|Ai) for all i, the theorem enables us to compute conditional probabilities “in the other direction”— that is, we can deduce P(Aj|B) from the P(B|Ai)’s.

Theorem 2.4.2

(Bayes’) Let {Ai}ni=1 be a set of n events, each with positive probability, that partition S in such a way that ∪ni=1Ai = S and Ai ∩ Aj = ∅ for i = j. For any event B (also defined on S), where P(B) > 0,

P(Aj|B) = P(B|Aj)P(Aj)n∑ i=1

P(B|Ai)P(Ai)

for any 1 ≤ j ≤ n.

46 Chapter 2 Probability

Proof From Definition 2.4.1,

P(Aj|B) = P(Aj ∩ B)P(B) = P(B|Aj)P(Aj)

P(B)

But Theorem 2.4.1 allows the denominator to be written as n∑

i=1 P(B|Ai)P(Ai), and

the result follows.

Problem-Solving Hints

(Working with Partitioned Sample Spaces)

Students sometimes have difficulty setting up problems that involve partitioned sample spaces—in particular, ones whose solution requires an application of ei- ther Theorem 2.4.1 or 2.4.2—because of the nature and amount of information that need to be incorporated into the answers. The “trick” is learning to identify which part of the “given” corresponds to B and which parts correspond to the Ai’s. The following hints may help.

1. As you read the question, pay particular attention to the last one or two sentences. Is the problem asking for an unconditional probability (in which case Theorem 2.4.1 applies) or a conditional probability (in which case Theorem 2.4.2 applies)?

2. If the question is asking for an unconditional probability, let B denote the event whose probability you are trying to find; if the question is asking for a conditional probability, let B denote the event that has already happened.

3. Once event B has been identified, reread the beginning of the question and assign the Ai’s.

Example 2.4.11

A biased coin, twice as likely to come up heads as tails, is tossed once. If it shows heads, a chip is drawn from urn I, which contains three white chips and four red chips; if it shows tails, a chip is drawn from urn II, which contains six white chips and three red chips. Given that a white chip was drawn, what is the probability that the coin came up tails (see Figure 2.4.9)?

Urn I

White is drawn

3 W

4 R

Head s Tails

Urn II

6 W

3 R

Figure 2.4.9

Since P(heads) = 2P(tails), it must be true that P(heads) = 23 and P(tails) = 13 . Define the events

B: white chip is drawn A1: coin came up heads (i.e., chip came from urn I) A2: coin came up tails (i.e., chip came from urn II)

Section 2.4 Conditional Probability 47

Our objective is to find P(A2|B). From Figure 2.4.9,

P(B|A1) = 37 P(B|A2) = 6 9

P(A1) = 23 P(A2) = 1 3

so

P(A2|B) = P(B|A2)P(A2)P(B|A1)P(A1) + P(B|A2)P(A2)

= (6/9)(1/3) (3/7)(2/3) + (6/9)(1/3)

= 7 16

Example 2.4.12

During a power blackout, one hundred persons are arrested on suspicion of looting. Each is given a polygraph test. From past experience it is known that the polygraph is 90% reliable when administered to a guilty suspect and 98% reliable when given to someone who is innocent. Suppose that of the one hundred persons taken into cus- tody, only twelve were actually involved in any wrongdoing. What is the probability that a given suspect is innocent given that the polygraph says he is guilty?

Let B be the event “Polygraph says suspect is guilty,” and let A1 and A2 be the events “Suspect is guilty” and “Suspect is not guilty,” respectively. To say that the polygraph is “90% reliable when administered to a guilty suspect” means that P(B|A1) = 0.90. Similarly, the 98% reliability for innocent suspects implies that P(BC|A2) = 0.98, or, equivalently, P(B|A2) = 0.02.

We also know that P(A1) = 12100 and P(A2) = 88100 . Substituting into Theo- rem 2.4.2, then, shows that the probability a suspect is innocent given that the poly- graph says he is guilty is 0.14:

P(A2|B) = P(B|A2)P(A2)P(B|A1)P(A1) + P(B|A2)P(A2)

= (0.02)(88/100) (0.90)(12/100) + (0.02)(88/100)

= 0.14

Example 2.4.13

As medical technology advances and adults become more health conscious, the demand for diagnostic screening tests inevitably increases. Looking for problems, though, when no symptoms are present can have undesirable consequences that may outweigh the intended benefits.

Suppose, for example, a woman has a medical procedure performed to see whether she has a certain type of cancer. Let B denote the event that the test says she has cancer, and let A1 denote the event that she actually does (and A2, the event that she does not). Furthermore, suppose the prevalence of the disease and the precision of the diagnostic test are such that

P(A1) = 0.0001 [and P(A2) = 0.9999] P(B|A1) = 0.90 = P(Test says woman has cancer when, in fact, she does) P(B|A2) = P

( B|AC1

) = 0.001 = P(False positive) = P(Test says woman has cancer when, in fact, she does not)

48 Chapter 2 Probability

What is the probability that she does have cancer, given that the diagnostic procedure says she does? That is, calculate P(A1|B).

Although the method of solution here is straightforward, the actual numerical answer is not what we would expect. From Theorem 2.4.2,

P(A1|B) = P(B|A1)P(A1) P(B|A1)P(A1) + P

( B|AC1

) P

( AC1

) = (0.9)(0.0001)

(0.9)(0.0001) + (0.001)(0.9999) = 0.08

So, only 8% of those women identified as having cancer actually do! Table 2.4.2 shows the strong dependence of P(A1|B) on P(A1) and P(B|AC1 ).

Table 2.4.2

P(A1) P(B|AC1 ) P(A1|B) 0.0001 0.001 0.08

0.0001 0.47 0.001 0.001 0.47

0.0001 0.90 0.01 0.001 0.90

0.0001 0.99

In light of these probabilities, the practicality of screening programs directed at diseases having a low prevalence is open to question, especially when the diagnostic procedure, itself, poses a nontrivial health risk. (For precisely those two reasons, the use of chest X-rays to screen for tuberculosis is no longer advocated by the medical community.)

Example 2.4.14

According to the manufacturer’s specifications, your home burglar alarm has a 95% chance of going off if someone breaks into your house. During the two years you have lived there, the alarm has gone off on five different nights, each time for no apparent reason. Suppose the alarm goes off tomorrow night. What is the probability that someone is trying to break into your house? (Note: Police statistics show that the chances of any particular house in your neighborhood being burglarized on any given night are two in ten thousand.)

Let B be the event “Alarm goes off tomorrow night,” and let A1 and A2 be the events “House is being burglarized” and “House is not being burglarized,” respec- tively. Then

P(B|A1) = 0.95 P(B|A2) = 5/730 (i.e., five nights in two years)

P(A1) = 2/10, 000 P(A2) = 1 − P(A1) = 9998/10, 000

The probability in question is P(A1|B). Intuitively, it might seem that P(A1|B) should be close to 1 because the alarm’s

“performance” probabilities look good—P(B|A1) is close to 1 (as it should be) and

Section 2.4 Conditional Probability 49

P(B|A2) is close to 0 (as it should be). Nevertheless, P(A1|B) turns out to be surpris- ingly small:

P(A1|B) = P(B|A1)P(A1)P(B|A1)P(A1) + P(B|A2)P(A2)

= (0.95)(2/10,000) (0.95)(2/10,000) + (5/730)(9998/10,000)

= 0.027 That is, if you hear the alarm going off, the probability is only 0.027 that your house is being burglarized.

Computationally, the reason P(A1|B) is so small is that P(A2) is so large. The lat- ter makes the denominator of P(A1|B) large and, in effect, “washes out” the numer- ator. Even if P(B|A1) were substantially increased (by installing a more expensive alarm), P(A1|B) would remain largely unchanged (see Table 2.4.3).

Table 2.4.3

P(B|A1) 0.95 0.97 0.99 0.999

P(A1|B) 0.027 0.028 0.028 0.028

Questions

2.4.40. Urn I contains two white chips and one red chip; urn II has one white chip and two red chips. One chip is drawn at random from urn I and transferred to urn II. Then one chip is drawn from urn II. Suppose that a red chip is selected from urn II. What is the probability that the chip transferred was white?

2.4.41. Urn I contains three red chips and five white chips; urn II contains four reds and four whites; urn III contains five reds and three whites. One urn is chosen at random and one chip is drawn from that urn. Given that the chip drawn was red, what is the probability that III was the urn sampled?

2.4.42. A dashboard warning light is supposed to flash red if a car’s oil pressure is too low. On a certain model, the probability of the light flashing when it should is 0.99; 2% of the time, though, it flashes for no apparent reason. If there is a 10% chance that the oil pressure really is low, what is the probability that a driver needs to be concerned if the warning light goes on?

2.4.43. Building permits were issued last year to three contractors starting up a new subdivision: Tara Con- struction built two houses; Westview, three houses; and Hearthstone, six houses. Tara’s houses have a 60% prob- ability of developing leaky basements; homes built by Westview and Hearthstone have that same problem 50% of the time and 40% of the time, respectively. Yesterday, the Better Business Bureau received a complaint from one of the new homeowners that his basement is leaking. Who is most likely to have been the contractor?

2.4.44. Two sections of a senior probability course are be- ing taught. From what she has heard about the two in- structors listed, Francesca estimates that her chances of passing the course are 0.85 if she gets Professor X and 0.60 if she gets Professor Y . The section into which she is put is determined by the registrar. Suppose that her chances of being assigned to Professor X are four out of ten. Fifteen weeks later we learn that Francesca did, indeed, pass the course. What is the probability she was enrolled in Profes- sor X ’s section?

2.4.45. A liquor store owner is willing to cash personal checks for amounts up to $50, but she has become wary of customers who wear sunglasses. Fifty percent of checks written by persons wearing sunglasses bounce. In contrast, 98% of the checks written by persons not wearing sun- glasses clear the bank. She estimates that 10% of her cus- tomers wear sunglasses. If the bank returns a check and marks it “insufficient funds,” what is the probability it was written by someone wearing sunglasses?

2.4.46. Brett and Margo have each thought about murder- ing their rich Uncle Basil in hopes of claiming their in- heritance a bit early. Hoping to take advantage of Basil’s predilection for immoderate desserts, Brett has put rat poison into the cherries flambé; Margo, unaware of Brett’s activities, has laced the chocolate mousse with cyanide. Given the amounts likely to be eaten, the probability of the rat poison being fatal is 0.60; the cyanide, 0.90. Based on other dinners where Basil was presented with the same dessert options, we can assume that he has a 50% chance

50 Chapter 2 Probability

of asking for the cherries flambé, a 40% chance of order- ing the chocolate mousse, and a 10% chance of skipping dessert altogether. No sooner are the dishes cleared away than Basil drops dead. In the absence of any other evi- dence, who should be considered the prime suspect?

2.4.47. Josh takes a twenty-question multiple-choice exam where each question has five possible answers. Some of the answers he knows, while others he gets right just by making lucky guesses. Suppose that the conditional prob- ability of his knowing the answer to a randomly selected question given that he got it right is 0.92. How many of the twenty questions was he prepared for?

2.4.48. Recently the U.S. Senate Committee on Labor and Public Welfare investigated the feasibility of setting up a national screening program to detect child abuse. A team of consultants estimated the following probabilities: (1) one child in ninety is abused, (2) a screening program can detect an abused child 90% of the time, and (3) a screen- ing program would incorrectly label 3% of all nonabused children as abused. What is the probability that a child is actually abused given that the screening program makes that diagnosis? How does the probability change if the incidence of abuse is one in one thousand? Or one in fifty?

2.4.49. At State University, 30% of the students are ma- joring in humanities, 50% in history and culture, and 20% in science. Moreover, according to figures released by the registrar, the percentages of women majoring in human- ities, history and culture, and science are 75%, 45%, and 30%, respectively. Suppose Justin meets Anna at a frater- nity party. What is the probability that Anna is a history and culture major?

2.4.50. An “eyes-only” diplomatic message is to be trans- mitted as a binary code of 0’s and 1’s. Past experience with the equipment being used suggests that if a 0 is sent, it will be (correctly) received as a 0 90% of the time (and mis- takenly decoded as a 1 10% of the time). If a 1 is sent, it will be received as a 1 95% of the time (and as a 0 5%

of the time). The text being sent is thought to be 70% 1’s and 30% 0’s. Suppose the next signal sent is received as a 1. What is the probability that it was sent as a 0?

2.4.51. When Zach wants to contact his girlfriend and he knows she is not at home, he is twice as likely to send her an e-mail as he is to leave a message on her phone. The probability that she responds to his e-mail within three hours is 80%; her chances of being similarly prompt in an- swering a phone message increase to 90%. Suppose she responded within two hours to the message he left this morning. What is the probability that Zach was commu- nicating with her via e-mail?

2.4.52. A dot-com company ships products from three dif- ferent warehouses (A, B, and C). Based on customer com- plaints, it appears that 3% of the shipments coming from A are somehow faulty, as are 5% of the shipments com- ing from B, and 2% coming from C. Suppose a customer is mailed an order and calls in a complaint the next day. What is the probability the item came from Warehouse C? Assume that Warehouses A, B, and C ship 30%, 20%, and 50% of the dot-com’s sales, respectively.

2.4.53. A desk has three drawers. The first contains two gold coins, the second has two silver coins, and the third has one gold coin and one silver coin. A coin is drawn from a drawer selected at random. Suppose the coin selected was silver. What is the probability that the other coin in that drawer is gold?

2.4.54. An automobile insurance company has compiled the information summarized below on its policy-holders. Suppose someone calls to file a claim. To which age group does he or she most likely belong?

% of % Involved in Age Group Policyholders Accidents Last Year

Young (< 30) 20 35 Middle-aged (30–64) 50 15 Elderly (65+) 30 25

2.5 Independence Section 2.4 dealt with the problem of reevaluating the probability of a given event in light of the additional information that some other event has already occurred. It often is the case, though, that the probability of the given event remains unchanged, regardless of the outcome of the second event—that is, P(A|B) = P(A) = P(A|BC). Events sharing this property are said to be independent. Definition 2.5.1 gives a nec- essary and sufficient condition for two events to be independent.

Definition 2.5.1 Two events A and B are said to be independent if P(A ∩ B) = P(A) · P(B).

Section 2.5 Independence 51

Comment The fact that the probability of the intersection of two independent events is equal to the product of their individual probabilities follows immediately from our first definition of independence, that P(A|B) = P(A). Recall that the def- inition of conditional probability holds true for any two events A and B [provided that P(B) > 0]:

P(A|B) = P(A ∩ B) P(B)

But P(A|B) can equal P(A) only if P(A ∩ B) factors into P(A) times P(B).

Example 2.5.1

Let A be the event of drawing a king from a standard poker deck and B, the event of drawing a diamond. Then, by Definition 2.5.1, A and B are independent be- cause the probability of their intersection—drawing a king of diamonds—is equal to P(A) · P(B):

P(A ∩ B) = 1 52

= 1 4

· 1 13

= P(A) · P(B)

Example 2.5.2

Suppose that A and B are independent events. Does it follow that AC and BC are also independent? That is, does P(A ∩ B) = P(A) · P(B) guarantee that P(AC ∩ BC) = P(AC) · P(BC)?

Yes. The proof is accomplished by equating two different expressions for P(AC ∪ BC). First, by Theorem 2.3.6,

P(AC ∪ BC) = P(AC) + P(BC) − P(AC ∩ BC) (2.5.1) But the union of two complements is the complement of their intersection (recall Question 2.2.32). Therefore,

P(AC ∪ BC) = 1 − P(A ∩ B) (2.5.2) Combining Equations 2.5.1 and 2.5.2, we get

1 − P(A ∩ B) = 1 − P(A) + 1 − P(B) − P(AC ∩ BC) Since A and B are independent, P(A ∩ B) = P(A) · P(B), so

P(AC ∩ BC) = 1 − P(A) + 1 − P(B) − [1 − P(A) · P(B)] = [1 − P(A)][1 − P(B)] = P(AC) · P(BC)

the latter factorization implying that AC and BC are, themselves, independent.

Example 2.5.3

Electronics Warehouse is responding to affirmative-action litigation by establishing hiring goals by race and sex for its office staff. So far they have agreed to employ the 120 people characterized in Table 2.5.1. How many black women do they need in order for the events A: Employee is female and B: Employee is black to be independent?

Let x denote the number of black women necessary for A and B to be indepen- dent. Then

P(A ∩ B) = P(Black female) = x/(120 + x) must equal

P(A)P(B) = P(female)P(black) = [(40 + x)/(120 + x)] · [(30 + x)/(120 + x)]

52 Chapter 2 Probability

Setting x/(120 + x) = [(40 + x)/(120 + x)] · [(30 + x)/(120 + x)] implies that x = 24 black women need to be on the staff in order for A and B to be independent.

Table 2.5.1

White Black

Male 50 30 Female 40

Comment Having shown that “Employee is female” and “Employee is black” are independent, does it follow that, say, “Employee is male” and “Employee is white” are independent? Yes. By virtue of the derivation in Example 2.5.2, the independence of events A and B implies the independence of events AC and BC (as well as A and BC

and AC and B). It follows, then, that the x = 24 black women not only makes A and B independent, it also implies, more generally, that “race” and “sex” are independent.

Example 2.5.4

Suppose that two events, A and B, each having nonzero probability, are mutually exclusive. Are they also independent?

No. If A and B are mutually exclusive, then P(A ∩ B) = 0. But P(A) · P(B) > 0 (by assumption), so the equality spelled out in Definition 2.5.1 that characterizes independence is not met.

DEDUCING INDEPENDENCE

Sometimes the physical circumstances surrounding two events make it obvious that the occurrence (or nonoccurrence) of one has absolutely no influence or effect on the occurrence (or nonoccurrence) of the other. If that should be the case, then the two events will necessarily be independent in the sense of Definition 2.5.1.

Suppose a coin is tossed twice. Clearly, whatever happens on the first toss has no physical connection or influence on the outcome of the second. If A and B, then, are events defined on the second and first tosses, respectively, it would have to be the case that P(A|B) = P(A|BC) = P(A). For example, let A be the event that the second toss of a fair coin is a head, and let B be the event that the first toss of that coin is a tail. Then

P(A|B) = P(Head on second toss | tail on first toss)

= P(Head on second toss) = 1 2

Being able to infer that certain events are independent proves to be of enor- mous help in solving certain problems. The reason is that many events of interest are, in fact, intersections. If those events are independent, then the probability of that intersection reduces to a simple product (because of Definition 2.5.1)—that is, P(A ∩ B) = P(A) · P(B). For the coin tosses just described,

P(A ∩ B) = P(head on second toss ∩ tail on first toss) = P(A) · P(B) = P(head on second toss) · P(tail on first toss)

= 1 2

· 1 2

= 1 4

Section 2.5 Independence 53

Example 2.5.5

Myra and Carlos are summer interns working as proofreaders for a local newspaper. Based on aptitude tests, Myra has a 50% chance of spotting a hyphenation error, while Carlos picks up on that same kind of mistake 80% of the time. Suppose the copy they are proofing contains a hyphenation error. What is the probability it goes undetected?

Let A and B be the events that Myra and Carlos, respectively, catch the mis- take. By assumption, P(A) = 0.50 and P(B) = 0.80. What we are looking for is the probability of the complement of a union. That is,

P(Error goes undetected) = 1 − P(Error is detected) = 1 − P(Myra or Carlos or both see the mistake) = 1 − P(A ∪ B) = 1 − [P(A) + P(B) − P(A ∩ B)] (from Theorem 2.3.6)

Since proofreaders invariably work by themselves, events A and B are necessarily independent, so P(A ∩ B) would reduce to the product P(A) · P(B). It follows that such an error would go unnoticed 10% of the time:

P(Error goes undetected) = 1 − {0.50 + 0.80 − (0.50)(0.80)} = 1 − 0.90 = 0.10

Example 2.5.6

Suppose that one of the genes associated with the control of carbohydrate metabolism exhibits two alleles—a dominant W and a recessive w. If the probabil- ities of the WW, Ww, and ww genotypes in the present generation are p, q, and r, respectively, for both males and females, what are the chances that an individual in the next generation will be a ww?

Let A denote the event that an offspring receives a w allele from her father; let B denote the event that she receives the recessive allele from her mother. What we are looking for is P(A ∩ B).

According to the information given,

p = P(Parent has genotype WW) = P(WW) q = P(Parent has genotype Ww) = P(Ww) r = P(Parent has genotype ww) = P(ww)

If an offspring is equally likely to receive either of her parent’s alleles, the probabil- ities of A and B can be computed using Theorem 2.4.1:

P(A) = P(A | WW)P(WW) + P(A | Ww)P(Ww) + P(A | ww)P(ww)

= 0 · p + 1 2

· q + 1 · r

= r + q 2

= P(B) Lacking any evidence to the contrary, there is every reason here to assume that A and B are independent events, in which case

P(A ∩ B) = P(Offspring has genotype ww) = P(A) · P(B)

= (

r + q 2

)2

54 Chapter 2 Probability

(This particular model for allele segregation, together with the independence assumption, is called random Mendelian mating.)

The last two examples focused on events for which an a priori assumption of independence was eminently reasonable. Sometimes, though, what seems eminently reasonable turns out to be surprisingly incorrect. The next example is a case in point.

Example 2.5.7

A crooked gambler has nine dice in her coat pocket. Three are fair and six are not. The biased ones are loaded in such a way that the probability of rolling a 6 is 1/2. She takes out one die at random and rolls it twice. Let A be the event “6 appears on first roll” and let B be the event “6 appears on second roll.” Are A and B independent?

Our intuition here would probably answer “yes”: How can two rolls of a die not be independent? For every dice problem we have encountered so far, they have been. But this is not a typical dice problem. Repeated throws of a die do qualify as independent events if the probabilities associated with the different faces are known. In this situation, though, those probabilities are not known and depend in a random way on which die the gambler draws from her pocket.

To see what effect not knowing which die is being tossed has on the relationship between A and B requires an application of Theorem 2.4.1. Let F and L denote the events “fair die is selected” and “loaded die is selected”, respectively. Then

P (A ∩ B) = P (6 on first roll ∩ 6 on second roll) = P (A ∩ B | F ) P (F ) + P (A ∩ B | L) P (L)

Conditional on either F or L, A and B are independent, so

P(A ∩ B) = (1/6)(1/6)(3/9) + (1/2)(1/2)(6/9) = 19/108 Similarly,

P (A) = P (A | F ) P (F ) + P (A | L) P (L) = (1/6) (3/9) + (1/2) (6/9) = 7/18 = P (B)

But note that

P (A ∩ B) = 19/108 = 57/324 = P (A) · P (B) = (7/18) (7/18) = 49/324 proving that A and B are not independent.

Questions

2.5.1. Suppose that P(A ∩ B) = 0.2, P(A) = 0.6, and P(B) = 0.5. (a) Are A and B mutually exclusive? (b) Are A and B independent? (c) Find P(AC ∪ BC). 2.5.2. Spike is not a terribly bright student. His chances of passing chemistry are 0.35; mathematics, 0.40; and both, 0.12. Are the events “Spike passes chemistry” and “Spike passes mathematics” independent? What is the probabil- ity that he fails both subjects?

2.5.3. Two fair dice are rolled. What is the probability that the number showing on one will be twice the number ap- pearing on the other?

2.5.4. Emma and Josh have just gotten engaged. What is the probability that they have different blood types? Assume that blood types for both men and women are distributed in the general population according to the fol- lowing proportions:

Blood Type Proportion

A 40% B 10%

AB 5% O 45%

2.5.5. Dana and Cathy are playing tennis. The probabil- ity that Dana wins at least one out of two games is 0.3.

Section 2.5 Independence 55

What is the probability that Dana wins at least one out of four?

2.5.6. Three points, X1, X2, and X3, are chosen at random in the interval (0, a). A second set of three points, Y1, Y2, and Y3, are chosen at random in the interval (0, b). Let A be the event that X2 is between X1 and X3. Let B be the event that Y1 < Y2 < Y3. Find P(A ∩ B). 2.5.7. Suppose that P(A) = 14 and P(B) = 18 . (a) What does P(A ∪ B) equal if

1. A and B are mutually exclusive? 2. A and B are independent?

(b) What does P(A | B) equal if 1. A and B are mutually exclusive? 2. A and B are independent?

2.5.8. Suppose that events A, B, and C are independent. (a) Use a Venn diagram to find an expression for P(A ∪ B ∪ C) that does not make use of a complement. (b) Find an expression for P(A ∪ B ∪ C) that does make use of a complement.

2.5.9. A fair coin is tossed four times. What is the proba- bility that the number of heads appearing on the first two tosses is equal to the number of heads appearing on the second two tosses?

2.5.10. Suppose that two cards are drawn simultaneously from a standard fifty-two-card poker deck. Let A be the event that both are either a jack, queen, king, or ace of hearts, and let B be the event that both are aces. Are A and B independent? (Note: There are 1326 equally likely ways to draw two cards from a poker deck.)

DEFINING THE INDEPENDENCE OF MORE THAN TWO EVENTS

It is not immediately obvious how to extend Definition 2.5.1 to, say, three events. To call A, B, and C independent, should we require that the probability of the three-way intersection factors into the product of the three original probabilities,

P(A ∩ B ∩ C) = P(A) · P(B) · P(C) (2.5.3) or should we impose the definition we already have on the three pairs of events:

P(A ∩ B) = P(A) · P(B) P(B ∩ C) = P(B) · P(C) (2.5.4) P(A ∩ C) = P(A) · P(C)

Actually, neither condition by itself is sufficient. If three events satisfy Equa- tions 2.5.3 and 2.5.4, we will call them independent (or mutually independent), but Equation 2.5.3 does not imply Equation 2.5.4, nor does Equation 2.5.4 imply Equa- tion 2.5.3 (see Questions 2.5.11 and 2.5.12).

More generally, the independence of n events requires that the probabilities of all possible intersections equal the products of all the corresponding individual prob- abilities. Definition 2.5.2 states the result formally. Analogous to what was true in the case of two events, the practical applications of Definition 2.5.2 arise when n events are mutually independent, and we can calculate P(A1 ∩ A2 ∩ · · · ∩ An) by computing the product P(A1) · P(A2) · · · P(An).

Definition 2.5.2 Events A1, A2, . . ., An are said to be independent if for every set of indices i1, i2, . . ., ik between 1 and n, inclusive,

P(Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P(Ai1 ) · P(Ai2 ) · · · · · P(Aik )

Example 2.5.8

An insurance company plans to assess its future liabilities by sampling the records of its current policyholders. A pilot study has turned up three clients—one living in Alaska, one in Missouri, and one in Vermont—whose estimated chances of surviving to the year 2020 are 0.7, 0.9, and 0.3, respectively. What is the probability that by the

56 Chapter 2 Probability

end of 2019 the company will have had to pay death benefits to exactly one of the three?

Let A1 be the event “Alaska client survives through 2019.” Define A2 and A3 analogously for the Missouri client and Vermont client, respectively. Then the event E: “Exactly one dies” can be written as the union of three intersections:

E = (A1 ∩ A2 ∩ AC3 ) ∪ (A1 ∩ AC2 ∩ A3) ∪ (AC1 ∩ A2 ∩ A3)

Since each of the intersections is mutually exclusive of the other two,

P(E) = P(A1 ∩ A2 ∩ AC3 ) + P(A1 ∩ AC2 ∩ A3) + P(AC1 ∩ A2 ∩ A3)

Furthermore, there is no reason to believe that for all practical purposes the fates of the three are not independent. That being the case, each of the intersection proba- bilities reduces to a product, and we can write

P(E) = P(A1) · P(A2)·P ( AC3

) + P(A1)·P(AC2 )·P(A3) + P(AC1 )·P(A2)·P(A3) = (0.7)(0.9)(0.7) + (0.7)(0.1)(0.3) + (0.3)(0.9)(0.3) = 0.543

Example 2.5.9

Protocol for making financial decisions in a certain corporation follows the “circuit” pictured in Figure 2.5.1. Any budget is first screened by 1. If he approves it, the plan is forwarded to 2, 3, and 5. If either 2 or 3 concurs, it goes to 4. If either 4 or 5 says “yes,” it moves on to 6 for a final reading. Only if 6 is also in agreement does the proposal pass. Suppose that 1, 5, and 6 each has a 50% chance of saying “yes,” whereas 2, 3, and 4 will each concur with a probability of 0.70. If everyone comes to a decision independently, what is the probability that a budget will pass?

2

3

4

6

5

1

Figure 2.5.1

Probabilities of this sort are calculated by reducing the circuit to its component unions and intersections. Moreover, if all decisions are made independently, which is the case here, then every intersection becomes a product.

Section 2.5 Independence 57

Let Ai be the event that person i approves the budget, i = 1, 2, . . . , 6. Looking at Figure 2.5.1, we see that

P(Budget passes) = P(A1 ∩ {[(A2 ∪ A3) ∩ A4] ∪ A5} ∩ A6) = P(A1)P{[(A2 ∪ A3) ∩ A4] ∪ A5}P(A6)

By assumption, P(A1) = 0.5, P(A2) = 0.7, P(A3) = 0.7, P(A4) = 0.7, P(A5) = 0.5, and P(A6) = 0.5, so

P{[(A2 ∪ A3) ∩ A4]} = [P(A2) + P(A3) − P(A2)P(A3)]P(A4) = [0.7 + 0.7 − (0.7)(0.7)](0.7) = 0.637

Therefore,

P(Budget passes) = (0.5){0.637 + 0.5 − (0.637)(0.5)}(0.5)

= 0.205

REPEATED INDEPENDENT EVENTS

We have already seen several examples where the event of interest was actually an intersection of independent simpler events (in which case the probability of the in- tersection reduced to a product). There is a special case of that basic scenario that deserves special mention because it applies to numerous real-world situations. If the events making up the intersection all arise from the same physical circumstances and assumptions (i.e., they represent repetitions of the same experiment), they are referred to as repeated independent trials. The number of such trials may be finite or infinite.

Example 2.5.10

Suppose the string of Christmas tree lights you just bought has twenty-four bulbs wired in series. If each bulb has a 99.9% chance of “working” the first time current is applied, what is the probability that the string itself will not work?

Let Ai be the event that the ith bulb fails, i = 1, 2, . . . , 24. Then P(String fails) = P(At least one bulb fails)

= P(A1 ∪ A2 ∪ · · · ∪ A24) = 1 − P(String works) = 1 − P(All twenty-four bulbs work) = 1 − P(AC1 ∩ AC2 ∩ · · · ∩ AC24)

If we assume that bulb failures are independent events,

P(String fails) = 1 − P(AC1 )P(AC2 ) · · · P(AC24) Moreover, since all the bulbs are presumably manufactured the same way, P(ACi ) is the same for all i, so

P(String fails) = 1 − [P(ACi )]24 = 1 − (0.999)24

= 1 − 0.98 = 0.02

58 Chapter 2 Probability

The chances are one in fifty, in other words, that the string will not work the first time current is applied.

Example 2.5.11

During the 1978 baseball season, Pete Rose of the Cincinnati Reds set a National League record by hitting safely in forty-four consecutive games. Assume that Rose was a .300 hitter and that he came to bat four times each game. If each at-bat is as- sumed to be an independent event, what probability might reasonably be associated with a hitting streak of that length?

For this problem we need to invoke the repeated independent trials model twice—once for the four at-bats making up a game and a second time for the forty- four games making up the streak. Let Ai denote the event “Rose hit safely in ith game,” i = 1, 2, . . . , 44. Then

P(Rose hit safely in forty-four consecutive games) = P(A1 ∩ A2 ∩ · · · ∩ A44) = P(A1) · P(A2) · · · · · P(A44)

(2.5.5)

Since all the P(Ai)’s are equal, we can further simplify Equation 2.5.5 by writing

P(Rose hit safely in forty-four consecutive games) = [P(A1)]44

To calculate P(A1) we should focus on the complement of A1. Specifically,

P(A1) = 1 − P ( AC1

) = 1 − P(Rose did not hit safely in game 1) = 1 − P(Rose made four outs) = 1 − (0.700)4 (Why?) = 0.76

Therefore, the probability of a .300 hitter putting together a forty-four-game streak (during a given set of forty-four games) is 0.0000057:

P(Rose hit safely in forty-four consecutive games) = (0.76)44

= 0.0000057

Comment The analysis described here has the basic “structure” of a repeated in- dependent trials problem, but the assumptions that the latter makes are not entirely satisfied by the data. Each at-bat, for example, is not really a repetition of the same experiment, nor is P(Ai) the same for all i. Rose would obviously have had dif- ferent probabilities of getting a hit against different pitchers. Moreover, although “four” was probably the typical number of official at-bats that he had during a game, there would certainly have been many instances where he had either fewer or more. Clearly, the day-to-day deviations from the assumed model would have sometimes been in Rose’s favor, sometimes not. Over the course of the hitting streak, the net ef- fect of those deviations would not be expected to have much effect on the 0.0000057 probability.

Example 2.5.12

In the game of craps, one of the ways a player can win is by rolling (with two dice) one of the sums 4, 5, 6, 8, 9, or 10, and then rolling that sum again before rolling a sum of 7. For example, the sequence of sums 6, 5, 8, 8, 6 would result in the player winning on his fifth roll. In gambling parlance, “6” is the player’s “point,” and he “made his point.” On the other hand, the sequence of sums 8, 4, 10, 7 would result in the player

Section 2.5 Independence 59

losing on his fourth roll: his point was an 8, but he rolled a sum of 7 before he rolled a second 8. What is the probability that a player wins with a point of 10?

Table 2.5.2 shows some of the ways a player can make a point of 10. Each se- quence, of course, is an intersection of independent events, so its probability becomes

Table 2.5.2

Sequence of Rolls Probability

(10, 10) (3/36)(3/36) (10, no 10 or 7, 10) (3/36)(27/36)(3/36)

(10, no 10 or 7, no 10 or 7, 10) (3/36)(27/36)(27/36)(3/36) ...

...

a product. The event “Player wins with a point of 10” is then the union of all the sequences that could have been listed in the first column. Since all those sequences are mutually exclusive, the probability of winning with a point of 10 reduces to the sum of an infinite number of products:

P(Player wins with a point of 10) = 3 36

· 3 36

+ 3 36

· 27 36

· 3 36

+ 3 36

· 27 36

· 27 36

· 3 36

+ · · ·

= 3 36

· 3 36

∞∑ k=0

( 27 36

)k (2.5.6)

Recall from algebra that if 0 < r < 1, ∞∑

k=0 rk = 1/(1 − r)

Applying the formula for the sum of a geometric series to Equation 2.5.6 shows that the probability of winning at craps with a point of 10 is 136 :

P(Player wins with a point of 10) = 3 36

· 3 36

· 1( 1 − 2736

) = 1

36

Table 2.5.3 shows the probabilities of a person “making” each of the possible six points—4, 5, 6, 8, 9, and 10. According to the rules of craps, a player wins by either

Table 2.5.3

Point P (makes point)

4 1/36 5 16/360 6 25/396 8 25/396 9 16/360

10 1/36

60 Chapter 2 Probability

(1) getting a sum of 7 or 11 on the first roll or (2) getting a 4, 5, 6, 8, 9, or 10 on the first roll and making the point. But P(sum = 7) = 6/36 and P(sum = 11) = 2/36, so

P(Player wins) = 6 36

+ 2 36

+ 1 36

+ 16 360

+ 25 396

+ 25 396

+ 16 360

+ 1 36

= 0.493 As even-money games go, craps is relatively fair—the probability of the shooter win- ning is not much less than 0.500.

Example 2.5.13

A transmitter is sending a binary code (+ and − signals) that must pass through three relay signals before being sent on to the receiver (see Figure 2.5.2). At each relay station, there is a 25% chance that the signal will be reversed—that is

P(+ is sent by relay i|− is received by relay i) = P(− is sent by relay i|+ is received by relay i) = 1/4, i = 1, 2, 3

Suppose + symbols make up 60% of the message being sent. If the signal + is re- ceived, what is the probability a + was sent?

1 2 3 (+ ?) (+)

Receiver

Figure 2.5.2

This is basically a Bayes’ theorem (Theorem 2.4.2) problem, but the three relay stations introduce a complex mechanism for transmission error. Let A be the event “+ is transmitted from tower” and B be the event “+ is received from relay 3.” Then

P(A|B) = P(B|A)P(A) P(B|A)P(A) + P(B|AC)P(AC)

Notice that a + can be received from relay 3 given that a + was initially sent from the tower if either (1) all relay stations function properly or (2) any two of the sta- tions make transmission errors. Table 2.5.4 shows the four mutually exclusive ways (1) and (2) can happen. The probabilities associated with the message transmissions at each relay station are shown in parentheses. Assuming the relay station outputs are independent events, the probability of an entire transmission sequence is simply the product of the probabilities in parentheses in any given row. These overall proba- bilities are listed in the last column; their sum, 36/64, is P(B|A). By a similar analysis, we can show that

P(B|AC) = P(+ is received from relay 3|− is transmitted from tower) = 28/64 Finally, since P(A) = 0.6 and P(AC) = 0.4, the conditional probability we are

looking for is 0.66:

P(A|B) = ( 36

64

) (0.6)( 36

64

) (0.6) + ( 2864) (0.4) = 0.66

Section 2.5 Independence 61

Table 2.5.4

Signal Transmitted by

Tower Relay 1 Relay 2 Relay 3 Probability

+ +(3/4) −(1/4) +(1/4) 3/64 + −(1/4) −(3/4) +(1/4) 3/64 + −(1/4) +(1/4) +(3/4) 3/64 + +(3/4) +(3/4) +(3/4) 27/64

36/64

Example 2.5.14

Andy, Bob, and Charley have gotten into a disagreement over a female acquaintance, Donna, and decide to settle their dispute with a three-cornered pistol duel. Of the three, Andy is the worst shot, hitting his target only 30% of the time. Charley, a little better, is on-target 50% of the time, while Bob never misses (see Figure 2.5.3). The rules they agree to are simple: They are to fire at the targets of their choice in succession, and cyclically, in the order Andy, Bob, Charley, and so on, until only one of them is left standing. On each “turn,” they get only one shot. If a combatant is hit, he no longer participates, either as a target or as a shooter.

Andy

P (Hits target) 0.3

CharleyBob

P (Hits target) 1.0 P (Hits target) 0.5

Figure 2.5.3

As Andy loads his revolver, he mulls over his options (his objective is clear—to max- imize his probability of survival). According to the rule he can shoot either Bob or Charley, but he quickly rules out shooting at the latter because it would be counter- productive to his future well-being: If he shot at Charley and had the misfortune of hitting him, it would be Bob’s turn, and Bob would have no recourse but to shoot at Andy. From Andy’s point of view, this would be a decidedly grim turn of events, since Bob never misses. Clearly, Andy’s only option is to shoot at Bob. This leaves two scenarios: (1) He shoots at Bob and hits him, or (2) he shoots at Bob and misses.

Consider the first possibility. If Andy hits Bob, Charley will proceed to shoot at Andy, Andy will shoot back at Charley, and so on, until one of them hits the other. Let CHi and CMi denote the events “Charley hits Andy with the ith shot” and “Charley misses Andy with the ith shot,” respectively. Define AHi and AMi analogously. Then Andy’s chances of survival (given that he has killed Bob) reduce to a countably infi- nite union of intersections:

P(Andy survives) = P[(CM1 ∩ AH1) ∪ (CM1 ∩ AM1 ∩ CM2 ∩ AH2) ∪ (CM1 ∩ AM1 ∩ CM2 ∩ AM2 ∩ CM3 ∩ AH3) ∪ · · · ]

62 Chapter 2 Probability

Note that each intersection is mutually exclusive of all of the others and that its com- ponent events are independent. Therefore,

P(Andy survives) = P(CM1)P(AH1) + P(CM1)P(AM1)P(CM2)P(AH2) + P(CM1)P(AM1)P(CM2)P(AM2)P(CM3)P(AH3) + · · ·

= (0.5)(0.3) + (0.5)(0.7)(0.5)(0.3) + (0.5)(0.7)(0.5)(0.7)(0.5)(0.3) + · · ·

= (0.5)(0.3) ∞∑

k=0 (0.35)k

= (0.15) (

1 1 − 0.35

) = 3

13

Now consider the second scenario. If Andy shoots at Bob and misses, Bob will undoubtedly shoot and hit Charley, since Charley is the more dangerous adversary. Then it will be Andy’s turn again. Whether he would see another tomorrow would depend on his ability to make that very next shot count. Specifically,

P(Andy survives) = P(Andy hits Bob on second turn) = 3/10 But 310 >

3 13 , so Andy is better off not hitting Bob with his first shot. And because we

have already argued that it would be foolhardy for Andy to shoot at Charley, Andy’s optimal strategy is clear—deliberately miss both Bob and Charley with the first shot.

Example 2.5.15

Scientists estimate that the Earth’s atmosphere contains on the order of 1044 molecules of which roughly 78% are Nitrogen (N2), 21% Oxygen (O2), and 1% Argon (A). Some 2 × 1022 of those molecules make up each and every breath you take. Rooted in those not particularly interesting facts is a bizarre question, one that has an equally bizarre answer.

On March 15, 44 b.c., Julius Caesar was assassinated by a group of Roman sen- ators, a mutiny led by one of his dearest friends, Marcus Brutus. In Shakespeare’s play describing the attack, the dying Caesar calls out his friend’s treachery with the famous lament, “Et tu, Brute.” In that final breath, Caesar presumably exhaled 2 × 1022 molecules.

So, here is the question, framed in the context of a typical balls-in-boxes problem. Suppose those last 2 × 1022 molecules that Caesar exhaled could all be colored red; and suppose the remaining 1044 − 2×1022 molecules in the atmosphere could all be colored blue. Furthermore, suppose the entire set of red and blue molecules remains unchanged over time, but the box (that is, the atmosphere) is stirred and shaken every day for the next 2060 years. By 2016, then, we can assume that Caesar’s last breath (of red molecules) has become randomly scattered throughout Earth’s atmosphere (of blue molecules). Given those assumptions, what is the probability that in your next breath there will be at least one molecule from Caesar’s last breath?

At first blush, everyone’s intuition would dismiss the question as absurd—using a sample as small as a single breath to “recapture” something from another breath that could be anywhere in the atmosphere would seem to be analogous to a blind person searching for an infinitesimally small needle in a Godzilla-sized haystack. For all practical purposes, the answer surely must be 0. Not so.

Imagine taking a random sample of 2 × 1022 molecules (your next breath) one- at-a-time from the 1044 molecules in the atmosphere (see Figure 2.5.4). Let Ai, i = 1, 2, . . . , 2×1022 be the event that the ith molecule taken in your next breath is not a

Section 2.5 Independence 63

Molecules from Caesar’s last breath (2 3 1022)

Molecules not from Caesar’s last breath (1044 – 2 3 1022)

Figure 2.5.4

“Caesar” molecule, and let A denote the event that your last breath ultimately does contain something from Caesar’s last breath. Then

P (A) = 1 − P (A1 ∩ A2 ∩ A3 ∩ · · · ∩ A2 ×1022 ) Clearly,

P (A1) = ( 1044 − 2 × 1022) /1044

P (A2|A1) = ( 1044 − 2 × 1022 − 1) / (1044 − 1)

P (A3|A1 ∩ A2) = ( 1044 − 2 × 1022 − 2) / (1044 − 2), and so on.

Also, from Equation 2.4.3,

P (A) = 1 − P (A1) P (A2|A1) P (A3|A1 ∩ A2) . . . P (A2 ×1022 |A1 ∩ A2 ∩ . . . A2 ×1022−1) Because of the huge disparity, though, between 1044 and 2 × 1022, all the condi-

tional probabilities are essentially equal to P(A1), which implies that

P (A) =̇ 1 − [P (A1)]2×10 22 = 1 − [(1044 − 2 × 1022)/1044]2×1022

= 1 − [(1 − 2/1022)]2×1022 Recall from calculus that if x is a very small number, 1 − x =̇ e−x. Therefore,

P (A) =̇ 1 − (e−2/1022 )2×1022 = 1 − e−4 = 0.98 (2.5.7) Equation 2.5.7 is a shocker to say the least: Given the assumptions that were

made, it shows that the probability your next breath contains something from Caesar’s last breath is almost a certainty. How can something so unbelievable have such a high probability of happening? The answer in a word is persistence. Think of your next breath as a raffle with anything from Caesar’s last breath being the top prize. What makes this “game” different than a weekly LOTTO drawing where you might have purchased a half-dozen tickets is that here your next breath, in effect, has purchased 20,000,000,000,000,000,000,000 tickets. Moreover, there is not just one jackpot, there are 20,000,000,000,000,000,000,000 jackpots.

The great Roman poet and philosopher Ovid was born a year before Caesar was murdered. Among his voluminous writings was an interesting reflection on the nature of probability, one suggesting that he might not have been surprised by the answer given in Equation 2.5.7. “Chance,” he wrote, “is always powerful. Let your hook be always cast; in the pool where you least expect it, there will be a fish.”

64 Chapter 2 Probability

Questions

2.5.11. Suppose that two fair dice (one red and one green) are rolled. Define the events

A: a 1 or a 2 shows on the red die B: a 3, 4, or 5 shows on the green die C: the dice total is 4, 11, or 12

Show that these events satisfy Equation 2.5.3 but not Equation 2.5.4.

2.5.12. A roulette wheel has thirty-six numbers colored red or black according to the pattern indicated below:

Roulette wheel pattern 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

R R R R R B B B B R R R R B B B B B

36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19

Define the events

A: red number appears B: even number appears C: number is less than or equal to 18

Show that these events satisfy Equation 2.5.4 but not Equation 2.5.3.

2.5.13. How many probability equations need to be veri- fied to establish the mutual independence of four events?

2.5.14. In a roll of a pair of fair dice (one red and one green), let A be the event the red die shows a 3, 4, or 5; let B be the event the green die shows a 1 or a 2; and let C be the event the dice total is 7. Show that A, B, and C are independent.

2.5.15. In a roll of a pair of fair dice (one red and one green), let A be the event of an odd number on the red die, let B be the event of an odd number on the green die, and let C be the event that the sum is odd. Show that any pair of these events is independent but that A, B, and C are not mutually independent.

2.5.16. On her way to work, a commuter encounters four traffic signals. Assume that the distance between each of the four is sufficiently great that her probability of getting a green light at any intersection is independent of what happened at any previous intersection. The first two lights are green for forty seconds of each minute; the last two, for thirty seconds of each minute. What is the probability that the commuter has to stop at least three times?

2.5.17. School board officials are debating whether to re- quire all high school seniors to take a proficiency exam before graduating. A student passing all three parts (mathematics, language skills, and general knowledge) would be awarded a diploma; otherwise, he or she would receive only a certificate of attendance. A practice test given to this year’s ninety-five hundred seniors resulted in the following numbers of failures:

Subject Area Number of Students Failing

Mathematics 3325

Language skills 1900

General knowledge 1425

If “Student fails mathematics,” “Student fails language skills,” and “Student fails general knowledge” are inde- pendent events, what proportion of next year’s seniors can be expected to fail to qualify for a diploma? Does inde- pendence seem a reasonable assumption in this situation?

2.5.18. Consider the following four-switch circuit:

A

In

1 A2

A3 A4

Out

If all switches operate independently and P(Switch closes) = p, what is the probability the circuit is com- pleted?

2.5.19. A fast-food chain is running a new promotion. For each purchase, a customer is given a game card that may win $10. The company claims that the probability of a per- son winning at least once in five tries is 0.32. What is the probability that a customer wins $10 on his or her first purchase?

2.5.20. Players A, B, and C toss a fair coin in order. The first to throw a head wins. What are their respective chances of winning?

2.5.21. In a certain developing nation, statistics show that only two out of ten children born in the early 1980s reached the age of twenty-one. If the same mortality rate is operative over the next generation, how many children does a woman need to bear if she wants to have at least a 75% probability that at least one of her offspring survives to adulthood?

2.5.22. According to an advertising study, 15% of televi- sion viewers who have seen a certain automobile commer- cial can correctly identify the actor who does the voice- over. Suppose that ten such people are watching TV and the commercial comes on. What is the probability that at least one of them will be able to name the actor? What is the probability that exactly one will be able to name the actor?

2.5.23. A fair die is rolled and then n fair coins are tossed, where n is the number showing on the die. What is the probability that no heads appear?

2.5.24. Each of m urns contains three red chips and four white chips. A total of r samples with replacement are

Section 2.6 Combinatorics 65

taken from each urn. What is the probability that at least one red chip is drawn from at least one urn?

2.5.25. If two fair dice are tossed, what is the smallest number of throws, n, for which the probability of getting at least one double 6 exceeds 0.5? (Note: This was one of the first problems that de Méré communicated to Pascal in 1654.)

2.5.26. A pair of fair dice are rolled until the first sum of 8 appears. What is the probability that a sum of 7 does not precede that first sum of 8?

2.5.27. An urn contains w white chips, b black chips, and r red chips. The chips are drawn out at random, one at a time, with replacement. What is the probability that a white appears before a red?

2.5.28. A Coast Guard dispatcher receives an SOS from a ship that has run aground off the shore of a small island. Before the captain can relay her exact position, though, her radio goes dead. The dispatcher has n helicopter crews he can send out to conduct a search. He suspects the ship is somewhere either south in area I (with probability p) or north in area II (with probability 1 − p). Each of the n rescue parties is equally competent and has probabil- ity r of locating the ship given it has run aground in the sector being searched. How should the dispatcher deploy the helicopter crews to maximize the probability that one of them will find the missing ship? (Hint: Assume that m search crews are sent to area I and n − m are sent to area II. Let B denote the event that the ship is found, let A1 be

the event that the ship is in area I, and let A2 be the event that the ship is in area II. Use Theorem 2.4.1 to get an expression for P(B); then differentiate with respect to m.)

2.5.29. A computer is instructed to generate a random sequence using the digits 0 through 9; repetitions are per- missible. What is the shortest length the sequence can be and still have at least a 70% probability of containing at least one 4?

2.5.30. A box contains a two-headed coin and eight fair coins. One coin is drawn at random and tossed n times. Suppose all n tosses come up heads. Show that the limit of the probability that the coin is fair is 0 as n goes to infinity.

2.5.31. Stanley’s statistics seminar is graded on a Pass/Fail basis. At the end of the semester each student is given the option of taking either a two-question exam (Final A) or a three-question exam (Final B). To pass the course, students must answer at least one question correctly on whichever exam they choose. The professor estimates that a typical student has a 45% chance of correctly answering each of the two questions on Final A and a 30% chance of correctly answering each of the three questions on Final B. Which exam should Stanley choose? Answer the question two different ways.

2.5.32. What is the smallest number of switches wired in parallel that will give a probability of at least 0.98 that a circuit will be completed? Assume that each switch op- erates independently and will function properly 60% of the time.

2.6 Combinatorics Combinatorics is a time-honored branch of mathematics concerned with counting, arranging, and ordering. While blessed with a wealth of early contributors (there are references to combinatorial problems in the Old Testament), its emergence as a separate discipline is often credited to the German mathematician and philoso- pher Gottfried Wilhelm Leibniz (1646–1716), whose 1666 treatise, Dissertatio de arte combinatoria, was perhaps the first monograph written on the subject (114).

Applications of combinatorics are rich in both diversity and number. Users range from the molecular biologist trying to determine how many ways genes can be po- sitioned along a chromosome, to a computer scientist studying queuing priorities, to a psychologist modeling the way we learn, to a weekend poker player wondering whether he should draw to a straight, or a flush, or a full house. Surprisingly enough, despite the considerable differences that seem to distinguish one question from an- other, solutions to all of these questions are rooted in the same set of four basic theorems and rules.

COUNTING ORDERED SEQUENCES: THE MULTIPLICATION RULE

More often than not, the relevant “outcomes” in a combinatorial problem are or- dered sequences. If two dice are rolled, for example, the outcome (4, 5)—that is, the first die comes up 4 and the second die comes up 5—is an ordered sequence of length two. The number of such sequences is calculated by using the most fundamental result in combinatorics, the multiplication rule.

66 Chapter 2 Probability

Multiplication Rule If operation A can be performed in m different ways and oper- ation B in n different ways, the sequence (operation A, operation B) can be performed in m · n different ways.

Proof At the risk of belaboring the obvious, we can verify the multiplication rule by considering a tree diagram (see Figure 2.6.1). Since each version of A can be followed by any of n versions of B, and there are m of the former, the total number of “A, B” sequences that can be pieced together is obviously the product m · n.

1

1

Operation BOperation A

2

1 2

n

2 n

1

m 2

n

Figure 2.6.1

Corollary 2.6.1

If operation Ai, i = 1, 2, . . . , k, can be performed in ni ways, i = 1, 2, . . . , k, respec- tively, then the ordered sequence (operation A1, operation A2, . . ., operation Ak) can be performed in n1 · n2 · · · · · nk ways.

Example 2.6.1

The combination lock on a briefcase has two dials, each marked off with sixteen notches (see Figure 2.6.2). To open the case, a person first turns the left dial in a certain direction for two revolutions and then stops on a particular mark. The right dial is set in a similar fashion, after having been turned in a certain direction for two revolutions. How many different settings are possible?

A

C

D B

A

C

D B

Figure 2.6.2

In the terminology of the multiplication rule, opening the briefcase corresponds to the four-step sequence (A1, A2, A3, A4) detailed in Table 2.6.1. Applying the pre- vious corollary, we see that one thousand twenty-four different settings are possible:

number of different settings = n1 · n2 · n3 · n4 = 2 · 16 · 2 · 16 = 1024

Section 2.6 Combinatorics 67

Table 2.6.1

Operation Purpose Number of Options

A1 Rotating the left dial in a particular direction 2

A2 Choosing an endpoint for the left dial 16

A3 Rotating the right dial in a particular direction 2

A4 Choosing an endpoint for the right dial 16

Comment The number of dials, as opposed to the number of notches on each dial, is the critical factor in determining how many different settings are possible. A two-dial lock, for example, where each dial has twenty notches, gives rise to only 2 ·20 ·2 ·20 = 1600 settings. If those forty notches, though, are distributed among four dials (ten to each dial), the number of different settings increases a hundredfold to 160,000 (= 2 · 10 · 2 · 10 · 2 · 10 · 2 · 10).

Example 2.6.2

Alphonse Bertillon, a nineteenth-century French criminologist, developed an identi- fication system based on eleven anatomical variables (height, head width, ear length, etc.) that presumably remain essentially unchanged during an individual’s adult life. The range of each variable was divided into three subintervals: small, medium, and large. A person’s Bertillon configuration is an ordered sequence of eleven letters, say,

s, s, m, m, l, s, l, s, s, m, s

where a letter indicates the individual’s “size” relative to a particular variable. How populated does a city have to be before it can be guaranteed that at least two citizens will have the same Bertillon configuration?

Viewed as an ordered sequence, a Bertillon configuration is an eleven-step classi- fication system, where three options are available at each step. By the multiplication rule, a total of 311, or 177,147, distinct sequences are possible. Therefore, any city with at least 177,148 adults would necessarily have at least two residents with the same pattern. (The limited number of possibilities generated by the configuration’s vari- ables proved to be one of its major weaknesses. Still, it was widely used in Europe for criminal identification before the development of fingerprinting.)

Example 2.6.3

In 1824 Louis Braille invented what would eventually become the standard alphabet for the blind. Based on an earlier form of “night writing” used by the French army for reading battlefield communiqués in the dark, Braille’s system replaced each written character with a six-dot matrix:

• • • • • •

where certain dots were raised, the choice depending on the character being tran- scribed. The letter e, for example, has two raised dots and is written

• • • • • •

68 Chapter 2 Probability

Punctuation marks, common words, suffixes, and so on, also have specified dot pat- terns. In all, how many different characters can be enciphered in Braille?

Think of the dots as six distinct operations, numbered 1 to 6 (see Figure 2.6.3). In forming a Braille letter, we have two options for each dot: We can raise it or not raise it. The letter e, for example, corresponds to the six-step sequence (raise, do not raise, do not raise, do not raise, raise, do not raise). The number of such sequences, with k = 6 and n1 = n2 = · · · = n6 = 2, is 26, or 64. One of those sixty-four configurations, though, has no raised dots, making it of no use to a blind person. Figure 2.6.4 shows the entire sixty-three-character Braille alphabet.

1 • 4 •

2 • 5 •

3 • 6 •

2 Sequences (2) 1

(2) 2

(2) 3

(2) 4

(2) 5

(2) 6

Options

Dot number

6

Figure 2.6.3

a 1

b 2

c 3

d 4

e 5

f 6

g 7

h 8

i 9

j 0

k l m n o p q r s t

u v x y z and for of the with

ch gh sh th wh ed er ou ow w

, ; : . en ! () "/? in ..

st ing # ar ' -

General accent

sign Used for

two-celled contractions

Italic sign;

decimal point

Letter sign

Capital sign

Figure 2.6.4

Section 2.6 Combinatorics 69

Example 2.6.4

The annual NCAA (“March Madness”) basketball tournament starts with a field of sixty-four teams. After six rounds of play, the squad that remains unbeaten is declared the national champion. How many different configurations of winners and losers are possible, starting with the first round? Assume that the initial pairing of the sixty-four invited teams into thirty-two first-round matches has already been done.

Counting the number of ways a tournament of this sort can play out is an exer- cise in applying the multiplication rule twice. Notice, first, that the thirty-two first- round games can be decided in 232 ways. Similarly, the resulting sixteen second-round games can generate 216 different winners, and so on. Overall, the tournament can be pictured as a six-step sequence, where the number of possible outcomes at the six steps are 232, 216, 28, 24, 22, and 21, respectively. It follows that the number of possi- ble tournaments (not all of which, of course, would be equally likely!) is the product 232 · 216 · 28 · 24 · 22 · 21, or 263.

Example 2.6.5

When they were first introduced, postal zip codes were five-digit numbers, theoreti- cally ranging from 00000 to 99999. (In reality, the lowest zip code was 00601 for San Juan, Puerto Rico; the highest was 99950 for Ketchikan, Alaska.) An additional four digits have since been added, so each zip code is now a nine-digit number: an initial five digits, followed by a hyphen, followed by a final four digits.

Let N(A) denote the number of zip codes in the set A, which contains all possible zip codes having at least one 7 among the first five digits; let N(B) denote the number of zip codes in the set B, which contains all possible zip codes having at least one 7 among the final four digits; and let N(T) denote the number of all possible nine-digit zip codes.

Find N(T), N(A), N(B), N(A ∩ B), and N(A ∪ B). Assume that any digit from 0 to 9 can appear any number of times in a zip code.

Since each of the nine positions in a zip code can be occupied by any of ten digits, N(T) = 109. Figure 2.6.5 shows examples of zip codes belonging to A, B, A ∩ B, and A ∪ B.

3 7 2 1 7 − 4 4 1 6 ∈ A 1 6 7 9 4 − 0 7 2 1 ∈ B 7 0 6 2 1 − 7 7 3 7 ∈ A ∩ B 2 9 7 5 5 − 6 6 7 4 ∈ A ∪ B

Figure 2.6.5

Notice that the number of zip codes in the set A is necessarily equal to the total number of zip codes minus all the zip codes that have no 7’s in the first five positions. That is,

N(A) = N(T) − 95 · 104 = 109 − 95 · 104

Likewise,

N(B) = N(T) − 105 · 94 = 109 − 105 · 94

By definition, A ∩ B is the set of outcomes having at least one 7 in the first five positions and at least one 7 in the last four positions (see Figure 2.6.6). By the

at leastone 7︷ ︸︸ ︷ 1 2 3 4 5

at leastone 7︷ ︸︸ ︷ 6 7 8 9

No. of ways: 105− 95 104− 94

Figure 2.6.6

70 Chapter 2 Probability

Multiplication rule, then,

N(A ∩ B) = (105 − 95) · (104 − 94) Finally,

N(A ∪ B) = N(A) + N(B) − N(A ∩ B) = 109 − 95 · 104 + 109 − 105 · 94 − (105 − 95)(104 − 94) = 612,579,511

As a partial check, N (A ∪ B) should be equal to the total number of zip codes minus the number of zip codes with no 7’s. But 109 − 99 = 612,579,511.

Problem-Solving Hints

(Doing combinatorial problems)

Combinatorial questions sometimes call for problem-solving techniques that are not routinely used in other areas of mathematics. The three listed below are especially helpful.

1. Draw a diagram that shows the structure of the outcomes that are being counted. Be sure to include (or indicate) all relevant variations. A case in point is Figure 2.6.3. Almost invariably, diagrams such as these will suggest the formula, or combination of formulas, that should be applied.

2. Use enumerations to “test” the appropriateness of a formula. Typically, the answer to a combinatorial problem—that is, the number of ways to do something—will be so large that listing all possible outcomes is not feasible. It often is feasible, though, to construct a simple, but analogous, problem for which the entire set of outcomes can be identified (and counted). If the pro- posed formula does not agree with the simple-case enumeration, we know that our analysis of the original question is incorrect.

3. If the outcomes to be counted fall into structurally different categories, the total number of outcomes will be the sum (not the product) of the number of outcomes in each category.

Questions

2.6.1. A chemical engineer wishes to observe the effects of temperature, pressure, and catalyst concentration on the yield resulting from a certain reaction. If she intends to in- clude two different temperatures, three pressures, and two levels of catalyst, how many different runs must she make in order to observe each temperature-pressure-catalyst combination exactly twice?

2.6.2. A coded message from a CIA operative to his Russian KGB counterpart is to be sent in the form Q4ET, where the first and last entries must be consonants; the second, an integer 1 through 9; and the third, one of the six vowels. How many different ciphers can be transmitted?

2.6.3. How many terms will be included in the expansion of

(a + b + c)(d + e + f )(x + y + u + v + w) Which of the following will be included in that number: aeu, cdx, bef, xvw?

2.6.4. Suppose that the format for license plates in a cer- tain state is two letters followed by four numbers. (a) How many different plates can be made? (b) How many different plates are there if the letters can be repeated but no two numbers can be the same?

Section 2.6 Combinatorics 71

(c) How many different plates can be made if repetitions of numbers and letters are allowed except that no plate can have four zeros?

2.6.5. How many integers between 100 and 999 have dis- tinct digits, and how many of those are odd numbers?

2.6.6. A fast-food restaurant offers customers a choice of eight toppings that can be added to a hamburger. How many different hamburgers can be ordered?

2.6.7. In baseball there are twenty-four different “base- out” configurations (runner on first—two outs, bases loaded—none out, and so on). Suppose that a new game, sleazeball, is played where there are seven bases (exclud- ing home plate) and each team gets five outs an inning. How many base-out configurations would be possible in sleazeball?

2.6.8. Recall the postal zip codes described in Example 2.6.5. (a) If viewed as nine-digit numbers, how many zip codes are greater than 700,000,000? (b) How many zip codes will the digits in the nine posi- tions alternate between even and odd? (c) How many zip codes will have the first five digits be all different odd numbers and the last four digits be two 2’s and two 4’s?

2.6.9. A restaurant offers a choice of four appetizers, four- teen entrees, six desserts, and five beverages. How many different meals are possible if a diner intends to order only three courses? (Consider the beverage to be a “course.”)

2.6.10. An octave contains twelve distinct notes (on a pi- ano, five black keys and seven white keys). How many different eight-note melodies within a single octave can be written if the black keys and white keys need to alternate?

2.6.11. Residents of a condominium have an automatic garage door opener that has a row of eight buttons. Each garage door has been programmed to respond to a par- ticular set of buttons being pushed. If the condominium

houses 250 families, can residents be assured that no two garage doors will open on the same signal? If so, how many additional families can be added before the eight- button code becomes inadequate? (Note: The order in which the buttons are pushed is irrelevant.)

2.6.12. In international Morse code, each letter in the al- phabet is symbolized by a series of dots and dashes: the letter a, for example, is encoded as “· –”. What is the min- imum number of dots and/or dashes needed to represent any letter in the English alphabet?

2.6.13. The decimal number corresponding to a sequence of n binary digits a0, a1, . . . , an−1, where each ai is either 0 or 1, is defined to be

a020 + a121 + · · · + an−12n−1

For example, the sequence 0 1 1 0 is equal to 6 (= 0 · 20 + 1 · 21 + 1 · 22 + 0 · 23). Suppose a fair coin is tossed nine times. Replace the resulting sequence of H’s and T’s with a binary sequence of 1’s and 0’s (1 for H, 0 for T). For how many sequences of tosses will the deci- mal corresponding to the observed set of heads and tails exceed 256?

2.6.14. Given the letters in the word

Z O M B I E S

in how many ways can two of the letters be arranged such that one is a vowel and one is a consonant?

2.6.15. Suppose that two cards are drawn—in order— from a standard 52-card poker deck. In how many ways can the first card be a club and the second card be an ace?

2.6.16. Monica’s vacation plans require that she fly from Nashville to Chicago to Seattle to Anchorage. Accord- ing to her travel agent, there are three available flights from Nashville to Chicago, five from Chicago to Seat- tle, and two from Seattle to Anchorage. Assume that the numbers of options she has for return flights are the same. How many round-trip itineraries can she schedule?

COUNTING PERMUTATIONS (WHEN THE OBJECTS ARE ALL DISTINCT)

Ordered sequences arise in two fundamentally different ways. The first is the scenario addressed by the multiplication rule—a process is comprised of k operations, each allowing ni options, i = 1, 2, . . . , k; choosing one version of each operation leads to n1n2 . . . nk possibilities.

The second occurs when an ordered arrangement of some specified length k is formed from a finite collection of objects. Any such arrangement is referred to as a permutation of length k. For example, given the three objects A, B, and C, there are six different permutations of length two that can be formed if the objects cannot be repeated: AB, AC, BC, BA, CA, and CB.

72 Chapter 2 Probability

Theorem 2.6.1

The number of permutations of length k that can be formed from a set of n distinct elements, repetitions not allowed, is denoted by the symbol nPk, where

nPk = n(n − 1)(n − 2) · · · (n − k + 1) = n!(n − k)!

Proof Any of the n objects may occupy the first position in the arrangement, any of n − 1 the second, and so on—the number of choices available for filling the kth position will be n − k + 1 (see Figure 2.6.7). The theorem follows, then, from the multiplication rule: There will be n(n − 1) · · · (n − k + 1) ordered arrangements.

n 1

n – 1 2

n – (k – 2) k – 1

n – (k – 1) k

Choices:

Position in sequence

Figure 2.6.7

Corollary 2.6.2

The number of ways to permute an entire set of n distinct objects is nPn = n(n − 1)(n − 2) · · · 1 = n!.

Example 2.6.6

How many permutations of length k = 3 can be formed from the set of n = 4 distinct elements, A, B,C, and D?

According to Theorem 2.6.1, the number should be 24:

n! (n − k)! =

4! (4 − 3)! =

4 · 3 · 2 · 1 1

= 24

Confirming that figure, Table 2.6.2 lists the entire set of twenty-four permutations and illustrates the argument used in the proof of the theorem.

Table 2.6.2

B C 1. (ABC) D 2. (ABD)

A C B 3. (ACB) D 4. (ACD)

D B 5. (ADB) C 6. (ADC)

A C 7. (BAC) D 8. (BAD)

B C A 9. (BCA) D 10. (BCD)

D A 11. (BDA) C 12. (BDC)

A B 13. (CAB) D 14. (CAD)

C B A 15. (CBA) D 16. (CBD)

D A 17. (CDA) B 18. (CDB)

A B 19. (DAB) C 20. (DAC)

D B A 21. (DBA) C 22. (DBC)

C A 23. (DCA) B 24. (DCB)

Section 2.6 Combinatorics 73

Example 2.6.7

In her sonnet with the famous first line, “How do I love thee? Let me count the ways,” Elizabeth Barrett Browning listed eight. Suppose Ms. Browning had decided that writing greeting cards afforded her a better format for expressing her feelings. For how many years could she have corresponded with her favorite beau on a daily basis and never sent the same card twice? Assume that each card contains exactly four of the eight “ways” and that order matters.

In selecting the verse for a card, Ms. Browning would be creating a permutation of length k = 4 from a set of n = 8 distinct objects. According to Theorem 2.6.1,

number of different cards = 8P4 = 8!(8 − 4)! = 8 · 7 · 6 · 5

= 1680 At the rate of a card a day, she could have kept the correspondence going for more than four and one-half years.

Example 2.6.8

Years ago—long before Rubik’s Cubes and electronic games had become epidemic—puzzles were much simpler. One of the more popular combinatorial- related diversions was a four-by-four grid consisting of fifteen movable squares and one empty space. The object was to maneuver as quickly as possible an arbitrary con- figuration (Figure 2.6.8a) into a specific pattern (Figure 2.6.8b). How many different ways could the puzzle be arranged?

Take the empty space to be square number 16 and imagine the four rows of the grid laid end to end to make a sixteen-digit sequence. Each permutation of that sequence corresponds to a different pattern for the grid. By the corollary to Theo- rem 2.6.1, the number of ways to position the tiles is 16!, or more than twenty trillion (20,922,789,888,000, to be exact). (Note: Not all of the 16! permutations can be gener- ated without physically removing some of the tiles. Think of the two-by-two version of Figure 2.6.8 with tiles numbered 1 through 3. How many of the 4! theoretical con- figurations can actually be formed?)

13 1 8 7

6 9 3 11

2 10 4

5 12 15 14

(a)

1 2 3 4

5 6 7 8

9 10 12

13 14 15

11

(b)

Figure 2.6.8

Example 2.6.9

A deck of fifty-two cards is shuffled and dealt face up in a row. For how many arrangements will the four aces be adjacent?

This is a good example illustrating the problem-solving benefits that come from drawing diagrams, as mentioned earlier. Figure 2.6.9 shows the basic structure that needs to be considered: The four aces are positioned as a “clump” somewhere between or around the forty-eight non-aces.

Clearly, there are forty-nine “spaces” that could be occupied by the four aces (in front of the first non-ace, between the first and second non-aces, and so on). Further- more, by the corollary to Theorem 2.6.1, once the four aces are assigned to one of those forty-nine positions, they can still be permuted in 4P4 = 4! ways. Similarly, the

74 Chapter 2 Probability

Non-aces

1 2 3 4 48

4 aces

Figure 2.6.9

forty-eight non-aces can be arranged in 48P48 = 48! ways. It follows from the multi- plication rule, then, that the number of arrangements having consecutive aces is the product 49 · 4! · 48!, or, approximately, 1.46 × 1064.

Comment Computing n! can be quite cumbersome, even for n’s that are fairly small: We saw in Example 2.6.8, for instance, that 16! is already in the trillions. For- tunately, an easy-to-use approximation is available. According to Stirling’s formula,

n! .= √

2πnn+1/2e−n

In practice, we apply Stirling’s formula by writing

log10(n!) .= log10

(√ 2π

) + (n + 1 2

) log10(n) − n log10(e)

and then exponentiating the right-hand side. In Example 2.6.9, the number of arrangements was calculated to be 49 · 4! · 48!,

or 24 · 49!. Substituting into Stirling’s formula, we can write

log10(49!) .= log10

(√ 2π

) + (49 + 1 2

) log10(49) − 49 log10(e)

≈ 62.783366 Therefore,

24 · 49! .= 24 · 1062.78337

= 1.46 × 1064

Example 2.6.10

In chess a rook can move vertically and horizontally (see Figure 2.6.10). It can cap- ture any unobstructed piece located anywhere in its own row or column. In how many ways can eight distinct rooks be placed on a chessboard (having eight rows and eight columns) so that no two can capture one another?

Figure 2.6.10

Section 2.6 Combinatorics 75

To start with a simpler problem, suppose that the eight rooks are all identical. Since no two rooks can be in the same row or same column (why?), it follows that each row must contain exactly one. The rook in the first row, however, can be in any of eight columns; the rook in the second row is then limited to being in one of seven columns, and so on. By the multiplication rule, then, the number of noncapturing configurations for eight identical rooks is 8P8, or 8! (see Figure 2.6.11).

Choices

8

7

6

5

4

3

2

1

Total number = 8 7 6 5 4 3 2 1

Figure 2.6.11 Now imagine the eight rooks to be distinct—they might be numbered, for ex-

ample, 1 through 8. The rook in the first row could be marked with any of eight numbers; the rook in the second row with any of the remaining seven numbers; and so on. Altogether, there would be 8! numbering patterns for each configuration. The total number of ways to position eight distinct, noncapturing rooks, then, is 8! · 8!, or 1,625,702,400.

Example 2.6.11

A group of n families, each having m members, are to be lined up in a row for a pho- tograph at a family reunion. In how many ways can those nm individuals be arranged if members of a family must stay together?

Figure 2.6.12 shows one such arrangement. Notice, first of all, that the families— as n distinct groups—can be positioned in n! ways. Also, the individual members

Family 1

1 2 m

Family 2

1 2 m

Family n

1 2 m

Figure 2.6.12

of each family can be rearranged in m! ways. By the multiplication rule, then, the total number of different pictures the photographer could take would be n!(m!)n. Is the latter a big number? Yes, indeed. If only n = 5 families attended, each hav- ing only m = 4 members, the photographer would have almost a billion different arrangements to choose from:

5! (4!)5 = 955,514,880 Even more amazing, though, is how the number of different arrangements would

increase if members of different families were allowed to intermingle. Under those rules, each possible photograph would simply be a permutation of nm people— where in this case nm = 5(4) = 20. But the number of permutations of 20 distinct objects is 20!, or 2.4329 × 1018.

Currently, the total world population is approximately eight billion people, or 8.0 × 109. Since

2.4329 × 1018/8.0 × 109 = 3.04 × 108

76 Chapter 2 Probability

it would be cheaper—in terms of film costs—to take 300,000,000 pictures of every person living on the planet than it would be to take the picture of all possible ways for twenty people to be lined up in a row.

Example 2.6.12

Consider the set of nine-digit numbers that can be formed by rearranging without repetition the integers 1 through 9. For how many of those permutations will the 1 and the 2 precede the 3 and the 4? That is, we want to count sequences like 7 2 5 1 3 6 9 4 8 but not like 6 8 1 5 4 2 7 3 9.

At first glance, this seems to be a problem well beyond the scope of Theo- rem 2.6.1. With the help of a symmetry argument, though, its solution is surprisingly simple.

Think of just the digits 1 through 4. By the corollary on p. 72, those four numbers give rise to 4!(= 24) permutations. Of those twenty-four, only four—(1, 2, 3, 4), (2, 1, 3, 4), (1, 2, 4, 3), and (2, 1, 4, 3)—have the property that the 1 and the 2 come before the 3 and the 4. It follows that 424 of the total number of nine-digit permutations should satisfy the condition being imposed on 1, 2, 3, and 4. Therefore,

number of permutations where 1 and 2 precede 3 and 4 = 4 24

· 9!

= 60,480

Questions

2.6.17. The board of a large corporation has six members willing to be nominated for office. How many different “president/vice president/treasurer” slates could be sub- mitted to the stockholders?

2.6.18. How many ways can a set of four tires be put on a car if all the tires are interchangeable? How many ways are possible if two of the four are snow tires?

2.6.19. Use Stirling’s formula to approximate 30!. (Note: The exact answer is 265,252,859,812,191,058,636, 308,480,000,000.)

2.6.20. The nine members of the music faculty baseball team, the Mahler Maulers, are all incompetent, and each can play any position equally poorly. In how many differ- ent ways can the Maulers take the field?

2.6.21. A three-digit number is to be formed from the dig- its 1 through 7, with no digit being used more than once. How many such numbers would be less than 289?

2.6.22. Four men and four women are to be seated in a row of chairs numbered 1 through 8. (a) How many total arrangements are possible? (b) How many arrangements are possible if the men are required to sit in alternate chairs?

2.6.23. An engineer needs to take three technical elec- tives sometime during his final four semesters. The three are to be selected from a list of ten. In how many ways can he schedule those classes, assuming that he never wants to take more than one technical elective in any given term?

2.6.24. How many ways can a twelve-member cheerlead- ing squad (six men and six women) pair up to form six male-female teams? How many ways can six male- female teams be positioned along a sideline? What might the number 6!6!26 represent? What might the number 6!6!26212 represent?

2.6.25. Suppose that a seemingly interminable German opera is recorded on all six sides of a three-record album. In how many ways can the six sides be played so that at least one is out of order?

2.6.26. A new horror movie, Friday the 13th, Part X, will star Jason’s great-grandson (also named Jason) as a psy- chotic trying to dispatch (as gruesomely as possible) eight camp counselors, four men and four women. (a) How many scenarios (i.e., victim orders) can the screenwriters devise, assuming they want Jason to do away with all the men before going after any of the women? (b) How many scripts are possible if the only restriction imposed on Jason is that he save Muffy for last?

2.6.27. Suppose that ten people, including you and a friend, line up for a group picture. How many ways can the photographer rearrange the line if she wants to keep exactly three people between you and your friend?

2.6.28. Use an induction argument to prove Theo- rem 2.6.1. (Note: This was the first mathematical result known to have been proved by induction. It was done in 1321 by Levi ben Gerson.)

2.6.29. In how many ways can a pack of fifty-two cards be dealt to thirteen players, four to each, so that every player has one card of each suit?

Section 2.6 Combinatorics 77

2.6.30. If the definition of n! is to hold for all nonnegative integers n, show that it follows that 0! must equal 1.

2.6.31. The crew of Apollo 17 consisted of a pilot, a copi- lot, and a geologist. Suppose that NASA had actually trained nine aviators and four geologists as candidates for the flight. How many different crews could they have assembled?

2.6.32. Uncle Harry and Aunt Minnie will both be attend- ing your next family reunion. Unfortunately, they hate each other. Unless they are seated with at least two people between them, they are likely to get into a shouting match. The side of the table at which they will be seated has seven

chairs. How many seating arrangements are available for those seven people if a safe distance is to be maintained between your aunt and your uncle?

2.6.33. In how many ways can the digits 1 through 9 be arranged such that (a) all the even digits precede all the odd digits? (b) all the even digits are adjacent to each other? (c) two even digits begin the sequence and two even digits end the sequence? (d) the even digits appear in either ascending or descend- ing order?

COUNTING PERMUTATIONS (WHEN THE OBJECTS ARE NOT ALL DISTINCT)

The corollary to Theorem 2.6.1 gives a formula for the number of ways an entire set of n objects can be permuted if the objects are all distinct. Fewer than n! permutations are possible, though, if some of the objects are identical. For example, there are 3! = 6 ways to permute the three distinct objects A, B, and C:

ABC ACB BAC BCA CAB CBA

If the three objects to permute, though, are A, A, and B—that is, if two of the three are identical—the number of permutations decreases to three:

AAB ABA BAA

As we will see, there are many real-world applications where the n objects to be permuted belong to r different categories, each category containing one or more identical objects.

Theorem 2.6.2

The number of ways to arrange n objects, n1 being of one kind, n2 of a second kind, . . . , and nr of an rth kind, is

n! n1! n2! · · · nr!

where r∑

i=1 ni = n.

Proof Let N denote the total number of such arrangements. For any one of those N, the similar objects (if they were actually different) could be arranged in n1! n2! · · · nr! ways. (Why?) It follows that N · n1! n2! · · · nr! is the total number of ways to arrange n (distinct) objects. But n! equals that same number. Setting N · n1! n2! · · · nr! equal to n! gives the result.

78 Chapter 2 Probability

Comment Ratios like n!/(n1! n2! · · · nr!) are called multinomial coefficients because the general term in the expansion of

(x1 + x2 + · · · + xr)n is

n! n1!n2! · · · nr!x

n1 1 x

n2 2 · · · xnrr

Example 2.6.13

A pastry in a vending machine costs 85¢. In how many ways can a customer put in two quarters, three dimes, and one nickel?

Order in which coins are deposited

1 2 3 5 64

Figure 2.6.13

If all coins of a given value are considered identical, then a typical deposit se- quence, say, QDDQND (see Figure 2.6.13), can be thought of as a permutation of n = 6 objects belonging to r = 3 categories, where

n1 = number of nickels = 1 n2 = number of dimes = 3 n3 = number of quarters = 2

By Theorem 2.6.2, there are sixty such sequences:

n! n1!n2!n3!

= 6! 1!3!2!

= 60

Of course, had we assumed the coins were distinct (having been minted at different places and different times), the number of distinct permutations would have been 6!, or 720.

Example 2.6.14

Prior to the seventeenth century there were no scientific journals, a state of affairs that made it difficult for researchers to document discoveries. If a scientist sent a copy of his work to a colleague, there was always a risk that the colleague might claim it as his own. The obvious alternative—wait to get enough material to publish a book— invariably resulted in lengthy delays. So, as a sort of interim documentation, scien- tists would sometimes send each other anagrams—letter puzzles that, when properly unscrambled, summarized in a sentence or two what had been discovered.

When Christiaan Huygens (1629–1695) looked through his telescope and saw the ring around Saturn, he composed the following anagram (203):

aaaaaaa, ccccc, d, eeeee, g, h, iiiiiii, llll, mm,

nnnnnnnnn, oooo, pp, q, rr, s, ttttt, uuuuu

How many ways can the sixty-two letters in Huygens’s anagram be arranged? Let n1(= 7) denote the number of a’s, n2(= 5) the number of c’s, and so on.

Substituting into the appropriate multinomial coefficient, we find

N = 62! 7!5!1!5!1!1!7!4!2!9!4!2!1!2!1!5!5!

Section 2.6 Combinatorics 79

as the total number of arrangements. To get a feeling for the magnitude of N, we need to apply Stirling’s formula to the numerator. Since

62! .= √

2πe−626262.5

then

log(62!) .= log (√2π) − 62 · log(e) + 62.5 · log(62) .= 85.49731

The antilog of 85.49731 is 3.143 × 1085, so

N .= 3.143 × 10 85

7!5!1!5!1!1!7!4!2!9!4!2!1!2!1!5!5!

is a number on the order of 3.6 × 1060. Huygens was clearly taking no chances! (Note: When appropriately rearranged, the anagram becomes “Annulo cingitur tenui, plano, nusquam cohaerente, ad eclipticam inclinato,” which translates to “Surrounded by a thin ring, flat, suspended nowhere, inclined to the ecliptic.”)

Example 2.6.15

What is the coefficient of x23 in the expansion of (1 + x5 + x9)100? To understand how this question relates to permutations, consider the simpler

problem of expanding (a + b)2: (a + b)2 = (a + b)(a + b)

= a · a + a · b + b · a + b · b = a2 + 2ab + b2

Notice that each term in the first (a + b) is multiplied by each term in the second (a+b). Moreover, the coefficient that appears in front of each term in the expansion corresponds to the number of ways that term can be formed. For example, the 2 in the term 2ab reflects the fact that the product ab can result from two different multiplications:

(a + b)(a + b︸ ︷︷ ︸ ab

) or (a + b) (a︸︷︷︸ ab

+ b)

By analogy, the coefficient of x23 in the expansion of (1 + x5 + x9)100 will be the number of ways that one term from each of the one hundred factors (1+x5 +x9) can be multiplied together to form x23. The only factors that will produce x23, though, are the set of two x9’s, one x5, and ninety-seven 1’s:

x23 = x9 · x9 · x5 · 1 · 1 · · · 1 It follows that the coefficient of x23 is the number of ways to permute two x9’s, one x5, and ninety-seven 1’s. So, from Theorem 2.6.2,

coefficient of x23 = 100! 2!1!97!

= 485,100

Example 2.6.16

A palindrome is a phrase whose letters are in the same order whether they are read backward or forward, such as Napoleon’s lament

Able was I ere I saw Elba.

80 Chapter 2 Probability

or the often-cited

Madam, I’m Adam.

Words themselves can become the units in a palindrome, as in the sentence

Girl, bathing on Bikini, eyeing boy,

finds boy eyeing bikini on bathing girl.

Suppose the members of a set consisting of four objects of one type, six of a sec- ond type, and two of a third type are to be lined up in a row. How many of those permutations are palindromes?

Think of the twelve objects to arrange as being four A’s, six B’s, and two C’s. If the arrangement is to be a palindrome, then half of the A’s, half of the B’s, and half of the C’s must occupy the first six positions in the permutation. Moreover, the final six members of the sequence must be in the reverse order of the first six. For example, if the objects comprising the first half of the permutation were

C A B A B B

then the last six would need to be in the order

B B A B A C

It follows that the number of palindromes is the number of ways to permute the first six objects in the sequence, because once the first six are positioned, there is only one arrangement of the last six that will complete the palindrome. By Theorem 2.6.2, then,

number of palindromes = 6!/(2!3!1!) = 60

Example 2.6.17

A deliveryman is currently at Point X and needs to stop at Point 0 before driving through to Point Y (see Figure 2.6.14). How many different routes can he take with- out ever going out of his way?

X

O

Y

Figure 2.6.14

Notice that any admissible path from, say, X to 0 is an ordered sequence of 11 “moves”—nine east and two north. Pictured in Figure 2.6.14, for example, is the particular X to 0 route

E E N E E E E N E E E

Similarly, any acceptable path from 0 to Y will necessarily consist of five moves east and three moves north (the one indicated is E E N N E N E E).

Since each path from X to 0 corresponds to a unique permutation of nine E’s and two N’s, the number of such paths (from Theorem 2.6.2) is the quotient

11!/(9!2!) = 55

Section 2.6 Combinatorics 81

For the same reasons, the number of different paths from 0 to Y is

8!/(5!3!) = 56 By the multiplication rule, then, the total number of admissible routes from X to Y that pass through 0 is the product of 55 and 56, or 3080.

Example 2.6.18

Stacy is worried that her recently-released-from-prison stalker-out-for-revenge ex- boyfriend Psycho Bob is trying to hack into her e-mail account. Crazy, but not stupid, Bob has programmed an algorithm capable of checking three billion passwords a second, which he intends to run 24/7. Suppose Stacy intends to change her password every month. What is the probability her e-mail gets broken into at least once in the next six months?

Stacy’s e-mail provider requires 10-digit passwords—four letters (each having two options, lower case or upper case), four numbers, and two symbols (chosen from a list of eight). Figure 2.6.15 shows one such admissible sequence.

7 3 B * Q a # 6 a 1

Figure 2.6.15

Counting the number of admissible passwords is an exercise in using the mul- tiplication rule. Clearly, the identities of the four numbers and two symbols can be chosen in 10 · 10 · 10 · 10 and 8 · 8 ways, respectively, while the letters can be cho- sen in 26 · 26 · 26 · 26 · 24 ways. Moreover, by Theorem 2.6.2 the positions for the four numbers, two symbols, and four letters can be assigned in 10!/(4!4!2!) ways. The total number of different passwords, then, is the product

104 · 82 · (26)4 · 24 · (10!/(4!4!2!)), or 1.474 × 1016

The probability that Bob’s cyberattack identifies whatever password Stacy is using in a given month is simply equal to the number of passwords his algorithm can check in thirty days divided by the total number of passwords admissible. The former is equal to

3,000,000,000 passwords/sec × 60 sec/min × 60 min /hr × 24 hrs/day × 30 days/mo. = 7.776 × 1015 passwords/mo

so the probability that Stacy’s e-mail privacy is compromised in any given month is the quotient

7.776 × 1015/1.474 × 1016, or 0.53 Of course, Bob’s success (or failure) in one month is independent of what hap-

pens in any other month, so

P(e-mail is hacked into at least once in six months) = 1 − P(e-mail is not hacked into for any of the six months) = 1 − (0.47)6 = 0.989

CIRCULAR PERMUTATIONS

Thus far the enumeration results we have seen have dealt with what might be called linear permutations—objects being lined up in a row. This is the typical context in which permutation problems arise, but sometimes nonlinear arrangements of one

82 Chapter 2 Probability

sort or another need to be counted. The next theorem gives the basic result associated with circular permutations.

Theorem 2.6.3

There are (n − 1)! ways to arrange n distinct objects in a circle. Proof Fix any object at the “top” of the circle. The remaining n – 1 objects can then be permuted in (n – 1)! ways. Since any arrangement with a different object at the top can be reproduced by simply rotating one of the original (n − 1)! per- mutations, the statement of the theorem holds.

Example 2.6.19

How many different firing orders are theoretically possible in a six-cylinder engine? (If the cylinders are numbered from 1 to 6, a firing order is a list such as 1, 4, 2, 5, 3, 6 giving the sequence in which fuel is ignited in the six cylinders.)

By a direct application of Theorem 2.6.3, the number of distinct firing orders is (6 – 1)!, or 120.

Comment According to legend (84), perhaps the first person for whom the problem of arranging objects in a circle took on life or death significance was Flavius Josephus, an early Jewish scholar and historian. In 66 a.d., Josephus found himself a somewhat reluctant leader of an attempt to overthrow the Roman administration in the town of Judea. But the coup failed, and Josephus and forty of his comrades ended up trapped in a cave, surrounded by an angry Roman army.

Faced with the prospect of imminent capture, others in the group were intent on committing mass suicide, but Josephus’s devotion to the cause did not extend quite that far. Still, he did not want to appear cowardly, so he proposed an alternate plan: All forty one would arrange themselves in a circle; then, one by one, the group would go around the circle and kill every seventh remaining person, starting with whoever was seated at the head of the circle. That way, only one of them would have to commit suicide, and the entire action would make more of an impact on the Romans.

To his relief, the group accepted his suggestion and began forming a circle. Jose- phus, who was reputed to have had some genuine mathematical ability, quickly made his way to the twenty-fifth position. Forty murders later, he was the only person left alive!

Is the story true? Maybe yes, maybe no. Any conclusion would be little more than idle speculation. It is known, though, that Josephus was the sole survivor of the siege, and that he surrendered and eventually rose to a position of considerable in- fluence in the Roman government. And, whether true or not, the legend has given rise to some mathematical terminology: Cyclic cancellations of a fixed set of num- bers having the property that a specified number is left at the end are referred to as Josephus permutations.

Questions

2.6.34. Which state name can generate more permuta- tions, TENNESSEE or FLORIDA?

2.6.35. How many numbers greater than four million can be formed from the digits 2, 3, 4, 4, 5, 5, 5?

2.6.36. An interior decorator is trying to arrange a shelf containing eight books, three with red covers, three with blue covers, and two with brown covers. (a) Assuming the titles and the sizes of the books are irrel- evant, in how many ways can she arrange the eight books?

Section 2.6 Combinatorics 83

(b) In how many ways could the books be arranged if they were all considered distinct? (c) In how many ways could the books be arranged if the red books were considered indistinguishable, but the other five were considered distinct?

2.6.37. Four Nigerians (A, B, C, D), three Chinese (#, ∗, &), and three Greeks (α, β, γ) are lined up at the box of- fice, waiting to buy tickets for the World’s Fair. (a) How many ways can they position themselves if the Nigerians are to hold the first four places in line; the Chinese, the next three; and the Greeks, the last three? (b) How many arrangements are possible if members of the same nationality must stay together? (c) How many different queues can be formed? (d) Suppose a vacationing Martian strolls by and wants to photograph the ten for her scrapbook. A bit myopic, the Martian is quite capable of discerning the more ob- vious differences in human anatomy but is unable to dis- tinguish one Nigerian (N) from another, one Chinese (C) from another, or one Greek (G) from another. Instead of perceiving a line to be B∗βAD#&Cαγ, for example, she would see NCGNNCCNGG. From the Martian’s perspec- tive, in how many different ways can the ten funny-looking Earthlings line themselves up?

2.6.38. How many ways can the letters in the word

S LU M GU L L I O N

be arranged so that the three L’s precede all the other consonants?

2.6.39. A tennis tournament has a field of 2n entrants, all of whom need to be scheduled to play in the first round. How many different pairings are possible?

2.6.40. What is the coefficient of x12 in the expansion of (1 + x3 + x6)18? 2.6.41. In how many ways can the letters of the word

E L E E M O SY N A RY

be arranged so that the S is always immediately followed by a Y?

2.6.42. In how many ways can the word ABRA- CADABRA be formed in the array pictured above? Assume that the word must begin with the top A and progress diagonally downward to the bottom A.

A

B B

R R R

A A A A

C C C C C

A A A A A A

D D D D D

A A A A

B B B

R R

A

2.6.43. Suppose a pitcher faces a batter who never swings. For how many different ball/strike sequences will the bat- ter be called out on the fifth pitch?

2.6.44. What is the coefficient of w2x3yz3 in the expansion of (w + x + y + z)9? 2.6.45. Imagine six points in a plane, no three of which lie on a straight line. In how many ways can the six points be used as vertices to form two triangles? (Hint: Number the points 1 through 6. Call one of the triangles A and the other B. What does the permutation

A A B B A B 1 2 3 4 5 6

represent?)

2.6.46. Show that (k!)! is divisible by k!(k−1)!. (Hint: Think of a related permutation problem whose solution would require Theorem 2.6.2.)

2.6.47. In how many ways can the letters of the word

B R O B D I N G N A G I A N

be arranged without changing the order of the vowels?

2.6.48. Make an anagram out of the familiar expression STATISTICS IS FUN. In how many ways can the letters in the anagram be permuted?

2.6.49. Linda is taking a five-course load her first semester: English, math, French, psychology, and history. In how many different ways can she earn three A’s and two B’s? Enumerate the entire set of possibilities. Use Theorem 2.6.2 to verify your answer.

COUNTING COMBINATIONS

Order is not always a meaningful characteristic of a collection of elements. Consider a poker player being dealt a five-card hand. Whether he receives a 2 of hearts, 4 of clubs, 9 of clubs, jack of hearts, and ace of diamonds in that order, or in any one of the other 5! − 1 permutations of those particular five cards is irrelevant, the hand is still the same. As the last set of examples in this section bears out, there are many such

84 Chapter 2 Probability

situations—problems where our only legitimate concern is with the composition of a set of elements, not with any particular arrangement of them.

We call a collection of k unordered elements a combination of size k. For exam- ple, given a set of n = 4 distinct elements—A, B, C, and D—there are six ways to form combinations of size 2:

A and B B and C A and C B and D A and D C and D

A general formula for counting combinations can be derived quite easily from what we already know about counting permutations.

Theorem 2.6.4

The number of ways to form combinations of size k from a set of n distinct objects, repetitions not allowed, is denoted by the symbols

( n k

) or nCk, where(n

k

) = nCk = n!k!(n − k)!

Proof Let the symbol ( n

k

) denote the number of combinations satisfying the con-

ditions of the theorem. Since each of those combinations can be ordered in k! ways, the product k!

( n k

) must equal the number of permutations of length k that can be

formed from n distinct elements. But n distinct elements can be formed into per- mutations of length k in n(n − 1) · · · (n − k + 1) = n!/(n − k)! ways. Therefore,

k! (n

k

) = n!

(n − k)! Solving for

( n k

) gives the result.

Comment It often helps to think of combinations in the context of drawing ob- jects out of an urn. If an urn contains n chips labeled 1 through n, the number of ways we can reach in and draw out different samples of size k is

( n k

) . In deference

to this sampling interpretation for the formation of combinations, ( n

k

) is usually read

“n things taken k at a time” or “n choose k.”

Comment The symbol ( n

k

) appears in the statement of a familiar theorem from

algebra,

(x + y)n = n∑

k=0

(n k

) xkyn−k

Since the expression being raised to a power involves two terms, x and y, the con- stants

( n k

) , k = 0, 1, . . ., n, are commonly referred to as binomial coefficients.

Example 2.6.20

Eight politicians meet at a fund-raising dinner. How many greetings can be ex- changed if each politician shakes hands with every other politician exactly once?

Imagine the politicians to be eight chips—1 through 8—in an urn. A handshake corresponds to an unordered sample of size 2 chosen from that urn. Since repeti- tions are not allowed (even the most obsequious and overzealous of campaigners

Section 2.6 Combinatorics 85

would not shake hands with himself!), Theorem 2.6.4 applies, and the total number of handshakes is (

8 2

) = 8!

2!6!

or 28.

Example 2.6.21

A chemist is trying to synthesize a part of a straight-chain aliphatic hydrocarbon polymer that consists of twenty-one radicals—ten ethyls (E), six methyls (M), and five propyls (P). Assuming all arrangements of radicals are physically possible, how many different polymers can be formed if no two of the methyl radicals are to be adjacent?

Imagine arranging the E’s and the P’s without the M’s. Figure 2.6.16 shows one such possibility. Consider the sixteen “spaces” between and outside the E’s and P’s as indicated by the arrows in Figure 2.6.16. In order for the M’s to be nonadjacent, they must occupy any six of these locations. But those six spaces can be chosen in(

16 6

) ways. And for each of the

( 16 6

) positionings of the M’s, the E’s and P’s can be

permuted in 15!10!5! ways (Theorem 2.6.2).

E E P P P P PE E E E E E E E

Figure 2.6.16

So, by the multiplication rule, the total number of polymers having nonadjacent methyl radicals is 24,048,024:(

16 6

) · 15!

10!5! = 16!

10!6! 15!

10!5! = (8008)(3003) = 24, 048, 024

Example 2.6.22

Binomial coefficients have many interesting properties. Perhaps the most familiar is Pascal’s triangle,1 a numerical array where each entry is equal to the sum of the two numbers appearing diagonally above it (see Figure 2.6.17). Notice that each entry in Pascal’s triangle can be expressed as a binomial coefficient, and the relationship just described appears to reduce to a simple equation involving those coefficients:(

n + 1 k

) =

(n k

) +

( n

k − 1 )

(2.6.1)

Prove that Equation 2.6.1 holds for all positive integers n and k.

1

1 1

1 2 1

1 3 3 1

1 4 6 4 1

0

1

2

3

4

Row 0 0( )

1 0( )

1 1( )

2 0( )

2 1( )

2 2( )

3 0( )

3 1( )( )

4 0( )

4 1( )

4 2( )

3 2( )

3 3( )( )

4 3( )

4 4( )

Figure 2.6.17 1 Despite its name, Pascal’s triangle was not discovered by Pascal. Its basic structure had been known hundreds of years before the French mathematician was born. It was Pascal, though, who first made extensive use of its properties.

86 Chapter 2 Probability

Consider a set of n + 1 distinct objects A1, A2, . . ., An+1. We can obviously draw samples of size k from that set in

(n+1 k

) different ways. Now, consider any particular

object—for example, A1. Relative to A1, each of those (n+1

k

) samples belongs to one

of two categories: those containing A1 and those not containing A1. To form samples containing A1, we need to select k − 1 additional objects from the remaining n. This can be done in

( n k−1

) ways. Similarly, there are

(n k

) ways to form samples not containing

A1. Therefore, (n+1

k

) must equal

( n k

) + ( nk−1). Example 2.6.23

The answers to combinatorial questions can sometimes be obtained using quite dif- ferent approaches. What invariably distinguishes one solution from another is the way in which outcomes are characterized.

For example, suppose you have just ordered a roast beef sub at a sandwich shop, and now you need to decide which, if any, of the available toppings (lettuce, tomato, onions, etc.) to add. If the shop has eight “extras” to choose from, how many different subs can you order?

One way to answer this question is to think of each sub as an ordered sequence of length eight, where each position in the sequence corresponds to one of the top- pings. At each of those positions, you have two choices—“add” or “do not add” that particular topping. Pictured in Figure 2.6.18 is the sequence corresponding to the sub that has lettuce, tomato, and onion but no other toppings. Since two choices (“add” or “do not add”) are available for each of the eight toppings, the multiplication rule tells us that the number of different roast beef subs that could be requested is 28, or 256.

Add? Y Y Y N N N N N

Lettuce Tomato Onion Mustard Relish Mayo Pickles Peppers

Figure 2.6.18

An ordered sequence of length 8, though, is not the only model capable of char- acterizing a roast beef sandwich. We can also distinguish one roast beef sub from another by the particular combination of toppings that each one has. For example,

there are (

8 4

) = 70 different subs having exactly four toppings. It follows that the

total number of different sandwiches is the total number of different combinations of size k, where k ranges from 0 to 8. Reassuringly, that sum agrees with the ordered sequence answer:

total number of different roast beef subs = (

8 0

) +

( 8 1

) +

( 8 2

) + · · · +

( 8 8

) = 1 + 8 + 28 + · · · + 1 = 256

What we have just illustrated here is another property of binomial coefficients, namely, that

n∑ k=0

(n k

) = 2n (2.6.2)

The proof of Equation 2.6.2 is a direct consequence of Newton’s binomial expansion (see the second comment following Theorem 2.6.4).

Section 2.6 Combinatorics 87

Example 2.6.24

Recall Theorem 2.3.7, the formula for the probability of the union of n events, P(A1 ∪ A2 ∪ · · · ∪ An). As the special-case discussion on p. 28 indicated, it needs to be proved in general that the formula adds—once and only once—the probability of every outcome represented in the Venn diagram for A1 ∪ A2 ∪ · · · ∪ An.

To that end, consider the set of outcomes in A1 ∪ A2 ∪ · · · ∪ An that belong to a specified k of the Ai’s and to no others. If it can be established that the right-hand- side of Theorem 2.3.7 counts that particular set of outcomes one time, the theorem is proved since k was arbitrary.

Consider such a set of outcomes. Relative to the summations making up the right-hand side of Theorem 2.3.7, those outcomes get counted ( k1 ) times in

n∑ i=1

P(Ai), ( k 2 ) times in

∑ i< j

P(Ai ∩ Aj), (k3) times in ∑

i< j<k P(Ai ∩ Aj ∩ Ak), and so on.

According to the theorem, then, they will be counted a total of( k 1

) −

( k 2

) +

( k 3

) − · · · + (−1)k+1

( k k

) times.

Expressing that sum as a binomial expansion, we can write

(−1 + 1)k = 0k = k∑

j=0

( k j

) (−1) j(1)k− j

= (

k 0

) −

( k 1

) +

( k 2

) − · · · + (−1)k

( k k

) or, equivalently, (

k 1

) −

( k 2

) + · · · + (−1)k+1

( k k

) =

( k 0

) = 1

and the theorem is proved.

Example 2.6.25

In Example 2.6.23, the fact that n∑

k=0 (nk) is equal to 2

n was established by showing that

both expressions count the number of ways to complete the same task. Analytically, Equation 2.6.2 could have been derived by simplifying the expansion of (x + y)n. That is,

(x + y)n = n∑

k=0

(n k

) xkyn−k

Letting x = y = 1 gives

(1 + 1)n = 2n = n∑

k=0

(n k

) 1k · 1n–k =

n∑ k=0

(n k

) Concluding this section are two other examples of binomial coefficient identi-

ties. The first is proven analytically; the second is done by appealing to a sampling experiment.

a. Prove that (n 1

) + 2

(n 2

) + · · · + n

(n n

) = n2n−1

Consider the expansion of (1 + x)n:

(1 + x)n = n∑

k=0

(n k

) xk(1)n−k (2.6.3)

88 Chapter 2 Probability

Differentiating both sides of Equation 2.6.3 with respect to x gives

n(1 + x)n−1 = n∑

x=0

(n k

) kxk−1 (2.6.4)

Let x = 1. Then Equation 2.6.4 reduces to n · 2n−1 = 1

(n 1

) + 2

(n 2

) + · · · + n

(n n

) b. Prove that (n

0

)2 +

(n 1

)2 + · · · +

(n n

)2 =

( 2n n

) Imagine drawing a sample of size n from a population of 2n objects, divided into a first set of n objects and a second set of n objects. The desired sample could be formed by drawing zero objects from the first set and n objects from the second set or one object from the first set and n − 1 objects from the second set, and so on. Clearly(n

0

)(n n

) +

(n 1

)( n n − 1

) +

(n 2

)( n n − 2

) + · · · +

(n n

)(n 0

) =

( 2n n

) But ( nk ) = ( nn − k ) for all k, so(n

0

)2 +

(n 1

)2 + · · · +

(n n

)2 =

( 2n n

)

Questions

2.6.50. How many straight lines can be drawn between five points (A, B, C, D, and E), no three of which are collinear?

2.6.51. The Alpha Beta Zeta sorority is trying to fill a pledge class of nine new members during fall rush. Among the twenty-five available candidates, fifteen have been judged marginally acceptable and ten highly desirable. How many ways can the pledge class be chosen to give a two-to-one ratio of highly desirable to marginally accept- able candidates?

2.6.52. A boat has a crew of eight: Two of those eight can row only on the stroke side, while three can row only on the bow side. In how many ways can the two sides of the boat be manned?

2.6.53. Nine students, five men and four women, interview for four summer internships sponsored by a city newspaper. (a) In how many ways can the newspaper choose a set of four interns? (b) In how many ways can the newspaper choose a set of four interns if it must include two men and two women in each set? (c) How many sets of four can be picked such that not ev- eryone in a set is of the same sex?

2.6.54. The final exam in History 101 consists of five essay questions that the professor chooses from a pool of seven that are given to the students a week in advance. For how

many possible sets of questions does a student need to be prepared? In this situation, does order matter?

2.6.55. Ten basketball players meet in the school gym for a pickup game. How many ways can they form two teams of five each?

2.6.56. Your statistics teacher announces a twenty-page reading assignment on Monday that is to be finished by Thursday morning. You intend to read the first x1 pages Monday, the next x2 pages Tuesday, and the final x3 pages Wednesday, where x1 + x2 + x3 = 20, and each xi ≥ 1. In how many ways can you complete the assignment? That is, how many different sets of values can be chosen for x1, x2, and x3?

2.6.57. In how many ways can the letters in

M I S S I S S I P P I

be arranged so that no two I’s are adjacent?

2.6.58. Prove that( n + 1

k

) =

(n k

) +

( n

k − 1 )

directly without appealing to any combinatorial argu- ments.

2.6.59. Find a recursion formula for ( nk+1 ) in terms of ( n k).

2.6.60. Show that n(n − 1)2n−2 = n∑

k=2 k(k − 1)(nk).

Section 2.7 Combinatorial Probability 89

2.6.61. Prove that successive terms in the sequence ( n

0

) ,( n

1

) , . . .,

( n n

) first increase and then decrease. [Hint: Exam-

ine the ratio of two successive terms, (

n j+1

)/( n j

) .]

2.6.62. Mitch is trying to add a little zing to his cabaret act by telling four jokes at the beginning of each show. His current engagement is booked to run four months. If he gives one performance a night and never wants to

repeat the same set of jokes on any two nights, what is the minimum number of jokes he needs in his repertoire?

2.6.63. Compare the coefficients of tk in (1 + t)d(1 + t)e = (1 + t)d+e to prove that

k∑ j=0

( d j

)( e

k − j )

= (

d + e k

)

2.7 Combinatorial Probability In Section 2.6 our concern focused on counting the number of ways a given opera- tion, or sequence of operations, could be performed. In Section 2.7 we want to couple those enumeration results with the notion of probability. Putting the two together makes a lot of sense—there are many combinatorial problems where an enumera- tion, by itself, is not particularly relevant. A poker player, for example, is not inter- ested in knowing the total number of ways he can draw to a straight; he is interested, though, in his probability of drawing to a straight.

In a combinatorial setting, making the transition from an enumeration to a prob- ability is easy. If there are n ways to perform a certain operation and a total of m of those satisfy some stated condition—call it A—then P(A) is defined to be the ratio m/n. This assumes, of course, that all possible outcomes are equally likely.

Historically, the “m over n” idea is what motivated the early work of Pascal, Fermat, and Huygens (recall Section 1.3). Today we recognize that not all probabil- ities are so easily characterized. Nevertheless, the m/n model—the so-called classi- cal definition of probability—is entirely appropriate for describing a wide variety of phenomena.

Example 2.7.1

An urn contains eight chips, numbered 1 through 8. A sample of three is drawn with- out replacement. What is the probability that the largest chip in the sample is a 5?

Let A be the event “Largest chip in sample is a 5.” Figure 2.7.1 shows what must happen in order for A to occur: (1) the 5 chip must be selected, and (2) two chips must be drawn from the subpopulation of chips numbered 1 through 4. By the mul- tiplication rule, the number of samples satisfying event A is the product

(1 1

) · (42). 6 8

Choose 1

1

5

34 2

7

Choose 2

Figure 2.7.1

The sample space S for the experiment of drawing three chips from the urn con-

tains (

8 3

) outcomes, all equally likely. In this situation, then, m =

( 1 1

) · (

4 2

) , n =

( 8 3

) ,

and

P(A) = (1

1

) · (42)(8 3

) = 0.11

90 Chapter 2 Probability

Example 2.7.2

An urn contains n red chips numbered 1 through n, n white chips numbered 1 through n, and n blue chips numbered 1 through n (see Figure 2.7.2). Two chips are drawn at random and without replacement. What is the probability that the two drawn are either the same color or the same number?

Draw two

without replacement

r1

rn

r2

w1

wn

w2

b1

bn

b2

Figure 2.7.2

Let A be the event that the two chips drawn are the same color; let B be the event that they have the same number. We are looking for P(A ∪ B).

Since A and B here are mutually exclusive,

P(A ∪ B) = P(A) + P(B) With 3n chips in the urn, the total number of ways to draw an unordered sample of

size 2 is (

3n 2

) . Moreover,

P(A) = P(2 reds ∪ 2 whites ∪ 2 blues) = P(2 reds) + P(2 whites) + P(2 blues)

= 3 (n

2

) /(3n 2

) and

P(B) = P(two 1’s ∪ two 2’s ∪ · · · ∪ two n’s)

= n (

3 2

) / (3n 2

) Therefore,

P(A ∪ B) = 3

(n 2

) + n

( 3 2

) (

3n 2

)

= n + 1 3n − 1

Example 2.7.3

Twelve fair dice are rolled. What is the probability that

a. the first six dice all show one face and the last six dice all show a second face?

b. not all the faces are the same?

c. each face appears exactly twice?

a. The sample space that corresponds to the “experiment” of rolling twelve dice is the set of ordered sequences of length twelve, where the outcome at every position in the sequence is one of the integers 1 through 6. If the dice are fair, all 612 such sequences are equally likely.

Section 2.7 Combinatorial Probability 91

Let A be the set of rolls where the first six dice show one face and the second six show another face. Figure 2.7.3 shows one of the sequences in the event A. Clearly, the face that appears for the first half of the sequence could be any of the six integers from 1 through 6.

Faces 2 2 2 2 2 2 4 4 4 4 4 4

1 2 3 4 5 6 7 8 9 10 11 12 Position in sequence

Figure 2.7.3

Five choices would be available for the last half of the sequence (since the two faces cannot be the same). The number of sequences in the event A, then, is 6P2 = 6 · 5 = 30. Applying the “m/n” rule gives

P(A) = 30/612 = 1.4 × 10−8

b. Let B be the event that not all the faces are the same. Then

P(B) = 1 − P(BC) = 1 − 6/126

since there are six sequences—(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,), . . ., (6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,)—where the twelve faces are all the same.

c. Let C be the event that each face appears exactly twice. From Theorem 2.6.2, the number of ways each face can appear exactly twice is 12!/(2! · 2! · 2! · 2! · 2! · 2!). Therefore,

P(C) = 12!/(2! · 2! · 2! · 2! · 2! · 2!) 612

= 0.0034

Example 2.7.4

A fair die is tossed n times. What is the probability that the sum of the faces showing is n + 2?

The sample space associated with rolling a die n times has 6n outcomes, all of which in this case are equally likely because the die is presumed fair. There are two “types” of outcomes that will produce a sum of n + 2: (a) n − 1 1’s and one 3 and (b) n − 2 1’s and two 2’s (see Figure 2.7.4). By Theorem 2.6.2 the number of sequences having n − 1 1’s and one 3 is n!1!(n−1)! = n; likewise, there are n!2!(n−2)! =

( n 2

) outcomes

having n − 2 1’s and two 2’s. Therefore,

P(sum = n + 2) = n + (n

2

) 6n

1 1

1 2

1 3

1 n – 1

3 n

Sum = n + 2

1 1

1 2

1 3

1 n – 2

2 n – 1

2 n

Sum = n + 2

Figure 2.7.4

92 Chapter 2 Probability

Example 2.7.5

Two monkeys, Mickey and Marian, are strolling along a moonlit beach when Mickey sees an abandoned Scrabble set. Investigating, he notices that some of the letters are missing, and what remain are the following fifty-nine tiles:

A B C D E F G H I J K L M 4 1 2 2 7 1 1 3 5 0 3 5 1

N O P Q R S T U V W X Y Z 3 2 0 0 2 8 4 2 0 1 0 2 0

Mickey, being of a romantic bent, would like to impress Marian, so he rearranges the letters hoping to spell something endearing. For some unknown reason, Marian can read, but Mickey is dumb as dirt, so all he can do is scramble the fifty-nine tiles at random and hope for the best. What is the probability he gets lucky and spells out

She walks in beauty, like the night Of cloudless climes and starry skies

As we might imagine, Mickey would have to get very lucky. The total number of ways to permute fifty-nine letters—four A’s, one B, two C’s, and so on—is a direct application of Theorem 2.6.2:

59! 4!1!2! . . . 2!0!

But of that number of ways, only one is the couplet he is hoping for. So, since he is ar- ranging the letters randomly, making all permutations equally likely, the probability of his spelling out Byron’s lines is

1 59!

4!1!2! . . . 2!0!

or, using Stirling’s formula, about 1.7×10−61. Love may conquer all, but it won’t beat those odds: Mickey would be well advised to start working on Plan B.

Example 2.7.6

Suppose k people are selected at random. What are the chances that at least two of those k were born on the same day of the year? Known as the birthday problem, this is a particularly intriguing example of combinatorial probability because its statement is so simple, its analysis is straightforward, yet its solution is strongly contrary to our intuition.

Picture the k individuals lined up in a row to form an ordered sequence. If leap year is omitted, each person might have any of 365 birthdays. By the multiplication rule, the group as a whole generates a sample space of 365k birthday sequences (see Figure 2.7.5).

(365) 1

(365) 2

(365) k

Person

Possible birthdays: 365 k different

sequences

Figure 2.7.5

Define A to be the event “At least two people have the same birthday.” If each person is assumed to have the same chance of being born on any given day, the 365k

sequences in Figure 2.7.5 are equally likely, and

P(A) = number of sequences in A 365k

Section 2.7 Combinatorial Probability 93

Counting the number of sequences in the numerator here is prohibitively diffi- cult because of the complexity of the event A; fortunately, counting the number of sequences in Ac is quite easy. Notice that each birthday sequence in the sample space belongs to exactly one of two categories (see Figure 2.7.6):

1. At least two people have the same birthday.

2. All k people have different birthdays.

(July 13, Sept. 2, , July 13) • (April 4, April 4, , Aug. 17) •

Sequences where at least two people have the same birthday

(June 14, Jan. 10, , Oct. 28) • (Aug. 10, March 1, , Sept. 8) •

A

Ac

Sequences where all k people have different birthdays

Sample space: all birthday sequences of length k (contains 365k outcomes).

Figure 2.7.6

It follows that

number of sequences in A = 365k − number of sequences where all k people have different birthdays

The number of ways to form birthday sequences for k people subject to the re- striction that all k birthdays must be different is simply the number of ways to form permutations of length k from a set of 365 distinct objects:

365Pk = 365(364) · · · (365 − k + 1) Therefore,

P(A) = P(At least two people have the same birthday)

= 365 k − 365(364) · · · (365 − k + 1)

365k

Table 2.7.1 shows P(A) for k values of 15, 22, 23, 40, 50, and 70. Notice how the P(A)’s greatly exceed what our intuition would suggest.

Table 2.7.1

k P(A) = P (At least two have same birthday) 15 0.253 22 0.476 23 0.507 40 0.891 50 0.970 70 0.999

Comment The values for P(A) in Table 2.7.1 are actually slight underestimates for the true probabilities that at least two of k people will be born on the same day. The assumption made earlier that all 365k birthday sequences are equally likely is not

94 Chapter 2 Probability

entirely correct: Births are somewhat more common during the summer than they are during the winter. It has been proven, though, that any sort of deviation from the equally likely model will serve only to increase the chances that two or more people will share the same birthday. So, if k = 40, for example, the probability is slightly greater than 0.891 that at least two were born on the same day.

Comment Presidential biographies offer one opportunity to “confirm” the unex- pectedly large values that Table 2.7.1 gives for P(A). Among our first k = 40 presi- dents, two did have the same birthday: Harding and Polk were both born on Novem- ber 2. More surprising, though, are the death dates of the presidents: John Adams, Jefferson, and Monroe all died on July 4, and Fillmore and Taft both died on March 8.

Example 2.7.7

One of the more instructive—and to some, one of the more useful—applications of combinatorics is the calculation of probabilities associated with various poker hands. It will be assumed in what follows that five cards are dealt from a poker deck and that no other cards are showing, although some may already have been dealt. The sample space is the set of

(52 5

) = 2,598,960 different hands, each having probability 1/2,598,960. What are the chances of being dealt (a) a full house, (b) one pair, and (c) a straight? [Probabilities for the various other kinds of poker hands (two pairs, three-of-a-kind, flush, and so on) are gotten in much the same way.]

a. Full house. A full house consists of three cards of one denomination and two of another. Figure 2.7.7 shows a full house consisting of three 7’s and two queens. Denominations for the three-of-a-kind can be chosen in

(13 1

) ways. Then, given that

a denomination has been decided on, the three requisite suits can be selected in (4

3

) ways. Applying the same reasoning to the pair gives

(12 1

) available denominations,

each having (4

2

) possible choices of suits. Thus, by the multiplication rule,

P(full house) =

( 13 1

)( 4 3

)( 12 1

)( 4 2

) (

52 5

) = 0.00144 2 3 4 5 6 7 8 9 10 J Q K A

D H × × C × S × ×

Figure 2.7.7

b. One pair. To qualify as a one-pair hand, the five cards must include two of the same denomination and three “single” cards—cards whose denominations match neither the pair nor each other. Figure 2.7.8 shows a pair of 6’s. For the pair, there are

(13 1

) possible denominations and, once selected,

(4 2

) possible

suits. Denominations for the three single cards can be chosen (12

3

) ways (see

Question 2.7.16), and each card can have any of (4

1

) suits. Multiplying these fac-

tors together and dividing by (52

2

) gives a probability of 0.42:

P(one pair) =

( 13 1

)( 4 2

)( 12 3

)( 4 1

)( 4 1

)( 4 1

) (

52 5

) = 0.42

Section 2.7 Combinatorial Probability 95

2 3 4 5 6 7 8 9 10 J Q K A

D × × H × × C × S

Figure 2.7.8

c. Straight. A straight is five cards having consecutive denominations but not all in the same suit—for example, a 4 of diamonds, 5 of hearts, 6 of hearts, 7 of clubs, and 8 of diamonds (see Figure 2.7.9). An ace may be counted “high” or “low,” which means that (10, jack, queen, king, ace) is a straight and so is (ace, 2, 3, 4, 5). (If five consecutive cards are all in the same suit, the hand is called a straight flush. The latter is considered a fundamentally different type of hand in the sense that a straight flush “beats” a straight.) To get the numerator for P(straight), we will first ignore the condition that all five cards not be in the same suit and simply count the number of hands having consecutive denominations. Note there are ten sets of consecutive denominations of length five: (ace, 2, 3, 4, 5), (2, 3, 4, 5, 6), . . ., (10, jack, queen, king, ace). With no restrictions on the suits, each card can be either a diamond, heart, club, or spade. It follows, then, that the number of

five-card hands having consecutive denominations is 10 · (41)5. But forty (= 10 ·4) of those hands are straight flushes. Therefore,

P(straight) = 10 ·

( 4 1

)5 − 40(

52 5

) = 0.00392 Table 2.7.2 shows the probabilities associated with all the different poker hands. Hand i beats hand j if P(hand i) < P(hand j).

2 3 4 5 6 7 8 9 10 J Q K A

D × × H × × C × S

Figure 2.7.9

Table 2.7.2

Hand Probability

One pair 0.42 Two pairs 0.048 Three-of-a-kind 0.021 Straight 0.0039 Flush 0.0020 Full house 0.0014 Four-of-a-kind 0.00024 Straight flush 0.000014 Royal flush 0.0000015

96 Chapter 2 Probability

Example 2.7.8

A somewhat inebriated conventioneer finds himself in the embarrassing position of being unable to discern whether he is walking forward or backward—or, what is worst, to predict in which of those directions his next step will be. If he is equally likely to walk forward or backward, what is the probability that after hazarding n such maneuvers, he will have moved forward a distance of r steps?

Let x denote the number of steps he takes forward and y denote the number backward. Then

x + y = n and

x − y = r Solving these equations simultaneously, we get x = (n+r)/2 and y = (n−r)/2. Thus, out of the 2n total ways he can take n steps, the number of permutations for which he ends up r steps forward is n!

/[( n+r 2

) ! ( n−r

2

) ! ]

(recall Theorem 2.6.2). The probability that he shows a net advance of r steps is the quotient(

n n + r

2

)

2n

Comment Example 2.7.8 is describing a one-dimensional random walk problem. Over the years, the mathematical properties of random walks in various numbers of dimensions have been a useful tool for modeling such disparate phenomena as Brownian motion, the movements of individual animals and populations of animals, and daily fluctuations in the stock market. The term random walk was coined in 1905 by the renowned British statistician, Karl Pearson.

Example 2.7.9

To a purist, buying a LOTTO ticket at your local supermarket is not quite the gam- bling equivalent of playing a round of Texas Hold’Em in some smoke-filled road- house, but there is no denying that the amount of money spent on lotteries each year (estimated to be upwards of $50 billion in the United States alone) is major-league. And to be fair, lotteries do have an interesting history that goes back thousands of years, long before the first poker game was ever played. Evidence suggests that lot- teries helped finance the building of the Great Wall of China. Closer to home, the first permanent English settlement on the North American continent–Jamestown, founded in 1607–was funded by the Virginia Company of London, a group of in- vestors who acquired the rights to sponsor the expedition by winning a lottery.

In recent years, one of the most popular versions of “drawing lots” has been Powerball. For the price of a ticket, a player gets to pick five (distinct) numbers from the integers 1 through 59 and one additional number from the integers 1 through 35. Then at a regularly-scheduled time, five white balls are drawn from a drum containing fifty-nine balls, numbered 1 through 59, and a sixth selection (the Powerball) is drawn from a second drum containing thirty-five balls, numbered 1 through 35. If and how much a player wins depends on the nature of the matches between the ticket numbers picked and the balls actually drawn.

For example, the fourth prize is worth $100 and awarded to anyone whose ticket correctly matches four of the five white balls but not the Powerball. Calculating the probability of that happening is a straightforward exercise in applying the multipli- cation rule to the numbers of ways to form various combinations. Since(

59 5

) = number of ways to choose five white balls from the first drum

Section 2.7 Combinatorial Probability 97

and ( 35 1

) = number of ways to choose a Powerball from the second drum

the product ( 595 )( 35 1 ) = 175, 223, 510 is the total number of (equally-likely) outcomes

possible in a LOTTO drawing. Your particular ticket has the potential to (1) match exactly four of the white balls drawn in ( 54 ) = 5 different ways, (2) not match the fifth white ball in ( 541 ) = 54 different ways and (3) not match the Powerball in ( 341 ) = 34 different ways. The probability, then, that your ticket will win the $100 prize is the ratio (

5 4

) ( 54 1

) ( 34 1

) [( 59 5

) ( 35 1

)] = 9180/175,223,510 = 0.00005239/

The entire set of winning combinations, associated probabilities, and payoffs are listed in Table 2.7.3. Adding the entries in the middle column would show that your probability of winning something is 0.0314.

Table 2.7.3

Winning Pick Probability Payoff

1. Match all five white balls and Powerball [( 55 )( 1 1 )]/[(

59 5 )(

35 1 )] Depends on the

number of winners 2. Match all five white balls but not Powerball [( 55 )(

34 1 )]/[(

59 5 )(

35 1 )] $1,000,000

3. Match four white balls and Powerball [( 54 )( 54 1 )(

1 1 )]/[(

59 5 )(

35 1 )] $10,000

4. Match four white balls but not Powerball [( 54 )( 54 1 )(

34 1 )]/[(

59 5 )(

35 1 )] $100

5. Match three white balls and Powerball [( 53 )( 54 2 )(

1 1 )]/[(

59 5 )(

35 1 )] $100

6. Match three white balls but not Powerball [( 53 )( 54 2 )(

34 1 )]/[(

59 5 )(

35 1 )] $7

7. Match two white balls and Powerball [( 52 )( 54 3 )(

1 1 )]/[(

59 5 )(

35 1 )] $7

8. Match one white ball and Powerball [( 51 )( 54 4 )(

1 1 )]/[(

59 5 )(

35 1 )] $4

9. Match zero white balls and Powerball [( 50 )( 54 5 )(

1 1 )]/[(

59 5 )(

35 1 )] $4

Comment While the six numbers drawn in Powerball are random, records show that the six numbers picked by ticket-buyers are not random. Why? Because Power- ball players have a tendency to bet on their birthdays, which results in their entries tending to have smaller digits and be disproportionately over-represented (and du- plicated) in the population of all possible entries. Does that matter? It depends. If one ticket or one hundred thousand tickets qualify for the fourth prize, for example, each of those players will still receive $100. But if n players qualify for the jackpot, each will receive only l/nth of the jackpot money. So, unless you feel a burning desire to be magnanimous and share your possible (albeit highly improbable) multi-millions of dollars of jackpot winnings with total strangers, betting on your birthday is a bad idea.

Problem-Solving Hints

(Doing combinatorial probability problems)

Listed on p. 70 are several hints that can be helpful in counting the number of ways to do something. Those same hints apply to the solution of combinatorial probability problems, but a few others should be kept in mind as well.

(Continued on next page)

98 Chapter 2 Probability

(Continued)

1. The solution to a combinatorial probability problem should be set up as a quotient of numerator and denominator enumerations. Avoid the tempta- tion to multiply probabilities associated with each position in the sequence. The latter approach will always “sound” reasonable, but it will frequently oversimplify the problem and give the wrong answer.

2. Keep the numerator and denominator consistent with respect to order—if permutations are being counted in the numerator, be sure that permutations are being counted in the denominator; likewise, if the outcomes in the numerator are combinations, the outcomes in the denominator must also be combinations.

3. The number of outcomes associated with any problem involving the rolling of n six-sided dice is 6n; similarly, the number of outcomes associated with tossing a coin n times is 2n. The number of outcomes associated with dealing a hand of n cards from a standard fifty-two-card poker deck is 52Cn.

Questions

2.7.1. Ten equally qualified marketing assistants are can- didates for promotion to associate buyer; seven are men and three are women. If the company intends to promote four of the ten at random, what is the probability that exactly two of the four are women?

2.7.2. An urn contains six chips, numbered 1 through 6. Two are chosen at random and their numbers are added together. What is the probability that the resulting sum is equal to 5?

2.7.3. An urn contains twenty chips, numbered 1 through 20. Two are drawn simultaneously. What is the probability that the numbers on the two chips will differ by more than 2?

2.7.4. A bridge hand (thirteen cards) is dealt from a stan- dard fifty-two-card deck. Let A be the event that the hand contains four aces; let B be the event that the hand con- tains four kings. Find P(A ∪ B). 2.7.5. Consider a set of ten urns, nine of which contain three white chips and three red chips each. The tenth con- tains five white chips and one red chip. An urn is picked at random. Then a sample of size 3 is drawn without replace- ment from that urn. If all three chips drawn are white, what is the probability that the urn being sampled is the one with five white chips?

2.7.6. A committee of fifty politicians is to be chosen from among our one hundred U.S. senators. If the selection is done at random, what is the probability that each state will be represented?

2.7.7. Suppose that n fair dice are rolled. What are the chances that all n faces will be the same?

2.7.8. Five fair dice are rolled. What is the probability that the faces showing constitute a “full house”—that is, three faces show one number and two faces show a second number?

2.7.9. Imagine that the test tube pictured contains 2n grains of sand, n white and n black. Suppose the tube is vigorously shaken. What is the probability that the two colors of sand will completely separate; that is, all of one color fall to the bottom, and all of the other color lie on top? (Hint: Consider the 2n grains to be aligned in a row. In how many ways can the n white and the n black grains be permuted?)

2.7.10. Does a monkey have a better chance of rearranging

AC C L LU U S to spell C A LC U LU S

or

A A B E G L R to spell A L G E B R A?

2.7.11. An apartment building has eight floors. If seven people get on the elevator on the first floor, what is the probability they all want to get off on different floors? On the same floor? What assumption are you making? Does it seem reasonable? Explain.

2.7.12. If the letters in the phrase

A R O L L I N G S T O N E G A T H E R S N O M O S S

are arranged at random, what are the chances that not all the S’s will be adjacent?

2.7.13. Suppose each of ten sticks is broken into a long part and a short part. The twenty parts are arranged into ten pairs and glued back together so that again there are ten sticks. What is the probability that each long part will be paired with a short part? (Note: This problem is a model for the effects of radiation on a living cell. Each

Section 2.8 Taking a Second Look at Statistics (Monte Carlo Techniques) 99

chromosome, as a result of being struck by ionizing radi- ation, breaks into two parts, one part containing the cen- tromere. The cell will die unless the fragment containing the centromere recombines with a fragment not contain- ing a centromere.)

2.7.14. Six dice are rolled one time. What is the probability that each of the six faces appears?

2.7.15. Suppose that a randomly selected group of k peo- ple are brought together. What is the probability that exactly one pair has the same birthday?

2.7.16. For one-pair poker hands, why is the number of denominations for the three single cards

(12 3

) rather than(12

1

)(11 1

)(10 1

) ?

2.7.17. Dana is not the world’s best poker player. Dealt a 2 of diamonds, an 8 of diamonds, an ace of hearts, an ace of clubs, and an ace of spades, she discards the three aces. What are her chances of drawing to a flush?

2.7.18. A poker player is dealt a 7 of diamonds, a queen of diamonds, a queen of hearts, a queen of clubs, and an ace of hearts. He discards the 7. What is his probability of drawing to either a full house or four-of-a-kind?

2.7.19. Tim is dealt a 4 of clubs, a 6 of hearts, an 8 of hearts, a 9 of hearts, and a king of diamonds. He discards the 4

and the king. What are his chances of drawing to a straight flush? To a flush?

2.7.20. Five cards are dealt from a standard 52-card deck. What is the probability that the sum of the faces on the five cards is 48 or more?

2.7.21. Nine cards are dealt from a 52-card deck. Write a formula for the probability that three of the five even nu- merical denominations are represented twice, one of the three face cards appears twice, and a second face card appears once. (Note: Face cards are the jacks, queens, and kings; 2, 4, 6, 8, and 10 are the even numerical denominations.)

2.7.22. A coke hand in bridge is one where none of the thirteen cards is an ace or is higher than a 9. What is the probability of being dealt such a hand?

2.7.23. A pinochle deck has forty-eight cards, two of each of six denominations (9, J, Q, K, 10, A) and the usual four suits. Among the many hands that count for meld is a roundhouse, which occurs when a player has a king and queen of each suit. In a hand of twelve cards, what is the probability of getting a “bare” roundhouse (a king and queen of each suit and no other kings or queens)?

2.8 Taking a Second Look at Statistics (Monte Carlo Techniques)

Recall the von Mises definition of probability given on p. 16. If an experiment is repeated n times under identical conditions, and if the event E occurs on m of those repetitions, then

P(E) = lim n→∞

m n

(2.8.1)

To be sure, Equation 2.8.1 is an asymptotic result, but it suggests an obvious (and very useful) approximation—if n is finite,

P(E) .= m n

In general, efforts to estimate probabilities by simulating repetitions of an ex- periment (usually with a computer) are referred to as Monte Carlo studies. Usually the technique is used in situations where an exact probability is difficult to calculate. It can also be used, though, as an empirical justification for choosing one proposed solution over another.

For example, consider the game described in Example 2.4.10. An urn contains a red chip, a blue chip, and a two-color chip (red on one side, blue on the other). One chip is drawn at random and placed on a table. The question is, if blue is showing, what is the probability that the color underneath is also blue?

Pictured in Figure 2.8.1 are two ways of conceptualizing the question just posed. The outcomes in (a) are assuming that a chip was drawn. Starting with that premise,

100 Chapter 2 Probability

the answer to the question is 12 —the red chip is obviously eliminated and only one of the two remaining chips is blue on both sides.

Chip drawn

red blue

two-color

Side drawn

red/red blue/blue red/blue

P(B|B) = 1/2 P(B|B) = 2/3

(a) (b)

Figure 2.8.1

By way of contrast, the outcomes in (b) are assuming that the side of a chip was drawn. If so, the blue color showing could be any of three blue sides, two of which are blue underneath. According to model (b), then, the probability of both sides being blue is 23 .

The formal analysis on p. 43, of course, resolves the debate—the correct answer is 23 . But suppose that such a derivation was unavailable. How might we assess the relative plausibilities of 12 and

2 3 ? The answer is simple—just play the game a

number of times and see what proportion of outcomes that show blue on top have blue underneath.

To that end, Table 2.8.1 summarizes the results of one hundred random drawings. For a total of fifty-two trials, blue was showing (S) when the chip was placed on a

Table 2.8.1

Trial # S U Trial # S U Trial # S U Trial # S U

1 R B 26 B R 51 B R 76 B B* 2 B B* 27 R R 52 R B 77 B B* 3 B R 28 R B 53 B B* 78 R R 4 R R 29 R B 54 R B 79 B B* 5 R B 30 R R 55 R R 80 R R 6 R B 31 R B 56 R B 81 R B 7 R R 32 B B* 57 R R 82 R B 8 R R 33 R B 58 B B* 83 R R 9 B B* 34 B B* 59 B R 84 B R

10 B R 35 B B* 60 B B* 85 B R 11 R R 36 R R 61 B R 86 R R 12 B B* 37 B R 62 R B 87 B B* 13 R R 38 B B* 63 R R 88 R B 14 B R 39 R R 64 R R 89 B R 15 B B* 40 B B* 65 B B* 90 R R 16 B B* 41 B B* 66 B R 91 R B 17 R B 42 B R 67 R R 92 R R 18 B R 43 B B* 68 B B* 93 R R 19 B B* 44 B B* 69 B B* 94 R B 20 B B* 45 B B* 70 R R 95 B B* 21 R R 46 R R 71 R R 96 B B* 22 R R 47 B B* 72 B B* 97 B R 23 B B* 48 B B* 73 R B 98 R R 24 B R 49 R R 74 R R 99 B B* 25 B B* 50 R R 75 B B* 100 B B*

Section 2.8 Taking a Second Look at Statistics (Monte Carlo Techniques) 101

table; for thirty-six of the trials (those marked with an asterisk), the color underneath (U) was also blue. Using the approximation suggested by Equation 2.8.1,

P(Blue is underneath | Blue is on top) = P(B | B) .= 36 52

= 0.69

a figure much more consistent with 23 than with 1 2 .

The point of this example is not to downgrade the importance of rigorous deriva- tions and exact answers. Far from it. The application of Theorem 2.4.1 to solve the problem posed in Example 2.4.10 is obviously superior to the Monte Carlo approximation illustrated in Table 2.8.1. Still, replications of experiments can of- ten provide valuable insights and call attention to nuances that might otherwise go unnoticed.

Chapte r

3Random Variables CHAPTER OUTLINE

3.1 Introduction 3.2 Binomial and

Hypergeometric Probabilities

3.3 Discrete Random Variables

3.4 Continuous Random Variables

3.5 Expected Values 3.6 The Variance 3.7 Joint Densities 3.8 Transforming and

Combining Random Variables

3.9 Further Properties of the Mean and Variance

3.10 Order Statistics 3.11 Conditional Densities 3.12 Moment-Generating

Functions 3.13 Taking a Second

Look at Statistics (Interpreting Means)

One of a Swiss family producing eight distinguished scientists, Jakob (Jacques) Bernoulli (1654–1705) was forced by his father to pursue theological studies, but his love of mathematics eventually led him to a university career. He and his brother, Johann, were the most prominent champions of Leibniz’s calculus on continental Europe, the two using the new theory to solve numerous problems in physics and mathematics. Bernoulli’s main work in probability, Ars Conjectandi, was published after his death by his nephew, Nikolaus, in 1713.

3.1 INTRODUCTION Throughout Chapter 2, probabilities were assigned to events—that is, to sets of sam- ple outcomes. The events we dealt with were composed of either a finite or a count- ably infinite number of sample outcomes, in which case the event’s probability was simply the sum of the probabilities assigned to its outcomes. One particular proba- bility function that came up over and over again in Chapter 2 was the assignment of 1 n as the probability associated with each of the n points in a finite sample space. This is the model that typically describes games of chance (and all of our combinatorial probability problems in Chapter 2).

The first objective of this chapter is to look at several other useful ways for as- signing probabilities to sample outcomes. In so doing, we confront the desirability of “redefining” sample spaces using functions known as random variables. How and why these are used—and what their mathematical properties are—become the focus of virtually everything covered in Chapter 3.

As a case in point, suppose a medical researcher is testing eight elderly adults for their allergic reaction (yes or no) to a new drug for controlling blood pressure. One of the 28 = 256 possible sample points would be the sequence (yes, no, no, yes, no, no, yes, no), signifying that the first subject had an allergic reaction, the second did not, the third did not, and so on. Typically, in studies of this sort, the particular subjects experiencing reactions is of little interest: what does matter is the number who show a reaction. If that were true here, the outcome’s relevant information (i.e., the number of allergic reactions) could be summarized by the number 3.1

Suppose X denotes the number of allergic reactions among a set of eight adults. Then X is said to be a random variable and the number 3 is the value of the random variable for the outcome (yes, no, no, yes, no, no, yes, no).

1 By Theorem 2.6.2, of course, there would be a total of fifty-six (= 8!/3!5!) outcomes having exactly three yeses. All fifty-six would be equivalent in terms of what they imply about the drug’s likelihood of causing allergic reactions.

102

Section 3.2 Binomial and Hypergeometric Probabilities 103

In general, random variables are functions that associate numbers with some at- tribute of a sample outcome that is deemed to be especially important. If X denotes the random variable and s denotes a sample outcome, then X (s) = t, where t is a real number. For the allergy example, s = (yes, no, no, yes, no, no, yes, no) and t = 3.

Random variables can often create a dramatically simpler sample space. That certainly is the case here—the original sample space has 256 (= 28) outcomes, each being an ordered sequence of length eight. The random variable X , on the other hand, has only nine possible values, the integers from 0 to 8, inclusive.

In terms of their fundamental structure, all random variables fall into one of two broad categories, the distinction resting on the number of possible values the random variable can equal. If the latter is finite or countably infinite (which would be the case with the allergic reaction example), the random variable is said to be discrete; if the outcomes can be any real number in a given interval, the number of possibilities is uncountably infinite, and the random variable is said to be continuous. The difference between the two is critically important, as we will learn in the next several sections.

The purpose of Chapter 3 is to introduce the important definitions, concepts, and computational techniques associated with random variables, both discrete and continuous. Taken together, these ideas form the bedrock of modern probability and statistics.

3.2 Binomial and Hypergeometric Probabilities This section looks at two specific probability scenarios that are especially important, both for their theoretical implications as well as for their ability to describe real-world problems. What we learn in developing these two models will help us understand random variables in general, the formal discussion of which begins in Section 3.3.

THE BINOMIAL PROBABILITY DISTRIBUTION

Binomial probabilities apply to situations involving a series of independent and iden- tical trials, where each trial can have only one of two possible outcomes. Imagine three distinguishable coins being tossed, each having a probability p of coming up heads. The set of possible outcomes are the eight listed in Table 3.2.1. If the proba- bility of any of the coins coming up heads is p, then the probability of the sequence (H, H, H) is p3, since the coin tosses qualify as independent trials. Similarly, the probability of (T, H, H) is (1 − p)p2. The fourth column of Table 3.2.1 shows the probabilities associated with each of the three-coin sequences.

Table 3.2.1

1st Coin 2nd Coin 3rd Coin Probability Number of Heads

H H H p3 3 H H T p2(1 − p) 2 H T H p2(1 − p) 2 T H H p2(1 − p) 2 H T T p(1 − p)2 1 T H T p(1 − p)2 1 T T H p(1 − p)2 1 T T T (1 − p)3 0

Suppose our main interest in the coin tosses is the number of heads that oc- cur. Whether the actual sequence is, say, (H, H, T) or (H, T, H) is immaterial, since

104 Chapter 3 Random Variables

each outcome contains exactly two heads. The last column of Table 3.2.1 shows the number of heads in each of the eight possible outcomes. Notice that there are three outcomes with exactly two heads, each having an individual probability of p2(1 − p). The probability, then, of the event “two heads” is the sum of those three individual probabilities—that is, 3p2(1− p). Table 3.2.2 lists the probabilities of tossing k heads, where k = 0, 1, 2, or 3.

Table 3.2.2

Number of Heads Probability

0 (1 − p)3 1 3p(1 − p)2 2 3p2(1 − p) 3 p3

Now, more generally, suppose that n coins are tossed, in which case the number of heads can equal any integer from 0 through n. By analogy,

P(k heads) = ⎛ ⎝ number ofways to arrange k

heads and n − k tails

⎞ ⎠ ·

⎛ ⎜⎜⎝

probability of any particular sequence

having k heads and n − k tails

⎞ ⎟⎟⎠

= ⎛ ⎝ number of waysto arrange k

heads and n − k tails

⎞ ⎠ · pk(1 − p)n−k

The number of ways to arrange k H’s and n − k T’s, though, is n!k!(n−k)! , or (n

k

) (recall

Theorem 2.6.2).

Theorem 3.2.1

Consider a series of n independent trials, each resulting in one of two possible out- comes, “success” or “failure.” Let p = P (success occurs at any given trial) and assume that p remains constant from trial to trial. Then

P(k successes) = (n

k

) pk(1 − p)n−k, k = 0, 1, . . . , n

Comment The probability assignment given by the equation in Theorem 3.2.1 is known as the binomial distribution.

Example 3.2.1

In communications of various kinds, a sent message may not be received correctly be- cause the communications channel is “noisy.” In particular, a bit transmitted might be changed with probability p. One method of coping with this problem is to send the bit five times, then use the majority of these five received bits as the intended message. Under this scheme, what is the probability that the message is received correctly?

The five bits received form a sequence of five Bernoulli trials with probability p that the bit is changed (success). Then the message is received correctly if the number of changed bits (successes) is 0, 1, or 2. The probability of this is

2∑ k=0

( 5 k

) pk(1 − p)5−k

The following chart gives values of this sum for some choices for p.

Section 3.2 Binomial and Hypergeometric Probabilities 105

p Probability

0.01 0.99999 0.05 0.99884 0.10 0.99144 0.15 0.97339

Example 3.2.2

Kingwest Pharmaceuticals is experimenting with a new affordable AIDS medica- tion, PM-17, that may have the ability to strengthen a victim’s immune system. Thirty monkeys infected with the HIV complex have been given the drug. Researchers in- tend to wait six weeks and then count the number of animals whose immunological responses show a marked improvement. Any inexpensive drug capable of being ef- fective 60% of the time would be considered a major breakthrough; medications whose chances of success are 50% or less are not likely to have any commercial potential.

Yet to be finalized are guidelines for interpreting results. Kingwest hopes to avoid making either of two errors: (1) rejecting a drug that would ultimately prove to be marketable and (2) spending additional development dollars on a drug whose effectiveness, in the long run, would be 50% or less. As a tentative “decision rule,” the project manager suggests that unless sixteen or more of the monkeys show im- provement, research on PM-17 should be discontinued.

a. What are the chances that the “sixteen or more” rule will cause the company to reject PM-17, even if the drug is 60% effective?

b. How often will the “sixteen or more” rule allow a 50%-effective drug to be per- ceived as a major breakthrough?

a. Each of the monkeys is one of n = 30 independent trials, where the out- come is either a “success” (Monkey’s immune system is strengthened) or a “failure” (Monkey’s immune system is not strengthened). By assumption, the probability that PM-17 produces an immunological improvement in any given monkey is p = P (success) = 0.60.

By Theorem 3.2.1, the probability that exactly k monkeys (out of thirty)

will show improvement after six weeks is (

30 k

) (0.60)k(0.40)30−k. The proba-

bility, then, that the “sixteen or more” rule will cause a 60%-effective drug to be discarded is the sum of “binomial” probabilities for k values ranging from 0 to 15:

P(60%-effective drug fails “sixteen or more” rule) = 15∑

k=0

( 30 k

) (0.60)k(0.40)30−k

= 0.1754

Roughly 18% of the time, in other words, a “breakthrough” drug such as PM-17 will produce test results so mediocre (as measured by the “sixteen or more” rule) that the company will be misled into thinking it has no potential.

b. The other error Kingwest can make is to conclude that PM-17 warrants fur- ther study when, in fact, its value for p is below a marketable level. The chance that particular incorrect inference will be drawn here is the probability that

106 Chapter 3 Random Variables

the number of successes will be greater than or equal to sixteen when p = 0.5. That is,

P(50%-effective PM-17 appears to be marketable)

= P(Sixteen or more successes occur)

= 30∑

k=16

( 30 k

) (0.5)k(0.5)30−k

= 0.43 Thus, even if PM-17’s success rate is an unacceptably low 50%, it has a 43%

chance of performing sufficiently well in thirty trials to satisfy the “sixteen or more” criterion.

Example 3.2.3

The Stanley Cup playoff in professional hockey is a seven-game series, where the first team to win four games is declared the champion. The series, then, can last any- where from four to seven games (just like the World Series in baseball). Calculate the likelihoods that the series will last four, five, six, or seven games. Assume that (1) each game is an independent event and (2) the two teams are evenly matched.

Consider the case where Team A wins the series in six games. For that to happen, they must win exactly three of the first five games and they must win the sixth game. Because of the independence assumption, we can write

P(Team A wins in six games) = P(Team A wins three of first five) · P(Team A wins sixth)

= [(

5 3

) (0.5)3(0.5)2

] · (0.5) = 0.15625

Since the probability that Team B wins the series in six games is the same (why?),

P(Series ends in six games) = P(Team A wins in six games ∪ Team B wins in six games)

= P(A wins in six) + P(B wins in six) (why?) = 0.15625 + 0.15625 = 0.3125

A similar argument allows us to calculate the probabilties of four-, five-, and seven- game series:

P(four-game series) = 2(0.5)4 = 0.125

P(five-game series) = 2 [(

4 3

) (0.5)3(0.5)

] (0.5) = 0.25

P(seven-game series) = 2 [(

6 3

) (0.5)3(0.5)3

] (0.5) = 0.3125

Having calculated the “theoretical” probabilities associated with the possible lengths of a Stanley Cup playoff raises an obvious question: How do those likelihoods compare with the actual distribution of playoff lengths? Between 1947 and 2015 there were sixty-eight playoffs (the 2004–05 season was cancelled). Column 2 in Table 3.2.3 shows the proportion of playoffs that have lasted four, five, six, and seven games, respectively.

Section 3.2 Binomial and Hypergeometric Probabilities 107

Table 3.2.3

Series Length Observed Proportion Theoretical Probability

4 17/68 = 0.250 0.125 5 16/68 = 0.235 0.250 6 21/68 = 0.309 0.3125 7 14/68 = 0.206 0.3125

Data from: http://statshockey.homestead.com/trophies/stanleycup.html

Clearly, the agreement between the entries in columns 2 and 3 is not very good: Particularly noticeable is the excess of short playoffs (four games) and the deficit of long playoffs (seven games). What this “lack of fit” suggests is that one or more of the binomial distribution assumptions is not satisfied. Consider, for example, the param- eter p, which we assumed to equal 12 . In reality, its value might be something quite different—just because the teams playing for the championship won their respective divisions, it does not necessarily follow that the two are equally good. Indeed, if the two contending teams were frequently mismatched, the consequence would be an increase in the number of short playoffs and a decrease in the number of long play- offs. It may also be the case that momentum is a factor in a team’s chances of winning a given game. If so, the independence assumption implicit in the binomial model is rendered invalid.

Example 3.2.4

A university discovers that its colleges don’t each require a full time technical support person. In fact, it estimates that any one of the ten colleges has only a 0.2 probability of needing a technical support person in any given day. They decide to create a pool of five support personnel to be called upon by the colleges. What is the probability that a college will have to wait on someone to get free?

Assume the needs of the colleges for service are independent. Then the number of support persons needed per day is a sequence of n Bernoulli trials, where a success means that a college needs a technical support person. Here n = 10, and p = 0.2. A group has to wait if the number of “successes” is strictly greater than 5. But the

probability of more than five successes is 10∑

k=6

( 10 k

) (0.2)k(0.8)10−k = 0.006.

In a month, there are approximately two hundred college-days, so typically in only one of those college-days per month will there be some wait time.

Such analyses can easily be modified to observe the effects of other assumptions on n and p. (See Question 3.2.17.)

Questions

3.2.1. An investment analyst has tracked a certain blue- chip stock for the past six months and found that on any given day, it either goes up a point or goes down a point. Furthermore, it went up on 25% of the days and down on 75%. What is the probability that at the close of trading four days from now, the price of the stock will be the same as it is today? Assume that the daily fluctuations are inde- pendent events.

3.2.2. In a nuclear reactor, the fission process is controlled by inserting special rods into the radioactive core to ab- sorb neutrons and slow down the nuclear chain reaction.

When functioning properly, these rods serve as a first- line defense against a core meltdown. Suppose a reac- tor has ten control rods, each operating independently and each having an 0.80 probability of being properly in- serted in the event of an “incident.” Furthermore, suppose that a meltdown will be prevented if at least half the rods perform satisfactorily. What is the probability that, upon demand, the system will fail?

3.2.3. In 2009 a donor who insisted on anonymity gave seven-figure donations to twelve universities. A media report of this generous but somewhat mysterious act

108 Chapter 3 Random Variables

identified that all of the universities awarded had fe- male presidents. It went on to say that with about 23% of U.S. college presidents being women, the probabil- ity of a dozen randomly selected institutions having fe- male presidents is about 1/50,000,000. Is this probability approximately correct?

3.2.4. An entrepreneur owns six corporations, each with more than $10 million in assets. The entrepreneur con- sults the U.S. Internal Revenue Data Book and discovers that the IRS audits 15.3% of businesses of that size. What is the probability that two or more of these businesses will be audited?

3.2.5. The probability is 0.10 that ball bearings in a machine component will fail under certain adverse con- ditions of load and temperature. If a component contain- ing eleven ball bearings must have at least eight of them functioning to operate under the adverse conditions, what is the probability that it will break down?

3.2.6. Suppose that since the early 1950s some ten-thousand independent UFO sightings have been re- ported to civil authorities. If the probability that any sighting is genuine is on the order of one in one hun- dred thousand, what is the probability that at least one of the ten-thousand was genuine?

3.2.7. Doomsday Airlines (“Come Take the Flight of Your Life”) has two dilapidated airplanes, one with two engines, and the other with four. Each plane will land safely only if at least half of its engines are working. Each engine on each aircraft operates independently and each has probability p = 0.4 of failing. Assuming you wish to maximize your survival probability, which plane should you fly on?

3.2.8. Two lighting systems are being proposed for an employee work area. One requires fifty bulbs, each hav- ing a probability of 0.05 of burning out within a month’s time. The second has one hundred bulbs, each with a 0.02 burnout probability. Whichever system is installed will be inspected once a month for the purpose of replac- ing burned-out bulbs. Which system is likely to require less maintenance? Answer the question by comparing the probabilities that each will require at least one bulb to be replaced at the end of thirty days.

3.2.9. The great English diarist Samuel Pepys asked his friend Sir Isaac Newton the following question: Is it more likely to get at least one 6 when six dice are rolled, at least two 6’s when twelve dice are rolled, or at least three 6’s when eighteen dice are rolled? After considerable cor- respondence [see (167)], Newton convinced the skeptical Pepys that the first event is the most likely. Compute the three probabilities.

3.2.10. The gunner on a small assault boat fires six mis- siles at an attacking plane. Each has a 20% chance of being on-target. If two or more of the shells find their mark, the

plane will crash. At the same time, the pilot of the plane fires ten air-to-surface rockets, each of which has a 0.05 chance of critically disabling the boat. Would you rather be on the plane or the boat?

3.2.11. If a family has four children, is it more likely they will have two boys and two girls or three of one sex and one of the other? Assume that the probability of a child being a boy is 12 and that the births are independent events.

3.2.12. Experience has shown that only 13 of all patients having a certain disease will recover if given the standard treatment. A new drug is to be tested on a group of twelve volunteers. If the FDA requires that at least seven of these patients recover before it will license the new drug, what is the probability that the treatment will be discredited even if it has the potential to increase an individual’s recovery rate to 12 ?

3.2.13. Transportation to school for a rural county’s seventy-six children is provided by a fleet of four buses. Drivers are chosen on a day-to-day basis and come from a pool of local farmers who have agreed to be “on call.” What is the smallest number of drivers who need to be in the pool if the county wants to have at least a 95% proba- bility on any given day that all the buses will run? Assume that each driver has an 80% chance of being available if contacted.

3.2.14. The captain of a Navy gunboat orders a volley of twenty-five missiles to be fired at random along a five-hundred-foot stretch of shoreline that he hopes to establish as a beachhead. Dug into the beach is a thirty- foot-long bunker serving as the enemy’s first line of de- fense. The captain has reason to believe that the bunker will be destroyed if at least three of the missiles are on- target. What is the probability of that happening?

3.2.15. A computer has generated seven random numbers over the interval 0 to 1. Is it more likely that (a) exactly three will be in the interval 12 to 1 or (b) fewer than three will be greater than 34 ?

3.2.16. Listed in the following table is the length distribu- tion of World Series competition for the sixty-four series from 1950 to 2014 (there was no series in 1994).

World Series Lengths

Number of Games, k Number of Years

4 13 5 11 6 14 7 26

Data from: www.baseball-almanac.com

Assuming that each World Series game is an independent event and that the probability of either team’s winning

Section 3.2 Binomial and Hypergeometric Probabilities 109

any particular contest is 0.5, find the probability of each series length. How well does the model fit the data? (Compute the “expected” frequencies, that is, multiply the probability of a given-length series times 64).

3.2.17. Redo Example 3.2.4 assuming n = 12 and p = 0.3. 3.2.18. Suppose a series of n independent trials can end in one of three possible outcomes. Let k1 and k2 denote the number of trials that result in outcomes 1 and 2, re- spectively. Let p1 and p2 denote the probabilities associ- ated with outcomes 1 and 2. Generalize Theorem 3.2.1 to

deduce a formula for the probability of getting k1 and k2 occurrences of outcomes 1 and 2, respectively.

3.2.19. Repair calls for central air conditioners fall into three general categories: coolant leakage, compressor failure, and electrical malfunction. Experience has shown that the probabilities associated with the three are 0.5, 0.3, and 0.2, respectively. Suppose that a dispatcher has logged in ten service requests for tomorrow morning. Use the an- swer to Question 3.2.18 to calculate the probability that three of those ten will involve coolant leakage and five will be compressor failures.

THE HYPERGEOMETRIC DISTRIBUTION

The second ‘‘special’’ distribution that we want to look at formalizes the urn problems that frequented Chapter 2. Our solutions to those earlier problems tended to be enumerations in which we listed the entire set of possible samples, and then counted the ones that satisfied the event in question. The inefficiency and redundancy of that approach should now be painfully obvious. What we are seeking here is a general formula that can be applied to any and all such problems, much like the expression in Theorem 3.2.1 can handle the full range of questions arising from the binomial model.

In the binomial model, if p is a rational number, then the experiment can be cast as an urn model. Suppose that p = r/N, where r and N are positive integers with r < N. Consider an urn with r red balls and w white balls, where r + w = N. Draw a ball; note its color; return it to the urn; mix the urn; draw another ball. If we continue in this way for n trials, we have a Bernoulli experiment. Then the probability of drawing k red balls is binomial.

Now let us examine a variation of the above scheme. The urn is the same, but n balls are drawn from the urn simultaneously. We again count the number of red balls in the sample. The result is unordered sampling without replacement. The probabilities associated with the number of red balls drawn are known as hypergeometric, and not surprisingly, rely on combinations. We formalize this discussion in the following theorem.

Theorem 3.2.2

Suppose an urn contains r red chips and w white chips, where r + w = N. If n chips are drawn out at random, without replacement, and if k denotes the number of red chips selected, then

P(k red chips are chosen) = ( r

k

)( w

n−k )

(N n

) (3.2.1) where k varies over all the integers for which

( r k

) and

( w

n−k )

are defined. The proba- bilities appearing on the right-hand side of Equation 3.2.1 are known as the hyper- geometric distribution.

Proof Since this model concerns unordered selections, Theorem 2.6.3 applies. The number of ways to select a sample of size n from N elements is

( N n

) . Now,

ignoring the white balls, the number of ways to select k red balls from the r in the urn is

( r k

) . Similarly, the selection of white balls can be done in

( w

n−k )

ways.

The number of ways to choose the red and white balls together is (

r k

) · ( wn−k ) by the multiplication principle. Finally, the desired probability is

( r k

)( w

n−k )(

N n

) .

110 Chapter 3 Random Variables

Comment A third urn model is to draw the sample in order but without replace- ment. In this case, the probabilities are also hypergeometric (See Question 3.2.28).

Comment The name hypergeometric derives from a series introduced in 1769 by the Swiss mathematician and physicist Leonhard Euler:

1 + ab c

x + a(a + 1)b(b + 1) 2!c(c + 1) x

2 + a(a + 1)(a + 2)b(b + 1)(b + 2) 3!c(c + 1)(c + 2) x

3 + · · ·

This is an expansion of considerable flexibility: Given appropriate values for a, b, and c, it reduces to many of the standard infinite series used in analysis. In particular, if a is set equal to 1, and b and c are set equal to each other, it reduces to the familiar geometric series,

1 + x + x2 + x3 + · · · hence the name hypergeometric. The relationship of the probability function in Theo- rem 3.2.2 to Euler’s series becomes apparent if we set a = −n, b = −r, c = w − n + 1, and multiply the series by

( w

n

) / (N

n

) . Then the coefficient of xk will be(r

k

)( w

n−k )

(N n

) the value the theorem gives for P(k red chips are chosen).

Example 3.2.5

A hung jury is one that is unable to reach a unanimous decision. Suppose that a pool of twenty-five potential jurors is assigned to a murder case where the evidence is so overwhelming against the defendant that twenty-three of the twenty-five would return a guilty verdict. The other two potential jurors would vote to acquit regardless of the facts. What is the probability that a twelve-member panel chosen at random from the pool of twenty-five will be unable to reach a unanimous decision?

Think of the jury pool as an urn containing twenty-five chips, twenty-three of which correspond to jurors who would vote “guilty” and two of which correspond to jurors who would vote “not guilty.” If either or both of the jurors who would vote “not guilty” are included in the panel of twelve, the result would be a hung jury. Applying Theorem 3.2.2 (twice) gives 0.74 as the probability that the jury impanelled would not reach a unanimous decision:

P(Hung jury) = P(Decision is not unanimous)

= (

2 1

)( 23 11

)/( 25 12

) + (

2 2

)( 23 10

)/( 25 12

) = 0.74

Example 3.2.6

The Florida Lottery features a number of games of chance, one of which is called Fantasy Five. The player chooses five numbers from a card containing the numbers 1 through 36. Each day five numbers are chosen at random, and if the player matches all five, the winnings can be as much as $200,000 for a $1 bet.

Lottery games like this one have spawned a mini-industry looking for biases in the selection of the winning numbers. Websites post various “analyses” claiming cer- tain numbers are “hot” and should be played. One such examination focused on the frequency of winning numbers between 1 and 12. The probability of such occurrences

Section 3.2 Binomial and Hypergeometric Probabilities 111

fits the hypergeometric distribution, where r = 12, w = 24, n = 5, and N = 36. For example, the probability that three of the five numbers are 12 or less is(

12 3

) ( 24 2

) (

36 5

) = 60,720 376,992

= 0.161

Notice how that compares to the observed proportion of drawings with exactly three numbers between 1 and 12. Of one leap year’s daily drawings—366 of them —there were sixty-five with three numbers 12 or less, giving a relative frequency of 65/366 = 0.178.

The full breakdown of observed and expected probabilities for winning numbers between 1 and 12 is given in Table 3.2.4.

Table 3.2.4

No. Drawn ≤ 12 Observed Proportion Hypergeometric Probability 0 0.128 0.113 1 0.372 0.338 2 0.279 0.354 3 0.178 0.161 4 0.038 0.032 5 0.005 0.002

Data from: www.flalottery.com/exptkt/ff.html

The naive or dishonest commentator might claim that the lottery “likes” num- bers ≤ 12 since the proportion of tickets drawn with three, four, or five numbers ≤ 12 is

0.178 + 0.038 + 0.005 = 0.221 This figure is in excess of the sum of the hypergeometric probabilities for k = 3,

4, and 5:

0.161 + 0.032 + 0.002 = 0.195 However, we shall see in Chapter 10 that such variation is well within the random fluctuations expected for truly random drawings. No bias can be inferred from these results.

Example 3.2.7

When a bullet is fired it becomes scored with minute striations produced by imper- fections in the gun barrel. Appearing as a series of parallel lines, these striations have long been recognized as a basis for matching a bullet with a gun, since repeated firings of the same weapon will produce bullets having substantially the same configuration of markings. Until recently, deciding how close two patterns had to be before it could be concluded the bullets came from the same weapon was largely subjective. A bal- listics expert would simply look at the two bullets under a microscope and make an informed judgment based on past experience. Today, however, criminologists are beginning to address the problem more quantitatively, partly with the help of the hypergeometric distribution.

Suppose a bullet is recovered from the scene of a crime, along with the suspect’s gun. Under a microscope, a grid of m cells, numbered 1 to m, is superimposed over the bullet. If m is chosen large enough that the width of the cells is sufficiently small, each of that evidence bullet’s ne striations will fall into a different cell (see Figure 3.2.1a). Then the suspect’s gun is fired, yielding a test bullet, which will have a total of nt striations located in a possibly different set of cells (see Figure 3.2.1b). How might we assess the similarities in cell locations for the two striation patterns?

112 Chapter 3 Random Variables

1 2 3 4 5 m

Striations (total of ne)

Evidence bullet

(a)

1 2 3 4 5 m

Striations (total of nt)

Test bullet

(b)

Figure 3.2.1

As a model for the striation pattern on the evidence bullet, imagine an urn con- taining m chips, with ne corresponding to the striation locations. Now, think of the striation pattern on the test bullet as representing a sample of size nt from the evi- dence urn. By Theorem 3.2.2, the probability that k of the cell locations will be shared by the two striation patterns is (ne

k

)(m−ne nt−k

) (m

nt

) Suppose the bullet found at a murder scene is superimposed with a grid having

m = 25 cells, ne = 4 of which contain striations. The suspect’s gun is fired and the bullet is found to have nt = 3 striations, one of which matches the location of one of the striations on the evidence bullet. What do you think a ballistics expert would conclude?

Intuitively, the similarity between the two bullets would be reflected in the prob- ability that one or more striations in the suspect’s bullet match the evidence bullet. The smaller that probability is, the stronger would be our belief that the two bullets were fired by the same gun. Based on the values given for m, ne, and nt ,

P(one or more matches) = (4

1

)(21 2

) (25

3

) + (4

2

)(21 1

) (25

3

) + (4

3

)(21 0

) (25

3

) = 0.42

If P(one or more matches) had been a very small number—say, 0.001—the infer- ence would have been clear-cut: The same gun fired both bullets. But, here with the probability of one or more matches being so large, we cannot rule out the possibility that the bullets were fired by two different guns (and, presumably, by two different people).

Example 3.2.8

A tax collector, finding himself short of funds, delayed depositing a large property tax payment ten different times. The money was subsequently repaid, and the whole amount deposited in the proper account. The tip-off to this behavior was the delay of the deposit. During the period of these irregularities, there was a total of 470 tax collections.

An auditing firm was preparing to do a routine annual audit of these transactions. They decided to randomly sample nineteen of the collections (approximately 4%) of the payments. The auditors would assume a pattern of malfeasance only if they

Section 3.2 Binomial and Hypergeometric Probabilities 113

saw three or more irregularities. What is the probability that three or more of the delayed deposits would be chosen in this sample?

This kind of audit sampling can be considered a hypergeometric experiment. Here, N = 470, n = 19, r = 10, and w = 460. In this case it is better to calculate the desired probability via the complement—that is,

1 − (

10 0

) ( 460 19

) (

470 19

) − (

10 1

) ( 460 18

) (

470 19

) − (

10 2

) ( 460 17

) (

470 19

) The calculation of the first hypergeometric term is(

10 0

) ( 460 19

) (

470 19

) = 1 · 460! 19!441!

· 19!451 470!

= 451 470

· 450 469

· . . . · 442 461

= 0.6592

To compute hypergeometric probabilities where the numbers are large, a useful device is a recursion formula. To that end, note that the ratio of the k + 1 term to the k term is ( r

k+1 ) (

w

n−k−1 )(

N n

) ÷ ( r

k

) ( w

n−k )(

N n

) = n − k k + 1 ·

r − k w − n + k + 1

(See Question 3.2.30.) Therefore,(

10 1

) ( 460 18

) (

470 19

) = 0.6592 · 19 + 0 1 + 0 ·

10 − 0 460 − 19 + 0 + 1 = 0.2834

and ( 10 2

) ( 460 17

) (

470 19

) = 0.2834 · 19 − 1 1 + 1 ·

10 − 1 460 − 19 + 1 + 1 = 0.0518

The desired probability, then, is 1 − 0.6592 − 0.2834 − 0.0518 = 0.0056, which shows that a larger audit sample would be necessary to have a reasonable chance of detect- ing this sort of impropriety.

CASE STUDY 3.2.1

Biting into a plump, juicy apple is one of the innocent pleasures of autumn. Crit- ical to that enjoyment is the firmness of the apple, a property that growers and shippers monitor closely. The apple industry goes so far as to set a lowest ac- ceptable limit for firmness, which is measured (in lbs) by inserting a probe into the apple. For the Red Delicious variety, for example, firmness is supposed to be at least 12 lbs; in the state of Washington, wholesalers are not allowed to sell apples if more than 10% of their shipment falls below that 12-lb limit.

All of this raises an obvious question: How can shippers demonstrate that their apples meet the 10% standard? Testing each one is not an option—the probe that measures firmness renders an apple unfit for sale. That leaves sam- pling as the only viable strategy.

(Continued on next page)

114 Chapter 3 Random Variables

(Case Study 3.2.1 continued)

Suppose, for example, a shipper has a supply of one hundred forty-four ap- ples. She decides to select fifteen at random and measure each one’s firmness, with the intention of selling the remaining apples if two or fewer in the sample are substandard. What are the consequences of her plan? More specifically, does it have a good chance of “accepting” a shipment that meets the 10% rule and “rejecting” one that does not? (If either or both of those objectives are not met, the plan is inappropriate.)

For example, suppose there are actually ten defective apples among the orig- inal one hundred forty-four. Since 10144 × 100 = 6.9%, that shipment would be suitable for sale because fewer than 10% failed to meet the firmness standard. The question is, how likely is it that a sample of fifteen chosen at random from that shipment will pass inspection?

Notice, here, that the number of substandard apples in the sample has a hy- pergeometric distribution with r = 10, w = 134, n = 15, and N = 144. Therefore,

P(Sample passes inspection) = P(Two or fewer substandard apples are found)

= (10

0

)(134 15

) (144

15

) + (10

1

)(134 14

) (144

15

) + (10

2

)(134 13

) (144

15

) = 0.320 + 0.401 + 0.208 = 0.929

So, the probability is reassuringly high that a supply of apples this good would, in fact, be judged acceptable to ship. Of course, it also follows from this calcula- tion that roughly 7% of the time, the number of substandard apples found will be greater than two, in which case the apples would be (incorrectly) assumed to be unsuitable for sale (earning them an undeserved one-way ticket to the apple- sauce factory . . . ).

How good is the proposed sampling plan at recognizing apples that would, in fact, be inappropriate to ship? Suppose, for example, that 30, or 21%, of the one hundred forty-four apples would fall below the 12-lb limit. Ideally, the probability here that a sample passes inspection should be small. The number of substandard apples found in this case would be hypergeometric with r = 30, w = 114, n = 15, and N = 144, so

P(Sample passes inspection) = (30

0

)(114 15

) (144

15

) + (30

1

)(114 14

) (144

15

) + (30

2

)(114 13

) (144

15

) = 0.024 + 0.110 + 0.221 = 0.355

Here the bad news is that the sampling plan will allow a 21% defective supply to be shipped 36% of the time. The good news is that 64% of the time, the number of substandard apples in the sample will exceed two, meaning that the correct decision “not to ship” will be made.

Figure 3.2.2 shows P(Sample passes) plotted against the percentage of de- fectives in the entire supply. Graphs of this sort are called operating characteristic (or OC) curves: They summarize how a sampling plan will respond to all possible levels of quality.

Section 3.2 Binomial and Hypergeometric Probabilities 115

1

0.8

P (

Sa m

pl e

pa ss

es )

0.6

0.4

0.2

0 0 10 20 30 40 50 60

Presumed percent defective 70 80 90 100

Figure 3.2.2

Comment Every sampling plan invariably allows for two kinds of errors— rejecting shipments that should be accepted and accepting shipments that should be rejected. In practice, the probabilities of committing these errors can be manipulated by redefining the decision rule and/or changing the sample size. Some of these options will be explored in Chapter 6.

Questions

3.2.20. A corporate board contains twelve members. The board decides to create a five-person Committee to Hide Corporation Debt. Suppose four members of the board are accountants. What is the probability that the Committee will contain two accountants and three nonaccountants?

3.2.21. One of the popular tourist attractions in Alaska is watching black bears catch salmon swimming upstream to spawn. Not all “black” bears are black, though—some are tan-colored. Suppose that six black bears and three tan- colored bears are working the rapids of a salmon stream. Over the course of an hour, six different bears are sighted. What is the probability that those six include at least twice as many black bears as tan-colored bears?

3.2.22. A city has 4050 children under the age of ten, in- cluding 514 who have not been vaccinated for measles. Sixty-five of the city’s children are enrolled in the ABC Day Care Center. Suppose the municipal health depart- ment sends a doctor and a nurse to ABC to immunize any child who has not already been vaccinated. Find a formula for the probability that exactly k of the children at ABC have not been vaccinated.

3.2.23. Country A inadvertently launches ten guided missiles—six armed with nuclear warheads—at Country B. In response, Country B fires seven antiballis- tic missiles, each of which will destroy exactly one of the incoming rockets. The antiballistic missiles have no way of detecting, though, which of the ten rockets are carrying nuclear warheads. What are the chances that Country B will be hit by at least one nuclear missile?

3.2.24. Anne is studying for a history exam covering the French Revolution that will consist of five essay ques- tions selected at random from a list of ten the professor

has handed out to the class in advance. Not exactly a Napoleon buff, Anne would like to avoid researching all ten questions but still be reasonably assured of getting a fairly good grade. Specifically, she wants to have at least an 85% chance of getting at least four of the five ques- tions right. Will it be sufficient if she studies eight of the ten questions?

3.2.25. Each year a college awards five merit-based schol- arships to members of the entering freshman class who have exceptional high school records. The initial pool of applicants for the upcoming academic year has been reduced to a “short list” of eight men and ten women, all of whom seem equally deserving. If the awards are made at random from among the eighteen finalists, what are the chances that both men and women will be represented?

3.2.26. Keno is a casino game in which the player has a card with the numbers 1 through 80 on it. The player selects a set of k numbers from the card, where k can range from one to fifteen. The “caller” announces twenty winning numbers, chosen at random from the eighty. The amount won depends on how many of the called numbers match those the player chose. Suppose the player picks ten numbers. What is the probability that among those ten are six winning numbers?

3.2.27. A display case contains thirty-five gems, of which ten are real diamonds and twenty-five are fake diamonds. A burglar removes four gems at random, one at a time and without replacement. What is the probability that the last gem she steals is the second real diamond in the set of four?

3.2.28. Consider an urn with r red balls and w white balls, where r + w = N. Draw n balls in order without

116 Chapter 3 Random Variables

replacement. Show that the probability of k red balls is hypergeometric.

3.2.29. Show directly that the set of probabilities associ- ated with the hypergeometric distribution sum to 1. (Hint: Expand the identity

(1 + μ)N = (1 + μ)r(1 + μ)N−r and equate coefficients.)

3.2.30. Show that the ratio of two successive hypergeo- metric probability terms satisfies the following equation,( r

k+1 ) (

w

n−k−1 )(

N n

) ÷ ( r

k

) ( w

n−k )(

N n

) = n − k k + 1 ·

r − k w − n + k + 1

for any k where both numerators are defined.

3.2.31. Urn I contains five red chips and four white chips; urn II contains four red and five white chips. Two chips are drawn simultaneously from urn I and placed into urn II. Then a single chip is drawn from urn II. What is the proba- bility that the chip drawn from urn II is white? (Hint: Use Theorem 2.4.1.)

3.2.32. As the owner of a chain of sporting goods stores, you have just been offered a “deal” on a shipment of one hundred robot table tennis machines. The price is right, but the prospect of picking up the merchandise at mid- night from an unmarked van parked on the side of the New Jersey Turnpike is a bit disconcerting. Being of low repute yourself, you do not consider the legality of the transaction to be an issue, but you do have concerns about being cheated. If too many of the machines are in poor working order, the offer ceases to be a bargain. Sup- pose you decide to close the deal only if a sample of ten

machines contains no more than one defective. Construct the corresponding operating characteristic curve. For ap- proximately what incoming quality will you accept a ship- ment 50% of the time?

3.2.33. Suppose that r of N chips are red. Divide the chips into three groups of sizes n1, n2, and n3, where n1 + n2 + n3 = N. Generalize the hypergeometric distribution to find the probability that the first group contains r1 red chips, the second group r2 red chips, and the third group r3 red chips, where r1 + r2 + r3 = r. 3.2.34. Some nomadic tribes, when faced with a life- threatening contagious disease, try to improve their chances of survival by dispersing into smaller groups. Sup- pose a tribe of twenty-one people, of whom four are car- riers of the disease, split into three groups of seven each. What is the probability that at least one group is free of the disease? (Hint: Find the probability of the complement.)

3.2.35. Suppose a population contains n1 objects of one kind, n2 objects of a second kind, . . . , and nt objects of a tth kind, where n1 + n2 + · · · + nt = N. A sample of size n is drawn at random and without replacement. Deduce an expression for the probability of drawing k1 objects of the first kind, k2 objects of the second kind, . . . , and kt objects of the tth kind by generalizing Theorem 3.2.2.

3.2.36. Sixteen students—five freshmen, four sopho- mores, four juniors, and three seniors—have applied for membership in their school’s Communications Board, a group that oversees the college’s newspaper, literary mag- azine, and radio show. Eight positions are open. If the se- lection is done at random, what is the probability that each class gets two representatives? (Hint: Use the generalized hypergeometric model asked for in Question 3.2.35.)

3.3 Discrete Random Variables The binomial and hypergeometric distributions described in Section 3.2 are special cases of some important general concepts that we want to explore more fully in this section. Previously in Chapter 2, we studied in depth the situation where every point in a sample space is equally likely to occur (recall Section 2.6). The sample space of independent trials that ultimately led to the binomial distribution presented a quite different scenario: specifically, individual points in S had different probabilities. For example, if n = 4 and p = 13 , the probabilities assigned to the sample points (s, f, s, f ) and ( f, f, f, f ) are (1/3)2(2/3)2 = 481 and (2/3)4 = 1681 , respectively. Allowing for the possibility that different outcomes may have different probabilities will obviously broaden enormously the range of real-world problems that probability models can address.

How to assign probabilities to outcomes that are not binomial or hypergeomet- ric is one of the major questions investigated in this chapter. A second critical issue is the nature of the sample space itself and whether it makes sense to redefine the outcomes and create, in effect, an alternative sample space. Why we would want to do that has already come up in our discussion of independent trials. The “original” sample space in such cases is a set of ordered sequences, where the ith member of a

Section 3.3 Discrete Random Variables 117

sequence is either an “s” or an “ f ,” depending on whether the ith trial ended in suc- cess or failure, respectively. However, knowing which particular trials ended in suc- cess is typically less important than knowing the number that did (recall the medical researcher discussion on p. 102). That being the case, it often makes sense to replace each ordered sequence with the number of successes that sequence contains. Doing so collapses the original set of 2n ordered sequences (i.e., outcomes) in S to the set of n + 1 integers ranging from 0 to n. The probabilities assigned to those integers, of course, are given by the binomial formula in Theorem 3.2.1.

In general, a function that assigns numbers to outcomes is called a random vari- able. The purpose of such functions in practice is to define a new sample space whose outcomes speak more directly to the objectives of the experiment. That was the ratio- nale that ultimately motivated both the binomial and hypergeometric distributions.

The purpose of this section is to (1) outline the general conditions under which probabilities can be assigned to sample spaces and (2) explore the ways and means of redefining sample spaces through the use of random variables. The notation intro- duced in this section is especially important and will be used throughout the remain- der of the book.

ASSIGNING PROBABILITIES: THE DISCRETE CASE

We begin with the general problem of assigning probabilities to sample outcomes, the simplest version of which occurs when the number of points in S is either finite or countably infinite. The probability functions, p(s), that we are looking for in those cases satisfy the conditions in Definition 3.3.1.

Definition 3.3.1 Suppose that S is a finite or countably infinite sample space. Let p be a real- valued function defined for each element of S such that

a. 0 ≤ p(s) for each s ∈ S b. ∑ s∈S

p(s) = 1

Then p is said to be a discrete probability function.

Comment Once p(s) is defined for all s, it follows that the probability of any event A—that is, P(A)—is the sum of the probabilities of the outcomes comprising A:

P(A) = ∑ s∈A

p(s) (3.3.1)

Defined in this way, the function P(A) satisfies the probability axioms given in Sec- tion 2.3. The next several examples illustrate some of the specific forms that p(s) can have and how P(A) is calculated.

Example 3.3.1

Ace-six flats are a type of crooked dice where the cube is foreshortened in the one- six direction, the effect being that 1’s and 6’s are more likely to occur than any of the other four faces. Let p(s) denote the probability that the face showing is s. For many ace-six flats, the “cube” is asymmetric to the extent that p(1) = p(6) = 14 , while p(2) = p(3) = p(4) = p(5) = 18 . Notice that p(s) here qualifies as a discrete probability function because each p(s) is greater than or equal to 0 and the sum of p(s), over all s, is 1

[ = 2( 14)+ 4( 18)].

118 Chapter 3 Random Variables

Suppose A is the event that an even number occurs. It follows from Equa- tion 3.3.1 that P(A) = P(2) + P(4) + P(6) = 18 + 18 + 14 = 12 .

Comment If two ace-six flats are rolled, the probability of getting a sum equal to 7 is equal to 2p(1)p(6) + 2p(2)p(5) + 2p(3)p(4) = 2( 14)2 + 4( 18)2 = 316 . If two fair dice are rolled, the probability of getting a sum equal to 7 is 2p(1)p(6)+2p(2)p(5)+ 2p(3)p(4) = 6( 16)2 = 16 , which is less than 316 . Gamblers cheat with ace-six flats by switching back and forth between fair dice and ace-six flats, depending on whether or not they want a sum of 7 to be rolled.

Example 3.3.2

Suppose a fair coin is tossed until a head comes up for the first time. What are the chances of that happening on an odd-numbered toss?

Note that the sample space here is countably infinite and so is the set of outcomes making up the event whose probability we are trying to find. The P(A) that we are looking for, then, will be the sum of an infinite number of terms.

Let p(s) be the probability that the first head appears on the sth toss. Since the coin is presumed to be fair, p(1) = 12 . Furthermore, we would expect that half the time, when a tail appears, the next toss would be a head, so p(2) = 12 · 12 = 14 . In general, p(s) = ( 12)s, s = 1, 2, . . . .

Does p(s) satisfy the conditions stated in Definition 3.3.1? Yes. Clearly, p(s) ≥ 0 for all s. To see that the sum of the probabilities is 1, recall the formula for the sum of a geometric series: If 0 < r < 1,

∞∑ s=0

rs = 1 1 − r (3.3.2)

Applying Equation 3.3.2 to the sample space here confirms that P(S) = 1:

P(S) = ∞∑

s=1 p(s) =

∞∑ s=1

( 1 2

)s =

∞∑ s=0

( 1 2

)s − (

1 2

)0 = 1

/( 1 − 1

2

) − 1 = 1

Now, let A be the event that the first head appears on an odd-numbered toss. Then P(A) = p(1) + p(3) + p(5) + · · · But

p(1) + p(3) + p(5) + · · · = ∞∑

s=0 p(2s + 1) =

∞∑ s=0

( 1 2

)2s+1 = (

1 2

) ∞∑ s=0

( 1 4

)s

= (

1 2

)[ 1 /(

1 − 1 4

)] = 2

3

CASE STUDY 3.3.1

For good pedagogical reasons, the principles of probability are always intro- duced by considering events defined on familiar sample spaces generated by simple experiments. To that end, we toss coins, deal cards, roll dice, and draw chips from urns. It would be a serious error, though, to infer that the impor- tance of probability extends no further than the nearest casino. In its infancy,

Section 3.3 Discrete Random Variables 119

gambling and probability were, indeed, intimately related: Questions arising from games of chance were often the catalyst that motivated mathematicians to study random phenomena in earnest. But more than three hundred forty years have passed since Huygens published De Ratiociniis. Today, the application of probability to gambling is relatively insignificant (the NCAA March basketball tournament notwithstanding) compared to the depth and breadth of uses the subject finds in business, medicine, engineering, and science.

Probability functions—properly chosen—can “model” complex real-world phenomena every bit as well as P(heads) = 12 describes the behavior of a fair coin. The following set of actuarial data is a case in point. Over a period of three years (= 1096 days) in London, records showed that a total of nine hun- dred three deaths occurred among males eighty-five years of age and older (191). Columns 1 and 2 of Table 3.3.1 give the breakdown of those 903 deaths accord- ing to the number occurring on a given day. Column 3 gives the proportion of days for which exactly s elderly men died.

Table 3.3.1

(1) (2) (3) (4) Number of Deaths, s

Number of Days

Proportion [= Col.(2)/1096]

p(s)

0 484 0.442 0.440 1 391 0.357 0.361 2 164 0.150 0.148 3 45 0.041 0.040 4 11 0.010 0.008 5 1 0.001 0.003 6+ 0 0.000 0.000

1096 1 1

For reasons that we will go into at length in Chapter 4, the probability func- tion that describes the behavior of this particular phenomenon is

p(s) = P(s elderly men die on a given day)

= e −0.82(0.82)s

s! , s = 0, 1, 2, . . . (3.3.3)

How do we know that the p(s) in Equation 3.3.3 is an appropriate way to assign probabilities to the “experiment” of elderly men dying? Because it accurately predicts what happened. Column 4 of Table 3.3.1 shows p(s) evaluated for s = 0, 1, 2, . . . . To two decimal places, the agreement between the entries in Column 3 and Column 4 is perfect.

Example 3.3.3

Consider the following experiment: Every day for the next month you copy down each number that appears in the stories on the front pages of your hometown news- paper. Those numbers would necessarily be extremely diverse: One might be the age of a celebrity who had just died, another might report the interest rate currently paid on government Treasury bills, and still another might give the number of square feet of retail space recently added to a local shopping mall.

120 Chapter 3 Random Variables

Suppose you then calculated the proportion of those numbers whose leading digit was a 1, the proportion whose leading digit was a 2, and so on. What relationship would you expect those proportions to have? Would numbers starting with a 2, for example, occur as often as numbers starting with a 6?

Let p(s) denote the probability that the first significant digit of a “newspaper number” is s, s = 1, 2, . . . , 9. Our intuition is likely to tell us that the nine first digits should be equally probable—that is, p(1) = p(2) = · · · = p(9) = 19 . Given the diversity and the randomness of the numbers, there is no obvious reason why one digit should be more common than another. Our intuition, though, would be wrong— first digits are not equally likely. Indeed, they are not even close to being equally likely!

Credit for making this remarkable discovery goes to Simon Newcomb, a math- ematician who observed more than a hundred years ago that some portions of log- arithm tables are used more than others (85). Specifically, pages at the beginning of such tables are more dog-eared than pages at the end, suggesting that users have more occasion to look up logs of numbers starting with small digits than they do numbers starting with large digits.

Almost fifty years later, a physicist, Frank Benford, reexamined Newcomb’s claim in more detail and looked for a mathematical explanation. What is now known as Benford’s law asserts that the first digits of many different types of mea- surements, or combinations of measurements, often follow the discrete probability model:

p(s) = P(First significant digit is s) = log (

1 + 1 s

) , s = 1, 2, . . . , 9

Table 3.3.2 compares Benford’s law to the uniform assumption that p(s) = 19 , for all s. The differences are striking. According to Benford’s law, for example, 1’s are the most frequently occurring first digit, appearing 6.5 times (= 0.301/0.046) as often as 9’s.

Table 3.3.2

s “Uniform” Law Benford’s Law

1 0.111 0.301 2 0.111 0.176 3 0.111 0.125 4 0.111 0.097 5 0.111 0.079 6 0.111 0.067 7 0.111 0.058 8 0.111 0.051 9 0.111 0.046

Comment A key to why Benford’s law is true is the differences in proportional changes associated with each leading digit. To go from one thousand to two thou- sand, for example, represents a 100% increase; to go from eight thousand to nine thousand, on the other hand, is only a 12.5% increase. That would suggest that evo- lutionary phenomena such as stock prices would be more likely to start with 1’s and 2’s than with 8’s and 9’s—and they are. Still, the precise conditions under which p(s) = log (1 + 1s ) , s = 1, 2, . . . , 9, are not fully understood and remain a topic of research.

Section 3.3 Discrete Random Variables 121

Example 3.3.4

Is p(s) as defined below a discrete probability function? Why or why not?

p(s) = 1 1 + λ

( λ

1 + λ )s

, s = 0, 1, 2, . . . ; λ > 0

To qualify as a discrete probability function, a given p(s) needs to satisfy parts (a) and (b) of Definition 3.3.1. A simple inspection shows that part (a) is satisfied. Since λ > 0, p(s) is, in fact, greater than or equal to 0 for all s = 0, 1, 2, . . . . Part (b) is satisfied if the sum of all the probabilities defined on the outcomes in S is 1. But

∑ all s∈S

p(s) = ∞∑

s=0

1 1 + λ

( λ

1 + λ )s

= 1 1 + λ

( 1

1 − λ1+λ

) (why?)

= 1 1 + λ ·

1 + λ 1

= 1

The answer, then, is “yes”—p(s) = 11+λ (

λ 1+λ

)s, s = 0, 1, 2, . . . ; λ > 0 does qual- ify as a discrete probability function. Of course, whether it has any practical value depends on whether the set of values for p(s) actually do describe the behavior of real-world phenomena.

DEFINING “NEW” SAMPLE SPACES

We have seen how the function p(s) associates a probability with each outcome, s, in a sample space. Related is the key idea that outcomes can often be grouped or reconfigured in ways that may facilitate problem solving. Recall the sample space associated with a series of n independent trials, where each s is an ordered se- quence of successes and failures. The most relevant information in such outcomes is often the number of successes that occur, not a detailed listing of which trials ended in success and which ended in failure. That being the case, it makes sense to define a “new” sample space by grouping the original outcomes according to the number of successes they contained. The outcome ( f , f , . . . , f ), for example, had 0 successes. On the other hand, there were n outcomes that yielded 1 success— (s, f , f , . . . , f ), ( f , s, f , . . . , f ), . . . , and ( f , f , . . . , s). As we saw earlier in this chapter, that particular regrouping of outcomes ultimately led to the binomial distribution.

The function that replaces the outcome (s, f , f , . . . , f ) with the numerical value 1 is called a random variable. We conclude this section with a discussion of some of the concepts, terminology, and applications associated with random variables.

Definition 3.3.2 A function whose domain is a sample space S and whose values form a finite or countably infinite set of real numbers is called a discrete random variable. We denote random variables by uppercase letters, often X or Y .

122 Chapter 3 Random Variables

Example 3.3.5

Consider tossing two dice, an experiment for which the sample space is a set of or- dered pairs, S = {(i, j) | i = 1, 2, . . . , 6; j = 1, 2, . . . , 6}. For a variety of games ranging from Monopoly to craps, the sum of the numbers showing is what matters on a given turn. That being the case, the original sample space S of thirty-six ordered pairs would not provide a particularly convenient backdrop for discussing the rules of those games. It would be better to work directly with the sums. Of course, the eleven possible sums (from 2 to 12) are simply the different values of the random variable X , where X (i, j) = i + j.

Comment In the above example, suppose we define a random variable X1 that gives the result on the first die and a random variable X2 that gives the result on the second die. Then X = X1 + X2. Note how easily we could extend this idea to the toss of three dice, or ten dice. The ability to conveniently express complex events in terms of simpler ones is an advantage of the random variable concept that we will see playing out over and over again.

THE PROBABILITY DENSITY FUNCTION

We began this section discussing the function p(s), which assigns a probability to each outcome s in S. Now, having introduced the notion of a random variable X as a real-valued function defined on S, that is, X (s) = k, we need to find a mapping analogous to p(s) that assigns probabilities to the different values of k.

Definition 3.3.3 Associated with every discrete random variable X is a probability density func- tion (or pdf ), denoted pX (k), where

pX (k) = P({s ∈ S | X (s) = k}) Note that pX (k) = 0 for any k not in the range of X . For notational simplicity, we will usually delete all references to s and S and write pX (k) = P(X = k).

Comment We have already discussed at length two examples of the function pX (k). Recall the binomial distribution derived in Section 3.2. If we let the random vari- able X denote the number of successes in n independent trials, then Theorem 3.2.1 states that

P(X = k) = pX (k) = (

n k

) pk(1 − p)n−k, k = 0, 1, . . . , n

A similar result was given in that same section in connection with the hyper- geometric distribution. If a sample of size n is drawn without replacement from an urn containing r red chips and w white chips, and if we let the random vari- able X denote the number of red chips included in the sample, then (according to Theorem 3.2.2),

P(X = k) = pX (k) = (

r k

)( w

n − k )/(

r + w n

) where k ranges over the values for which the numerator terms are defined.

Section 3.3 Discrete Random Variables 123

Example 3.3.6

Consider again the rolling of two dice as described in Example 3.3.5. Let i and j denote the faces showing on the first and second die, respectively, and define the random variable X to be the sum of the two faces: X (i, j) = i + j. Find pX (k).

According to Definition 3.3.3, each value of pX (k) is the sum of the probabilities of the outcomes that get mapped by X onto the value k. For example,

P(X = 5) = pX (5) = P({s ∈ S | X (s) = 5}) = P[(1, 4), (4, 1), (2, 3), (3, 2)] = P(1, 4) + P(4, 1) + P(2, 3) + P(3, 2)

= 1 36

+ 1 36

+ 1 36

+ 1 36

= 4 36

assuming the dice are fair. Values of pX (k) for other k are calculated similarly. Table 3.3.3 shows the random variable’s entire pdf.

Table 3.3.3

k pX (k) k pX (k)

2 1/36 8 5/36 3 2/36 9 4/36 4 3/36 10 3/36 5 4/36 11 2/36 6 5/36 12 1/36 7 6/36

Example 3.3.7

Acme Industries typically produces three electric power generators a day; some pass the company’s quality-control inspection on their first try and are ready to be shipped; others need to be retooled. The probability of a generator needing further work is 0.05. If a generator is ready to ship, the firm earns a profit of $10,000. If it needs to be retooled, it ultimately costs the firm $2,000. Let X be the random vari- able quantifying the company’s daily profit. Find pX (k).

The underlying sample space here is a set of n = 3 independent trials, where p = P(Generator passes inspection) = 0.95. If the random variable X is to measure the company’s daily profit, then

X = $10,000 × (no. of generators passing inspection) − $2,000 × (no. of generators needing retooling)

For instance, X (s, f, s) = 2($10,000) − 1($2,000) = $18,000. Moreover, the random variable X equals $18,000 whenever the day’s output consists of two successes and one failure. That is, X (s, f, s) = X (s, s, f ) = X ( f, s, s). It follows that

P(X = $18,000) = pX (18,000) = (

3 2

) (0.95)2(0.05)1 = 0.135375

Table 3.3.4 shows pX (k) for the four possible values of k ($30,000, $18,000, $6,000, and −$6,000).

124 Chapter 3 Random Variables

Table 3.3.4

No. Defectives k = Profit pX (k) 0 $30,000 0.857375 1 $18,000 0.135375 2 $6,000 0.007125 3 −$6,000 0.000125

Example 3.3.8

As part of her warm-up drill, each player on State’s basketball team is required to shoot free throws until two baskets are made. If Rhonda has a 65% success rate at the foul line, what is the pdf of the random variable X that describes the number of throws it takes her to complete the drill? Assume that individual throws constitute independent events.

Figure 3.3.1 illustrates what must occur if the drill is to end on the kth toss, k = 2, 3, 4, . . . First, Rhonda needs to make exactly one basket sometime during the first k − 1 attempts, and, second, she needs to make a basket on the kth toss. Written formally,

pX (k) = P(X = k) = P(Drill ends on kth throw) = P[(1 basket and k − 2 misses in first k − 1 throws) ∩ (basket on kth throw)] = P(1 basket and k − 2 misses) · P(basket)

Exactly one basket Second basket

Miss 1

Basket 2

Miss 3

· · · Miss k − 1

Basket k

Attempts

Figure 3.3.1

Notice that k − 1 different sequences have the property that exactly one of the first k − 1 throws results in a basket:

k − 1 sequences

⎧⎪⎪⎪⎪⎪⎨ ⎪⎪⎪⎪⎪⎩

B 1

M 2

M 3

M 4 · · · Mk−1

M 1

B 2

M 3

M 4 · · · Mk−1

... M 1

M 2

M 3

M 4 · · · Bk−1

Since each sequence has probability (0.35)k−2(0.65),

P(1 basket and k − 2 misses) = (k − 1)(0.35)k−2(0.65) Therefore,

pX (k) = (k − 1)(0.35)k−2(0.65) · (0.65) = (k − 1)(0.35)k−2(0.65)2, k = 2, 3, 4, . . . (3.3.4)

Table 3.3.5 shows the pdf evaluated for specific values of k. Although the range of k is infinite, the bulk of the probability associated with X is concentrated in the values 2 through 7: It is highly unlikely, for example, that Rhonda would need more than seven shots to complete the drill.

Section 3.3 Discrete Random Variables 125

Table 3.3.5

k pX (k)

2 0.4225 3 0.2958 4 0.1553 5 0.0725 6 0.0317 7 0.0133 8+ 0.0089

THE CUMULATIVE DISTRIBUTION FUNCTION

In working with random variables, we frequently need to calculate the probability that the value of a random variable is somewhere between two numbers. For exam- ple, suppose we have an integer-valued random variable. We might want to calculate an expression like P(s ≤ X ≤ t). If we know the pdf for X , then

P(s ≤ X ≤ t) = t∑

k=s pX (k)

But depending on the nature of pX (k) and the number of terms that need to be added, calculating the sum of pX (k) from k = s to k = t may be quite difficult. An alternate strategy is to use the fact that

P(s ≤ X ≤ t) = P(X ≤ t) − P(X ≤ s − 1) where the two probabilities on the right represent cumulative probabilities of the random variable X . If the latter were available (and they often are), then evaluating P(s ≤ X ≤ t) by one simple subtraction would clearly be easier than doing all the calculations implicit in

t∑ k=s

pX (k).

Definition 3.3.4 Let X be a discrete random variable. For any real number t, the probability that X takes on a value ≤ t is the cumulative distribution function (cdf ) of X [written FX (t)]. In formal notation, FX (t) = P({s ∈ S | X (s) ≤ t}). As was the case with pdfs, references to s and S are typically deleted, and the cdf is written FX (t) = P(X ≤ t).

Example 3.3.9

Suppose we wish to compute P(21 ≤ X ≤ 40) for a binomial random variable X with n = 50 and p = 0.6. From Theorem 3.2.1, we know the formula for pX (k), so P(21 ≤ X ≤ 40) can be written as a simple, although computationally cumbersome, sum:

P(21 ≤ X ≤ 40) = 40∑

k=21

( 50 k

) (0.6)k(0.4)50−k

Equivalently, the probability we are looking for can be expressed as the difference between two cdfs:

P(21 ≤ X ≤ 40) = P(X ≤ 40) − P(X ≤ 20) = FX (40) − FX (20)

126 Chapter 3 Random Variables

As it turns out, values of the cdf for a binomial random variable are widely available, both in books and in computer software. Here, for example, FX (40) = 0.9992 and FX (20) = 0.0034, so

P(21 ≤ X ≤ 40) = 0.9992 − 0.0034 = 0.9958

Example 3.3.10

Suppose that two fair dice are rolled. Let the random variable X denote the larger of the two faces showing: (a) Find FX (t) for t = 1, 2, . . . , 6 and (b) find FX (2.5).

a. The sample space associated with the experiment of rolling two fair dice is the set of ordered pairs s = (i, j), where the face showing on the first die is i and the face showing on the second die is j. By assumption, all thirty-six possible outcomes are equally likely. Now, suppose t is some integer from 1 to 6, inclusive. Then

FX (t) = P(X ≤ t) = P[Max (i, j) ≤ t] = P(i ≤ t and j ≤ t) (why?) = P(i ≤ t) · P( j ≤ t) (why?)

= t 6

· t 6

= t 2

36 , t = 1, 2, 3, 4, 5, 6

b. Even though the random variable X has nonzero probability only for the inte- gers 1 through 6, the cdf is defined for any real number from −∞ to +∞. By definition, FX (2.5) = P(X ≤ 2.5). But

P(X ≤ 2.5) = P(X ≤ 2) + P(2 < X ≤ 2.5) = FX (2) + 0

so

FX (2.5) = FX (2) = 2 2

36 = 1

9 What would the graph of FX (t) as a function of t look like?

Questions

3.3.1. An urn contains five balls numbered 1 to 5. Two balls are drawn simultaneously. (a) Let X be the larger of the two numbers drawn. Find pX (k). (b) Let V be the sum of the two numbers drawn. Find pV (k).

3.3.2. Repeat Question 3.3.1 for the case where the two balls are drawn with replacement.

3.3.3. Suppose a fair die is tossed three times. Let X be the largest of the three faces that appear. Find pX (k).

3.3.4. Suppose a fair die is tossed three times. Let X be the number of different faces that appear (so X = 1, 2, or 3). Find pX (k).

3.3.5. A fair coin is tossed three times. Let X be the num- ber of heads in the tosses minus the number of tails. Find pX (k).

3.3.6. Suppose die one has spots 1, 2, 2, 3, 3, 4 and die two has spots 1, 3, 4, 5, 6, 8. If both dice are rolled, what is the sample space? Let X = total spots showing. Show that the pdf for X is the same as for normal dice.

3.3.7. Suppose a particle moves along the x-axis beginning at 0. It moves one integer step to the left or right with equal probability. What is the pdf of its position after four steps?

3.3.8. How would the pdf asked for in Question 3.3.7 be affected if the particle was twice as likely to move to the right as to the left?

Section 3.4 Continuous Random Variables 127

3.3.9. Suppose that five people, including you and a friend, line up at random. Let the random variable X de- note the number of people standing between you and your friend. What is pX (k)?

3.3.10. Urn I and urn II each have two red chips and two white chips. Two chips are drawn simultaneously from each urn. Let X1 be the number of red chips in the first sample and X2 the number of red chips in the second sam- ple. Find the pdf of X1 + X2. 3.3.11. Suppose X is a binomial random variable with n = 4 and p = 23 . What is the pdf of 2X + 1? 3.3.12. Find the cdf for the random variable X in Ques- tion 3.3.3.

3.3.13. A fair die is rolled four times. Let the random vari- able X denote the number of 6’s that appear. Find and graph the cdf for X .

3.3.14. At the points x = 0, 1, . . . , 6, the cdf for the discrete random variable X has the value FX (x) = x(x + 1)/42. Find the pdf for X . 3.3.15. Find the pdf for the infinite-valued discrete ran- dom variable X whose cdf at the points x = 1, 2, 3, . . . is given by FX (x) = 1 − (1 − p)x, where 0 < p < 1. 3.3.16. Recall the game of Fantasy Five from Example 3.2.6. For any Fantasy Five draw of five balls, let the ran- dom variable X be the largest number drawn.

(a) Find FX (k) (b) Find P(X = 36) (c) In one hundred eight plays of Fantasy Five, thirty- six was the largest number drawn fifteen times. Com- pare this observation to the theoretical probability in part (b).

3.4 Continuous Random Variables The statement was made in Chapter 2 that all sample spaces belong to one of two generic types—discrete sample spaces are ones that contain a finite or a countably infinite number of outcomes and continuous sample spaces are those that contain an uncountably infinite number of outcomes. Rolling a pair of dice and recording the faces that appear is an experiment with a discrete sample space; choosing a number at random from the interval [0, 1] would have a continuous sample space.

How we assign probabilities to these two types of sample spaces is different. Section 3.3 focused on discrete sample spaces. Each outcome s is assigned a probability by the discrete probability function p(s). If a random variable X is defined on the sample space, the probabilities associated with its outcomes are assigned by the probability density function pX (k). Applying those same definitions, though, to the outcomes in a continuous sample space will not work. The fact that a continuous sample space has an uncountably infinite number of outcomes eliminates the option of assigning a probability to each point as we did in the discrete case with the function p(s). We begin this section with a particular pdf defined on a discrete sample space that suggests how we might define probabilities, in general, on a continuous sample space.

Suppose an electronic surveillance monitor is turned on briefly at the beginning of every hour and has a 0.905 probability of working properly, regardless of how long it has remained in service. If we let the random variable X denote the hour at which the monitor first fails, then pX (k) is the product of k individual probabilities:

pX (k) = P(X = k) = P(Monitor fails for the first time at the kth hour) = P(Monitor functions properly for first k − 1 hours ∩ Monitor fails at the kth hour) = (0.905)k−1(0.095), k = 1, 2, 3, . . .

Figure 3.4.1 shows a probability histogram of pX (k) for k values ranging from 1 to 21. Here the height of the kth bar is pX (k), and since the width of each bar is 1, the area of the kth bar is also pX (k).

Now, look at Figure 3.4.2, where the exponential curve y = 0.1e−0.1x is superim- posed on the graph of pX (k). Notice how closely the area under the curve approxi- mates the area of the bars. It follows that the probability that X lies in some given interval will be numerically similar to the integral of the exponential curve above that same interval.

128 Chapter 3 Random Variables

0.1

0.09

0.08

0.07

0.06

0.05

0.04

0.03

0.02

0.01

1 0

2 3 4 5

Hour when monitor first fails, k

pX(k)

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Figure 3.4.1

0.1

0.09

0.08

0.07

0.06

0.05

0.04

0.03

0.02

0.01

1 0

2 3 4 5

Hour when monitor first fails, k

pX(k)

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

y = 0.1e–0.1x

Figure 3.4.2

For example, the probability that the monitor fails sometime during the first four hours would be the sum

P(1 ≤ X ≤ 4) = 4∑

k=1 pX (k)

= 4∑

k=1 (0.905)k−1(0.095)

= 0.3297 To four decimal places, the corresponding area under the exponential curve is the same:∫ 4

0 0.1e−0.1x dx = 0.3297

Implicit in the similarity here between pX (k) and the exponential curve y = 0.1e−0.1x is our sought-after alternative to p(s) for continuous sample spaces. Instead of defining probabilities for individual points, we will define probabilities for intervals of points, and those probabilities will be areas under the graph of some function (such

Section 3.4 Continuous Random Variables 129

as y = 0.1e−0.1x), where the shape of the function will reflect the desired probability “measure” to be associated with the sample space.

Definition 3.4.1 A probability function P on a set of real numbers S is called continuous if there exists a function f (t) such that for any closed interval [a, b] ⊂ S, P([a, b]) =∫ b

a f (t) dt. The function f (t) must have the following two properties:

a. f (t) ≥ 0 for all t. b.

∫ ∞ −∞ f (t)dt = 1.

Then the probability P(A) = ∫A f (t)dt for any set A where the integral is defined. Comment Using the −∞ and ∞ limits on the integral is simply a convention to mean the function should be integrated over its entire domain. The examples below will make this clear.

Comment If a probability function P satisfies Definition 3.4.1, then it will satisfy the probability axioms given in Section 2.3.

Comment Replacing a sum for discrete probability with an integral is not without its own logic. One of the founders of calculus, Gottfried Wilhelm Leibniz, regarded the integral as an infinite sum of infinitesimal summands. The integral sign was based on a version of the German letter long s (for the German Summe).

CHOOSING THE FUNCTION f(t)

We have seen that the probability structure of any sample space with a finite or count- ably infinite number of outcomes is defined by the function p(s) = P(Outcome is s). For sample spaces having an uncountably infinite number of possible outcomes, the function f (t) serves an analogous purpose. Specifically, f (t) defines the probability structure of S in the sense that the probability of any interval in the sample space is the integral of f (t). The next set of examples illustrate several different choices for f (t).

Example 3.4.1

The continuous equivalent of the equiprobable probability model on a discrete sam- ple space is the function f (t) defined by f (t) = 1/(b−a) for all t in the interval [a, b] (and f (t) = 0, otherwise). This particular f (t) places equal probability weighting on every closed interval of the same length contained in the interval [a, b]. For example, suppose a = 0 and b = 10, and let A = [1, 3] and B = [6, 8]. Then f (t) = 110 , and

P(A) = ∫ 3

1

( 1

10

) dt = 2

10 = P(B) =

∫ 8 6

( 1

10

) dt

(See Figure 3.4.3.)

0 1 3 6 8 10

A B

P(A) = P(B) =

f(t) =

t

1 10

2 10

2 10

1 10

Probability density

Figure 3.4.3

130 Chapter 3 Random Variables

Example 3.4.2

Could f (t) = 3t2, 0 ≤ t ≤ 1, be used to define the probability function for a con- tinuous sample space whose outcomes consist of all the real numbers in the interval [0, 1]? Yes, because (1) f (t) ≥ 0 for all t, and (2) ∫ 10 f (t) dt = ∫ 10 3t2 dt = t3∣∣10 = 1.

Notice that the shape of f (t) (see Figure 3.4.4) implies that outcomes close to 1 are more likely to occur than are outcomes close to 0. For example, P

([ 0, 13

]) =∫ 1/3 0 3t

2 dt = t3∣∣1/30 = 127 , while P ([ 23 , 1]) = ∫ 12/3 3t2 dt = t3∣∣12/3 = 1 − 827 = 1927 .

0

1

2

3

1 t

f(t) = 3t2

Area = Area = Probability

density 1 27

19 27

1 3

2 3

Figure 3.4.4

Example 3.4.3

By far the most important of all continuous probability functions is the “bell-shaped” curve, known more formally as the normal (or Gaussian) distribution. The sample space for the normal distribution is the entire real line; its probability function is given by

f (t) = 1√ 2πσ

exp

[ −1

2

( t − μ

σ

)2] , −∞ < t < ∞, −∞ < μ < ∞, σ > 0

Depending on the values assigned to the parameters μ and σ, f (t) can take on a variety of shapes and locations; three are illustrated in Figure 3.4.5.

m = –4 s = 0.5

m = 0 s = 1.5

m = 3 s = 1

0 t

f(t)

–4 3

Figure 3.4.5

FITTING f(t) TO DATA: THE DENSITY-SCALED HISTOGRAM

The notion of using a continuous probability function to approximate an integer- valued discrete probability model has already been discussed (recall Figure 3.4.2). The “trick” there was to replace the spikes that define pX (k) with rectangles whose heights are pX (k) and whose widths are 1. Doing that makes the sum of the areas of the rectangles corresponding to pX (k) equal to 1, which is the same as the total area under the approximating continuous probability function. Because of the equality of those two areas, it makes sense to superimpose (and compare) the “histogram” of pX (k) and the continuous probability function on the same set of axes.

Now, consider the related, but slightly more general, problem of using a con- tinuous probability function to model the distribution of a set of n measurements,

Section 3.4 Continuous Random Variables 131

y1, y2, . . . , yn. Following the approach taken in Figure 3.4.2, we would start by mak- ing a histogram of the n observations. The problem, though, is that the sum of the areas of the bars comprising that histogram would not necessarily equal 1.

As a case in point, Table 3.4.1 shows a set of forty observations. Grouping those yi’s into five classes, each of width 10, produces the distribution and histogram pictured in Figure 3.4.6. Furthermore, suppose we have reason to believe that these forty yi’s may be a random sample from a uniform probability function defined over the interval [20, 70], that is,

f (t) = 1 70 − 20 =

1 50

, 20 ≤ t ≤ 70

Table 3.4.1

33.8 62.6 42.3 62.9 32.9 58.9 60.8 49.1 42.6 59.8 41.6 54.5 40.5 30.3 22.4 25.0 59.2 67.5 64.1 59.3 24.9 22.3 69.7 41.2 64.5 33.4 39.0 53.1 21.6 46.0 28.1 68.7 27.6 57.6 54.8 48.9 68.4 38.4 69.0 46.6

Recall Example 3.4.1. How can we appropriately draw the distribution of the yi’s and the uniform probability model on the same graph?

Class Frequency

20 ≤ y < 30 7 30 ≤ y < 40 6 40 ≤ y < 50 9 50 ≤ y < 60 8 60 ≤ y < 70 10

40 0

4

8

12

3020 7050 6040

Frequency

y

Figure 3.4.6

Note, first, that f (t) and the histogram are not compatible in the sense that the area under f (t) is (necessarily) 1 (= 50 × 150 ), but the sum of the areas of the bars making up the histogram is 400:

histogram area = 10(7) + 10(6) + 10(9) + 10(8) + 10(10) = 400

Nevertheless, we can “force” the total area of the five bars to match the area under f (t) by redefining the scale of the vertical axis on the histogram. Specifically, fre- quency needs to be replaced with the analog of probability density, which would be the scale used on the vertical axis of any graph of f (t). Intuitively, the density asso- ciated with, say, the interval [20, 30) would be defined as the quotient

7 40 × 10

because integrating that constant over the interval [20, 30) would give 740 , and the latter does represent the estimated probability that an observation belongs to the interval [20, 30).

Figure 3.4.7 shows a histogram of the data in Table 3.4.1, where the height of each bar has been converted to a density, according to the formula

density (of a class) = class frequency total no. of observations × class width

132 Chapter 3 Random Variables

Superimposed is the uniform probability model, f (t) = 150 , 20 ≤ t ≤ 70. Scaled in this fashion, areas under both f (t) and the histogram are 1.

Class Density

20 ≤ y < 30 7/[40(10)] = 0.0175 30 ≤ y < 40 6/[40(10)] = 0.0150 40 ≤ y < 50 9/[40(10)] = 0.0225 50 ≤ y < 60 8/[40(10)] = 0.0200 60 ≤ y < 70 10/[40(10)] = 0.0250

0

0.01

0.02

0.03

3020 7050 6040

Density

Uniform probability function

y

Figure 3.4.7

In practice, density-scaled histograms offer a simple, but effective, format for examining the “fit” between a set of data and a presumed continuous model. We will use it often in the chapters ahead. Applied statisticians have especially embraced this particular graphical technique. Indeed, computer software packages that include Histograms on their menus routinely give users the choice of putting either frequency or density on the vertical axis.

CASE STUDY 3.4.1

Years ago, the V805 transmitter tube was standard equipment on many aircraft radar systems. Table 3.4.2 summarizes part of a reliability study done on the V805; listed are the lifetimes (in hrs) recorded for nine hundred three tubes (38). Grouped into intervals of width 80, the densities for the nine classes are shown in the last column.

Table 3.4.2

Lifetime (hrs) Number of Tubes Density

0–80 317 0.0044 80–160 230 0.0032

160–240 118 0.0016 240–320 93 0.0013 320–400 49 0.0007 400–480 33 0.0005 480–560 17 0.0002 560–700 26 0.0002 700+ 20 0.0002

903

Experience has shown that lifetimes of electrical equipment can often be nicely modeled by the exponential probability function,

f (t) = λe−λt , t > 0 where the value of λ (for reasons explained in Chapter 5) is set equal to the reciprocal of the average lifetime of the tubes in the sample. Can the distribution of these data also be described by the exponential model?

Section 3.4 Continuous Random Variables 133

One way to answer such a question is to superimpose the proposed model on a graph of the density-scaled histogram. The extent to which the two graphs are similar then becomes an obvious measure of the appropriateness of the model.

For these data, λ would be 0.0056. Figure 3.4.8 shows the function

f (t) = 0.0056e−0.0056t

0.004

0.001

80 240 400 560500 700 0

V805 lifetimes (hrs)

f(t) = 0.0056e–0.0056t

Shaded area = P (lifetime > 500)

Probability density 0.002

0.003

Figure 3.4.8

plotted on the same axes as the density-scaled histogram. Clearly, the agreement is excellent, and we would have no reservations about using areas under f (t) to estimate lifetime probabilities. How likely is it, for example, that a V805 tube will last longer than five hundred hrs? Based on the exponential model, that probability would be 0.0608:

P(V805 lifetime exceeds 500 hrs) = ∫ ∞

500 0.0056e−0.0056ydy

= −e−0.0056y∣∣∞500 = e−0.0056(500) = e−2.8 = 0.0608

CONTINUOUS PROBABILITY DENSITY FUNCTIONS

We saw in Section 3.3 how the introduction of discrete random variables facilitates the solution of certain problems. The same sort of function can also be defined on sample spaces with an uncountably infinite number of outcomes. Usually, the sample space is an interval of real numbers—finite or infinite. The notation and techniques for this type of random variable replace sums with integrals.

Definition 3.4.2 Let Y be a function from a sample space S to the real numbers. The function Y is called a continuous random variable if there exists a function fY (y) such that for any real numbers a and b with a < b

P(a ≤ Y ≤ b) = ∫ b

a fY (y)dy

134 Chapter 3 Random Variables

The function fY (y) is the probability density function (pdf ) for Y . As in the discrete case, the cumulative distribution function (cdf ) is defined

by

FY (y) = P(Y ≤ y)

The cdf in the continuous case is just an integral of fY (y), that is,

FY (y) = ∫ y

−∞ fY (t)dt

Let f (y) be an arbitrary real-valued function defined on some subset S of the real numbers. If f (y) satisfies Definition 3.4.1, then f (y) = fY (y) for all y, where the random variable Y is the identity mapping.

Example 3.4.4

We saw in Case Study 3.4.1 that lifetimes of V805 radar tubes can be nicely modeled by the exponential probability function

f (t) = 0.0056e−0.0056t , t > 0 To couch that statement in random variable notation would simply require that we define Y to be the life of a V805 radar tube. Then Y would be the identity mapping, and the pdf for the random variable Y would be the same as the probability function, f (t). That is, we would write

fY (y) = 0.0056e−0.0056y, y ≥ 0 Similarly, when we work with the bell-shaped normal distribution in later chapters, we will write the model in random variable notation as

fY (y) = 1√ 2πσ

e− 1 2 ( y−μσ )

2

, −∞ < y < ∞

Example 3.4.5

Suppose we would like a continuous random variable Y to “select” a number be- tween 0 and 1 in such a way that intervals near the middle of the range would be more likely to be represented than intervals near either 0 or 1. One pdf having that property is the function fY (y) = 6y(1−y), 0 ≤ y ≤ 1 (see Figure 3.4.9). Do we know for certain that the function pictured in Figure 3.4.9 is a “legitimate” pdf? Yes, be- cause fY (y) ≥ 0 for all y between 0 and 1, and

∫ 1 0 6y(1 − y) dy = 6[y2/2 − y3/3]

∣∣1 0 = 1.

Comment To simplify the way pdfs are written, it will be assumed that fY (y) = 0 for all y outside the range actually specified in the function’s definition. In Example 3.4.5,

0

1

1

1 y

fY(y) = 6y(1 – y)

Probability density

1 2

1 2

1 2

3 4

1 4

Figure 3.4.9

Section 3.4 Continuous Random Variables 135

for instance, the statement fY (y) = 6y(1 − y), 0 ≤ y ≤ 1, is to be interpreted as an abbreviation for

fY (y) =

⎧⎪⎨ ⎪⎩

0, y < 0 6y(1 − y), 0 ≤ y ≤ 1 0, y > 1

CONTINUOUS CUMULATIVE DISTRIBUTION FUNCTIONS

Associated with every random variable, discrete or continuous, is a cumulative dis- tribution function. For discrete random variables (recall Definition 3.3.4), the cdf is a nondecreasing step function, where the “jumps” occur at the values of t for which the pdf has positive probability. For continuous random variables, the cdf is a mono- tonically nondecreasing continuous function. In both cases, the cdf can be helpful in calculating the probability that a random variable takes on a value in a given interval. As we will see in later chapters, there are also several important relationships that hold for continuous cdfs and pdfs. One such relationship is cited in Theorem 3.4.1.

Definition 3.4.3 The cdf for a continuous random variable Y is an indefinite integral of its pdf:

FY (y) = ∫ y

−∞ fY (t) dt = P({s ∈ S | Y(s) ≤ y}) = P(Y ≤ y)

Theorem 3.4.1

Let FY (y) be the cdf of a continuous random variable Y. Then

d dy

FY (y) = fY (y)

Proof The statement of Theorem 3.4.1 follows immediately from the Fundamen- tal Theorem of Calculus.

Theorem 3.4.2

Let Y be a continuous random variable with cdf FY (y). Then a. P(Y > s) = 1 − FY (s) b. P(r < Y ≤ s) = FY (s) − FY (r) c. lim

y→∞ FY (y) = 1 d. lim

y→−∞ FY (y) = 0

Proof

a. P(Y > s) = 1 − P(Y ≤ s) since (Y > s) and (Y ≤ s) are complementary events. But P(Y ≤ s) = FY (s), and the conclusion follows.

b. Since the set (r < Y ≤ s) = (Y ≤ s) − (Y ≤ r), P(r < Y ≤ s) = P(Y ≤ s) − P(Y ≤ r) = FY (s) − FY (r).

c. Let {yn} be a set of values of Y, n = 1, 2, 3, . . . , where yn < yn+1 for all n, and lim

n→∞ yn = ∞. If limn→∞ FY (yn) = 1 for every such sequence {yn}, then limy→∞ FY (y) = 1. To that end, set A1 = (Y ≤ y1), and let An = (yn−1 < Y ≤ yn)

(Continued on next page)

136 Chapter 3 Random Variables

(Theorem 3.4.2 continued)

for n = 2, 3, . . . . Then FY (yn) = P(∪nk=1 Ak) = n∑

k=1 P(Ak), since the Ak’s

are disjoint. Also, the sample space S = ∪∞k=1 Ak, and by Axiom 4, 1 = P(S) = P(∪∞k=1 Ak) =

∞∑ k=1

P(Ak). Putting these equalities together gives

1 = ∞∑

k=1 P(Ak) = lim

n→∞

n∑ k=1

P(Ak) = lim n→∞ FY (yn).

d. lim y→−∞ FY (y) = limy→−∞ P(Y ≤ y) = limy→−∞ P(−Y ≥ −y) = limy→−∞[1 − P(−Y ≤ −y)]

= 1 − lim y→−∞ P(−Y ≤ −y) = 1 − limy→∞ P(−Y ≤ y)

= 1 − lim y→∞ F−Y (y) = 0

Questions

3.4.1. Suppose fY (y) = 4y3, 0 ≤ y ≤ 1. Find P ( 0 ≤

Y ≤ 12 ) .

3.4.2. For the random variable Y with pdf fY (y) = 23 + 2 3 y, 0 ≤ y ≤ 1, find P

( 3 4 ≤ Y ≤ 1

) .

3.4.3. Let fY (y) = 32 y2,−1 ≤ y ≤ 1. Find P (|Y − 12 | < 14 ).

Draw a graph of fY (y) and show the area representing the desired probability.

3.4.4. For persons infected with a certain form of malaria, the length of time spent in remission is described by the continuous pdf fY (y) = 19 y2, 0 ≤ y ≤ 3, where Y is measured in years. What is the probability that a malaria patient’s remission lasts longer than one year?

3.4.5. For a high-risk driver, the time in days between the beginning of a year and an accident has an exponential pdf. Suppose an insurance company believes the proba- bility that such a driver will be involved in an accident in the first forty days is 0.25. What is the probability that such a driver will be involved in an accident during the first seventy-five days of the year?

3.4.6. Let n be a positive integer. Show that fY (y) = (n + 2)(n + 1)yn(1 − y), 0 ≤ y ≤ 1, is a pdf. 3.4.7. Find the cdf for the random variable Y given in Question 3.4.1. Calculate P

( 0 ≤ Y ≤ 12

) using FY (y).

3.4.8. If Y is an exponential random variable, fY (y) = λe−λy, y ≥ 0, find FY (y). 3.4.9. If the pdf for Y is

fY (y) = {

0, |y| > 1 1 − |y|, |y| ≤ 1

find and graph FY (y).

3.4.10. A continuous random variableY has a cdf given by

FY (y) = ⎧⎨ ⎩

0 y < 0 y2 0 ≤ y < 1 1 y ≥ 1

Find P ( 1

2 < Y ≤ 34 )

two ways—first, by using the cdf and second, by using the pdf.

3.4.11. A random variable Y has cdf

FY (y) = ⎧⎨ ⎩

0 y < 1 ln y 1 ≤ y ≤ e 1 e < y

Find (a) P(Y < 2) (b) P

( 2 < Y ≤ 2 12

) (c) P

( 2 < Y < 2 12

) (d) fY (y)

3.4.12. The cdf for a random variable Y is defined by FY (y) = 0 for y < 0; FY (y) = 4y3 − 3y4 for 0 ≤ y ≤ 1; and FY (y) = 1 for y > 1. Find P

( 1 4 ≤ Y ≤ 34

) by integrat-

ing fY (y).

3.4.13. Suppose FY (y) = 112 (y2 + y3), 0 ≤ y ≤ 2. Find fY (y).

3.4.14. In a certain country, the distribution of a family’s disposable income, Y , is described by the pdf fY (y) = ye−y, y ≥ 0. Find FY (y). 3.4.15. The logistic curve F (y) = 11+e−y ,−∞ < y < ∞, can represent a cdf since it is increasing, lim

y→−∞ 1

1+e−y = 0, and lim

y→+∞ 1

1+e−y = 1. Verify these three assertions and also find the associated pdf.

Section 3.5 Expected Values 137

3.4.16. Let Y be the random variable described in Ques- tion 3.4.1. Define W = 2Y + 1. Find fW (w). For which values of w is fW (w) �= 0? 3.4.17. Suppose that fY (y) is a continuous and symmetric pdf, where symmetry is the property that fY (y) = fY (−y) for all y. Show that P(−a ≤ Y ≤ a) = 2FY (a) − 1. 3.4.18. Let Y be a random variable denoting the age at which a piece of equipment fails. In reliability theory, the

probability that an item fails at time y given that it has survived until time y is called the hazard rate, h(y). In terms of the pdf and cdf,

h(y) = fY (y) 1 − FY (y)

Find h(y) if Y has an exponential pdf (see Question 3.4.8).

3.5 Expected Values Probability density functions, as we have already seen, provide a global overview of a random variable’s behavior. If X is discrete, pX (k) gives P(X = k) for all k; if Y is continuous, and A is any interval or a countable union of intervals, P(Y � A) =∫

A fY (y) dy. Detail that explicit, though, is not always necessary—or even helpful. There are times when a more prudent strategy is to focus the information contained in a pdf by summarizing certain of its features with single numbers.

The first such feature that we will examine is central tendency, a term referring to the “average” value of a random variable. Consider the pdfs pX (k) and fY (y) pictured in Figure 3.5.1. Although we obviously cannot predict with certainty what values any future X ’s and Y ’s will take on, it seems clear that X values will tend to lie somewhere near μX , and Y values somewhere near μY . In some sense, then, we can characterize pX (k) by μX , and fY (y) by μY .

mX my

pX (k) fY (y)

Figure 3.5.1

The most frequently used measure for describing central tendency—that is, for quantifying μX and μY —is the expected value. Discussed at some length in this section and in Section 3.9, the expected value of a random variable is a slightly more abstract formulation of what we are already familiar with in simple discrete settings as the arithmetic average. Here, though, the values included in the average are “weighted” by the pdf.

Gambling affords a familiar illustration of the notion of an expected value. Con- sider the game of roulette. After bets are placed, the croupier spins the wheel and declares one of thirty-eight numbers, 00, 0, 1, 2, . . ., 36, to be the winner. Disregarding what seems to be a perverse tendency of many roulette wheels to land on numbers for which no money has been wagered, we will assume that each of these thirty-eight numbers is equally likely (although only the eighteen numbers 1, 3, 5, . . ., 35 are con- sidered to be odd and only the eighteen numbers 2, 6, 4, . . ., 36 are considered to be even). Suppose that our particular bet (at “even money”) is $1 on odds. If the ran- dom variable X denotes our winnings, then X takes on the value 1 if an odd number occurs, and −1 otherwise. Therefore,

pX (1) = P(X = 1) = 1838 = 9 19

138 Chapter 3 Random Variables

and

pX (−1) = P(X = −1) = 2038 = 10 19

Then 919 of the time we will win $1 and 10 19 of the time we will lose $1. Intuitively, then,

if we persist in this foolishness, we stand to lose, on the average, a little more than 5¢ each time we play the game:

“expected” winnings = $1 · 9 19

+ (−$1) · 10 19

= −$0.053 .= −5¢

The number −0.053 is called the expected value of X . Physically, an expected value can be thought of as a center of gravity. Here, for

example, imagine two bars of height 1019 and 9 19 positioned along a weightless X -axis

at the points −1 and +1, respectively (see Figure 3.5.2). If a fulcrum were placed at the point −0.053, the system would be in balance, implying that we can think of that point as marking off the center of the random variable’s distribution.

10 19

9 19

–1 0

–0.053

1

Figure 3.5.2

If X is a discrete random variable taking on each of its values with the same probability, the expected value of X is simply the everyday notion of an arithmetic average or mean:

expected value of X = ∑ all k

k · 1 n

= 1 n

∑ all k

k

Extending this idea to a discrete X described by an arbitrary pdf, pX (k), gives

expected value of X = ∑ all k

k · pX (k) (3.5.1)

For a continuous random variable Y , the summation in Equation 3.5.1 is replaced by an integration and k · pX (k) becomes y · fY (y).

Definition 3.5.1 Let X be a discrete random variable with probability function pX (k). The expected value of X is denoted E(X ) (or sometimes μ or μX ) and is given by

E(X ) = μ = μX = ∑ all k

k · pX (k)

Similarly, if Y is a continuous random variable with pdf fY (y),

E(Y) = μ = μY = ∫ ∞

−∞ y · fY (y) dy

Section 3.5 Expected Values 139

Comment We assume that both the sum and the integral in Definition 3.5.1 con- verge absolutely:

∑ all k

|k|pX (k) < ∞ ∫ ∞

−∞ |y| fY (y) dy < ∞

If not, we say that the random variable has no finite expected value. One immediate reason for requiring absolute convergence is that a convergent sum that is not ab- solutely convergent depends on the order in which the terms are added, and order should obviously not be a consideration when defining an average.

Example 3.5.1

Suppose X is a binomial random variable with p = 59 and n = 3. Then pX (k) = P(X = k) = (3k)( 59)k( 49)3−k, k = 0, 1, 2, 3. What is the expected value of X?

Applying Definition 3.5.1 gives

E(X ) = 3∑

k=0 k ·

( 3 k

)( 5 9

)k (4 9

)3−k

= (0) (

64 729

) + (1)

( 240 729

) + (2)

( 300 729

) + (3)

( 125 729

) = 1215

729 = 5

3 = 3

( 5 9

)

Comment Notice that the expected value here reduces to five-thirds, which can be written as three times five-ninths, the latter two factors being n and p, respectively. As the next theorem proves, that relationship is not a coincidence.

Theorem 3.5.1

Suppose X is a binomial random variable with parameters n and p. Then E(X ) = np. Proof According to Definition 3.5.1, E(X ) for a binomial random variable is the sum

E(X ) = n∑

k=0 k · pX (k) =

n∑ k=0

k (

n k

) pk(1 − p)n−k

= n∑

k=0

k · n! k!(n − k)! p

k(1 − p)n−k

= n∑

k=1

n! (k − 1)!(n − k)! p

k(1 − p)n−k (3.5.2)

At this point, a trick is called for. If E(X ) = ∑ all k

g (k) can be factored in such a

way that E(X ) = h ∑ all k

pX ∗(k), where pX ∗(k) is the pdf for some random variable

X ∗, then E(X ) = h, since the sum of a pdf over its entire range is 1. Here, suppose that np is factored out of Equation 3.5.2. Then

E(X ) = np n∑

k=1

(n − 1)! (k − 1)!(n − k)! p

k−1(1 − p)n−k

= np n∑

k=1

( n − 1 k − 1

) pk−1(1 − p)n−k

(Continued on next page)

140 Chapter 3 Random Variables

(Theorem 3.5.1 continued)

Now, let j = k − 1. It follows that

E(X ) = np n−1∑ j=0

( n − 1

j

) pj(1 − p)n− j−1

Finally, letting m = n − 1 gives

E(X ) = np m∑

j=0

( m j

) pj(1 − p)m− j

and, since the value of the sum is 1 (why?),

E(X ) = np (3.5.3)

Comment The statement of Theorem 3.5.1 should come as no surprise. If a multiple-choice test, for example, has one hundred questions, each with five pos- sible answers, we would “expect” to get twenty correct, just by guessing. But if the random variable X denotes the number of correct answers (out of one hundred), 20 = E(X ) = 100( 15) = np.

Example 3.5.2

An urn contains nine chips, five red and four white. Three are drawn out at random without replacement. Let X denote the number of red chips in the sample. Find E(X ).

From Section 3.2, we recognize X to be a hypergeometric random variable, where

P(X = k) = pX (k) = (5

k

)( 4 3−k

) (9

3

) , k = 0, 1, 2, 3 Therefore,

E(X ) = 3∑

k=0 k ·

(5 k

)( 4 3−k

) (9

3

) = (0)

( 4 84

) + (1)

( 30 84

) + (2)

( 40 84

) + (3)

( 10 84

)

= 5 3

Comment As was true in Example 3.5.1, the value found here for E(X ) suggests a general formula—in this case, for the expected value of a hypergeometric random variable.

Theorem 3.5.2

Suppose X is a hypergeometric random variable with parameters r, w, and n. That is, suppose an urn contains r red balls and w white balls. A sample of size n is drawn simultaneously from the urn. Let X be the number of red balls in the sample. Then E(X ) = rnr+w . Proof See Question 3.5.25.

Section 3.5 Expected Values 141

Comment Let p represent the proportion of red balls in an urn, that is, p = rr+w . The formula, then, for the expected value of a hypergeometric random variable has the same structure as the formula for the expected value of a binomial random variable:

E(X ) = rn r + w = n

r r + w = np

Example 3.5.3

Among the more common versions of the “numbers” racket is a game called D.J., its name deriving from the fact that the winning ticket is determined from Dow Jones averages. Three sets of stocks are used: Industrials, Transportations, and Utilities. Traditionally, the three are quoted at two different times, 11 a.m. and noon. The last digits of the earlier quotation are arranged to form a three-digit number; the noon quotation generates a second three-digit number, formed the same way. Those two numbers are then added together and the last three digits of that sum become the winning pick. Figure 3.5.3 shows a set of quotations for which 906 would be declared the winner.

Industrials Transportation Utilities

845.6 1 375.2 7 110.6 3

11 A.M. quotation

Industrials Transportation Utilities

848.1 7 376.7 3 110.6 3

173 733

906 = Winning number

+

Noon quotation

=

Figure 3.5.3

The payoff in D.J. is 700 to 1. Suppose that we bet $5. How much do we stand to win, or lose, on the average?

Let p denote the probability of our number being the winner and let X denote our earnings. Then

X = {

$3500 with probability p −$5 with probability 1 − p

and

E(X ) = $3500 · p − $5 · (1 − p) Our intuition would suggest (and this time it would be correct!) that each of the possible winning numbers, 000 through 999, is equally likely. That being the case, p = 1/1000 and

E(X ) = $3500 · (

1 1000

) − $5 ·

( 999

1000

) = −$1.50

On the average, then, we lose $1.50 on a $5.00 bet.

Example 3.5.4

Suppose that fifty people are to be given a blood test to see who has a certain disease. The obvious laboratory procedure is to examine each person’s blood individually, meaning that fifty tests would eventually be run. An alternative strategy is to divide each person’s blood sample into two parts—say, A and B. All of the A’s would then be mixed together and treated as one sample. If that “pooled” sample proved to be

142 Chapter 3 Random Variables

negative for the disease, all fifty individuals must necessarily be free of the infection, and no further testing would need to be done. If the pooled sample gave a positive reading, of course, all fifty B samples would have to be analyzed separately. Under what conditions would it make sense for a laboratory to consider pooling the fifty samples?

In principle, the pooling strategy is preferable (i.e., more economical) if it can substantially reduce the number of tests that need to be performed. Whether or not it can do so depends ultimately on the probability p that a person is infected with the disease.

Let the random variable X denote the number of tests that will have to be per- formed if the samples are pooled. Clearly,

X = {

1 if none of the fifty is infected 51 if at least one of the fifty is infected

But

P(X = 1) = pX (1) = P(None of the fifty is infected) = (1 − p)50

(assuming independence), and

P(X = 51) = pX (51) = 1 − P(X = 1) = 1 − (1 − p)50

Therefore,

E(X ) = 1 · (1 − p)50 + 51 · [1 − (1 − p)50] Table 3.5.1 shows E(X ) as a function of p. As our intuition would suggest, the

pooling strategy becomes increasingly feasible as the prevalence of the disease di- minishes. If the chance of a person being infected is 1 in 1000, for example, the pooling strategy requires an average of only 3.4 tests, a dramatic improvement over the fifty tests that would be needed if the samples were tested one by one. On the other hand, if 1 in 10 individuals is infected, pooling would be clearly inappropriate, requiring more than fifty tests [E(X ) = 50.7].

Table 3.5.1

p E(X)

0.5 51.0 0.1 50.7 0.01 20.8 0.001 3.4 0.0001 1.2

Example 3.5.5

Consider the following game. A fair coin is flipped until the first tail appears; we win $2 if it appears on the first toss, $4 if it appears on the second toss, and, in general, $2k if it first occurs on the kth toss. Let the random variable X denote our winnings. How much should we have to pay in order for this to be a fair game? (Note: A fair game is one where the difference between the ante and E(X ) is 0.)

Known as the St. Petersburg paradox, this problem has a rather unusual answer. First, note that

pX (2k) = P(X = 2k) = 12k , k = 1, 2, . . .

Section 3.5 Expected Values 143

Therefore,

E(X ) = ∑ all k

2k pX (2k) = ∞∑

k=1 2k · 1

2k = 1 + 1 + 1 + · · ·

which is a divergent sum. That is, X does not have a finite expected value, so in order for this game to be fair, our ante would have to be an infinite amount of money!

Comment Mathematicians have been trying to “explain” the St. Petersburg para- dox for almost two hundred years (61). The answer seems clearly absurd—no gam- bler would consider paying even $25 to play such a game, much less an infinite amount—yet the computations involved in showing that X has no finite expected value are unassailably correct. Where the difficulty lies, according to one common theory, is with our inability to put in perspective the very small probabilities of win- ning very large payoffs. Furthermore, the problem assumes that our opponent has infinite capital, which is an impossible state of affairs. We get a much more reason- able answer for E(X ) if the stipulation is added that our winnings can be at most, say, $1000 (see Question 3.5.19) or if the payoffs are assigned according to some formula other than 2k (see Question 3.5.20).

Comment There are two important lessons to be learned from the St. Petersburg paradox. First is the realization that E(X ) is not necessarily a meaningful charac- terization of the “location” of a distribution. Question 3.5.24 shows another situa- tion where the formal computation of E(X ) gives a similarly inappropriate answer. Second, we need to be aware that the notion of expected value is not necessarily synonymous with the concept of worth. Just because a game, for example, has a posi- tive expected value—even a very large positive expected value—does not imply that someone would want to play it. Suppose, for example, that you had the opportunity to spend your last $10,000 on a sweepstakes ticket where the prize was $1 billion but the probability of winning was only one in ten thousand. The expected value of such a bet would be over $90,000,

E(X ) = $1,000,000,000 (

1 10,000

) + (−$10,000)

( 9,999

10,000

) = $90,001

but it is doubtful that many people would rush out to buy a ticket. (Economists have long recognized the distinction between a payoff’s numerical value and its perceived desirability. They refer to the latter as utility.)

Example 3.5.6

The distance, Y , that a molecule in a gas travels before colliding with another molecule can be modeled by the exponential pdf

fY (y) = 1 μ

e−y/μ, y ≥ 0

where μ is a positive constant known as the mean free path. Find E(Y). Since the random variable here is continuous, its expected value is an integral:

E(Y) = ∫ ∞

0 y

1 μ

e−y/μ dy

144 Chapter 3 Random Variables

Let w = y/μ, so that dw = 1/μ dy. Then E(Y) = μ ∫∞0 we−wdw. Setting u = w and dv = e−wdw and integrating by parts gives

E(Y) = μ[−we−w − e−w]∣∣∞0 = μ (3.5.4) Equation 3.5.4 shows that μ is aptly named—it does, in fact, represent the av-

erage distance a molecule travels, free of any collisions. Nitrogen (N2), for example, at room temperature and standard atmospheric pressure has μ = 0.00005 cm. An N2 molecule, then, travels that far before colliding with another N2 molecule, on the average.

Example 3.5.7

One continuous pdf that has a number of interesting applications in physics is the Rayleigh distribution, where the pdf is given by

fY (y) = ya2 e −y2/2a2 , a > 0; 0 ≤ y < ∞ (3.5.5)

Calculate the expected value for a random variable having a Rayleigh distribution. From Definition 3.5.1,

E(Y) = ∫ ∞

0 y · y

a2 e−y

2/2a2 dy

Let v = y/(√2a). Then

E(Y) = 2 √

2a ∫ ∞

0 v2e−v

2 dv

The integrand here is a special case of the general form v2ke−v 2 . For k = 1,∫ ∞

0 v2ke−v

2 dv =

∫ ∞ 0

v2e−v 2 dv = 1

4

√ π

Therefore,

E(Y) = 2 √

2a · 1 4

√ π

= a √

π/2

Comment The pdf here is named for John William Strutt, Baron Rayleigh, the nineteenth- and twentieth-century British physicist who showed that Equation 3.5.5 is the solution to a problem arising in the study of wave motion. If two waves are superimposed, it is well known that the height of the resultant at any time t is sim- ply the algebraic sum of the corresponding heights of the waves being added (see Figure 3.5.4). Seeking to extend that notion, Rayleigh posed the following question: If n waves, each having the same amplitude h and the same wavelength, are super- imposed randomly with respect to phase, what can we say about the amplitude R of the resultant? Clearly, R is a random variable, its value depending on the particu- lar collection of phase angles represented by the sample. What Rayleigh was able to show in his 1880 paper (177) is that when n is large, the probabilistic behavior of R is described by the pdf

fR(r) = 2rnh2 · e −r2/nh2 , r > 0

which is just a special case of Equation 3.5.5 with a = √

2/nh2.

Section 3.5 Expected Values 145

Resultant

Wave 1

Wave 2

Figure 3.5.4

A SECOND MEASURE OF CENTRAL TENDENCY: THE MEDIAN

While the expected value is the most frequently used measure of a random variable’s central tendency, it does have a weakness that sometimes makes it misleading and inappropriate. Specifically, if one or several possible values of a random variable are either much smaller or much larger than all the others, the value of μ can be distorted in the sense that it no longer reflects the center of the distribution in any meaningful way. For example, suppose a small community consists of a homogeneous group of middle-range salary earners, and then Ms. Rich moves to town. Obviously, the town’s average salary before and after the multibillionaire arrives will be quite different, even though she represents only one new value of the “salary” random variable.

It would be helpful to have a measure of central tendency that is not so sensi- tive to “outliers” or to probability distributions that are markedly skewed. One such measure is the median, which, in effect, divides the area under a pdf into two equal areas.

Definition 3.5.2 If X is a discrete random variable, the median, m, is that point for which P(X < m) = P(X > m). In the event that P(X ≤ m) = 0.5 and P(X ≥ m′) = 0.5, the median is defined to be the arithmetic average, (m + m′)/2.

If Y is a continuous random variable, its median is the solution to the integral equation

∫ m −∞ fY (y) dy = 0.5.

Example 3.5.8

If a random variable’s pdf is symmetric, μ and m will be equal. Should pX (k) or fY (y) not be symmetric, though, the difference between the expected value and the median can be considerable, especially if the asymmetry takes the form of extreme skewness. The situation described here is a case in point.

Soft-glow makes a 60-watt light bulb that is advertised to have an average life of one thousand hours. Assuming that the performance claim is valid, is it reasonable for consumers to conclude that the Soft-glow bulbs they buy will last for approxi- mately one thousand hours?

No! If the average life of a bulb is one thousand hours, the (continuous) pdf, fY (y), modeling the length of time, Y , that it remains lit before burning out is likely to have the form

fY (y) = 0.001e−0.001y, y > 0 (3.5.6) (for reasons explained in Chapter 4). But Equation 3.5.6 is a very skewed pdf, having a shape much like the curve drawn in Figure 3.4.8. The median for such a distribution will lie considerably to the left of the mean.

146 Chapter 3 Random Variables

More specifically, the median lifetime for these bulbs—according to Defini- tion 3.5.2—is the value m for which∫ m

0 0.001e−0.001ydy = 0.5

But ∫ m

0 0.001e −0.001ydy = 1 − e−0.001m. Setting the latter equal to 0.5 implies that

m = (1/−0.001) ln(0.5) = 693 So, even though the average life of one of these bulbs is one thousand hours, there is a 50% chance that the one you buy will last less than six hundred ninety-three hours.

Questions

3.5.1. Recall the game of Keno described in Ques- tion 3.2.26. The following are all the payoffs on a $1 wa- ger where the player has bet on ten numbers. Calculate E(X ), where the random variable X denotes the amount of money won.

Number of Correct Guesses Payoff Probability

< 5 −$ 1 .935 5 2 .0514 6 18 .0115 7 180 .0016 8 1,300 1.35 × 10−4 9 2,600 6.12 × 10−6

10 10,000 1.12 × 10−7

3.5.2. The roulette wheels in Monte Carlo typically have a 0 but not a 00. What is the expected value of betting on red in this case? If a trip to Monte Carlo costs $3000, how much would a player have to bet to justify gambling there rather than Las Vegas?

3.5.3. The pdf describing the daily profit, X , earned by Acme Industries was derived in Example 3.3.7. Find the company’s average daily profit.

3.5.4. In the game of redball, two drawings are made with- out replacement from a bowl that has four white ping- pong balls and two red ping-pong balls. The amount won is determined by how many of the red balls are selected. For a $5 bet, a player can opt to be paid under either Rule A or Rule B, as shown. If you were playing the game, which would you choose? Why?

A B

No. of Red No. of Red Balls Drawn Payoff Balls Drawn Payoff

0 0 0 0 1 $2 1 $1 2 $10 2 $20

3.5.5. Suppose a life insurance company sells a $50,000, five-year term policy to a twenty-five-year-old woman. At the beginning of each year the woman is alive, the com- pany collects a premium of $P. The probability that the woman dies and the company pays the $50,000 is given in the table below. So, for example, in Year 3, the company loses $50,000 – $P with probability 0.00054 and gains $P with probability 1 − 0.00054 = 0.99946. If the company expects to make $1000 on this policy, what should P be?

Year Probability of Payoff

1 0.00051 2 0.00052 3 0.00054 4 0.00056 5 0.00059

3.5.6. A manufacturer has one hundred memory chips in stock, 4% of which are likely to be defective (based on past experience). A random sample of twenty chips is se- lected and shipped to a factory that assembles laptops. Let X denote the number of computers that receive faulty memory chips. Find E(X ). 3.5.7. Records show that 642 new students have just en- tered a certain Florida school district. Of those 642, a total of 125 are not adequately vaccinated. The district’s physi- cian has scheduled a day for students to receive what- ever shots they might need. On any given day, though, 12% of the district’s students are likely to be absent. How many new students, then, can be expected to remain inad- equately vaccinated?

3.5.8. Calculate E(Y) for the following pdfs: (a) fY (y) = 3(1 − y)2, 0 ≤ y ≤ 1 (b) fY (y) = 4ye−2y, y ≥ 0

(c) fY (y) =

⎧⎪⎪⎨ ⎪⎪⎩

3 4 , 0 ≤ y ≤ 1 1 4 , 2 ≤ y ≤ 3 0, elsewhere

(d) fY (y) = sin y, 0 ≤ y ≤ π2

Section 3.5 Expected Values 147

3.5.9. Recall Question 3.4.4, where the length of time Y (in years) that a malaria patient spends in remission has pdf fY (y) = 19 y2, 0 ≤ y ≤ 3. What is the average length of time that such a patient spends in remission?

3.5.10. Let the random variable Y have the uniform dis- tribution over [a, b]; that is, fY (y) = 1b−a for a ≤ y ≤ b. Find E(Y) using Definition 3.5.1. Also, deduce the value of E(Y), knowing that the expected value is the center of gravity of fY (y).

3.5.11. Show that the expected value associated with the exponential distribution, fY (y) =λe−λy, y > 0, is 1/λ, where λ is a positive constant.

3.5.12. Show that

fY (y) = 1y2 , y ≥ 1

is a valid pdf but that Y does not have a finite expected value.

3.5.13. Based on recent experience, ten-year-old passen- ger cars going through a motor vehicle inspection station have an 80% chance of passing the emissions test. Sup- pose that two hundred such cars will be checked out next week. Write two formulas that show the number of cars that are expected to pass.

3.5.14. Suppose that fifteen observations are chosen at random from the pdf fY (y) = 3y2, 0 ≤ y ≤ 1. Let X de- note the number that lie in the interval

( 1 2 , 1

) . Find E(X ).

3.5.15. A city has 74,806 registered automobiles. Each is required to display a bumper decal showing that the owner paid an annual wheel tax of $50. By law, new de- cals need to be purchased during the month of the owner’s birthday. How much wheel tax revenue can the city expect to receive in November?

3.5.16. Regulators have found that twenty-three of the sixty-eight investment companies that filed for bankruptcy in the past five years failed because of fraud, not for reasons related to the economy. Suppose that nine additional firms will be added to the bankruptcy rolls dur- ing the next quarter. How many of those failures are likely to be attributed to fraud?

3.5.17. An urn contains four chips numbered 1 through 4. Two are drawn without replacement. Let the random vari- able X denote the larger of the two. Find E(X ).

3.5.18. A fair coin is tossed three times. Let the random variable X denote the total number of heads that appear times the number of heads that appear on the first and third tosses. Find E(X ).

3.5.19. How much would you have to ante to make the St. Petersburg game “fair” (recall Example 3.5.5) if the

most you could win was $1000? That is, the payoffs are $2k for 1 ≤ k ≤ 9, and $1000 for k ≥ 10.

3.5.20. For the St. Petersburg problem (Example 3.5.5), find the expected payoff if (a) the amounts won are ck instead of 2k, where 0 < c < 2. (b) the amounts won are log 2k. [This was a modifi- cation suggested by D. Bernoulli (a nephew of James Bernoulli) to take into account the decreasing marginal utility of money—the more you have, the less useful a bit more is.]

3.5.21. A fair die is rolled three times. Let X de- note the number of different faces showing, X = 1, 2, 3. Find E(X ).

3.5.22. Two distinct integers are chosen at random from the first five positive integers. Compute the expected value of the absolute value of the difference of the two numbers.

3.5.23. Suppose that two evenly matched teams are play- ing in the World Series. On the average, how many games will be played? (The winner is the first team to get four victories.) Assume that each game is an independent event.

3.5.24. An urn contains one white chip and one black chip. A chip is drawn at random. If it is white, the “game” is over; if it is black, that chip and another black one are put into the urn. Then another chip is drawn at random from the “new” urn and the same rules for ending or con- tinuing the game are followed (i.e., if the chip is white, the game is over; if the chip is black, it is placed back in the urn, together with another chip of the same color). The drawings continue until a white chip is selected. Show that the expected number of drawings necessary to get a white chip is not finite.

3.5.25. A random sample of size n is drawn without re- placement from an urn containing r red chips and w white chips. Define the random variable X to be the number of red chips in the sample. Use the summation tech- nique described in Theorem 3.5.1 to prove that E(X ) = rn/(r + w).

3.5.26. Given that X is a nonnegative, integer-valued ran- dom variable, show that

E(X ) = ∞∑

k=1 P(X ≥ k)

3.5.27. Find the median for each of the following pdfs: (a) fY (y) = (θ + 1)yθ, 0 ≤ y ≤ 1, where θ > 0 (b) fY (y) = y + 12 , 0 ≤ y ≤ 1

148 Chapter 3 Random Variables

THE EXPECTED VALUE OF A FUNCTION OF A RANDOM VARIABLE

There are many situations that call for finding the expected value of a function of a random variable—say, Y = g (X ). One common example would be change of scale problems, where g (X ) = aX + b for constants a and b. Sometimes the pdf of the new random variable Y can be easily determined, in which case E(Y) can be cal- culated by simply applying Definition 3.5.1. Often, though, fY (y) can be difficult to derive, depending on the complexity of g (X ). Fortunately, Theorem 3.5.3 allows us to calculate the expected value of Y without knowing the pdf for Y .

Theorem 3.5.3

Suppose X is a discrete random variable with pdf pX (k). Let g (X ) be a function of X. Then the expected value of the random variable g (X ) is given by

E[g (X )] = ∑ all k

g (k) · pX (k)

provided that ∑ all k

|g (k)|pX (k) < ∞.

If Y is a continuous random variable with pdf fY (y), and if g (Y) is a continuous function, then the expected value of the random variable g(Y) is

E[g (Y)] = ∫ ∞

−∞ g (y) · fY (y) dy

provided that ∫∞ −∞ |g (y)| fY (y) dy < ∞.

Proof We will prove the result for the discrete case. See (155) for details showing how the argument is modified when the pdf is continuous. Let W = g (X ). The set of all possible k values, k1, k2, . . ., of X will give rise to a set of w values, w1,w2, . . . , where, in general, more than one k may be associated with a given w. Let Sj be the set of k’s for which g (k) = w j [so ∪ jS j is the entire set of k values for which pX (k) is defined]. We obviously have that P(W = w j) = P(X ∈ Sj), and we can write

E(W) = ∑

j

w j · P(W = w j) = ∑

j

w j · P(X ∈ Sj)

= ∑

j

w j ∑ k∈Sj

pX (k)

= ∑

j

∑ k∈Sj

w j · pX (k)

= ∑

j

∑ k∈Sj

g (k)pX (k) (why?)

= ∑ all k

g (k)pX (k)

Since it is being assumed that ∑ all k

|g (k)|pX (k) < ∞, the statement of the theorem holds.

Corollary 3.5.1

For any random variable W, E(aW +b) = aE(W)+b, where a and b are constants. Proof Suppose W is continuous; the proof for the discrete case is similar. By The- orem 3.5.3, E(aW + b) = ∫∞−∞(aw + b) fW (w) dw, but the latter can be written a ∫∞ −∞ w · fW (w) dw + b

∫∞ −∞ fW (w) dw = aE(W) + b · 1 = aE(W) + b.

Section 3.5 Expected Values 149

Example 3.5.9

Suppose that X is a random variable whose pdf is nonzero only for the three values −2, 1, and +2:

k pX (k)

−2 5 8

1 1 8

2 2 8 1

Let W = g (X ) = X 2. Verify the statement of Theorem 3.5.3 by computing E(W) two ways—first, by finding pW (w) and summing w · pW (w) over w and, second, by summing g (k) · pX (k) over k.

By inspection, the pdf for W is defined for only two values, 1 and 4:

w (= k2) pW (w)

1 1 8

4 7 8 1

Taking the first approach to find E(W) gives

E(W) = ∑ w

w · pW (w) = 1 · (

1 8

) + 4 ·

( 7 8

)

= 29 8

To find the expected value via Theorem 3.5.3, we take

E[g (X )] = ∑

k

k2 · pX (k) = (−2)2 · 58 + (1) 2 · 1

8 + (2)2 · 2

8

with the sum here reducing to the answer we already found, 298 . For this particular situation, neither approach was easier than the other. In gen-

eral, that will not be the case. Finding pW (w) is often quite difficult, and on those occasions Theorem 3.5.3 can be of great benefit.

Example 3.5.10

Suppose the amount of propellant, Y , put into a can of spray paint is a random vari- able with pdf

fY (y) = 3y2, 0 < y < 1 Experience has shown that the largest surface area that can be painted by a can having Y amount of propellant is twenty times the area of a circle generated by a radius of Y ft. If the Purple Dominoes, a newly formed urban gang, have just stolen

150 Chapter 3 Random Variables

their first can of spray paint, can they expect to have enough to cover a 5′ ×8′ subway panel with grafitti?

No. By assumption, the maximum area (in ft2) that can be covered by a can of paint is described by the function

g (Y) = 20πY 2

According to the second statement in Theorem 3.5.3, though, the average value for g(Y) is slightly less than the desired 40 ft2:

E[g (Y)] = ∫ 1

0 20πy2 · 3y2 dy

= 60πy 5

5

∣∣∣∣1 0

= 12π = 37.7 ft2

Example 3.5.11

A fair coin is tossed until a head appears. You will be given ( 1

2

)k dollars if that first head occurs on the kth toss. How much money can you expect to be paid?

Let the random variable X denote the toss at which the first head appears. Then

pX (k) = P(X = k) = P(First k − 1 tosses are tails and kth toss is a head)

= (

1 2

)k−1 · 1

2

= (

1 2

)k , k = 1, 2, . . .

Moreover,

E(amount won) = E [(

1 2

)X] = E[g (X )] =

∑ all k

g (k) · pX (k)

= ∞∑

k=1

( 1 2

)k · (

1 2

)k

= ∞∑

k=1

( 1 2

)2k =

∞∑ k=1

( 1 4

)k

= ∞∑

k=0

( 1 4

)k − (

1 4

)0

= 1 1 − 14

− 1

= $0.33

Example 3.5.12

In one of the early applications of probability to physics, James Clerk Maxwell (1831–1879) showed that the speed S of a molecule in a perfect gas has a density function given by

fS(s) = 4 √

a3

π s2e−as

2 , s > 0

Section 3.5 Expected Values 151

where a is a constant depending on the temperature of the gas and the mass of the particle. What is the average energy of a molecule in a perfect gas?

Let m denote the molecule’s mass. Recall from physics that energy (W), mass (m), and speed (S) are related through the equation

W = 1 2

mS2 = g (S) To find E(W) we appeal to the second part of Theorem 3.5.3:

E(W) = ∫ ∞

0 g (s) fS(s) ds

= ∫ ∞

0

1 2

ms2 · 4 √

a3

π s2e−as

2 ds

= 2m √

a3

π

∫ ∞ 0

s4e−as 2 ds

We make the substitution t = as2. Then E(W) = m

a √

π

∫ ∞ 0

t3/2e−tdt

But ∫ ∞ 0

t3/2e−tdt = (

3 2

)( 1 2

)√ π (see Section 4.4.6)

so

E(energy) = E(W) = m a √

π

( 3 2

)( 1 2

)√ π

= 3m 4a

Example 3.5.13

Consolidated Industries is planning to market a new product and they are trying to decide how many to manufacture. They estimate that each item sold will return a profit of m dollars; each one not sold represents an n-dollar loss. Furthermore, they suspect the demand for the product, V , will have an exponential distribution,

fV (v) = (

1 λ

) e−v/λ, v > 0

How many items should the company produce if they want to maximize their ex- pected profit? (Assume that n, m, and λ are known.)

If a total of x items are made, the company’s profit can be expressed as a function Q(v), where

Q(v) = {

mv − n(x − v) if v < x mx if v ≥ x

and v is the number of items sold. It follows that their expected profit is

E[Q(V)] = ∫ ∞

0 Q(v) · fV (v) dv

= ∫ x

0 [(m + n)v − nx]

( 1 λ

) e−v/λ dv +

∫ ∞ x

mx · (

1 λ

) e−v/λ dv (3.5.7)

152 Chapter 3 Random Variables

The integration here is straightforward, though a bit tedious. Equation 3.5.7 eventu- ally simplifies to

E[Q(V)] = λ · (m + n) − λ · (m + n)e−x/λ − nx To find the optimal production level, we need to solve dE[Q(V)]/dx = 0 for x. But

dE[Q(V)] dx

= (m + n)e−x/λ − n

and the latter equals zero when

x = −λ · ln (

n m + n

)

Example 3.5.14

A point, y, is selected at random from the interval [0, 1], dividing the line into two segments (see Figure 3.5.5). What is the expected value of the ratio of the shorter segment to the longer segment?

0 11 2

y

Figure 3.5.5

Notice, first, that the function

g (Y) = shorter segment longer segment

has two expressions, depending on the location of the chosen point:

g (Y) = {

y/(1 − y), 0 ≤ y ≤ 12 (1 − y)/y, 12 < y ≤ 1

By assumption, fY (y) = 1, 0 ≤ y ≤ 1, so

E[g (Y)] = ∫ 1

2

0

y 1 − y · 1 dy +

∫ 1 1 2

1 − y y

· 1 dy

Writing the second integrand as (1/y − 1) gives∫ 1 1 2

1 − y y

· 1 dy = ∫ 1

1 2

( 1 y

− 1 )

dy = (ln y − y) ∣∣∣∣1

1 2

= ln 2 − 1 2

By symmetry, though, the two integrals are the same, so

E (

shorter segment longer segment

) = 2 ln 2 − 1

= 0.39 On the average, then, the longer segment will be a little more than 2 12 times the length of the shorter segment.

Section 3.6 The Variance 153

Questions

3.5.28. Suppose X is a binomial random variable with n = 10 and p = 25 . What is the expected value of 3X − 4? 3.5.29. A typical day’s production of a certain electronic component is twelve. The probability that one of these components needs rework is 0.11. Each component need- ing rework costs $100. What is the average daily cost for defective components?

3.5.30. Let Y have probability density function

fY (y) = 2(1 − y), 0 ≤ y ≤ 1 Suppose that W = Y 2, in which case

fW (w) = 1√ w

− 1, 0 ≤ w ≤ 1

Find E(W) in two different ways.

3.5.31. A tool and die company makes castings for steel stress-monitoring gauges. Their annual profit, Q, in hun- dreds of thousands of dollars, can be expressed as a function of product demand, y:

Q(y) = 2(1 − e−2y) Suppose that the demand (in thousands) for their castings follows an exponential pdf, fY (y) = 6e−6y, y > 0. Find the company’s expected profit.

3.5.32. A box is to be constructed so that its height is five inches and its base is Y inches by Y inches, where Y is a random variable described by the pdf, fY (y) = 6y(1 − y), 0 < y < 1. Find the expected volume of the box.

3.5.33. Grades on the last Economics 301 exam were not very good. Graphed, their distribution had a shape similar to the pdf

fY (y) = 15000 (100 − y), 0 ≤ y ≤ 100

As a way of “curving” the results, the professor announces that he will replace each person’s grade, Y , with a new grade, g (Y), where g (Y) = 10√Y . Will the profes- sor’s strategy be successful in raising the class average above 60?

3.5.34. If Y has probability density function

fY (y) = 2y, 0 ≤ y ≤ 1

then E(Y) = 23 . Define the random variable W to be the squared deviation of Y from its mean, that is, W =( Y − 23

)2 . Find E(W).

3.5.35. The hypotenuse, Y , of the isosceles right triangle shown is a random variable having a uniform pdf over the interval [6, 10]. Calculate the expected value of the triangle’s area. Do not leave the answer as a function of a.

0

Y

a

a

3.5.36. An urn contains n chips numbered 1 through n. Assume that the probability of choosing chip i is equal to ki, i = 1, 2, . . . , n. If one chip is drawn, calculate E( 1X ), where the random variable X denotes the number show- ing on the chip selected. [Hint: Recall that the sum of the first n integers is n(n + 1)/2.]

3.6 The Variance We saw in Section 3.5 that the location of a distribution is an important characteristic and that it can be effectively measured by calculating either the mean or the median. A second feature of a distribution that warrants further scrutiny is its dispersion— that is, the extent to which its values are spread out. The two properties are totally different: Knowing a pdf’s location tells us absolutely nothing about its dispersion. Table 3.6.1, for example, shows two simple discrete pdfs with the same expected value (equal to zero), but with vastly different dispersions.

Table 3.6.1

k pX1 (k) k pX2 (k)

−1 12 −1,000,000 12 1 12 1,000,000

1 2

154 Chapter 3 Random Variables

It is not immediately obvious how the dispersion in a pdf should be quantified. Suppose that X is any discrete random variable. One seemingly reasonable approach would be to average the deviations of X from their mean—that is, calculate the ex- pected value of X − μ. As it happens, that strategy will not work because the neg- ative deviations will exactly cancel the positive deviations, making the numerical value of such an average always zero, regardless of the amount of spread present in pX (k):

E(X − μ) = E(X ) − μ = μ − μ = 0 (3.6.1) Another possibility would be to modify Equation 3.6.1 by making all the devia- tions positive, that is, to replace E(X − μ) with E(|X − μ|). This does work, and it is sometimes used to measure dispersion, but the absolute value is somewhat troublesome mathematically: It does not have a simple arithmetic formula, nor is it a differentiable function. Squaring the deviations proves to be a much better approach.

Definition 3.6.1 The variance of a random variable is the expected value of its squared deviations from μ. If X is discrete, with pdf pX (k),

Var(X ) = σ2 = E[(X − μ)2] = ∑ all k

(k − μ)2 · pX (k)

If Y is continuous, with pdf fY (y),

Var(Y) = σ2 = E[(Y − μ)2] = ∫ ∞

−∞ (y − μ)2 · fY (y) dy

[If E(X 2) or E(Y 2) is not finite, the variance is not defined.]

Comment One unfortunate consequence of Definition 3.6.1 is that the units for the variance are the square of the units for the random variable: If Y is measured in inches, for example, the units for Var(Y) are inches squared. This causes obvi- ous problems in relating the variance back to the sample values. For that reason, in applied statistics, where unit compatibility is especially important, dispersion is mea- sured not by the variance but by the standard deviation, which is defined to be the square root of the variance. That is,

σ = standard deviation =

⎧⎪⎪⎪⎪⎨ ⎪⎪⎪⎪⎩

√∑ all k

(k − μ)2 · pX (k) if X is discrete√∫ ∞ −∞

(y − μ)2 · fY (y) dy if Y is continuous

Comment The analogy between the expected value of a random variable and the center of gravity of a physical system was pointed out in Section 3.5. A similar equiv- alency holds between the variance and what engineers call a moment of inertia. If a set of weights having masses m1, m2, . . . are positioned along a (weightless) rigid bar at distances r1, r2, . . . from an axis of rotation (see Figure 3.6.1), the moment of inertia of the system is defined to be value

∑ i

mir2i . Notice, though, that if the masses

were the probabilities associated with a discrete random variable and if the axis of

Section 3.6 The Variance 155

rotation were actually μ, then r1, r2, . . . could be written (k1 − μ), (k2 − μ), . . . and∑ i

mir2i would be the same as the variance, ∑ all k

(k − μ)2 · pX (k).

Axis of rotationm1

m2 m3

r3

r1 r2

Figure 3.6.1

Definition 3.6.1 gives a formula for calculating σ2 in both the discrete and the continuous cases. An equivalent, but easier-to-use, formula is given in The- orem 3.6.1.

Theorem 3.6.1

Let W be any random variable, discrete or continuous, having mean μ and for which E(W 2) is finite. Then

Var(W) = σ2 = E(W 2) − μ2

Proof We will prove the theorem for the continuous case. The argument for dis- crete W is similar. In Theorem 3.5.3, let g (W) = (W − μ)2. Then

Var(W) = E[(W − μ)2] = ∫ ∞

−∞ g (w) fW (w) dw =

∫ ∞ −∞

(w − μ)2 fW (w) dw

Squaring out the term (w − μ)2 that appears in the integrand and using the additive property of integrals gives

∫ ∞ −∞

(w − μ)2 fW (w) dw = ∫ ∞

−∞ (w2 − 2μw + μ2) fW (w) dw

= ∫ ∞

−∞ w2 fW (w) dw − 2μ

∫ ∞ −∞

w fW (w) dw + ∫ ∞

−∞ μ2 fW (w) dw

= E(W 2) − 2μ2 + μ2 = E(W 2) − μ2

Note that the equality ∫∞ −∞ w

2 fW (w) dw = E(W 2) also follows from Theorem 3.5.3.

Example 3.6.1

An urn contains five chips, two red and three white. Suppose that two are drawn out simultaneously. Let X denote the number of red chips in the sample. Find Var(X ).

Note, first, that since the chips are not being replaced from drawing to drawing, X is a hypergeometric random variable. Moreover, we need to find μ, regardless of which formula is used to calculate σ2. In the notation of Theorem 3.5.2, r = 2, w = 3, and n = 2, so

μ = rn/(r + w) = 2 · 2/(2 + 3) = 0.8

156 Chapter 3 Random Variables

To find Var(X ) using Definition 3.6.1, we write

Var(X ) = E[(X − μ)2] = ∑ all x

(x − μ)2 · fX (x)

= (0 − 0.8)2 · (2

0

)(3 2

) (5

2

) + (1 − 0.8)2 · (2

1

)(3 1

) (5

2

) + (2 − 0.8)2 · (2

2

)(3 0

) (5

2

) = 0.36

To use Theorem 3.6.1, we would first find E(X 2). From Theorem 3.5.3,

E(X 2) = ∑ all x

x2 · fX (x) = 02 · (2

0

)(3 2

) (5

2

) + 12 · (2

1

)(3 1

) (5

2

) + 22 · (2

2

)(3 0

) (5

2

) = 1.00

Then

Var(X ) = E(X 2) − μ2 = 1.00 − (0.8)2

= 0.36 confirming what we calculated earlier.

In Section 3.5 we encountered a change of scale formula that applied to expected values. For any constants a and b and any random variable W , E(aW + b) = aE(W) + b. A similar issue arises in connection with the variance of a linear transformation: If Var(W) = σ2, what is the variance of aW + b?

Theorem 3.6.2

Let W be any random variable having mean μ and where E(W 2) is finite. Then Var(aW + b) = a2Var(W). Proof Using the same approach taken in the proof of Theorem 3.6.1, it can be shown that E[(aW + b)2] = a2E(W 2) + 2abμ + b2. We also know from the corol- lary to Theorem 3.5.3 that E(aW + b) = aμ + b. Using Theorem 3.6.1, then, we can write

Var(aW + b) = E[(aW + b)2] − [E(aW + b)]2

= [a2E(W 2) + 2abμ + b2] − [aμ + b]2

= [a2E(W 2) + 2abμ + b2] − [a2μ2 + 2abμ + b2] = a2[E(W 2) − μ2] = a2Var(W)

Example 3.6.2

A random variable Y is described by the pdf

fY (y) = 2y, 0 ≤ y ≤ 1 What is the standard deviation of 3Y + 2?

First, we need to find the variance of Y . But

E(Y) = ∫ 1

0 y · 2y dy = 2

3

and

E(Y 2) = ∫ 1

0 y2 · 2y dy = 1

2

Section 3.6 The Variance 157

so

Var(Y) = E(Y 2) − μ2 = 1 2

− (

2 3

)2

= 1 18

Then, by Theorem 3.6.2,

Var(3Y + 2) = (3)2 · Var(Y) = 9 · 1 18

= 1 2

which makes the standard deviation of 3Y + 2 equal to √

1 2 or 0.71.

Comment When the median is used as the measure of central tendency, the usual measure of dispersion is called the interquartile range. It marks off the middle 50% of the data. In other words, let Q1 represent the first quartile; that is, P(Y ≤ Q1) = 0.25 and Q3 represent the third quartile, that is P(Y ≤ Q3) = 0.75. Then Q3 − Q1 is the interquartile range.

Example 3.6.3

For fY (y) = 3y2, 0 ≤ y ≤ 1, Q1 is the solution to the equation 0.25 = ∫ Q1

0 3y 2dy,

or 0.25 = Q31, giving Q1 = 3 √

0.25 = 0.630. Similarly Q3 = 3 √

0.75 = 0.909, so the interquartile range is 0.909 − 0.630 = 0.279.

Questions

3.6.1. Find Var(X ) for the urn problem of Example 3.6.1 if the sampling is done with replacement.

3.6.2. Find the variance of Y if

fY (y) =

⎧⎪⎨ ⎪⎩

3 4 , 0 ≤ y ≤ 1 1 4 , 2 ≤ y ≤ 3 0, elsewhere

3.6.3. Ten equally qualified applicants, six men and four women, apply for three lab technician positions. Unable to justify choosing any of the applicants over all the oth- ers, the personnel director decides to select the three at random. Let X denote the number of men hired. Com- pute the standard deviation of X .

3.6.4. A certain hospitalization policy pays a cash benefit for up to five days in the hospital. It pays $250 per day for the first three days and $150 per day for the next two. The number of days of hospitalization, X , is a discrete random variable with probability function P(X = k) = 115 (6 − k) for k = 1, 2, 3, 4, 5. Find Var(X ).

3.6.5. Use Theorem 3.6.1 to find the variance of the ran- dom variable Y , where

fY (y) = 3(1 − y)2, 0 < y < 1

3.6.6. If

fY (y) = 2yk2 , 0 ≤ y ≤ k

for what value of k does Var(Y) = 2? 3.6.7. Calculate the standard deviation, σ, for the random variable Y whose pdf has the graph shown below:

1 2

1 2 30

1

y

f (y)Y

3.6.8. Consider the pdf defined by

fY (y) = 2y3 , y ≥ 1

Show that (a) ∫∞

1 fY (y) dy = 1, (b) E(Y) = 2, and (c) Var(Y) is not finite.

3.6.9. Frankie and Johnny play the following game. Frankie selects a number at random from the interval [a, b]. Johnny, not knowing Frankie’s number, is to pick a second number from that same interval and pay Frankie an amount, W , equal to the squared difference between

158 Chapter 3 Random Variables

the two [so 0 ≤ W ≤ (b − a)2]. What should be Johnny’s strategy if he wants to minimize his expected loss?

3.6.10. Let Y be a random variable whose pdf is given by fY (y) = 5y4, 0 ≤ y ≤ 1. Use Theorem 3.6.1 to find Var(Y).

3.6.11. Suppose that Y is an exponential random variable, so fY (y) = λe−λy, y ≥ 0. Show that the variance of Y is 1/λ2.

3.6.12. Suppose that Y is an exponential random vari- able with λ = 2 (recall Question 3.6.11). Find P[Y > E(Y) + 2√Var(Y)]. 3.6.13. Let X be a random variable with finite mean μ. Define for every real number a, g (a) = E[(X −a)2]. Show that

g (a) = E[(X − μ)2] + (μ − a)2. What is another name for min g (a)?

3.6.14. Suppose the charge for repairing an automobile averages $200 with a standard deviation of $16. If a 10% tax is added to the charge and then a $15 flat fee for en- vironmental impact, what is the standard deviation of the charge to the car owner?

3.6.15. If Y denotes a temperature recorded in degrees Fahrenheit, then 59 (Y −32) is the corresponding tempera- ture in degrees Celsius. If the standard deviation for a set of temperatures is 15.7◦F, what is the standard deviation of the equivalent Celsius temperatures?

3.6.16. If E(W) = μ and Var(W) = σ2, show that

E (

W − μ σ

) = 0 and Var

( W − μ

σ

) = 1

3.6.17. Suppose U is a uniform random variable over [0, 1]. (a) Show that Y = (b − a)U + a is uniform over [a, b]. (b) Use part (a) and Theorem 3.6.2 to find the variance of Y .

3.6.18. Recovering small quantities of calcium in the pres- ence of magnesium can be a difficult problem for an an- alytical chemist. Suppose the amount of calcium Y to be recovered is uniformly distributed between 4 and 7 mg. The amount of calcium recovered by one method is the random variable

W1 = 0.2281 + (0.9948)Y + E1 where the error term E1 has mean 0 and variance 0.0427 and is independent of Y .

A second procedure has random variable

W2 = −0.0748 + (1.0024)Y + E2 where the error term E2 has mean 0 and variance 0.0159 and is independent of Y .

The better technique should have a mean as close as possible to the mean of Y(=5.5), and a variance as small as possible. Compare the two methods on the basis of mean and variance.

HIGHER MOMENTS

The quantities we have identified as the mean and the variance are actually special cases of what are referred to more generally as the moments of a random variable. More precisely, E(W) is the first moment about the origin and σ2 is the second mo- ment about the mean. As the terminology suggests, we will have occasion to define higher moments of W . Just as E(W) and σ2 reflect a random variable’s location and dispersion, so it is possible to characterize other aspects of a distribution in terms of other moments. We will see, for example, that the skewness of a distribution—that is, the extent to which it is not symmetric around μ—can be effectively measured in terms of a third moment. Likewise, there are issues that arise in certain applied statistics problems that require a knowledge of the flatness of a pdf, a property that can be quantified by the fourth moment.

Definition 3.6.2 Let W be any random variable with pdf fW (w). For any positive integer r,

a. The rth moment of W about the origin, μr, is given by

μr = E(W r) provided

∫ ∞ −∞ |w|r · fW (w) dw < ∞ (or provided the analogous condition on

the summation of |w|r holds, if W is discrete). When r = 1, we usually delete the subscript and write E(W) as μ rather than μ1.

Section 3.6 The Variance 159

b. The rth moment of W about the mean, μ′r, is given by

μ′r = E[(W − μ)r] provided the finiteness conditions of part 1 hold.

Comment We can express μ′r in terms of μ j, j = 1, 2, . . . , r, by simply writing out the binomial expansion of (W − μ)r:

μ′r = E[(W − μ)r] = r∑

j=0

( r j

) E(W j)(−μ)r− j

Thus,

μ′2 = E[(W − μ)2] = σ2 = μ2 − μ21 μ′3 = E[(W − μ)3] = μ3 − 3μ1μ2 + 2μ31 μ′4 = E[(W − μ)4] = μ4 − 4μ1μ3 + 6μ21μ2 − 3μ41

and so on.

Example 3.6.4

The skewness of a pdf can be measured in terms of its third moment about the mean. If a pdf is symmetric, E[(W − μ)3] will obviously be zero; for pdfs not symmetric, E[(W − μ)3] will not be zero. In practice, the symmetry (or lack of symmetry) of a pdf is often measured by the coefficient of skewness, γ1, where

γ1 = E[(W − μ) 3]

σ3

Dividing μ′3 by σ 3 makes γ1 dimensionless.

A second “shape” parameter in common use is the coefficient of kurtosis, γ2, which involves the fourth moment about the mean. Specifically,

γ2 = E[(W − μ) 4]

σ4 − 3

For certain pdfs, γ2 is a useful measure of the probability of outliers (“fat tails”). Relatively flat pdfs are said to be platykurtic; more peaked pdfs are called leptokurtic.

Earlier in this chapter we encountered random variables whose means do not exist— recall, for example, the St. Petersburg paradox. More generally, there are random variables having certain of their higher moments finite and certain others, not finite. Addressing the question of whether or not a given E(W j) is finite is the following existence theorem.

Theorem 3.6.3

If the kth moment of a random variable exists, all moments of order less than k exist.

Proof Let fY (y) be the pdf of a continuous random variable Y . By Defini- tion 3.6.2, E(Y k) exists if and only if∫ ∞

−∞ |y|k · fY (y) dy < ∞ (3.6.2)

(Continued on next page)

160 Chapter 3 Random Variables

(Theorem 3.6.3 continued)

Let 1 ≤ j < k. To prove the theorem we must show that∫ ∞ −∞

|y| j · fY (y) dy < ∞

is implied by Inequality 3.6.2. But∫ ∞ −∞

|y| j · fY (y) dy = ∫

|y|≤1 |y| j · fY (y) dy +

∫ |y|>1

|y| j · fY (y) dy

≤ ∫

|y|≤1 fY (y) dy +

∫ |y|>1

|y| j · fY (y) dy

≤ 1 + ∫

|y|>1 |y| j · fY (y) dy

≤ 1 + ∫

|y|>1 |y|k · fY (y) dy < ∞

Therefore, E(Y j) exists, j = 1, 2, . . . , k − 1. The proof for discrete random vari- ables is similar.

Questions

3.6.19. Let Y be a uniform random variable defined over the interval (0, 2). Find an expression for the rth moment of Y about the origin. Also, use the binomial expansion as described in the Comment to find E[(Y − μ)6]. 3.6.20. Find the coefficient of skewness for an exponential random variable having the pdf

fY (y) = e−y, y > 0 3.6.21. Calculate the coefficient of kurtosis for a uniform random variable defined over the unit interval, fY (y) = 1, for 0 ≤ y ≤ 1. 3.6.22. Suppose that W is a random variable for which E[(W−μ)3] = 10 and E(W 3) = 4. Is it possible that μ = 2? 3.6.23. If Y = aX + b, a > 0, show that Y has the same coefficients of skewness and kurtosis as X .

3.6.24. Let Y be the random variable of Question 3.4.6, where for a positive integer n, fY (y) = (n + 2)(n + 1)yn(1 − y), 0 ≤ y ≤ 1. (a) Find Var(Y). (b) For any positive integer k, find the kth moment around the origin.

3.6.25. Suppose that the random variable Y is described by the pdf

fY (y) = c · y−6, y > 1

(a) Find c. (b) What is the highest moment of Y that exists?

3.7 Joint Densities Sections 3.3 and 3.4 introduced the basic terminology for describing the proba- bilistic behavior of a single random variable. Such information, while adequate for many problems, is insufficient when more than one variable are of interest to the ex- perimenter. Medical researchers, for example, continue to explore the relationship between blood cholesterol and heart disease, and, more recently, between “good” cholesterol and “bad” cholesterol. And more than a little attention—both political and pedagogical—is given to the role played by K–12 funding in the performance of would-be high school graduates on exit exams. On a smaller scale, electronic equip- ment and systems are often designed to have built-in redundancy: Whether or not that equipment functions properly ultimately depends on the reliability of two dif- ferent components.

Section 3.7 Joint Densities 161

The point is, there are many situations where two relevant random variables, say, X and Y , are defined on the same sample space.2 Knowing only fX (x) and fY (y), though, does not necessarily provide enough information to characterize the all-important simultaneous behavior of X and Y . The purpose of this section is to introduce the concepts, definitions, and mathematical techniques associated with distributions based on two (or more) random variables.

DISCRETE JOINT PDFS

As we saw in the single-variable case, the pdf is defined differently depending on whether the random variable is discrete or continuous. The same distinction applies to joint pdfs. We begin with a discussion of joint pdfs as they apply to two discrete random variables.

Definition 3.7.1 Suppose S is a discrete sample space on which two random variables, X and Y , are defined. The joint probability density function of X and Y (or joint pdf) is denoted pX,Y (x, y), where

pX,Y (x, y) = P({s|X (s) = x and Y(s) = y})

Comment A convenient shorthand notation for the meaning of pX,Y (x, y), consis- tent with what we used earlier for pdfs of single discrete random variables, is to write pX,Y (x, y) = P(X = x,Y = y).

Example 3.7.1

A supermarket has two express lines. Let X and Y denote the number of customers in the first and in the second, respectively, at any given time. During nonrush hours, the joint pdf of X and Y is summarized by the following table:

X 0 1 2 3

0 0.1 0.2 0 0 1 0.2 0.25 0.05 0

Y 2 0 0.05 0.05 0.025 3 0 0 0.025 0.05

Find P(|X − Y | = 1), the probability that X and Y differ by exactly 1. By definition,

P(|X − Y | = 1) = ∑

|x−y|=1

∑ pX,Y (x, y)

= pX,Y (0, 1) + pX,Y (1, 0) + pX,Y (1, 2) + pX,Y (2, 1) + pX,Y (2, 3) + pX,Y (3, 2)

= 0.2 + 0.2 + 0.05 + 0.05 + 0.025 + 0.025 = 0.55

[Would you expect pX,Y (x, y) to be symmetric? Would you expect the event |X − Y | ≥ 2 to have zero probability?]

2 For the next several sections we will suspend our earlier practice of using X to denote a discrete random variable and Y to denote a continuous random variable. The category of the random variables will need to be determined from the context of the problem. Typically, though, X and Y will either be both discrete or both continuous.

162 Chapter 3 Random Variables

Example 3.7.2

Suppose two fair dice are rolled. Let X be the sum of the numbers showing, and let Y be the larger of the two. So, for example,

pX,Y (2, 3) = P(X = 2,Y = 3) = P(∅) = 0

pX,Y (4, 3) = P(X = 4,Y = 3) = P({(1, 3)(3, 1)}) = 236 and

pX,Y (6, 3) = P(X = 6,Y = 3) = P({(3, 3)}) = 136 The entire joint pdf is given in Table 3.7.1.

Table 3.7.1 �����x

y 1 2 3 4 5 6 Row totals

2 1/36 0 0 0 0 0 1/36 3 0 2/36 0 0 0 0 2/36 4 0 1/36 2/36 0 0 0 3/36 5 0 0 2/36 2/36 0 0 4/36 6 0 0 1/36 2/36 2/36 0 5/36 7 0 0 0 2/36 2/36 2/36 6/36 8 0 0 0 1/36 2/36 2/36 5/36 9 0 0 0 0 2/36 2/36 4/36

10 0 0 0 0 1/36 2/36 3/36 11 0 0 0 0 0 2/36 2/36 12 0 0 0 0 0 1/36 1/36

Col. totals 1/36 3/36 5/36 7/36 9/36 11/36

Notice that the row totals in the right-hand margin of the table give the pdf for X . Similarly, the column totals along the bottom detail the pdf for Y . Those are not coincidences. Theorem 3.7.1 gives a formal statement of the relationship between the joint pdf and the individual pdfs.

Theorem 3.7.1

Suppose that pX,Y (x, y) is the joint pdf of the discrete random variables X and Y. Then

pX (x) = ∑ all y

pX,Y (x, y) and pY (y) = ∑ all x

pX,Y (x, y)

Proof We will prove the first statement. Note that the collection of sets (Y = y) for all y forms a partition of S; that is, they are disjoint and

⋃ all y(Y = y) = S. The set

(X = x) = (X = x) ∩ S = (X = x) ∩⋃all y(Y = y) = ⋃all y[(X = x) ∩ (Y = y)], so pX (x) = P(X = x) = P

⎛ ⎝⋃

all y

[(X = x) ∩ (Y = y)] ⎞ ⎠

= ∑ all y

P(X = x,Y = y) = ∑ all y

pX,Y (x, y)

Section 3.7 Joint Densities 163

Definition 3.7.2 An individual pdf obtained by summing a joint pdf over all values of the other random variable is called a marginal pdf.

CONTINUOUS JOINT PDFS

If X and Y are both continuous random variables, Definition 3.7.1 does not apply because P(X = x,Y = y) will be identically 0 for all (x, y). As was the case in single- variable situations, the joint pdf for two continuous random variables will be defined as a function that when integrated yields the probability that (X,Y) lies in a specified region of the xy-plane.

Definition 3.7.3 Two random variables defined on the same set of real numbers are jointly con- tinuous if there exists a function fX,Y (x, y) such that for any region R in the xy-plane, P[(X,Y) ∈ R] = ∫ ∫R fX,Y (x, y) dx dy. The function fX,Y (x, y) is the joint pdf of X and Y.

Comment Any function fX,Y (x, y) for which

1. fX,Y (x, y) ≥ 0 for all x and y 2.

∫ ∞ −∞

∫ ∞ −∞

fX,Y (x, y) dx dy = 1

qualifies as a joint pdf. We shall employ the convention of naming the domain only where the joint pdf is nonzero; everywhere else it will be assumed to be zero. This is analogous, of course, to the notation used earlier in describing the domain of single random variables. Also, for the functions used, the order of integration does not matter.

Example 3.7.3

Suppose that the variation in two continuous random variables, X and Y , can be modeled by the joint pdf fX,Y (x, y) = cxy, for 0 < y < x < 1. Find c.

By inspection, fX,Y (x, y) will be nonnegative as long as c ≥ 0. The particular c that qualifies fX,Y (x, y) as a joint pdf, though, is the one that makes the volume under fX,Y (x, y) equal to 1. But∫ ∫

S cxy dy dx = 1 = c

∫ 1 0

[∫ x 0

(xy) dy ]

dx = c ∫ 1

0 x (

y2

2

∣∣∣x 0

) dx

= c ∫ 1

0

( x3

2

) dx = cx

4

8

∣∣∣1 0

= (

1 8

) c

Therefore, c = 8.

Example 3.7.4

A study claims that the daily number of hours, X , a teenager watches television and the daily number of hours, Y , he works on his homework are approximated by the joint pdf

fX,Y (x, y) = xye−(x+y), x > 0, y > 0

164 Chapter 3 Random Variables

What is the probability that a teenager chosen at random spends at least twice as much time watching television as he does working on his homework?

The region, R, in the xy-plane corresponding to the event “X ≥ 2Y” is shown in Figure 3.7.1. It follows that P(X ≥ 2Y) is the volume under fX,Y (x, y) above the region R:

P(X ≥ 2Y) = ∫ ∞

0

∫ x/2 0

xye−(x+y) dy dx

0 x

x = 2y

y

R

Figure 3.7.1

Separating variables, we can write

P(X ≥ 2Y) = ∫ ∞

0 xe−x

[∫ x/2 0

ye−ydy

] dx

and the double integral reduces to 727 :

P(X ≥ 2Y) = ∫ ∞

0 xe−x

[ 1 −

(x 2

+ 1 )

e−x/2 ]

dx

= ∫ ∞

0 xe−xdx −

∫ ∞ 0

x2

2 e−3x/2 dx −

∫ ∞ 0

xe−3x/2dx

= 1 − 16 54

− 4 9

= 7 27

GEOMETRIC PROBABILITY

One particularly important special case of Definition 3.7.3 is the joint uniform pdf, which is represented by a surface having a constant height everywhere above a spec- ified rectangle in the xy-plane. That is,

fX,Y (x, y) = 1(b − a)(d − c) , a ≤ x ≤ b, c ≤ y ≤ d

If R is some region in the rectangle where X and Y are defined, P((X,Y) ∈ R) reduces to a simple ratio of areas:

P((X,Y) ∈ R) = area of R (b − a)(d − c) (3.7.1)

Calculations based on Equation 3.7.1 are referred to as geometric probabilities.

Section 3.7 Joint Densities 165

Example 3.7.5

Two friends agree to meet on the University Commons “sometime around 12:30.” But neither of them is particularly punctual—or patient. What will actually happen is that each will arrive at random sometime in the interval from 12:00 to 1:00. If one arrives and the other is not there, the first person will wait fifteen minutes or until 1:00, whichever comes first, and then leave. What is the probability that the two will get together?

To simplify notation, we can represent the time period from 12:00 to 1:00 as the interval from zero to sixty minutes. Then if x and y denote the two arrival times, the sample space is the 60 × 60 square shown in Figure 3.7.2. Furthermore, the event M, “The two friends meet,” will occur if and only if |x − y| ≤ 15 or, equivalently, if and only if −15 ≤ x − y ≤ 15. These inequalities appear as the shaded region in Figure 3.7.2.

0 x

y

60

(45, 60)

(15, 0)

x – y = –15

x – y = 15

60

(60, 45)

(0, 15) M

Figure 3.7.2

Notice that the areas of the triangles above and below M are each equal to 1 2 (45)(45). It follows that the two friends have a 44% chance of meeting:

P(M) = area of M area of S

= (60) 2 − 2[ 12 (45)(45)]

(60)2

= 0.44

Example 3.7.6

A carnival operator wants to set up a ringtoss game. Players will throw a ring of diameter d onto a grid of squares, the side of each square being of length s (see Fig- ure 3.7.3). If the ring lands entirely inside a square, the player wins a prize. To ensure a profit, the operator must keep the player’s chances of winning down to something less than one in five. How small can the operator make the ratio d/s?

d s s

Figure 3.7.3

First, assume that the player is required to stand far enough away that no skill is involved and the ring is falling at random on the grid. From Figure 3.7.4, we see that in order for the ring not to touch any side of the square, the ring’s center must be somewhere in the interior of a smaller square, each side of which is a distance d/2 from one of the grid lines.

166 Chapter 3 Random Variables

s

s d 2

Figure 3.7.4

Since the area of a grid square is s2 and the area of an interior square is (s − d)2, the probability of a winning toss can be written as the ratio:

P(Ring touches no lines) = (s − d) 2

s2

But the operator requires that

(s − d)2 s2

≤ 0.20 Solving for d/s gives

d s

≥ 1 − √

0.20 = 0.55 That is, if the diameter of the ring is at least 55% as long as the side of one of the squares, the player will have no more than a 20% chance of winning.

Questions

3.7.1. If pX,Y (x, y) = cxy at the points (1, 1), (2, 1), (2, 2), and (3, 1), and equals 0 elsewhere, find c.

3.7.2. Let X and Y be two continuous random vari- ables defined over the unit square. What does c equal if fX,Y (x, y) = c(x2 + y2)? 3.7.3. Suppose that random variables X and Y vary in ac- cordance with the joint pdf, fX,Y (x, y) = c(x+y), 0 < x < y < 1. Find c.

3.7.4. Find c if fX,Y (x, y) = cxy for X and Y defined over the triangle whose vertices are the points (0, 0), (0, 1), and (1, 1).

3.7.5. An urn contains four red chips, three white chips, and two blue chips. A random sample of size 3 is drawn without replacement. Let X denote the number of white chips in the sample and Y the number of blue chips. Write a formula for the joint pdf of X and Y .

3.7.6. Four cards are drawn from a standard poker deck. Let X be the number of kings drawn and Y the number of queens. Find pX,Y (x, y).

3.7.7. An advisor looks over the schedules of his fifty stu- dents to see how many math and science courses each has

registered for in the coming semester. He summarizes his results in a table. What is the probability that a student selected at random will have signed up for more math courses than science courses?

Number of math courses, X

0 1 2

Number of science courses, Y

0 11 6 4 1 9 10 3 2 5 0 2

3.7.8. Consider the experiment of tossing a fair coin three times. Let X denote the number of heads on the last flip, and let Y denote the total number of heads on the three flips. Find pX,Y (x, y).

3.7.9. Suppose that two fair dice are tossed one time. Let X denote the number of 2’s that appear, and Y the num- ber of 3’s. Write the matrix giving the joint probability density function for X and Y . Suppose a third random variable, Z, is defined, where Z = X + Y. Use pX,Y (x, y) to find pZ(z).

Section 3.7 Joint Densities 167

3.7.10. Let X be the time in days between a car accident and reporting a claim to the insurance company. Let Y be the time in days between the report and payment of the claim. Suppose that fX,Y (x, y) = c, 0 ≤ x ≤ 7, 0 ≤ y ≤ 7, and zero otherwise. (a) Find c. (b) Find P(0 ≤ X ≤ 2, 0 ≤ Y ≤ 4). 3.7.11. Let X and Y have the joint pdf

fX,Y (x, y) = 2e−(x+y), 0 < x < y, 0 < y Find P(Y < 3X ).

3.7.12. A point is chosen at random from the interior of a circle whose equation is x2 + y2 ≤ 4. Let the random

variables X and Y denote the x- and y-coordinates of the sampled point. Find fX,Y (x, y).

3.7.13. Find P(X < 2Y) if fX,Y (x, y) = x + y for X and Y each defined over the unit interval.

3.7.14. Suppose that five independent observations are drawn from the continuous pdf fT (t) = 2t, 0 ≤ t ≤ 1. Let X denote the number of t’s that fall in the interval 0 ≤ t < 13 and let Y denote the number of t’s that fall in the interval 13 ≤ t < 23 . Find pX,Y (1, 2). 3.7.15. A point is chosen at random from the interior of a right triangle with base b and height h. What is the prob- ability that the y value is between 0 and h/2?

MARGINAL PDFS FOR CONTINUOUS RANDOM VARIABLES

The notion of marginal pdfs in connection with discrete random variables was in- troduced in Theorem 3.7.1 and Definition 3.7.2. An analogous relationship holds in the continuous case—integration, though, replaces the summation that appears in Theorem 3.7.1.

Theorem 3.7.2

Suppose X and Y are jointly continuous with joint pdf fX,Y (x, y). Then the marginal pdfs, fX (x) and fY (y), are given by

fX (x) = ∫ ∞

−∞ fX,Y (x, y) dy and fY (y) =

∫ ∞ −∞

fX,Y (x, y) dx

Proof It suffices to verify the first of the theorem’s two equalities. As is often the case with proofs for continuous random variables, we begin with the cdf:

FX (x) = P(X ≤ x) = ∫ ∞

−∞

∫ x −∞

fX,Y (t, y) dt dy = ∫ x

−∞

∫ ∞ −∞

fX,Y (x, y) dy dt

Differentiating both ends of the equation above gives

fX (x) = ∫ ∞

−∞ fX,Y (x, y) dy

Recall Theorem 3.4.1.

Example 3.7.7

Suppose that two continuous random variables, X and Y , have the joint uniform pdf

fX,Y (x, y) = 16 , 0 ≤ x ≤ 3, 0 ≤ y ≤ 2

Find fX (x). Applying Theorem 3.7.2 gives

fX (x) = ∫ 2

0 fX,Y (x, y) dy =

∫ 2 0

1 6

dy = 1 3 , 0 ≤ x ≤ 3

Notice that X , by itself, is a uniform random variable defined over the interval [0, 3]; similarly, we would find that fY (y) is a uniform pdf over the interval [0, 2].

168 Chapter 3 Random Variables

Example 3.7.8

Consider the case where X and Y are two continuous random variables, jointly dis- tributed over the first quadrant of the xy-plane according to the joint pdf,

fX,Y (x, y) = {

y2e−y(x+1) x ≥ 0, y ≥ 0 0 elsewhere

Find the two marginal pdfs. First, consider fX (x). By Theorem 3.7.2,

fX (x) = ∫ ∞

−∞ fX,Y (x, y) dy =

∫ ∞ 0

y2e−y(x+1) dy

In the integrand, substitute

u = y(x + 1) making du = (x + 1) dy. This gives

fX (x) = 1x + 1 ∫ ∞

0

u2

(x + 1)2 e −u du = 1

(x + 1)3 ∫ ∞

0 u2e−u du

After applying integration by parts (twice) to ∫∞

0 u 2e−u du, we get

fX (x) = 1(x + 1)3 [−u2e−u − 2ue−u − 2e−u]∣∣∞0

= 1 (x + 1)3

[ 2 − lim

u→∞

( u2

eu + 2u

eu + 2

eu

)]

= 2 (x + 1)3 , x ≥ 0

Finding fY (y) is a bit easier:

fY (y) = ∫ ∞

−∞ fX,Y (x, y) dx =

∫ ∞ 0

y2e−y(x+1) dx

= y2e−y ∫ ∞

0 e−yx dx = y2e−y

( 1 y

)( −e−yx

∣∣∣∣∞ 0

) = ye−y, y ≥ 0

Questions

3.7.16. Find the marginal pdf of X for the joint pdf derived in Question 3.7.5.

3.7.17. Find the marginal pdfs of X and Y for the joint pdf derived in Question 3.7.8.

3.7.18. The campus recruiter for an international con- glomerate classifies the large number of students she in- terviews into three categories—the lower quarter, the middle half, and the upper quarter. If she meets six stu- dents on a given morning, what is the probability that they will be evenly divided among the three categories? What is the marginal probability that exactly two will belong to the middle half?

3.7.19. For each of the following joint pdfs, find fX (x) and fY (y). (a) fX,Y (x, y) = 12 , 0 ≤ x ≤ 2, 0 ≤ y ≤ 1 (b) fX,Y (x, y) = 32 y2, 0 ≤ x ≤ 2, 0 ≤ y ≤ 1 (c) fX,Y (x, y) = 23 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 (d) fX,Y (x, y) = c(x + y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 (e) fX,Y (x, y) = 4xy, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 (f) fX,Y (x, y) = xye−(x+y), 0 ≤ x, 0 ≤ y (g) fX,Y (x, y) = ye−xy−y, 0 ≤ x, 0 ≤ y 3.7.20. For each of the following joint pdfs, find fX (x) and fY (y).

Section 3.7 Joint Densities 169

(a) fX,Y (x, y) = 12 , 0 ≤ x ≤ y ≤ 2 (b) fX,Y (x, y) = 1x , 0 ≤ y ≤ x ≤ 1 (c) fX,Y (x, y) = 6x, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 − x 3.7.21. Suppose that fX,Y (x, y) = 6(1 − x − y) for x and y defined over the unit square, subject to the restriction that 0 ≤ x + y ≤ 1. Find the marginal pdf for X . 3.7.22. Find fY (y) if fX,Y (x, y) = 2e−xe−y for x and y de- fined over the shaded region pictured.

0 x

y

y = x

3.7.23. Suppose that X and Y are discrete random vari- ables with

pX,Y (x, y) = 4!x!y!(4 − x − y)! (

1 2

)x (1 3

)y (1 6

)4−x−y ,

0 ≤ x + y ≤ 4 Find pX (x) and pY (x).

3.7.24. A generalization of the binomial model occurs when there is a sequence of n independent trials with three outcomes,where p1 = P(outcome1)and p2 = P(outcome2). Let X and Y denote the number of trials (out of n) result- ing in outcome 1 and outcome 2, respectively.

(a) Show that pX,Y (x, y) = n!x!y!(n − x − y)! p x 1 p

y 2

(1 − p1 − p2)n−x−y, 0 ≤ x + y ≤ n (b) Find pX (x) and pY (x).

(Hint: See Question 3.7.23.)

JOINT CDFS

For a single random variable X , the cdf of X evaluated at some point x—that is, FX (x)—is the probability that the random variable X takes on a value less than or equal to x. Extended to two variables, a joint cdf [evaluated at the point (x, y)] is the probability that X ≤ x and, simultaneously, that Y ≤ y.

Definition 3.7.4 Let X and Y be any two random variables. The joint cumulative distribution function of X and Y (or joint cdf ) is denoted FX,Y (x, y), where

FX,Y (x, y) = P(X ≤ x and Y ≤ y)

Example 3.7.9

Find the joint cdf, FX,Y (x, y), for the two random variables X and Y with joint pdf fX,Y (x, y) = 43 (x + xy), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1.

If Definition 3.7.4 is applied, the probability that X ≤ x and Y ≤ y becomes a double integral of the pdf. In order to keep the cdf a function of x and y, use u and v as the variables of integration.

FX,Y (x, y) = 43 ∫ y

0

∫ x 0

(u + uv) du dv = 4 3

∫ y 0

(∫ x 0

(u + uv) du )

dv

= 4 3

∫ y 0

( u2

2 (1 + v)

∣∣∣∣ x

0

) dv = 4

3

∫ y 0

x2

2 (1 + v) dv

= 4 3

x2

2

( v + v

2

2

)∣∣∣∣ y

0 = 4

3 x2

2

( y + y

2

2

) ,

which simplifies to

FX,Y (x, y) = 13x 2(2y + y2).

[For what values of x and y is FX,Y (x, y) defined?]

170 Chapter 3 Random Variables

Theorem 3.7.3

Let FX,Y (x, y) be the joint cdf associated with the continuous random variables X and Y. Then the joint pdf of X and Y, fX,Y (x, y), is a second partial derivative of the

joint cdf, that is, fX,Y (x, y) = ∂ 2

∂x∂y FX,Y (x, y), provided FX,Y (x, y) has continuous

second partial derivatives.

Example 3.7.10

What is the joint pdf of the random variables X and Y whose joint cdf is FX,Y (x, y) = 1 3 x

2(2y + y2), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1? By Theorem 3.7.3,

fX,Y (x, y) = ∂ 2

∂x ∂y FX,Y (x, y) = ∂

2

∂x ∂y 1 3

x2(2y + y2)

= ∂ ∂y

2 3

x(2y + y2) = 2 3

x(2 + 2y) = 4 3

(x + xy), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1

Notice the similarity between Examples 3.7.9 and 3.7.10. fX,Y (x, y) is the same in both examples; so is FX,Y (x, y).

MULTIVARIATE DENSITIES

The definitions and theorems in this section extend in a very straightforward way to situations involving more than two variables. The joint pdf for n discrete random variables, for example, is denoted pX1,...,Xn (x1, . . . , xn) where

pX1,...,Xn (x1, . . . , xn) = P(X1 = x1, . . . , Xn = xn)

For n continuous random variables, the joint pdf is that function fX1,...,Xn (x1, . . . , xn) having the property that for any region R in n-space,

P[(X1, . . . , Xn) ∈ R] = ∫∫

R · · ·

∫ fX1,...,Xn (x1, . . . , xn) dx1 · · · dxn

And if FX1,...,Xn (x1, . . . , xn) is the joint cdf of continuous random variables X1, . . . , Xn—that is, FX1,...,Xn (x1, . . . , xn) = P(X1 ≤ x1, . . . , Xn ≤ xn)—then

fX1,...,Xn (x1, . . . , xn) = ∂n

∂x1 · · · ∂xn FX1,...,Xn (x1, . . . , xn)

The notion of a marginal pdf also extends readily, although in the n-variate case, a marginal pdf can, itself, be a joint pdf. Given X1, . . . , Xn, the marginal pdf of any subset of r of those variables (Xi1 , Xi2 , . . . , Xir ) is derived by integrating (or summing) the joint pdf with respect to the remaining n − r variables (Xj1 , Xj2 , . . . , Xjn−r ). If the Xi’s are all continuous, for example,

fXi1,...,Xir (xi1 , . . . , xir ) = ∫ ∞

−∞

∫ ∞ −∞

· · · ∫ ∞

−∞ fX1,...,Xn (x1, . . . , xn) dxj1 · · · dxjn−r

Section 3.7 Joint Densities 171

Questions

3.7.25. Consider the experiment of simultaneously toss- ing a fair coin and rolling a fair die. Let X denote the number of heads showing on the coin and Y the number of spots showing on the die. (a) List the outcomes in S. (b) Find FX,Y (1, 2).

3.7.26. An urn contains twelve chips—four red, three black, and five white. A sample of size 4 is to be drawn without replacement. Let X denote the number of white chips in the sample, Y the number of red. Find FX,Y (1, 2).

3.7.27. For each of the following joint pdfs, find FX,Y (x, y). (a) fX,Y (x, y) = 32 y2, 0 ≤ x ≤ 2, 0 ≤ y ≤ 1 (b) fX,Y (x, y) = 23 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 (c) fX,Y (x, y) = 4xy, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1

3.7.28. For each of the following joint pdfs, find FX,Y (x, y). (a) fX,Y (x, y) = 12 , 0 ≤ x ≤ y ≤ 2 (b) fX,Y (x, y) = 1x , 0 ≤ y ≤ x ≤ 1 (c) fX,Y (x, y) = 6x, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 − x

3.7.29. Find and graph fX,Y (x, y) if the joint cdf for ran- dom variables X and Y is

FX,Y (x, y) = xy, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1

3.7.30. Find the joint pdf associated with two random vari- ables X and Y whose joint cdf is

FX,Y (x, y) = (1 − e−λy)(1 − e−λx), x > 0, y > 0

3.7.31. Given that FX,Y (x, y) = k(4x2y2 + 5xy4), 0 < x < 1, 0 < y < 1, find the corresponding pdf and use it to cal- culate P(0 < X < 12 ,

1 2 < Y < 1).

3.7.32. Prove that

P(a < X ≤ b, c < Y ≤ d) = FX,Y (b, d) − FX,Y (a, d) − FX,Y (b, c) + FX,Y (a, c)

3.7.33. A certain brand of fluorescent bulbs will last, on the average, one thousand hours. Suppose that four of these bulbs are installed in an office. What is the proba- bility that all four are still functioning after one thousand fifty hours? If Xi denotes the ith bulb’s life, assume that

fX1,X2,X3,X4 (x1, x2, x3, x4) = 4∏

i=1

( 1

1000

) e−x/1000

for xi > 0, i = 1, 2, 3, 4. 3.7.34. A hand of six cards is dealt from a standard poker deck. Let X denote the number of aces, Y the number of kings, and Z the number of queens. (a) Write a formula for pX,Y,Z(x, y, z). (b) Find pX,Y (x, y) and pX,Z(x, z).

3.7.35. Calculate pX,Y (0, 1) if pX,Y,Z(x, y, z) = 3!

x!y!z!(3−x−y−z)! ( 1

2

)x ( 1 12

)y ( 1 6

)z · ( 14 )3−x−y−z for x, y, z = 0, 1, 2, 3 and 0 ≤ x + y + z ≤ 3. 3.7.36. Suppose that the random variables X , Y , and Z have the multivariate pdf

fX,Y,Z(x, y, z) = (x + y)e−z for 0 < x < 1, 0 < y < 1, and z > 0. Find (a) fX,Y (x, y), (b) fY,Z(y, z), and (c) fZ(z).

3.7.37. The four random variables W , X , Y , and Z have the multivariate pdf

fW,X,Y,Z(w, x, y, z) = 16wxyz for 0 ≤ w ≤ 1, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1, and 0 ≤ z ≤ 1. Find the marginal pdf, fW,X (w, x), and use it to compute P(0 ≤ W ≤ 12 , 12 ≤ X ≤ 1).

INDEPENDENCE OF TWO RANDOM VARIABLES

The concept of independent events that was introduced in Section 2.5 leads quite naturally to a similar definition for independent random variables.

Definition 3.7.5 Two discrete random variables X and Y are said to be independent if for every points a and b, P(X = a and Y = b) = P(X = a)P(Y = b). Two continuous random variables X and Y are said to be independent if for every interval A and every interval B, P(X ∈ A and Y ∈ B) = P(X ∈ A)P(Y ∈ B).

172 Chapter 3 Random Variables

Theorem 3.7.4

The continuous random variables X and Y are independent if and only if there are functions g (x) and h(y) such that

fX,Y (x, y) = g (x)h(y) f or all x and y (3.7.2) If Equation 3.7.2 holds, there is a constant k such that fX (x) = kg (x) and fY (y) = (1/k)h(y).

Proof First, suppose that X and Y are independent. Then FX,Y (x, y) = P(X ≤ x and Y ≤ y) = P(X ≤ x)P(Y ≤ y) = FX (x)FY (y), and we can write

fX,Y (x, y) = ∂ 2

∂x ∂y FX,Y (x, y) = ∂

2

∂x ∂y FX (x)FY (y) = ddxFX (x)

d dy

FY (y) = fX (x) fY (y)

Next we need to show that Equation 3.7.2 implies that X and Y are indepen- dent. To begin, note that

fX (x) = ∫ ∞

−∞ fX,Y (x, y) dy =

∫ ∞ −∞

g (x)h(y) dy = g (x) ∫ ∞

−∞ h(y) dy

Set k = ∫∞−∞ h(y) dy, so fX (x) = kg (x). Similarly, it can be shown that fY (y) = (1/k)h(y). Therefore,

P(X ∈ A and Y ∈ B) = ∫

A

∫ B

fX,Y (x, y) dx dy = ∫

A

∫ B

g (x)h(y) dx dy

= ∫

A

∫ B

kg (x)(1/k)h(y) dx dy = ∫

A fX (x) dx

∫ B

fY (y) dy

= P(X ∈ A)P(Y ∈ B) and the theorem is proved.

Comment Theorem 3.7.4 can be adapted to the case that X and Y are discrete.

Example 3.7.11

Suppose that the probabilistic behavior of two random variables X and Y is de- scribed by the joint pdf fX,Y (x, y) = 12xy(1 − y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1. Are X and Y independent? If they are, find fX (x) and fY (y).

According to Theorem 3.7.4, the answer to the independence question is “yes” if fX,Y (x, y) can be factored into a function of x times a function of y. There are such functions. Let g (x) = 12x and h(y) = y(1 − y).

To find fX (x) and fY (y) requires that the “12” appearing in fX,Y (x, y) be factored in such a way that g (x) · h(y) = fX (x) · fY (y). Let

k = ∫ ∞

−∞ h(y) dy =

∫ 1 0

y(1 − y) dy = [y2/2 − y3/3] ∣∣∣∣1 0

= 1 6

Therefore, fX (x) = kg (x) = 16 (12x) = 2x, 0 ≤ x ≤ 1 and fY (y) = (1/k)h(y) = 6y(1 − y), 0 ≤ y ≤ 1.

INDEPENDENCE OF n (>2) RANDOM VARIABLES

In Chapter 2, extending the notion of independence from two events to n events proved to be something of a problem. The independence of each subset of the n events had to be checked separately (recall Definition 2.5.2). This is not necessary in the case of n random variables. We simply use the extension of Theorem 3.7.4 to n random variables as the definition of independence in the multidimensional case.

Section 3.7 Joint Densities 173

The theorem that independence is equivalent to the factorization of the joint pdf holds in the multidimensional case.

Definition 3.7.6 The n random variables X1, X2, . . . , Xn are said to be independent if there are functions g1(x1), g2(x2), . . . , gn(xn) such that for every x1, x2, . . . , xn

fX1,X2,...,Xn (x1, x2, . . . , xn) = g1(x1)g2(x2) · · · gn(xn) A similar statement holds for discrete random variables, in which case f is re- placed with p.

Comment Analogous to the result for n = 2 random variables, the expression on the right-hand side of the equation in Definition 3.7.6 can also be written as the prod- uct of the marginal pdfs of X1, X2, . . . , and Xn.

Example 3.7.12

Consider k urns, each holding n chips numbered 1 through n. A chip is to be drawn at random from each urn. What is the probability that all k chips will bear the same number?

If X1, X2, . . . , Xk denote the numbers on the 1st, 2nd, . . ., and kth chips, respec- tively, we are looking for the probability that X1 = X2 = · · · = Xk. In terms of the joint pdf,

P(X1 = X2 = · · · = Xk) = ∑

x1=x2=···=xk pX1,X2,...,Xk (x1, x2, . . . , xk)

Each of the selections here is obviously independent of all the others, so the joint pdf factors according to Definition 3.7.6, and we can write

P(X1 = X2 = · · · = Xk) = n∑

i=1 pX1 (xi) · pX2 (xi) · · · pXk (xi)

= n · (

1 n

· 1 n

· · · · · 1 n

)

= 1 nk−1

RANDOM SAMPLES

Definition 3.7.6 addresses the question of independence as it applies to n random vari- ables having marginal pdfs—say, fX1 (x1), fX2 (x2), . . . , fXn (xn)—that might be quite different. A special case of that definition occurs for virtually every set of data col- lected for statistical analysis. Suppose an experimenter takes a set of n measurements, x1, x2, . . . , xn, under the same conditions. Those Xi’s, then, qualify as a set of indepen- dent random variables—moreover, each represents the same pdf. The special—but familiar—notation for that scenario is given in Definition 3.7.7. We will encounter it often in the chapters ahead.

Definition 3.7.7 Let W1,W2, . . . ,Wn be a set of n independent random variables, all having the same pdf. Then W1,W2, . . . ,Wn are said to be a random sample of size n.

174 Chapter 3 Random Variables

Questions

3.7.38. Two fair dice are tossed. Let X denote the num- ber appearing on the first die and Y the number on the second. Show that X and Y are independent.

3.7.39. Let fX,Y (x, y) = λ2e−λ(x+y), 0 ≤ x, 0 ≤ y. Show that X andY are independent. What are the marginal pdfs in this case?

3.7.40. Suppose that each of two urns has four chips, num- bered 1 through 4. A chip is drawn from the first urn and bears the number X . That chip is added to the second urn. A chip is then drawn from the second urn. Call its number Y . (a) Find pX,Y (x, y). (b) Show that pX (k) = pY (k) = 14 , k = 1, 2, 3, 4. (c) Show that X and Y are not independent.

3.7.41. Let X and Y be random variables with joint pdf

fX,Y (x, y) = k, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1, 0 ≤ x + y ≤ 1 Give a geometric argument to show that X and Y are not independent.

3.7.42. Are the random variables X and Y independent if fX,Y (x, y) = 23 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1? 3.7.43. Suppose that random variables X and Y are inde- pendent with marginal pdfs fX (x) = 2x, 0 ≤ x ≤ 1, and fY (y) = 3y2, 0 ≤ y ≤ 1. Find P(Y < X ). 3.7.44. Find the joint cdf of the independent random vari-

ables X and Y , where fX (x) = x2 , 0 ≤ x ≤ 2, and fY (y) = 2y, 0 ≤ y ≤ 1. 3.7.45. If two random variables X and Y are independent with marginal pdfs fX (x) = 2x, 0 ≤ x ≤ 1, and fY (y) = 1, 0 ≤ y ≤ 1, calculate P (YX > 2).

3.7.46. Suppose fX,Y (x, y) = xye−(x+y), x > 0, y > 0. Prove for any real numbers a, b, c, and d that

P(a < X < b, c < Y < d) = P(a < X < b) · P(c < Y < d) thereby establishing the independence of X and Y .

3.7.47. Given the joint pdf fX,Y (x, y) = 2x + y − 2xy, 0 < x < 1, 0 < y < 1, find numbers a, b, c, and d such that

P(a < X < b, c < Y < d) �= P(a < X < b) · P(c < Y < d) thus demonstrating that X and Y are not independent.

3.7.48. Prove that if X and Y are two independent ran- dom variables, then U = g (X ) and V = h(Y) are also independent.

3.7.49. If two random variables X and Y are defined over a region in the XY -plane that is not a rectangle (possibly infinite) with sides parallel to the coordinate axes, can X and Y be independent?

3.7.50. Write down the joint probability density function for a random sample of size n drawn from the exponential pdf, fX (x) = (1/λ)e−x/λ, x ≥ 0. 3.7.51. Suppose that X1, X2, X3, and X4 are independent random variables, each with pdf fXi (xi) = 4x3i , 0 ≤ xi ≤ 1. Find (a) P

( X1 < 12

) .

(b) P ( exactly one Xi < 12

) .

(c) fX1,X2,X3,X4 (x1, x2, x3, x4). (d) FX2,X3 (x2, x3).

3.7.52. A random sample of size n = 2k is taken from a uniform pdf defined over the unit interval. Calculate P ( X1 < 12 , X2 >

1 2 , X3 <

1 2 , X4 >

1 2 , . . . , X2k >

1 2

) .

3.8 Transforming and Combining Random Variables

TRANSFORMATIONS

Transforming a variable from one scale to another is a problem that is comfortably familiar. If a thermometer says the temperature outside is 83◦F, we know that the temperature in degrees Celsius is 28:

◦C = (

5 9

) (◦F − 32) =

( 5 9

) (83 − 32) = 28

An analogous question arises in connection with random variables. Suppose that X is a discrete random variable with pdf pX (k). If a second random variable, Y , is defined to be aX + b, where a and b are constants, what can be said about the pdf for Y? Recall Questions 3.3.11 and 3.4.16 as examples of such transformations.

Section 3.8 Transforming and Combining Random Variables 175

Theorem 3.8.1

Suppose X is a discrete random variable. Let Y = aX + b, where a and b are constants. Then pY (y) = pX

( y−b

a

) .

Proof pY (y) = P(Y = y) = P(aX + b = y) = P (

X = y − b a

) = pX

( y − b

a

)

Example 3.8.1

Let X be a random variable for which pX (k) = 110 , for k = 1, 2, . . . , 10. What is the probability distribution associated with the random variable Y , where Y = 4X − 1? That is, find pY (y).

From Theorem 3.8.1, P(Y = y) = P(4X − 1 = y) = P[X = (y + 1)/4] = pX

( y+1

4

) , which implies that pY (y) = 110 for the ten values of (y + 1)/4 that equal 1,

2, . . ., 10. But (y+1)/4 = 1 when y = 3, (y+1)/4 = 2 when y = 7, . . . , (y+1)/4 = 10 when y = 39. Therefore, pY (y) = 110 , for y = 3, 7, . . . , 39.

Next we give the analogous result for a linear transformation of a continuous random variable.

Theorem 3.8.2

Suppose X is a continuous random variable. Let Y = aX + b, where a �= 0 and b is a constant. Then

fY (y) = 1|a| fX (

y − b a

)

Proof We begin by writing an expression for the cdf of Y :

FY (y) = P(Y ≤ y) = P(aX + b ≤ y) = P(aX ≤ y − b) At this point we need to consider two cases, the distinction being the sign of a. Suppose, first, that a > 0. Then

FY (y) = P(aX ≤ y − b) = P (

X ≤ y − b a

) and differentiating FY (y) yields fY (y):

fY (y) = ddyFY (y) = d

dy FX

( y − b

a

) = 1

a fX

( y − b

a

) = 1|a| fX

( y − b

a

) If a < 0,

FY (y) = P(aX ≤ y − b) = P (

X > y − b

a

) = 1 − P

( X ≤ y − b

a

) Differentiation in this case gives

fY (y) = ddyFY (y) = d

dy

[ 1 − FX

( y − b

a

)] = −1

a fX

( y − b

a

) = 1|a| fX

( y − b

a

) and the theorem is proved.

Now, armed with the multivariable concepts and techniques covered in Sec- tion 3.7, we can extend the investigation of transformations to functions defined on sets of random variables. In statistics, the most important combination of a set of random variables is often their sum, so we continue this section with the problem of finding the pdf of X + Y .

176 Chapter 3 Random Variables

FINDING THE PDF OF A SUM

Theorem 3.8.3

Suppose that X and Y are independent random variables. Let W = X + Y. Then 1. If X and Y are discrete random variables with pdfs pX (x) and pY (y),

respectively,

pW (w) = ∑ all x

pX (x)pY (w − x)

2. If X and Y are continuous random variables with pdfs fX (x) and fY (y), respectively,

fW (w) = ∫ ∞

−∞ fX (x) fY (w − x) dx

Proof

1. pW (w) = P(W = w) = P(X + Y = w) = P

(⋃ all x

(X = x,Y = w − x) )

= ∑ all x

P(X = x,Y = w − x)

= ∑ all x

P(X = x)P(Y = w − x)

= ∑ all x

pX (x)pY (w − x)

where the next-to-last equality derives from the independence of X and Y .

2. Since X and Y are continuous random variables, we can find fW (w) by dif- ferentiating the corresponding cdf, FW (w). Here, FW (w) = P(X + Y ≤ w) is found by integrating fX,Y (x, y) = fX (x) · fY (y) over the shaded region R, as pictured in Figure 3.8.1.

0 x

w = x + y

y

R

w

w

Figure 3.8.1

By inspection,

Fw(w) = ∫ ∞

−∞

∫ w−x −∞

fX (x) fY (y) dy dx = ∫ ∞

−∞ fX (x)

[∫ w−x −∞

fY (y) dy ]

dx

= ∫ ∞

−∞ fX (x)FY (w − x) dx

Section 3.8 Transforming and Combining Random Variables 177

Assume that the integrand in the above equation is sufficiently smooth so that differentiation and integration can be interchanged. Then we can write

fW (w) = ddw FW (w) = d

dw

∫ ∞ −∞

fX (x)FY (w − x) dx = ∫ ∞

−∞ fX (x)

[ d

dw FY (w − x)

] dx

= ∫ ∞

−∞ fX (x) fY (w − x) dx

and the theorem is proved.

Comment The integral in part (2) above is referred to as the convolution of the functions fX and fY . Besides their frequent appearances in random variable prob- lems, convolutions turn up in many areas of mathematics and engineering.

Example 3.8.2

Suppose that X and Y are two independent binomial random variables, each with the same success probability but defined on m and n trials, respectively. Specifically,

pX (k) = (

m k

) pk(1 − p)m−k, k = 0, 1, . . . , m

and

pY (k) = (

n k

) pk(1 − p)n−k, k = 0, 1, . . . , n

Find pW (w), where W = X + Y . By Theorem 3.8.3, pW (w) =

∑ all x

pX (x)pY (w − x), but the summation over “all x” needs to be interpreted as the set of values for x and w − x such that pX (x) and pY (w−x), respectively, are both nonzero. But that will be true for all integers x from 0 to w. Therefore,

pW (w) = w∑

x=0 pX (x)pY (w − x) =

w∑ x=0

( m x

) px(1 − p)m−x

( n

w − x )

pw−x(1 − p)n−(w−x)

= w∑

x=0

( m x

)( n

w − x )

pw(1 − p)n+m−w

Now, consider an urn having m red chips and n white chips. If w chips are drawn out—without replacement—the probability that exactly x red chips are in the sample is given by the hypergeometric distribution,

P(x reds in sample) = (m

x

)( n w−x

)(m+n w

) (3.8.1) Summing Equation 3.8.1 from x = 0 to x = w must equal 1 (why?), in which case

w∑ x=0

( m x

)( n

w − x )

= (

m + n w

) so

pW (w) = (

m + n w

) pw(1 − p)n+m−w, w = 0, 1, . . . , n + m

Should we recognize pW (w)? Definitely. Compare the structure of pW (w) to the statement of Theorem 3.2.1: The random variable W has a binomial distribution where the probability of success at any given trial is p and the total number of trials is n + m.

178 Chapter 3 Random Variables

Comment Example 3.8.2 shows that the binomial distribution “reproduces” itself—that is, if X and Y are independent binomial random variables with the same value for p, their sum is also a binomial random variable. Not all random variables share that property. The sum of two independent uniform random variables, for ex- ample, is not a uniform random variable (see Question 3.8.5).

Example 3.8.3

Suppose a radiation monitor relies on an electronic sensor, whose lifetime X is mod- eled by the exponential pdf, fX (x) = λe−λx, x > 0. To improve the reliability of the monitor, the manufacturer has included an identical second sensor that is acti- vated only in the event the first sensor malfunctions. (This is called cold redundancy.) Let the random variable Y denote the operating lifetime of the second sensor, in which case the lifetime of the monitor can be written as the sum W = X + Y . Find fW (w).

Since X and Y are both continuous random variables,

fW (w) = ∫ ∞

−∞ fX (x) fY (w − x) dx (3.8.2)

Notice that fX (x) > 0 only if x > 0 and that fY (w − x) > 0 only if x < w. Therefore, the integral in Equation 3.8.2 that goes from −∞ to ∞ reduces to an integral from 0 to w, and we can write

fW (w) = ∫ w

0 fX (x) fY (w − x) dx =

∫ w 0

λe−λxλe−λ(w−x) dx = λ2 ∫ w

0 e−λxe−λ(w−x) dx

= λ2e−λw ∫ w

0 dx = λ2we−λw, w ≥ 0

Comment By integrating fX (x) and fW (w), we can assess the improvement in the monitor’s reliability afforded by the cold redundancy. Since X is an exponential ran- dom variable, E(X ) = 1/λ (recall Question 3.5.11). How different, for example, are P(X ≥ 1/λ) and P(W ≥ 1/λ)? A simple calculation shows that the latter is actually twice the magnitude of the former:

P(X ≥ 1/λ) = ∫ ∞

1/λ λe−λx dx = −e−u∣∣∞1 = e−1 = 0.37

P(W ≥ 1/λ) = ∫ ∞

1/λ λ2we−λw dw = e−u(−u − 1)∣∣∞1 = 2e−1 = 0.74

FINDING THE PDFS OF QUOTIENTS AND PRODUCTS

We conclude this section by considering the pdfs for the quotient and product of two independent random variables. That is, given X and Y , we are looking for fW (w), where (1) W = Y/X and (2) W = XY . Neither of the resulting formulas is as im- portant as the pdf for the sum of two random variables, but both formulas will play key roles in several derivations in Chapter 7. Example 3.8.4 gives an introduction to the theorems in this section.

Section 3.8 Transforming and Combining Random Variables 179

Example 3.8.4

Suppose that X and Y have a joint uniform density over the unit square:

fX,Y (x, y) = {

1 0 < x < 1, 0 < y < 1

0 elsewhere

Find the pdf for their product—that is, find fZ(z), where Z = XY . For 0 < z < 1, FZ(z) is the volume above the shaded region in Figure 3.8.2.

Specifically,

FZ(z) = P(Z ≤ z) = P(XY ≤ z) = ∫∫ R

fX,Y (x, y) dy dx

z = xy

X-axisz

Y R

0

1

Figure 3.8.2

By inspection, we see that the double integral over R can be split up into two double integrals—one letting x range from 0 to z, the other having x-values extend from z to 1:

FZ(z) = ∫ z

0

(∫ 1 0

1 dy

) dx +

∫ 1 z

(∫ z/x 0

1 dy )

dx

But ∫ z 0

(∫ 1 0

1 dy

) dx =

∫ z 0

1 dx = z

and ∫ 1 z

(∫ z/x 0

1 dy )

dx = ∫ 1

z

(z x

) dx = z ln x

∣∣∣1 z

= −z ln z

It follows that

FZ(z) = ⎧⎨ ⎩

0 z ≤ 0 z − z ln z 0 < z < 1 1 z ≥ 1

in which case

fZ(z) = {− ln z 0 < z < 1

0 elsewhere

Theorem 3.8.4

Let X and Y be independent continuous random variables, with pdfs fX (x) and fY (y), respectively. Assume that X is zero for at most a set of isolated points. Let W = Y/X. Then

fW (w) = ∫ ∞

−∞ |x| fX (x) fY (wx) dx

(Continued on next page)

180 Chapter 3 Random Variables

(Theorem 3.8.4 continued)

Proof

FW (w) = P(Y/X ≤ w) = P(Y/X ≤ w and X ≥ 0) + P(Y/X ≤ w and X < 0) = P(Y ≤ wX and X ≥ 0) + P(Y ≥ wX and X < 0) = P(Y ≤ wX and X ≥ 0) + 1 − P(Y ≤ wX and X < 0)

= ∫ ∞

0

∫ wx −∞

fX (x) fY (y) dy dx + 1 − ∫ 0

−∞

∫ wx −∞

fX (x) fY (y) dy dx

Then we differentiate FW (w) to obtain

fW (w) = ddw FW (w) = d

dw

∫ ∞ 0

∫ wx −∞

fX (x) fY (y) dy dx− ddw ∫ 0

−∞

∫ wx −∞

fX (x) fY (y) dy dx

= ∫ ∞

0 fX (x)

( d

dw

∫ wx −∞

fY (y) dy )

dx − ∫ 0

−∞ fX (x)

( d

dw

∫ wx −∞

fY (y) dy )

dx

(3.8.3)

(Note that we are assuming sufficient regularity of the functions to permit inter- change of integration and differentiation.)

To proceed, we need to differentiate the function G(w) = ∫ wx−∞ fY (y) dy with respect to w. By the Fundamental Theorem of Calculus and the chain rule, we find

d dw

G(w) = d dw

∫ wx −∞

fY (y) dy = fY (wx) ddwwx = x fY (wx)

Putting this result into Equation 3.8.3 gives

fW (w) = ∫ ∞

0 x fX (x) fY (wx) dx −

∫ 0 −∞

x fX (x) fY (wx) dx

= ∫ ∞

0 x fX (x) fY (wx) dx +

∫ 0 −∞

(−x) fX (x) fY (wx) dx

= ∫ ∞

0 |x| fX (x) fY (wx) dx +

∫ 0 −∞

|x| fX (x) fY (wx) dx

= ∫ ∞

−∞ |x| fX (x) fY (wx) dx

which completes the proof.

Example 3.8.5

Let X and Y be independent random variables with pdfs fX (x) = λe−λx, x > 0, and fY (y) = λe−λy, y > 0, respectively. Define W = Y/X . Find fW (w).

Substituting into the formula given in Theorem 3.8.4, we can write

fW (w) = ∫ ∞

0 x(λe−λx)(λe−λxw) dx = λ2

∫ ∞ 0

xe−λ(1+w)x dx

= λ 2

λ(1 + w) ∫ ∞

0 xλ(1 + w)e−λ(1+w)x dx

Section 3.8 Transforming and Combining Random Variables 181

Notice that the integral is the expected value of an exponential random variable with parameter λ(1 + w), so it equals 1/λ(1 + w) (recall Example 3.5.6). Therefore,

fW (w) = λ 2

λ(1 + w) 1

λ(1 + w) = 1

(1 + w)2 , w ≥ 0

Theorem 3.8.5

Let X and Y be independent continuous random variables with pdfs fX (x) and fY (y), respectively. Let W = XY. Then

fW (w) = ∫ ∞

−∞

1 |x| fX (x) fY (w/x) dx =

∫ ∞ −∞

1 |x| fX (w/x) fY (x) dx

Proof A line-by-line, straightforward modification of the proof of Theorem 3.8.4 will provide a proof of Theorem 3.8.5. The details are left to the reader.

Example 3.8.6

Suppose that X and Y are independent random variables with pdfs fX (x) = 1, 0 ≤ x ≤ 1, and fY (y) = 2y, 0 ≤ y ≤ 1, respectively. Find fW (w), where W = XY .

According to Theorem 3.8.5,

fW (w) = ∫ ∞

−∞

1 |x| fX (x) fY (w/x) dx

The region of integration, though, needs to be restricted to values of x for which the integrand is positive. But fY (w/x) is positive only if 0 ≤ w/x ≤ 1, which implies that x ≥ w. Moreover, for fX (x) to be positive requires that 0 ≤ x ≤ 1. Any x, then, from w to 1 will yield a positive integrand. Therefore,

fW (w) = ∫ 1

w

1 x

(1)(2w/x) dx = 2w ∫ 1

w

1 x2

dx = 2 − 2w, 0 ≤ w ≤ 1

Comment Theorems 3.8.3, 3.8.4, and 3.8.5 can be adapted to situations where X and Y are not independent by replacing the product of the marginal pdfs with the joint pdf.

Questions

3.8.1. Let Y be a continuous random variable with fY (y) = 12 (1+y),−1 ≤ y ≤ 1. Define the random variable W by W = −4Y + 7. Find fW (w). Be sure to specify those values of w for which fW (w) �= 0. 3.8.2. Let fY (y) = 314 (1 + y2), 0 ≤ y ≤ 2. Define the ran- dom variable W by W = 3Y + 2. Find fW (w). Be sure to specify the values of w for which fW (w) �= 0. 3.8.3. Let X and Y be two independent random variables. Given the marginal pdfs shown below, find the pdf of X + Y . In each case, check to see if X + Y belongs to the same family of pdfs as do X and Y .

(a) pX (k) = e−λ λ k

k! and pY (k) = e−μ μ

k

k! , k = 0, 1, 2, . . .

(b) pX (k) = pY (k) = (1 − p)k−1 p, k = 1, 2, . . .

3.8.4. Suppose fX (x) = xe−x, x ≥ 0, and fY (y) = e−y, y ≥ 0, where X and Y are independent. Find the pdf of X + Y . 3.8.5. Let X and Y be two independent random vari- ables, whose marginal pdfs are given below. Find the pdf of X + Y . (Hint: Consider two cases, 0 ≤ w < 1 and 1 ≤ w ≤ 2.)

fX (x) = 1, 0 ≤ x ≤ 1, and fY (y) = 1, 0 ≤ y ≤ 1 3.8.6. If a random variable V is independent of two in- dependent random variables X and Y , prove that V is independent of X + Y . 3.8.7. Let Y be a continuous nonnegative random vari- able. Show that W = Y 2 has pdf fW (w) = 12√w fY (

√ w).

(Hint: First find FW (w).)

182 Chapter 3 Random Variables

3.8.8. Let Y be a uniform random variable over the inter- val [0, 1]. Find the pdf of W = Y 2. 3.8.9. Let Y be a random variable with fY (y) = 6y(1−y), 0 ≤ y ≤ 1. Find the pdf of W = Y 2. 3.8.10. Suppose the velocity of a gas molecule of mass m is a random variable with pdf fY (y) = ay2e−by2 , y ≥ 0, where a and b are positive constants depending on the gas. Find the pdf of the kinetic energy, W = (m/2)Y 2, of such a molecule.

3.8.11. Given that X and Y are independent random vari- ables, find the pdf of XY for the following two sets of marginal pdfs:

(a) fX (x) = 1, 0 ≤ x ≤ 1, and fY (y) = 1, 0 ≤ y ≤ 1 (b) fX (x) = 2x, 0 ≤ x ≤ 1, and fY (y) = 2y, 0 ≤ y ≤ 1

3.8.12. Let X and Y be two independent random variables. Given the marginal pdfs indicated below, find the cdf of Y/X . (Hint: Consider two cases, 0 ≤ w ≤ 1 and 1 < w.) (a) fX (x) = 1, 0 ≤ x ≤ 1, and fY (y) = 1, 0 ≤ y ≤ 1 (b) fX (x) = 2x, 0 ≤ x ≤ 1, and fY (y) = 2y, 0 ≤ y ≤ 1

3.8.13. Suppose that X and Y are two independent ran- dom variables, where fX (x) = xe−x, x ≥ 0, and fY (y) = e−y, y ≥ 0. Find the pdf of Y/X .

3.9 Further Properties of the Mean and Variance Sections 3.5 and 3.6 introduced the basic definitions related to the expected value and variance of single random variables. We learned how to calculate E(W), E[g (W)], E(aW + b), Var(W), and Var(aW + b), where a and b are any constants and W could be either a discrete or a continuous random variable. The purpose of this section is to examine certain multivariable extensions of those results, based on the joint pdf material covered in Section 3.7.

We begin with a theorem that generalizes E[g (W)]. While it is stated here for the case of two random variables, it extends in a very straightforward way to include functions of n random variables.

Theorem 3.9.1

1. Suppose X and Y are discrete random variables with joint pdf pX,Y (x, y), and let g (X,Y) be a function of X and Y. Then the expected value of the random variable g (X,Y) is given by

E[g (X,Y)] = ∑ all x

∑ all y

g (x, y) · pX,Y (x, y)

provided ∑ all x

∑ all y

|g (x, y)| · pX,Y (x, y) < ∞.

2. Suppose X and Y are continuous random variables with joint pdf fX,Y (x, y), and let g (X,Y) be a continuous function. Then the expected value of the ran- dom variable g (X,Y) is given by

E[g (X,Y)] = ∫ ∞

−∞

∫ ∞ −∞

g (x, y) · fX,Y (x, y) dx dy

provided ∫∞ −∞

∫∞ −∞ |g (x, y)| · fX,Y (x, y) dx dy < ∞.

Proof The basic approach taken in deriving this result is similar to the method followed in the proof of Theorem 3.5.3. See (136) for details.

Example 3.9.1

Consider the two random variables X and Y whose joint pdf is detailed in the 2 × 4 matrix shown in Table 3.9.1. Let

g (X,Y) = 3X − 2XY + Y

Section 3.9 Further Properties of the Mean and Variance 183

Table 3.9.1

Y

0 1 2 3

0 1 8

1 4

1 8

0

X

1 0 1 8

1 4

1 8

Table 3.9.2 z 0 1 2 3

fZ(z) 14 1 2

1 4 0

Find E[g (X,Y)] two ways—first, by using the basic definition of an expected value, and second, by using Theorem 3.9.1.

Let Z = 3X − 2XY + Y . By inspection, Z takes on the values 0, 1, 2, and 3 according to the pdf fZ(z) shown in Table 3.9.2. Then from the basic definition that an expected value is a weighted average, we see that E[g (X,Y)] is equal to 1:

E[g (X,Y)] = E(Z) = ∑ all z

z · fZ(z)

= 0 · 1 4

+ 1 · 1 2

+ 2 · 1 4

+ 3 · 0

= 1 The same answer is obtained by applying Theorem 3.9.1 to the joint pdf given in Figure 3.9.1:

E[g (X,Y)] = 0 · 1 8

+ 1 · 1 4

+ 2 · 1 8

+ 3 · 0 + 3 · 0 + 2 · 1 8

+ 1 · 1 4

+ 0 · 1 8

= 1 The advantage, of course, enjoyed by the latter solution is that we avoid the inter- mediate step of having to determine fZ(z).

Example 3.9.2

An electrical circuit has three resistors, RX , RY , and RZ, wired in parallel (see Fig- ure 3.9.1). The nominal resistance of each is fifteen ohms, but their actual resistances, X , Y , and Z, vary between ten and twenty according to the joint pdf

fX,Y,Z(x, y, z) = 1675,000(xy + xz + yz), 10 ≤ x ≤ 20 10 ≤ y ≤ 20 10 ≤ z ≤ 20

What is the expected resistance for the circuit?

XR

YR

ZR

Figure 3.9.1

184 Chapter 3 Random Variables

Let R denote the circuit’s resistance. A well-known result in physics holds that

1 R

= 1 X

+ 1 Y

+ 1 Z

or, equivalently,

R = XYZ XY + XZ + YZ = R(X,Y, Z)

Integrating R(x, y, z) · fX,Y,Z(x, y, z) shows that the expected resistance is five:

E(R) = ∫ 20

10

∫ 20 10

∫ 20 10

xyz xy + xz + yz ·

1 675,000

(xy + xz + yz) dx dy dz

= 1 675,000

∫ 20 10

∫ 20 10

∫ 20 10

xyz dx dy dz

= 5.0

Theorem 3.9.2

Let X and Y be any two random variables (discrete or continuous, dependent or independent), and let a and b be any two constants. Then

E(aX + bY) = aE(X ) + bE(Y) provided E(X ) and E(Y) are both finite.

Proof Consider the continuous case (the discrete case is proved much the same way). Let fX,Y (x, y) be the joint pdf of X and Y , and define g (X,Y) = aX + bY . By Theorem 3.9.1,

E(aX + bY) = ∫ ∞

−∞

∫ ∞ −∞

(ax + by) fX,Y (x, y) dx dy

= ∫ ∞

−∞

∫ ∞ −∞

(ax) fX,Y (x, y) dx dy + ∫ ∞

−∞

∫ ∞ −∞

(by) fX,Y (x, y) dx dy

= a ∫ ∞

−∞ x [∫ ∞

−∞ fX,Y (x, y) dy

] dx + b

∫ ∞ −∞

y [∫ ∞

−∞ fX,Y (x, y) dx

] dy

= a ∫ ∞

−∞ x fX (x) dx + b

∫ ∞ −∞

y fY (y) dy

= aE(X ) + bE(Y)

Corollary 3.9.1

Let W1,W2, . . . ,Wn be any random variables for which E(|Wi|) < ∞, i = 1, 2, . . . , n, and let a1, a2, . . . , an be any set of constants. Then

E(a1W1 + a2W2 + · · · + anWn) = a1E(W1) + a2E(W2) + · · · + anE(Wn)

Example 3.9.3

Let X be a binomial random variable defined on n independent trials, each trial resulting in success with probability p. Find E(X ).

Note, first, that X can be thought of as a sum, X = X1 + X2 + · · · + Xn, where Xi represents the number of successes occurring at the ith trial:

Xi = {

1 if the ith trial produces a success 0 if the ith trial produces a failure

Section 3.9 Further Properties of the Mean and Variance 185

(Any Xi defined in this way on an individual trial is called a Bernoulli random variable. Every binomial random variable, then, can be thought of as the sum of n independent Bernoullis.) By assumption, pXi (1) = p and pXi (0) = 1 − p, i = 1, 2, . . . , n. Using the corollary,

E(X ) = E(X1) + E(X2) + · · · + E(Xn) = n · E(X1)

the last step being a consequence of the Xi’s having identical distributions. But

E(X1) = 1 · p + 0 · (1 − p) = p so E(X ) = np, which is what we found before (recall Theorem 3.5.1).

Comment The problem-solving implications of Theorem 3.9.2 and its corollary should not be underestimated. There are many real-world events that can be mod- eled as a linear combination a1W1 + a2W2 + · · · + anWn, where the Wi’s are relatively simple random variables. Finding E(a1W1 + a2W2 + · · · + anWn) directly may be pro- hibitively difficult because of the inherent complexity of the linear combination. It may very well be the case, though, that calculating the individual E(Wi)’s is easy. Compare, for instance, Example 3.9.3 with Theorem 3.5.1. Both derive the formula that E(X ) = np when X is a binomial random variable. However, the approach taken in Example 3.9.3 (i.e., using Theorem 3.9.2) is much easier. The next several examples further explore the technique of using linear combinations to facilitate the calculation of expected values.

Example 3.9.4

A disgruntled secretary is upset about having to stuff envelopes. Handed a box of n letters and n envelopes, she vents her frustration by putting the letters into the en- velopes at random. How many people, on the average, will receive their correct mail?

If X denotes the number of envelopes properly stuffed, what we want is E(X ). However, applying Definition 3.5.1 here would prove formidable because of the difficulty in getting a workable expression for pX (k). By using the corollary to Theorem 3.9.2, though, we can solve the problem quite easily.

Let Xi denote a random variable equal to the number of correct letters put into the ith envelope, i = 1, 2, . . . , n. Then Xi equals 0 or 1, and

pXi (k) = P(Xi = k) =

⎧⎪⎨ ⎪⎩

1 n

for k = 1 n − 1

n for k = 0

But X = X1 +X2 +· · ·+Xn and E(X ) = E(X1)+E(X2)+· · ·+E(Xn). Furthermore, each of the Xi’s has the same expected value, 1/n:

E(Xi) = 1∑

k=0 k · P(Xi = k) = 0 · n − 1n + 1 ·

1 n

= 1 n

It follows that

E(X ) = n∑

i=1 E(Xi) = n ·

( 1 n

)

= 1 showing that, regardless of n, the expected number of properly stuffed envelopes is one. (Are the Xi’s independent? Does it matter?)

186 Chapter 3 Random Variables

Example 3.9.5

Ten fair dice are rolled. Calculate the expected value of the sum of the faces showing. If the random variable X denotes the sum of the faces showing on the ten dice,

then

X = X1 + X2 + · · · + X10 where Xi is the number showing on the ith die, i = 1, 2, . . . , 10. By assumption, pXi (k) = 16 for k = 1, 2, 3, 4, 5, 6, so E(Xi) =

6∑ k=1

k · 16 = 16 6∑

k=1 k = 16 · 6(7)2 = 3.5. By

the corollary to Theorem 3.9.2,

E(X ) = E(X1) + E(X2) + · · · + E(X10) = 10(3.5) = 35

Notice that E(X ) can also be deduced here by appealing to the notion that ex- pected values are centers of gravity. It should be clear from our work with combina- torics that P(X = 10) = P(X = 60), P(X = 11) = P(X = 59), P(X = 12) = P(X = 58), and so on. In other words, the probability function pX (k) is symmetric, which implies that its center of gravity is the midpoint of the range of its X -values. It must be the case, then, that E(X ) equals 10+602 or 35.

Example 3.9.6

The honor count in a (thirteen-card) bridge hand can vary from zero to thirty-seven according to the formula:

honor count = 4 · (number of aces)+3 · (number of kings)+2 · (number of queens) + 1 · (number of jacks)

What is the expected honor count of North’s hand? The solution here is a bit unusual in that we use the corollary to Theorem 3.9.2

backward. If Xi, i = 1, 2, 3, 4, denotes the honor count for players North, South, East, and West, respectively, and if X denotes the analogous sum for the entire deck, we can write

X = X1 + X2 + X3 + X4 But

X = E(X ) = 4 · 4 + 3 · 4 + 2 · 4 + 1 · 4 = 40 By symmetry, E(Xi) = E(Xj), i �= j, so it follows that 40 = 4 · E(X1), which implies that ten is the expected honor count of North’s hand. (Try doing this problem directly, without making use of the fact that the deck’s honor count is forty.)

EXPECTED VALUES OF PRODUCTS: A SPECIAL CASE

We know from Theorem 3.9.1 that for any two random variables X and Y ,

E(XY) =

⎧⎪⎪⎨ ⎪⎪⎩ ∑ all x

∑ all y

xypX,Y (x, y) if X and Y are discrete∫ ∞ −∞

∫ ∞ −∞

xy fX,Y (x, y) dx dy if X and Y are continuous

If, however, X and Y are independent, there is an easier way to calculate E(XY).

Section 3.9 Further Properties of the Mean and Variance 187

Theorem 3.9.3

If X and Y are independent random variables,

E(XY) = E(X ) · E(Y) provided E(X ) and E(Y) both exist.

Proof Suppose X and Y are both discrete random variables. Then their joint pdf, pX,Y (x, y), can be replaced by the product of their marginal pdfs, pX (x)·pY (y), and the double summation required by Theorem 3.9.1 can be written as the product of two single summations:

E(XY) = ∑ all x

∑ all y

xy · pX,Y (x, y)

= ∑ all x

∑ all y

xy · pX (x) · pY (y)

= ∑ all x

x · pX (x) · ⎡ ⎣∑

all y

y · pY (y) ⎤ ⎦

= E(X ) · E(Y) The proof when X and Y are both continuous random variables is left as an exercise.

Questions

3.9.1. Suppose that r chips are drawn with replace- ment from an urn containing n chips, numbered 1 through n. Let V denote the sum of the numbers drawn. Find E(V).

3.9.2. Suppose that fX,Y (x, y) = λ2e−λ(x+y), 0 ≤ x, 0 ≤ y. Find E(X + Y). 3.9.3. Suppose that fX,Y (x, y) = 23 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 [recall Question 3.7.19(c)]. Find E(X + Y). 3.9.4. Marksmanship competition at a certain level re- quires each contestant to take ten shots with each of two different handguns. Final scores are computed by taking a weighted average of four times the number of bull’s-eyes made with the first gun plus six times the number gotten with the second. If Cathie has a 30% chance of hitting the bull’s-eye with each shot from the first gun and a 40% chance with each shot from the second gun, what is her expected score?

3.9.5. Suppose that Xi is a random variable for which E(Xi) = μ �= 0, i = 1, 2, . . . , n. Under what conditions will the following be true?

E

( n∑

i=1 aiXi

) = μ

3.9.6. Suppose that the daily closing price of stock goes up an eighth of a point with probability p and down an eighth of a point with probability q, where p > q. After n days how much gain can we expect the stock to have achieved? Assume that the daily price fluctuations are independent events.

3.9.7. An urn contains r red balls and w white balls. A sample of n balls is drawn in order and without replace- ment. Let Xi be 1 if the ith draw is red and 0 otherwise, i = 1, 2, . . . , n. (a) Show that E(Xi) = E(X1), i = 2, 3, . . . , n. (b) Use the corollary to Theorem 3.9.2 to show that the expected number of red balls is nr/(r + w). 3.9.8. Suppose two fair dice are tossed. Find the expected value of the product of the faces showing.

3.9.9. Find E(R) for a two-resistor circuit similar to the one described in Example 3.9.2, where fX,Y (x, y) = k(x+ y), 10 ≤ x ≤ 20, 10 ≤ y ≤ 20. 3.9.10. Suppose that X and Y are both uniformly dis- tributed over the interval [0, 1]. Calculate the expected value of the square of the distance of the random point (X,Y) from the origin; that is, find E(X 2 + Y 2). (Hint: See Question 3.8.8.)

188 Chapter 3 Random Variables

3.9.11. Suppose X represents a point picked at random from the interval [0, 1] on the x-axis, and Y is a point picked at random from the interval [0, 1] on the y-axis. Assume that X and Y are independent. What is the ex- pected value of the area of the triangle formed by the points (X, 0), (0,Y), and (0, 0)?

3.9.12. Suppose Y1,Y2, . . . ,Yn is a random sample from the uniform pdf over [0, 1]. The geometric mean of the numbers is the random variable n

√ Y1Y2 · · · · · Yn. Compare

the expected value of the geometric mean to that of the arithmetic mean Ȳ .

CALCULATING THE VARIANCE OF A SUM OF RANDOM VARIABLES

When random variables are not independent, a measure of the relationship between them, their covariance, enters into the picture.

Definition 3.9.1 Given random variables X and Y with finite variances, define the covariance of X and Y , written Cov(X,Y), as

Cov(X,Y) = E(XY) − E(X )E(Y)

Theorem 3.9.4

If X and Y are independent, then Cov(X,Y) = 0. Proof If X andY are independent, by Theorem 3.9.3, E(XY) = E(X )E(Y). Then

Cov(X,Y) = E(XY) − E(X )E(Y) = E(X )E(Y) − E(X )E(Y) = 0 The converse of Theorem 3.9.4 is not true. Just because Cov(X,Y) = 0, we

cannot conclude that X and Y are independent. Example 3.9.7 is a case in point.

Example 3.9.7

Consider the sample space S = {(−2, 4), (−1, 1), (0, 0), (1, 1), (2, 4)}, where each point is assumed to be equally likely. Define the random variable X to be the first component of a sample point and Y , the second. Then X (−2, 4) = −2,Y(−2, 4) = 4, and so on.

Notice that X and Y are dependent:

1 5

= P(X = 1,Y = 1) �= P(X = 1) · P(Y = 1) = 1 5

· 2 5

= 2 25

However, the convariance of X and Y is zero:

E(XY) = [(−8) + (−1) + 0 + 1 + 8] · 1 5

= 0

E(X ) = [(−2) + (−1) + 0 + 1 + 2] · 1 5

= 0 and

E(Y) = (4 + 1 + 0 + 1 + 4) · 1 5

= 2 so

Cov(X,Y) = E(XY) − E(X ) · E(Y) = 0 − 0 · 2 = 0 Theorem 3.9.5 demonstrates the role of the covariance in finding the variance of

a sum of random variables that are not necessarily independent.

Theorem 3.9.5

Suppose X and Y are random variables with finite variances, and a and b are con- stants. Then

Var(aX + bY) = a2Var(X ) + b2Var(Y) + 2ab Cov(X,Y)

Section 3.9 Further Properties of the Mean and Variance 189

Proof For convenience, denote E(X ) by μX and E(Y) by μY . Then E(aX + bY) = aμX + bμY and

Var(aX + bY) = E[(aX + bY)2] − (aμX + bμY )2

= E(a2X 2 + b2Y 2 + 2abXY) − (a2μ2X + b2μ2Y + 2abμX μY ) = [E(a2X 2) − a2μ2X ] + [E(b2Y 2) − b2μ2Y ] + [2abE(XY) − 2abμX μY ] = a2[E(X 2) − μ2X ] + b2[E(Y 2) − μ2Y ] + 2ab[E(XY) − μX μY ] = a2 Var(X) + b2 Var(Y) + 2abCov(X,Y)

Example 3.9.8

For the joint pdf fX,Y (x, y) = x+y, 0 ≤ x ≤ 1, 0 ≤ y ≤ 1, find the variance of X +Y . Since X and Y are not independent,

Var(X + Y) = Var(X ) + Var(Y) + 2Cov(X,Y) The pdf is symmetric in X and Y , so Var(X ) = Var(Y), and we can write Var(X + Y) = 2[Var(X ) + Cov(X,Y)].

To calculate Var(X ), the marginal pdf of X is needed. But

fX (x) = ∫ 1

0 (x + y)dy = x + 1

2

μX = ∫ 1

0 x(x + 1

2 )dx =

∫ 1 0

(x2 + x 2

)dx = 7 12

E(X 2) = ∫ 1

0 x2(x + 1

2 )dx =

∫ 1 0

(x3 + x 2

2 )dx = 5

12

Var(X ) = E(X 2) − μ2X = 5 12

− (

7 12

)2 = 11

144

Then

E(XY) = ∫ 1

0

∫ 1 0

xy(x + y)dydx = ∫ 1

0

( x2

2 + x

3

) dx = x

3

6 + x

2

6

∣∣∣∣1 0

= 1 3

so, putting all of the pieces together,

Cov(X,Y) = 1/3 − (7/12)(7/12) = −1/144 and, finally, Var(X + Y) = 2[11/144 + (−1/144)] = 5/36

The two corollaries that follow are straightforward extensions of Theorem 3.9.5 to n variables. The details of the proof will be left as an exercise.

Corollary Suppose that W1,W2, . . . ,Wn are random variables with finite variances. Then

Var

( a∑

i=1 aiWi

) =

n∑ i=1

a2i Var(Wi) + 2 ∑ i< j

aia jCov(Wi,Wj)

Corollary Suppose that W1,W2, . . . ,Wn are independent random variables with finite vari- ances. Then

Var(W1 + W2 + · · · + Wn) = Var(W1) + Var(W2) + · · · + Var(Wn)

More discussion of the covariance and its role in measuring the relationship be- tween random variables occurs in Section 11.4.

190 Chapter 3 Random Variables

Example 3.9.9

The binomial random variable, being a sum of n independent Bernoullis, is an obvi- ous candidate for the corollary to Theorem 3.9.5 on the sum of independent random variables. Let Xi denote the number of successes occurring on the ith trial. Then

Xi = {

1 with probability p 0 with probability 1 − p

and

X = X1 + X2 + · · · + Xn = total number of successes in n trials Find Var(X ).

Note that

E(Xi) = 1 · p + 0 · (1 − p) = p and

E ( X 2i

) = (1)2 · p + (0)2 · (1 − p) = p so

Var(Xi) = E ( X 2i

)− [E(Xi)]2 = p − p2 = p(1 − p)

It follows, then, that the variance of a binomial random variable is np(1 − p):

Var(X ) = n∑

i=1 Var(Xi) = np(1 − p)

Example 3.9.10

Recall the hypergeometric model—an urn contains N chips, r red and w white (r + w = N); a random sample of size n is selected without replacement and the random variable X is defined to be the number of red chips in the sample. As in the previous example, write X as a sum of simple random variables.

Xi = {

1 if the ith chip drawn is red 0 otherwise

Then X = X1 + X2 + · · · + Xn. Clearly,

E(Xi) = 1 · rN + 0 · w

N = r

N

and E(X ) = n ( rN ) = np, where p = rN . Since X 2i = Xi, E(X 2i ) = E(Xi) = rN and

Var(Xi) = E(X 2i ) − [E(Xi)]2 = r N

− ( r

N

)2 = p(1 − p)

Also, for any j �= k, Cov(Xj, Xk) = E(XjXk) − E(Xj)E(Xk)

= 1 · P(XjXk = 1) − ( r

N

)2 = r

N · r − 1

N − 1 − r2

N2 = − r

N · N − r

N · 1

N − 1

Section 3.9 Further Properties of the Mean and Variance 191

From the first corollary to Theorem 3.9.5, then,

Var(X ) = n∑

i=1 Var(Xi) + 2

∑ j<k

Cov(Xj, Xk)

= np(1 − p) − 2 (

n 2

) p(1 − p) · 1

N − 1

= p(1 − p) [

n − n(n − 1) N − 1

]

= np(1 − p) · N − n N − 1

Example 3.9.11

In statistics, it is often necessary to draw inferences based on W , the average com- puted from a random sample of n observations. Two properties of W are especially important. First, if theWi’s come from a population where the mean is μ, the corollary to Theorem 3.9.2 implies that E(W) = μ. Second, if the Wi’s come from a population whose variance is σ2, then Var(W) = σ2/n. To verify the latter, we can appeal again to Theorem 3.9.5. Write

W = 1 n

n∑ i=1

Wi = 1n · W1 + 1 n

· W2 + · · · + 1n · Wn

Then

Var(W) = (

1 n

)2 · Var(W1) +

( 1 n

)2 · Var(W2) + · · · +

( 1 n

)2 · Var(Wn)

= (

1 n

)2 σ2 +

( 1 n

)2 σ2 + · · · +

( 1 n

)2 σ2

= σ 2

n

Questions

3.9.13. Suppose that two dice are thrown. Let X be the number showing on the first die and let Y be the larger of the two numbers showing. Find Cov(X,Y).

3.9.14. Show that

Cov(aX + b, cY + d) = acCov(X,Y) for any constants a, b, c, and d.

3.9.15. Let U be a random variable uniformly distributed over [0, 2π]. Define X = cosU and Y = sinU . Show that X and Y are dependent but that Cov(X,Y) = 0. 3.9.16. Let X and Y be random variables with

fX,Y (x, y) = {

1, −y < x < y, 0 < y < 1 0, elsewhere

Show that Cov(X,Y) = 0 but that X and Y are depen- dent.

3.9.17. Suppose that fX,Y (x, y) = λ2e−λ(x+y), 0 ≤ x, 0 ≤ y. Find Var(X + Y). (Hint: See Questions 3.6.11 and 3.9.2.)

3.9.18. Suppose that fX,Y (x, y) = 23 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1. Find Var(X + Y). (Hint: See Question 3.9.3.) 3.9.19. Suppose that fX,Y (x, y) = 32 (x2 + y2), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1. Find Var(X + Y). 3.9.20. Let X be a binomial random variable based on n trials and a success probability of pX ; let Y be an inde- pendent binomial random variable based on m trials and a success probability of pY . Find E(W) and Var(W), where W = 4X + 6Y . 3.9.21. A Poisson random variable has pdf pX (k) = e−λ λkk! , k = 0, 1, 2, . . . and λ> 0 (see Section 4.2). Also, E(X ) = λ. Suppose the Poisson random variable U is the number of calls for technical assistance received by a computer company during the firm’s nine normal workday hours, with the average number of calls per hour equal 7.0. Also suppose each call costs the company $50. Let V be a Poisson random variable representing the number of calls for technical assistance received during a day’s remaining

192 Chapter 3 Random Variables

fifteen hours. Assume the average number of calls per hour is four for that time period and that each such call costs the company $60. Find the expected cost and the variance of the cost associated with the calls received dur- ing a twenty-four-hour day.

3.9.22. A mason is contracted to build a patio retaining wall. Plans call for the base of the wall to be a row of fifty 10-inch bricks, each separated by 12 -inch-thick mortar. Suppose that the bricks used are randomly chosen from a population of bricks whose mean length is 10 inches and whose standard deviation is 132 inch. Also, suppose that the mason, on the average, will make the mortar 12 inch thick, but that the actual dimension will vary from brick to brick, the standard deviation of the thicknesses being

1 16 inch. What is the standard deviation of L, the length of the first row of the wall? What assumption are you making?

3.9.23. An electric circuit has six resistors wired in series, each nominally being five ohms. What is the maximum standard deviation that can be allowed in the manufac- ture of these resistors if the combined circuit resistance is to have a standard deviation no greater than 0.4 ohm?

3.9.24. A gambler plays n hands of poker. If he wins the kth hand, he collects k dollars; if he loses the kth hand, he collects nothing. Let T denote his total winnings in n hands. Assuming that his chances of winning each hand are constant and independent of his success or failure at any other hand, find E(T) and Var(T).

3.10 Order Statistics The single-variable transformation taken up in Section 3.4 involved a standard linear operation, Y = aX + b. The bivariate transformations in Section 3.8 were similarly arithmetic, typically being concerned with either sums or products. In this section we will consider a different sort of transformation, one involving the ordering of an entire set of random variables. This particular transformation has wide applicability in many areas of statistics, and we will see some of its consequences in later chapters.

Definition 3.10.1 Let Y be a continuous random variable for which y1, y2, . . . , yn are the values of a random sample of size n. Reorder the yi’s from smallest to largest:

y′1 < y ′ 2 < · · · < y′n

(No two of the yi’s are equal, except with probability zero, since Y is continu- ous.) Define the random variable Y ′i to have the value y

′ i, 1 ≤ i ≤ n. Then Y ′i

is called the ith order statistic. Sometimes Y ′n and Y ′ 1 are denoted Ymax and Ymin,

respectively.

Example 3.10.1

Suppose that four measurements are made on the random variable Y : y1 = 3.4, y2 = 4.6, y3 = 2.6, and y4 = 3.2. The corresponding ordered sample would be

2.6 < 3.2 < 3.4 < 4.6

The random variable representing the smallest observation would be denoted Y ′1, with its value for this particular sample being 2.6. Similarly, the value for the second order statistic, Y ′2, is 3.2, and so on.

THE DISTRIBUTION OF EXTREME ORDER STATISTICS

By definition, every observation in a random sample has the same pdf. For example, if a set of four measurements is taken from a normal distribution with μ = 80 and σ = 15, then fY1 (y), fY2 (y), fY3 (y), and fY4 (y) are all the same—each is a normal pdf with μ = 80 and σ = 15. The pdf describing an ordered observation, though, is not the same as the pdf describing a random observation. Intuitively, that makes sense. If a single observation is drawn from a normal distribution with μ = 80 and σ = 15,

Section 3.10 Order Statistics 193

it would not be surprising if that observation were to take on a value near eighty. On the other hand, if a random sample of n = 100 observations is drawn from that same distribution, we would not expect the smallest observation—that is, Ymin—to be anywhere near eighty. Common sense tells us that that smallest observation is likely to be much smaller than eighty, just as the largest observation, Ymax, is likely to be much larger than eighty.

It follows, then, that before we can do any probability calculations—or any ap- plications whatsoever—involving order statistics, we need to know the pdf of Y ′i for i = 1, 2, . . . , n. We begin by investigating the pdfs of the “extreme” order statistics, fYmax (y) and fYmin (y). These are the simplest to work with. At the end of the section we return to the more general problems of finding (1) the pdf of Y ′i for any i and (2) the joint pdf of Y ′i and Y

′ j , where i < j.

Theorem 3.10.1

Suppose that Y1, Y2, . . ., Yn is a random sample of continuous random variables, each having pdf fY (y) and cdf FY (y). Then

a. The pdf of the largest order statistic is

fYmax (y) = fY ′n (y) = n[FY (y)]n−1 fY (y) b. The pdf of the smallest order statistic is

fYmin (y) = fY ′1 (y) = n[1 − FY (y)]n−1 fY (y)

Proof Finding the pdfs of Ymax and Ymin is accomplished by using the now- familiar technique of differentiating a random variable’s cdf. Consider, for example, the case of the largest order statistic, Y ′n:

FY ′n (y) = FYmax (y) = P(Ymax ≤ y) = P(Y1 ≤ y,Y2 ≤ y, · · · ,Yn ≤ y) = P(Y1 ≤ y) · P(Y2 ≤ y) · · · P(Yn ≤ y) (why?) = [FY (y)]n

Therefore,

fY ′n (y) = d/dy[[FY (y)]n] = n[FY (y)]n−1 fY (y) Similarly, for the smallest order statistic (i = 1),

FY ′1 (y) = FYmin (y) = P(Ymin ≤ y) = 1 − P(Ymin > y) = 1 − P(Y1 > y) · P(Y2 > y) · · · P(Yn > y)

= 1 − [1 − FY (y)]n

Therefore,

fY ′1 (y) = d/dy[1 − [1 − FY (y)]n] = n[1 − FY (y)]n−1 fY (y)

Example 3.10.2

Suppose a random sample of n = 3 observations, Y1, Y2, and Y3, is taken from the exponential pdf, fY (y) = e−y, y ≥ 0. Compare fY1 (y) with fY ′1 (y). Intuitively, which will be larger, P(Y1 < 1) or P(Y ′1 < 1)?

194 Chapter 3 Random Variables

The pdf for Y1, of course, is just the pdf of the distribution being sampled, that is,

fY1 (y) = fY (y) = e−y, y ≥ 0 To find the pdf for Y ′1 requires that we apply the formula given in the proof of The- orem 3.10.1 for fYmin (y). Note, first of all, that

FY (y) = ∫ y

0 e−tdt = −e−t ∣∣y0 = 1 − e−y

Then, since n = 3 (and i = 1), we can write fY ′1 (y) = 3[1 − (1 − e−y)]2e−y

= 3e−3y, y ≥ 0 Figure 3.10.1 shows the two pdfs plotted on the same set of axes. Compared to

fY1 (y), the pdf for Y ′ 1 has more of its area located above the smaller values of y (where

Y ′1 is more likely to lie). For example, the probability that the smallest observation (out of three) is less than 1 is 95%, while the probability that a random observation is less than 1 is only 63%:

P(Y ′1 < 1) = ∫ 1

0 3e−3y dy =

∫ 3 0

e−u du = −e−u ∣∣∣∣∣ 3

0

= 1 − e−3

= 0.95

P(Y1 < 1) = ∫ 1

0 e−y dy = −e−y

∣∣∣∣∣ 1

0

= 1 − e−1

= 0.63

3

2

1

0 1 2

Probability density

3 4 5

f (y) = 3eY9 1 –3y

f (y) = eY1 –y

y

Figure 3.10.1

Example 3.10.3

Suppose a random sample of size 10 is drawn from a continuous pdf fY (y). What is the probability that the largest observation, Y ′10, is less than the pdf’s median, m?

Using the formula for fY ′10 (y) = fYmax (y) given in the proof of Theorem 3.10.1, it is certainly true that

P(Y ′10 < m) = ∫ m

−∞ 10 fY (y)[FY (y)]9dy (3.10.1)

but the problem does not specify fY (y), so Equation 3.10.1 is of no help.

Section 3.10 Order Statistics 195

Fortunately, a much simpler solution is available, even if fY (y) were specified: The event “Y ′10 < m” is equivalent to the event “Y1 < m ∩ Y2 < m ∩ · · · ∩ Y10 < m.” Therefore,

P(Y ′10 < m) = P(Y1 < m,Y2 < m, . . . ,Y10 < m) (3.10.2) But the ten observations here are independent, so the intersection probability im- plicit on the right-hand side of Equation 3.10.2 factors into a product of ten terms. Moreover, each of those terms equals 12 (by definition of the median), so

P(Y ′10 < m) = P(Y1 < m) · P(Y2 < m) · · · P(Y10 < m) = ( 12)10 = 0.00098

Example 3.10.4

To find order statistics for discrete pdfs, the probability arguments of the type used in the proof of Theorem 3.10.1 can be be employed. The example of finding the pdf of Xmin for the discrete density function pX (k), k = 0, 1, 2, . . . suffices to demonstrate this point.

Given a random sample X1, X2, . . . , Xn from pX (k), choose an arbitrary non-

negative integer m. Recall that the cdf in this case is given by FX (m) = m∑

k=0 pk.

Consider the events

A =(m ≤ X1 ∩ m ≤ X2 ∩ · · · ∩ m ≤ Xn) and B =(m + 1 ≤ X1 ∩ m + 1 ≤ X2 ∩ · · · ∩ m + 1 ≤ Xn)

Then pXmin (m) = P(A ∩ BC) = P(A) − P(A ∩ B) = P(A) − P(B), where A ∩ B = B, since B ⊂ A.

Now P(A) = P(m ≤ X1) · P(m ≤ X2) · . . . · P(m ≤ Xn) = [1 − FX (m − 1)]n by the independence of the Xi. Similarly P(B) = [1 − FX (m)]n, so

pYmin (m) = [1 − FX (m − 1)]n − [1 − FX (m)]n

A GENERAL FORMULA FOR fYi'(y)

Having discussed two special cases of order statistics, Ymin and Ymax, we now turn to the more general problem of finding the pdf for the ith order statistic, where i can be any integer from 1 through n.

Theorem 3.10.2

Let Y1,Y2, . . . ,Yn be a random sample of continuous random variables drawn from a distribution having pdf fY (y) and cdf FY (y). The pdf of the ith order statistic is given by

fY ′i (y) = n!

(i − 1)!(n − i)! [FY (y)] i−1[1 − FY (y)]n−i fY (y)

for 1 ≤ i ≤ n. Proof We will give a heuristic argument that draws on the similarity between the statement of Theorem 3.10.2 and the binomial distribution. For a formal induction proof verifying the expression given for fY ′i (y), see (105).

Recall the derivation of the binomial probability function, pX (k) = P(X = k) =

( n k

) pk(1 − p)n−k, where X is the number of successes in n independent

(Continued on next page)

196 Chapter 3 Random Variables

(Theorem 3.10.2 continued)

trials, and p is the probability that any given trial ends in success. Central to that derivation was the recognition that the event “X = k” is actually a union of all the different (mutually exclusive) sequences having exactly k successes and n − k failures. Because the trials are independent, the probability of any such sequence is pk(1−p)n−k and the number of such sequences (by Theorem 2.6.2) is n!/[k!(n−k)!] (or

( n k

) ), so the probability that X = k is the product

( n k

) pk(1 − p)n−k.

Here we are looking for the pdf of the ith order statistic at some point y, that is, fY ′i (y). As was the case with the binomial, that pdf will reduce to a combina- torial term times the probability associated with an intersection of independent events. The only fundamental difference is that Y ′i is a continuous random vari- able, whereas the binomial X is discrete, which means that what we find here will be a probability density function.

By Theorem 2.6.2, there are n!/[(i − 1)!1!(n − i)!] ways that n observations can be parceled into three groups such that the ith largest is at the point y (see Fig- ure 3.10.2). Moreover, the likelihood associated with any particular set of points having the configuration pictured in Figure 3.10.2 will be the probability that i − 1 (independent) observations are all less than y, n − i observations are greater than y, and one observation is at y. The probability density associated with those con- straints for a given set of points would be [FY (y)]i−1[1−FY (y)]n−i fY (y). The proba- bility density, then, that the ith order statistic is located at the point y is the product

fY ′i (y) = n!

(i − 1)!(n − i)! [FY (y)] i−1[1 − FY (y)]n−i fY (y)

Y-axis y

i – 1 obs. 1 obs. n – i obs.

Figure 3.10.2

Example 3.10.5

Suppose that many years of observation have confirmed that the annual maximum flood tide Y (in feet) for a certain river can be modeled by the pdf

fY (y) = 120 , 20 < y < 40

(Note: It is unlikely that flood tides would be described by anything as simple as a uniform pdf. We are making that choice here solely to facilitate the mathematics.) The Army Corps of Engineers is planning to build a levee along a certain portion of the river, and they want to make it high enough so that there is only a 30% chance that the second worst flood in the next thirty-three years will overflow the embankment. How high should the levee be? (We assume that there will be only one potential flood per year.)

Let h be the desired height. If Y1,Y2, . . . ,Y33 denote the flood tides for the next n = 33 years, what we require of h is that

P(Y ′32 > h) = 0.30 As a starting point, notice that for 20 < y < 40,

FY (y) = ∫ y

20

1 20

dy = y 20

− 1

Section 3.10 Order Statistics 197

Therefore,

fY ′32 (y) = 33!

31!1!

( y 20

− 1 )31 (

2 − y 20

)1 · 1

20

and h is the solution of the integral equation∫ 40 h

(33)(32) ( y

20 − 1

)31 ( 2 − y

20

)1 · dy

20 = 0.30 (3.10.3)

If we make the substitution

u = y 20

− 1

Equation 3.10.3 simplifies to

P(Y ′32 > h) = 33(32) ∫ 1

(h/20)−1 u31(1 − u) du

= 1 − 33 (

h 20

− 1 )32

+ 32 (

h 20

− 1 )33

(3.10.4)

Setting the right-hand side of Equation 3.10.4 equal to 0.30 and solving for h by trial and error gives

h = 39.33 feet

JOINT PDFS OF ORDER STATISTICS

Finding the joint pdf of two or more order statistics is easily accomplished by gen- eralizing the argument that derived from Figure 3.10.2. Suppose, for example, that each of n observations in a random sample has pdf fY (y) and cdf FY (y). The joint pdf for order statistics Y ′i and Y

′ j at points u and v, where i < j and u < v, can be

deduced from Figure 3.10.3, which shows how the n points must be distributed if the ith and jth order statistics are to be located at points u and v, respectively.

Y-axis u v

i – 1 obs. Y 9 i Y9 jj – i – 1 obs. n – j obs.Figure 3.10.3

By Theorem 2.6.2, the number of ways to divide a set of n observations into groups of sizes i − 1, 1, j − i − 1, 1, and n − j is the quotient

n! (i − 1)!1!( j − i − 1)!1!(n − j)!

Also, given the independence of the n observations, the probability that i − 1 are less than u is [FY (u)]i−1, the probability that j − i − 1 are between u and v is [FY (v)−FY (u)] j−i−1, and the probability that n− j are greater than v is [1−FY (v)]n− j. Multiplying, then, by the pdfs describing the likelihoods that Y ′i and Y

′ j would be at

points u and v, respectively, gives the joint pdf of the two order statistics:

fY ′i ,Y ′j (u, v) = n!

(i − 1)!( j − i − 1)!(n − j)! [FY (u)] i−1[FY (v) − FY (u)] j−i−1 .

[1 − FY (v)]n− j fY (u) fY (v) (3.10.5) for i < j and u < v.

198 Chapter 3 Random Variables

Example 3.10.6

Let Y1, Y2, and Y3 be a random sample of size n = 3 from the uniform pdf defined over the unit interval, fY (y) = 1, 0 ≤ y ≤ 1. By definition, the range, R, of a sample is the difference between the largest and smallest order statistics, in this case

R = range = Ymax − Ymin = Y ′3 − Y ′1 Find fR(r), the pdf for the range.

We will begin by finding the joint pdf of Y ′1 and Y ′ 3. Then fY ′1,Y ′3 (u, v) is integrated

over the region Y ′3 − Y ′1 ≤ r to find the cdf, FR(r) = P(R ≤ r). The final step is to differentiate the cdf and make use of the fact that fR(r) = F ′R(r).

If fY (y) = 1, 0 ≤ y ≤ 1, it follows that

FY (y) = P(Y ≤ y) =

⎧⎪⎨ ⎪⎩

0, y < 0 y, 0 ≤ y ≤ 1 1, y > 1

Applying Equation 3.10.5, then, with n = 3, i = 1, and j = 3, gives the joint pdf of Y ′1 and Y

′ 3. Specifically,

fY ′1,Y ′3 (u, v) = 3!

0!1!0! u0(v − u)1(1 − v)0 · 1 · 1

= 6(v − u), 0 ≤ u < v ≤ 1

Moreover, we can write the cdf for R in terms of Y ′1 and Y ′ 3:

FR(r) = P(R ≤ r) = P(Y ′3 − Y ′1 ≤ r) = P(Y ′3 ≤ Y ′1 + r)

Figure 3.10.4 shows the region in the Y ′1Y ′ 3-plane corresponding to the event that

R ≤ r. Integrating the joint pdf of Y ′1 and Y ′3 over the shaded region gives

FR(r) = P(R ≤ r) = ∫ 1−r

0

∫ u+r u

6(v − u) dv du + ∫ 1

1−r

∫ 1 u

6(v − u) dv du

0 u-axis

n-axis

1Y19 = 1 – r

r

R#r

Y39 = Y19

Y39 = Y19 + r 1

Figure 3.10.4

The first double integral equals 3r2 − 3r3; the second equals r3. Therefore,

FR(r) = 3r2 − 3r3 + r3 = 3r2 − 2r3

which implies that

fR(r) = F ′R(r) = 6r − 6r2, 0 ≤ r ≤ 1

Section 3.11 Conditional Densities 199

Questions

3.10.1. Suppose the length of time, in minutes, that you have to wait at a bank teller’s window is uniformly dis- tributed over the interval (0, 10). If you go to the bank four times during the next month, what is the probabil- ity that your second longest wait will be less than five minutes?

3.10.2. A random sample of size n = 6 is taken from the pdf fY (y) = 3y2, 0 ≤ y ≤ 1. Find P(Y ′5 > 0.75). 3.10.3. What is the probability that the larger of two ran- dom observations drawn from any continuous pdf will exceed the sixtieth percentile?

3.10.4. A random sample of size 5 is drawn from the pdf fY (y) = 2y, 0 ≤ y ≤ 1. Calculate P(Y ′1 < 0.6 < Y ′5). (Hint: Consider the complement.)

3.10.5. Suppose that Y1, Y2, . . ., Yn is a random sample of size n drawn from a continuous pdf, fY (y), whose median is m. Is P(Y ′1 > m) less than, equal to, or greater than P(Y ′n > m)?

3.10.6. Let Y1, Y2, . . ., Yn be a random sample from the exponential pdf fy(y) = e−y, y ≥ 0. What is the smallest n for which P(Ymin < 0.2) > 0.9?

3.10.7. Calculate P(0.6 < Y ′4 < 0.7) if a random sample of size 6 is drawn from the uniform pdf defined over the interval [0, 1].

3.10.8. A random sample of size n = 5 is drawn from the pdf fY (y) = 2y, 0 ≤ y ≤ 1. On the same set of axes, graph the pdfs for Y2, Y ′1, and Y

′ 5.

3.10.9. Suppose that n observations are taken at random from the pdf

fY (y) = 1√ 2π(6)

e− 1 2 ( y−206 )

2

, − ∞ < y < ∞

What is the probability that the smallest observation is larger than twenty?

3.10.10. Suppose that n observations are chosen at ran- dom from a continuous pdf fY (y). What is the probabil- ity that the last observation recorded will be the smallest number in the entire sample?

3.10.11. In a certain large metropolitan area, the pro- portion, Y , of students bused varies widely from school to school. The distribution of proportions is roughly described by the following pdf:

0 y

1

1

2

f (y)Y

Suppose the enrollment figures for five schools selected at random are examined. What is the probability that the school with the fourth highest proportion of bused chil- dren will have a Y value in excess of 0.75? What is the probability that none of the schools will have fewer than 10% of their students bused?

3.10.12. Consider a system containing n components, where the lifetimes of the components are indepen- dent random variables and each has pdf fY (y) = λe−λy, y > 0. Show that the average time elapsing before the first component failure occurs is 1/nλ.

3.10.13. Let Y1, Y2, . . ., Yn be a random sample from a uniform pdf over [0, 1]. Use Theorem 3.10.2 to show that∫ 1

0 y i−1(1 − y)n−idy = (i − 1)!(n − i)!

n! .

3.10.14. Use Question 3.10.13 to find the expected value of Y ′i , where Y1, Y2, . . ., Yn is a random sample from a uniform pdf defined over the interval [0, 1].

3.10.15. Suppose three points are picked randomly from the unit interval. What is the probability that the three are within a half unit of one another?

3.10.16. Suppose a device has three independent compo- nents, all of whose lifetimes (in months) are modeled by the exponential pdf, fY (y) = e−y, y > 0. What is the probability that all three components will fail within two months of one another?

3.11 Conditional Densities We have already seen that many of the concepts defined in Chapter 2 relating to the probabilities of events—for example, independence—have random variable coun- terparts. Another of these carryovers is the notion of a conditional probability, or, in what will be our present terminology, a conditional probability density function. Applications of conditional pdfs are not uncommon. The height and girth of a tree, for instance, can be considered a pair of random variables. While it is easy to mea- sure girth, it can be difficult to determine height; thus it might be of interest to a

200 Chapter 3 Random Variables

lumberman to know the probabilities of a ponderosa pine’s attaining certain heights given a known value for its girth. Or consider the plight of a school board member agonizing over which way to vote on a proposed budget increase. Her task would be that much easier if she knew the conditional probability that x additional tax dol- lars would stimulate an average increase of y points among twelfth graders taking a standardized proficiency exam.

FINDING CONDITIONAL PDFS FOR DISCRETE RANDOM VARIABLES

In the case of discrete random variables, a conditional pdf can be treated in the same way as a conditional probability. Note the similarity between Definitions 3.11.1 and 2.4.1.

Definition 3.11.1 Let X and Y be discrete random variables. The conditional probability density function of Y given x—that is, the probability that Y takes on the value y given that X is equal to x—is denoted pY |x(y) and given by

pY |x(y) = P(Y = y | X = x) = pX,Y (x, y)pX (x) for pX (x) �= 0.

Example 3.11.1

A fair coin is tossed five times. Let the random variable Y denote the total number of heads that occur, and let X denote the number of heads occurring on the last two tosses. Find the conditional pdf pY |x(y) for all x and y.

Clearly, there will be three different conditional pdfs, one for each possible value of X (x = 0, x = 1, and x = 2). Moreover, for each value of x there will be four possi- ble values of Y , based on whether the first three tosses yield zero, one, two, or three heads.

For example, suppose no heads occur on the last two tosses. Then X = 0, and pY |0(y) = P(Y = y | X = 0) = P(y heads occur on first three tosses)

= (

3 y

)( 1 2

)y ( 1 − 1

2

)3−y

= (

3 y

)( 1 2

)3 , y = 0, 1, 2, 3

Now, suppose that X = 1. The corresponding conditional pdf in that case becomes

pY |x(y) = P(Y = y | X = 1) Notice that Y = 1 if zero heads occur in the first three tosses, Y = 2 if one head occurs in the first three trials, and so on. Therefore,

pY |1(y) = (

3 y − 1

)( 1 2

)y−1 ( 1 − 1

2

)3−(y−1)

= (

3 y − 1

)( 1 2

)3 , y = 1, 2, 3, 4

Section 3.11 Conditional Densities 201

Similarly,

pY |2(y) = P(Y = y | X = 2) = (

3 y − 2

)( 1 2

)3 , y = 2, 3, 4, 5

Figure 3.11.1 shows the three conditional pdfs. Each has the same shape, but the possible values of Y are different for each value of X .

0 1 2 3 4

pY|2(y)

pY|1(y)

pY|0(y)

5 Y-axis

x = 2

x = 1

x = 0

3 8 1 8

3 8 1 8

3 8 1 8

Figure 3.11.1

Example 3.11.2

Assume that the probabilistic behavior of a pair of discrete random variables X and Y is described by the joint pdf

pX,Y (x, y) = xy2/39 defined over the four points (1, 2), (1, 3), (2, 2), and (2, 3). Find the conditional probability that X = 1 given that Y = 2.

By definition,

pX |2(1) = P(X = 1 given that Y = 2)

= pX,Y (1, 2) pY (2)

= 1 · 2 2/39

1 · 22/39 + 2 · 22/39 = 1/3

Example 3.11.3

Suppose that X and Y are two independent binomial random variables, each defined on n trials and each having the same success probability p. Let Z = X + Y . Show that the conditional pdf pX |z(x) is a hypergeometric distribution.

We know from Example 3.8.2 that Z has a binomial distribution with parameters 2n and p. That is,

pZ(z) = P(Z = z) = (

2n z

) pz(1 − p)2n−z, z = 0, 1, . . . , 2n.

202 Chapter 3 Random Variables

By Definition 3.11.1,

pX |z(x) = P(X = x|Z = z) = pX,Z(x, z)pZ(z)

= P(X = x and Z = z) P(Z = z)

= P(X = x and Y = z − x) P(Z = z)

= P(X = x) · P(Y = z − x) P(Z = z) (because X and Y are independent)

=

( n x

) px(1 − p)n−x ·

( n

z − x )

pz−x(1 − p)n−(z−x)( 2n z

) pz(1 − p)2n−z

=

( n x

)( n

z − x )

( 2n z

) which we recognize as being the hypergeometric distribution.

Comment The notion of a conditional pdf generalizes easily to situations involving more than two discrete random variables. For example, if X , Y , and Z have the joint pdf pX,Y,Z(x, y, z), the joint conditional pdf of, say, X and Y given that Z = z is the ratio

pX,Y |z(x, y) = pX,Y,Z(x, y, z)pZ(z)

Example 3.11.4

Suppose that random variables X , Y , and Z have the joint pdf

pX,Y,Z(x, y, z) = xy/9z for points (1, 1, 1), (2, 1, 2), (1, 2, 2), (2, 2, 2), and (2, 2, 1). Find pX,Y |z(x, y) for all values of z.

To begin, we see from the points for which pX,Y,Z(x, y, z) is defined that Z has two possible values, 1 and 2. Suppose z = 1. Then

pX,Y |1(x, y) = pX,Y,Z(x, y, 1)pZ(1) But

pZ(1) = P(Z = 1) = P[(1, 1, 1) ∪ (2, 2, 1)]

= 1 · 1 9

· 1 + 2 · 2 9

· 1

= 5 9

Therefore,

pX,Y |1(x, y) = xy/95 9

= xy/5 for (x, y) = (1, 1) and (2, 2)

Section 3.11 Conditional Densities 203

Suppose z = 2. Then

pZ(2) = P(Z = 2) = P[(2, 1, 2) ∪ (1, 2, 2) ∪ (2, 2, 2)]

= 2 · 1 18

+ 1 · 2 18

+ 2 · 2 18

= 8 18

so

pX,Y |2(x, y) = pX,Y,Z(x, y, 2)pZ(2)

= x · y/188 18

= xy 8

for (x, y) = (2, 1), (1, 2), and (2, 2)

Questions

3.11.1. Suppose X and Y have the joint pdf pX,Y (x, y) = x+y+xy

21 for the points (1, 1), (1, 2), (2, 1), (2, 2), where X denotes a “message” sent (either x = 1 or x = 2) and Y denotes a “message” received. Find the probability that the message sent was the message received, that is, find pY |x(x).

3.11.2. Suppose a die is rolled six times. Let X be the total number of 4’s that occur and let Y be the number of 4’s in the first two tosses. Find pY |x(y).

3.11.3. An urn contains eight red chips, six white chips, and four blue chips. A sample of size 3 is drawn without replacement. Let X denote the number of red chips in the sample and Y , the number of white chips. Find an expres- sion for pY |x(y).

3.11.4. Five cards are dealt from a standard poker deck. Let X be the number of aces received, and Y the number of kings. Compute P(X = 2|Y = 2).

3.11.5. Given that two discrete random variables X and Y follow the joint pdf pX,Y (x, y) = k(x + y), for x = 1, 2, 3 and y = 1, 2, 3, (a) Find k. (b) Evaluate pY |x(1) for all values of x for which px(x) > 0.

3.11.6. Let X denote the number on a chip drawn at ran- dom from an urn containing three chips, numbered 1, 2, and 3. Let Y be the number of heads that occur when a fair coin is tossed X times.

(a) Find pX,Y (x, y). (b) Find the marginal pdf of Y by summing out the x values.

3.11.7. Suppose X , Y , and Z have a trivariate distribution described by the joint pdf

pX,Y,Z(x, y, z) = xy + xz + yz54 where x, y, and z can be 1 or 2. Tabulate the joint condi- tional pdf of X and Y given each of the two values of z.

3.11.8. In Question 3.11.7 define the random variable W to be the “majority” of x, y, and z. For example, W(2, 2, 1) = 2 and W(1, 1, 1) = 1. Find the pdf of W |x. 3.11.9. Let X and Y be independent Poisson random vari- ables where px(k) = e−λ λkk! and pY (k) = e−μ μ

k

k! for k = 0, 1, . . . . Show that the conditional pdf of X given that X + Y = n is binomial with parameters n and λ

λ+μ . (Hint: See Question 3.8.3.)

3.11.10. Suppose Compositor A is preparing a manuscript to be published. Assume that she makes X errors per page, where X has a Poisson pdf, with λ = 2 (see Ques- tion 3.9.21). A second compositor, B, is also working on the book. He makes Y errors on a page, where Y is Pois- son with λ = 3. Assume that Compositor A prepares the first one hundred pages of the text and Compositor B, the last one hundred pages. After the book is completed, re- viewers (with too much time on their hands!) find that the text contains a total of five hundred twenty errors. Write a formula for the exact probability that fewer than half of the errors are due to Compositor A.

204 Chapter 3 Random Variables

FINDING CONDITIONAL PDFS FOR CONTINUOUS RANDOM VARIABLES

If the variables X and Y are continuous, we can still appeal to the quotient fX,Y (x, y)/ fX (x) as the definition of fY |x(y) and argue its propriety by analogy. A more satisfying approach, though, is to arrive at the same conclusion by taking the limit of Y ’s “conditional” cdf.

If X is continuous, a direct evaluation of FY |x(y) = P(Y ≤ y|X = x), via Defini- tion 2.4.1, is impossible, since the denominator would be zero. Alternatively, we can think of P(Y ≤ y|X = x) as a limit:

P(Y ≤ y|X = x) = lim h→0

P(Y ≤ y|x ≤ X ≤ x + h)

= lim h→0

∫ x+h x

∫ y −∞

fX,Y (t, u) du dt∫ x+h x

fX (t) dt

Evaluating the quotient of the limits gives 00 , so l’Hôpital’s rule is indicated:

P(Y ≤ y|X = x) = lim h→0

d dh

∫ x+h x

∫ y −∞

fX,Y (t, u) du dt

d dh

∫ x+h x

fX (t) dt

(3.11.1)

By the Fundamental Theorem of Calculus,

d dh

∫ x+h x

g (t) dt = g (x + h)

which simplifies Equation 3.11.1 to

P(Y ≤ y|X = x) = lim h→0

∫ y −∞

fX,Y [(x + h), u] du fX (x + h)

=

∫ y −∞

lim h→0

fX,Y (x + h, u) du lim h→0

fX (x + h) = ∫ y

−∞

fX,Y (x, u) fX (x)

du

provided that the limit operation and the integration can be interchanged [see (9) for a discussion of when such an interchange is valid]. It follows from this last ex- pression that fX,Y (x, y)/ fX (x) behaves as a conditional probability density func- tion should, and we are justified in extending Definition 3.11.1 to the continuous case.

Example 3.11.5

Let X and Y be continuous random variables with joint pdf

fX,Y (x, y) = ⎧⎨ ⎩ (

1 8

) (6 − x − y), 0 ≤ x ≤ 2, 2 ≤ y ≤ 4

0, elsewhere

Find (a) fX (x), (b) fY |x(y), and (c) P(2 < Y < 3|x = 1).

Section 3.11 Conditional Densities 205

a. From Theorem 3.7.2,

fX (x) = ∫ ∞

−∞ fX,Y (x, y) dy =

∫ 4 2

( 1 8

) (6 − x − y) dy

= (

1 8

) (6 − 2x), 0 ≤ x ≤ 2

b. Substituting into the “continuous” statement of Definition 3.11.1, we can write

fY |x(y) = fX,Y (x, y)fX (x) = ( 1

8

) (6 − x − y)( 1

8

) (6 − 2x)

= 6 − x − y 6 − 2x , 0 ≤ x ≤ 2, 2 ≤ y ≤ 4

c. To find P(2 < Y < 3|x = 1), we simply integrate fY |1(y) over the interval 2 < Y < 3:

P(2 < Y < 3|x = 1) = ∫ 3

2 fY |1(y) dy

= ∫ 3

2

5 − y 4

dy

= 5 8

(A partial check that the derivation of a conditional pdf is correct can be performed by integrating fY |x(y) over the entire range of Y . That integral should be 1. Here, for example, when x = 1, ∫∞−∞ fY |1(y) dy = ∫ 42 [(5 − y)/4] dy does equal 1.)

Questions

3.11.11. Let X be a nonnegative random variable. We say that X is memoryless if

P(X > s + t|X > t) = P(X > s) for all s, t ≥ 0 Show that a random variable with pdf fX (x) = (1/λ)e−x/λ, x > 0, is memoryless.

3.11.12. Given the joint pdf

fX,Y (x, y) = 2e−(x+y), 0 ≤ x ≤ y, y ≥ 0 find (a) P(Y < 1|X < 1). (b) P(Y < 1|X = 1). (c) fY |x(y). (d) E(Y |x).

3.11.13. Find the conditional pdf of Y given x if

fX,Y (x, y) = x + y for 0 ≤ x ≤ 1 and 0 ≤ y ≤ 1.

3.11.14. If

fX,Y (x, y) = 2, x ≥ 0, y ≥ 0, x + y ≤ 1 show that the conditional pdf of Y given x is uniform.

3.11.15. Suppose that

fY |x(y) = 2y + 4x1 + 4x and fX (x) = 1 3

· (1 + 4x)

for 0 ≤ x ≤ 1 and 0 ≤ y ≤ 1. Find the marginal pdf for Y .

3.11.16. Suppose that X and Y are distributed according to the joint pdf

fX,Y (x, y) = 25 · (2x + 3y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1

Find (a) fX (x). (b) fY |x(y).

(c) P ( 1

4 ≤ Y ≤ 34 |X = 12 ) .

(d) E(Y |x).

206 Chapter 3 Random Variables

3.11.17. If X and Y have the joint pdf

fX,Y (x, y) = 2, 0 ≤ x < y ≤ 1 find P

( 0 < X < 12 |Y = 34

) .

3.11.18. Find P ( X < 1|Y = 1 12

) if X and Y have the

joint pdf fX,Y (x, y) = xy/2, 0 ≤ x < y ≤ 2

3.11.19. Suppose that X1, X2, X3, X4, and X5 have the joint pdf

fX1,X2,X3,X4,X5 (x1, x2, x3, x4, x5) = 32x1x2x3x4x5 for 0 ≤ xi ≤ 1, i = 1, 2, . . . , 5. Find the joint conditional pdf of X1, X2, and X3 given that X4 = x4 and X5 = x5.

3.11.20. Suppose the random variables X and Y are jointly distributed according to the pdf

fX,Y (x, y) = 67 (

x2 + xy 2

) , 0 ≤ x ≤ 1, 0 ≤ y ≤ 2

Find

(a) fX (x). (b) P(X > 2Y). (c) P

( Y > 1|X > 12

) .

3.11.21. For continuous random variables X and Y , prove that E(Y) = EX [E(Y |x)].

3.12 Moment-Generating Functions Finding moments of random variables directly, particularly the higher moments defined in Section 3.6, is conceptually straightforward but can be quite problematic: Depending on the nature of the pdf, integrals and sums of the form

∫∞ −∞ y

r fY (y) dy and

∑ all k

kr pX (k) can be very difficult to evaluate. Fortunately, an alternative method

is available. For many pdfs, we can find a moment-generating function (or mgf), MW (t), one of whose properties is that the rth derivative of MW (t) evaluated at zero is equal to E(W r).

CALCULATING A RANDOM VARIABLE’S MOMENT-GENERATING FUNCTION

In principle, what we call a moment-generating function is a direct application of Theorem 3.5.3.

Definition 3.12.1 Let W be a random variable. The moment-generating function (mgf) for W is denoted MW (t) and given by

MW (t) = E(etW ) =

⎧⎪⎪⎨ ⎪⎪⎩ ∑ all k

etk pW (k) ifW is discrete∫ ∞ −∞

etw fW (w) dw ifW is continuous

at all values of t for which the expected value exists.

Example 3.12.1

Suppose the random variable X has a geometric pdf,

pX (k) = (1 − p)k−1 p, k = 1, 2, . . .

[In practice, this is the pdf that models the occurrence of the first success in a series of independent trials, where each trial has a probability p of ending in success (recall Example 3.3.2)]. Find MX (t), the moment-generating function for X .

Section 3.12 Moment-Generating Functions 207

Since X is discrete, the first part of Definition 3.12.1 applies, so

MX (t) = E(etX ) = ∞∑

k=1 etk(1 − p)k−1 p

= p 1 − p

∞∑ k=1

etk(1 − p)k = p 1 − p

∞∑ k=1

[(1 − p)et ]k (3.12.1)

The t in MX (t) can be any number in a neighborhood of zero, as long as MX (t) < ∞. Here, MX (t) is an infinite sum of the terms [(1 − p)et ]k, and that sum will be finite only if (1 − p)et < 1, or, equivalently, if t < ln[1/(1 − p)]. It will be assumed, then, in what follows that 0 < t < ln[1/(1 − p)].

Recall that ∞∑

k=0 rk = 1

1 − r provided 0 < r < 1. This formula can be used on Equation 3.12.1, where r = (1−p)et and 0 < t < ln

[ 1 (1−p)

] . Specifically,

MX (t) = p1 − p

[ ∞∑ k=0

[(1 − p)et]k − [(1 − p)et]0 ]

= p 1 − p

[ 1

1 − (1 − p)et − 1 ]

= pe t

1 − (1 − p)et

Example 3.12.2

Suppose that X is a binomial random variable with pdf

pX (k) = (

n k

) pk(1 − p)n−k, k = 0, 1, . . . , n

Find MX (t). By Definition 3.12.1,

MX (t) = E(etX ) = n∑

k=0 etk

(n k

) pk(1 − p)n−k

= n∑

k=0

(n k

) (pet)k(1 − p)n−k (3.12.2)

To get a closed-form expression for MX (t)—that is, to evaluate the sum indicated in Equation 3.12.2—requires a (hopefully) familiar formula from algebra: According to Newton’s binomial expansion,

(x + y)n = n∑

k=0

( n k

) xkyn−k (3.12.3)

for any x and y. Suppose we let x = pet and y = 1−p. It follows from Equations 3.12.2 and 3.12.3, then, that

MX (t) = (1 − p + pet)n

Notice in this case that MX (t) is defined for all values of t.

208 Chapter 3 Random Variables

Example 3.12.3

Suppose that Y has an exponential pdf, where fY (y) = λe−λy, y > 0. Find MY (t). Since the exponential pdf describes a continuous random variable, MY (t) is an

integral:

MY (t) = E(etY ) = ∫ ∞

0 ety · λe−λy dy

= ∫ ∞

0 λe−(λ−t)y dy

After making the substitution u = (λ − t)y, we can write

MY (t) = ∫ ∞

u=0 λe−u

du λ − t

= λ λ − t

[−e−u∣∣∞u=0] = λ

λ − t [ 1 − lim

u→∞ e −u ]

= λ λ − t

Here, MY (t) is finite and nonzero only when u = (λ − t)y > 0, which implies that t must be less than λ. For t ≥ λ, MY (t) fails to exist.

Example 3.12.4

The normal (or bell-shaped) curve was introduced in Example 3.4.3. Its pdf is the rather cumbersome function

fY (y) = ( 1/

√ 2πσ

) exp

[ −1

2

( y − μ

σ

)2] , −∞ < y < ∞

where μ = E(Y) and σ2 = Var(Y). Derive the moment-generating function for this most important of all probability models.

Since Y is a continuous random variable,

MY (t) = E(etY ) = ( 1/

√ 2πσ

) ∫ ∞ −∞

exp(ty) exp

[ −1

2

( y − μ

σ

)2] dy

= (1/√2πσ) ∫ ∞ −∞

exp [ −y

2 − 2μy − 2σ2ty + μ2 2σ2

] dy (3.12.4)

Evaluating the integral in Equation 3.12.4 is best accomplished by completing the square of the numerator of the exponent (which means that the square of half the coefficient of y is added and subtracted). That is, we can write

y2 − (2μ + 2σ2t)y + (μ + σ2t)2 − (μ + σ2t)2 + μ2

= [y − (μ + σ2t)]2 − σ4t2 + 2μtσ2 (3.12.5) The last two terms on the right-hand side of Equation 3.12.5, though, do not

involve y, so they can be factored out of the integral, and Equation 3.12.4 reduces to

MY (t) = exp (

μt + σ 2t2

2

) ( 1/

√ 2πσ

) ∫ ∞ −∞

exp

[ −1

2

[ y − (μ + tσ2)

σ

]2] dy

But, together, the latter two factors equal 1 (why?), implying that the moment- generating function for a normally distributed random variable is given by

MY (t) = eμt+σ2t2/2

Section 3.12 Moment-Generating Functions 209

Questions

3.12.1. Let X be a random variable with pdf pX (k) = 1/n, for k = 0, 1, 2, . . . , n − 1 and 0 otherwise. Show that MX (t) = 1−entn(1−et ) .

3.12.2. Two chips are drawn at random and without re- placement from an urn that contains five chips, numbered 1 through 5. If the sum of the chips drawn is even, the ran- dom variable X equals 5; if the sum of the chips drawn is odd, X = −3. Find the moment-generating function for X .

3.12.3. Find the expected value of e3X if X is a binominal random variable with n = 10 and p = 13 .

3.12.4. Find the moment-generating function for the dis- crete random variable X whose probability function is given by

pX (k) = (

3 4

)k (1 4

) , k = 0, 1, 2, . . .

3.12.5. Which pdfs would have the following moment- generating functions?

(a) MY (t) = e6t2 (b) MY (t) = 2/(2 − t) (c) MX (t) =

( 1 2 + 12 et

)4 (d) MX (t) = 0.3et/(1 − 0.7et) 3.12.6. Let Y have pdf

fY (y) =

⎧⎪⎨ ⎪⎩

y, 0 ≤ y ≤ 1 2 − y, 1 ≤ y ≤ 2 0, elsewhere

Find MY (t).

3.12.7. The random variable X has a Poisson distribution pX (k) = e−λλk/k!, k = 0, 1, 2, . . . . Find the moment- generating function for a Poisson random variable. Recall that

er = ∞∑

k=0

rk

k!

3.12.8. Let Y be a continuous random variable with fY (y) = ye−y, 0 ≤ y. Show that MY (t) = 1(1−t)2 .

USING MOMENT-GENERATING FUNCTIONS TO FIND MOMENTS

Having practiced finding the functions MX (t) and MY (t), we now turn to the theorem that spells out their relationship to X r and Y r.

Theorem 3.12.1

Let W be a random variable with probability density function fW (w). (If W is con- tinuous, fW (w) must be sufficiently smooth to allow the order of differentiation and integration to be interchanged.) Let MW (t) be the moment-generating function for W. Then, provided the rth moment exists,

M(r)W (0) = E(W r)

Proof We will verify the theorem for the continuous case where r is either 1 or 2. The extensions to discrete random variables and to an arbitrary positive integer r are straightforward.

For r = 1,

M(1)Y (0) = d dt

∫ ∞ −∞

ety fY (y) dy ∣∣∣∣ t=0

= ∫ ∞

−∞

d dt

ety fY (y) dy ∣∣∣∣ t=0

= ∫ ∞

−∞ yety fY (y) dy

∣∣∣∣ t=0

= ∫ ∞

−∞ ye0·y fY (y) dy

= ∫ ∞

−∞ y fY (y) dy = E(Y)

(Continued on next page)

210 Chapter 3 Random Variables

(Theorem 3.12.1 continued)

For r = 2,

M(2)Y (0) = d2

dt2

∫ ∞ −∞

ety fY (y) dy ∣∣∣∣ t=0

= ∫ ∞

−∞

d2

dt2 ety fY (y) dy

∣∣∣∣ t=0

= ∫ ∞

−∞ y2ety fY (y) dy

∣∣∣∣ t=0

= ∫ ∞

−∞ y2e0 · y fY (y) dy

= ∫ ∞

−∞ y2 fY (y) dy = E(Y 2)

Example 3.12.5

For a geometric random variable X with pdf

pX (k) = (1 − p)k−1 p, k = 1, 2, . . . we saw in Example 3.12.1 that

MX (t) = pet[1 − (1 − p)et]−1 Find the expected value of X by differentiating its moment-generating function.

Using the product rule, we can write the first derivative of MX (t) as

M(1)X (t) = pet(−1)[1 − (1 − p)et]−2(−1)(1 − p)et + [1 − (1 − p)et]−1 pet

= p(1 − p)e 2t

[1 − (1 − p)et ]2 + pet

1 − (1 − p)et Setting t = 0 shows that E(X ) = 1p :

M(1)X (0) = E(X ) = p(1 − p)e2 · 0

[1 − (1 − p)e0]2 + pe0

1 − (1 − p)e0

= p(1 − p) p2

+ p p

= 1 p

Example 3.12.6

Find the expected value of an exponential random variable with pdf

fY (y) = λe−λy, y > 0 Use the fact that

MY (t) = λ(λ − t)−1 (as shown in Example 3.12.3).

Differentiating MY (t) gives

M(1)Y (t) = λ(−1)(λ − t)−2(−1)

= λ (λ − t)2

Set t = 0. Then M(1)Y (0) =

λ

(λ − 0)2 implying that

E(Y) = 1 λ

Section 3.12 Moment-Generating Functions 211

Example 3.12.7

Find an expression for E(X k) if the moment-generating function for X is given by

MX (t) = (1 − p1 − p2) + p1et + p2e2t

The only way to deduce a formula for an arbitrary moment such as E(X k) is to calculate the first couple moments and look for a pattern that can be generalized. Here,

M(1)X (t) = p1et + 2p2e2t

so

E(X ) = M(1)X (0) = p1e0 + 2p2e2 · 0

= p1 + 2p2 Taking the second derivative, we see that

M(2)X (t) = p1et + 22 p2e2t

implying that

E(X 2) = M(2)X (0) = p1e0 + 22 p2e2 · 0

= p1 + 22 p2 Clearly, each successive differentiation will leave the p1 term unaffected but will mul- tiply the p2 term by 2. Therefore,

E(X k) = M(k)X (0) = p1 + 2k p2

USING MOMENT-GENERATING FUNCTIONS TO FIND VARIANCES

In addition to providing a useful technique for calculating E(W r), moment- generating functions can also find variances, because

Var(W) = E(W 2) − [E(W)]2 (3.12.6) for any random variable W (recall Theorem 3.6.1). Other useful “descriptors” of pdfs can also be reduced to combinations of moments. The skewness of a distribution, for example, is a function of E[(W − μ)3], where μ = E(W). But

E[(W − μ)3] = E(W 3) − 3E(W 2)E(W) + 2[E(W)]3

In many cases, finding E[(W −μ)2] or E[(W −μ)3] could be quite difficult if moment- generating functions were not available.

Example 3.12.8

We know from Example 3.12.2 that if X is a binomial random variable with param- eters n and p, then

MX (t) = (1 − p + pet)n

Use MX (t) to find the variance of X . The first two derivatives of MX (t) are

M(1)X (t) = n(1 − p + pet)n−1 · pet

212 Chapter 3 Random Variables

and

M(2)X (t) = pet · n(n − 1)(1 − p + pet)n−2 · pet + n(1 − p + pet)n−1 · pet

Setting t = 0 gives

M(1)X (0) = np = E(X )

and

M(2)X (0) = n(n − 1)p2 + np = E(X 2)

From Equation 3.12.6, then,

Var(X ) = n(n − 1)p2 + np − (np)2

= np(1 − p)

(the same answer we found in Example 3.9.9).

Example 3.12.9

It can be shown (see Question 3.12.7) that the moment-generating function for a Poisson random variable is given by

MX (t) = e−λ+λet

Use MX (t) to find E(X ) and Var(X ). Taking the first derivative of MX (t) gives

M(1)X (t) = e−λ+λe t · λet

so

E(X ) = M(1)X (0) = e−λ+λe 0 · λe0

= λ

Applying the product rule to M(1)X (t) yields the second derivative,

M(2)X (t) = e−λ+λe t · λet + λete−λ+λet · λet

For t = 0,

M(2)X (0) = E(X 2) = e−λ+λe 0 · λe0 + λe0 · e−λ+λe0 · λe0

= λ + λ2

The variance of a Poisson random variable, then, proves to be the same as its mean:

Var(X ) = E(X 2) − [E(X )]2

= M(2)X (0) − [ M(1)X (0)

]2 = λ2 + λ − λ2

= λ

Section 3.12 Moment-Generating Functions 213

Questions

3.12.9. Calculate E(Y 3) for a random variable whose moment-generating function is MY (t) = et2/2. 3.12.10. Find E(Y 4) if Y is an exponential random vari- able with fY (y) = λe−λy, y > 0. 3.12.11. The form of the moment-generating function for a normal random variable is MY (t) = eat+b2t2/2 (recall Ex- ample 3.12.4). Differentiate MY (t) to verify that a = E(Y) and b2 = Var(Y). 3.12.12. What is E(Y 4) if the random variable Y has moment-generating function MY (t) = (1 − αt)−k?

3.12.13. Find E(Y 2) if the moment-generating function for Y is given by MY (t) = e−t+4t2 . Use Example 3.12.4 to find E(Y 2) without taking any derivatives. (Hint: Recall Theorem 3.6.1.)

3.12.14. Find an expression for E(Y k) if MY (t) = (1 − t/λ)−r, where λ is any positive real number and r is a pos- itive integer.

3.12.15. Use MY (t) to find the expected value of the uni- form random variable described in Example 3.4.1.

3.12.16. Find the variance of Y if MY (t) = e2t/(1 − t2).

USING MOMENT-GENERATING FUNCTIONS TO IDENTIFY PDFS

Finding moments is not the only application of moment-generating functions. They are also used to identify the pdf of sums of random variables—that is, finding fW (w), where W = W1 +W2 +· · ·+Wn. Their assistance in the latter is particularly important for two reasons: (1) Many statistical procedures are defined in terms of sums, and (2) alternative methods for deriving fW1+W2+···+Wn (w) are extremely cumbersome.

The next two theorems give the background results necessary for deriving fW (w). Theorem 3.12.2 states a key uniqueness property of moment-generating func- tions: If W1 and W2 are random variables with the same mgfs, they must necessarily have the same pdfs. In practice, applications of Theorem 3.12.2 typically rely on one or both of the algebraic properties cited in Theorem 3.12.3.

Theorem 3.12.2

Suppose that W1 and W2 are random variables for which MW1 (t) = MW2 (t) for some interval of t’s containing 0. Then fW1 (w) = fW2 (w). Proof See (103).

Theorem 3.12.3

a. Let W be a random variable with moment-generating function MW (t). Let V = aW + b. Then

MV (t) = ebtMW (at) b. Let W1,W2, . . . ,Wn be independent random variables with moment-generating

functions MW1 (t), MW2 (t), . . . , and MWn (t), respectively. Let W = W1 + W2 + · · · + Wn. Then

MW (t) = MW1 (t) · MW2 (t) · · · MWn (t)

Proof The proof is left as an exercise.

Example 3.12.10

Suppose that X1 and X2 are two independent Poisson random variables with param- eters λ1 and λ2, respectively. That is,

pX1 (k) = P(X1 = k) = e−λ1λk1

k! , k = 0, 1, 2, . . .

214 Chapter 3 Random Variables

and

pX2 (k) = P(X2 = k) = e−λ2λk2

k! , k = 0, 1, 2, . . .

Let X = X1 + X2. What is the pdf for X? According to Example 3.12.9, the moment-generating functions for X1 and

X2 are

MX1 (t) = e−λ1+λ1e t

and

MX2 (t) = e−λ2+λ2e t

Moreover, if X = X1 + X2, then by part (b) of Theorem 3.12.3, MX (t) = MX1 (t) · MX2 (t)

= e−λ1+λ1et · e−λ2+λ2et

= e−(λ1+λ2)+(λ1+λ2)et (3.12.7) But, by inspection, Equation 3.12.7 is the moment-generating function that a Poisson random variable with λ = λ1 + λ2 would have. It follows, then, by Theorem 3.12.2 that

pX (k) = e −(λ1+λ2)(λ1 + λ2)k

k! , k = 0, 1, 2, . . .

Comment The Poisson random variable reproduces itself in the sense that the sum of independent Poissons is also a Poisson. A similar property holds for independent normal random variables (see Question 3.12.19) and, under certain conditions, for independent binomial random variables (recall Example 3.8.2).

Example 3.12.11

We saw in Example 3.12.4 that a normal random variable, Y , with mean μ and vari- ance σ2 has pdf

fY (y) = ( 1/

√ 2πσ

) exp

[ −1

2

( y − μ

σ

)2] , −∞ < y < ∞

and mgf

MY (t) = eμt+σ2t2/2

By definition, a standard normal random variable is a normal random variable for which μ = 0 and σ = 1. Denoted Z, the pdf and mgf for a standard normal random variable are fZ(z) = (1/

√ 2π)e−z

2/2, −∞ < z < ∞, and MZ(t) = et2/2, respectively. Show that the ratio

Y − μ σ

is a standard normal random variable, Z. Write Y−μ

σ as 1

σ Y − μ

σ . By part (a) of Theorem 3.12.3,

M(Y−μ)/σ(t) = e−μt/σMY (

t σ

)

= e−μt/σe[μt/σ+σ2(t/σ)2/2]

= et2/2

Section 3.13 Taking a Second Look at Statistics (Interpreting Means) 215

But MZ(t) = et2/2 so it follows from Theorem 3.12.2 that the pdf for Y−μσ is the same as the pdf for fz(z). (We call

Y−μ σ

a Z transformation. Its importance will become evident in Chapter 4.)

Questions

3.12.17. Use Theorem 3.12.3(a) and Question 3.12.8 to find the moment-generating function of the random vari- able Y , where fY (y) = λ2ye−λy, 0 ≤ y.

3.12.18. Let Y1, Y2, and Y3 be independent random vari- ables, each having the pdf of Question 3.12.17. Use The- orem 3.12.3(b) to find the moment-generating function of Y1 + Y2 + Y3. Compare your answer to the moment- generating function in Question 3.12.14.

3.12.19. Use Theorems 3.12.2 and 3.12.3 to determine which of the following statements is true.

(a) The sum of two independent Poisson random vari- ables has a Poisson distribution. (b) The sum of two independent exponential random variables has an exponential distribution. (c) The sum of two independent normal random variables has a normal distribution.

3.12.20. Calculate P(X ≤ 2) if MX (t) = ( 1

4 + 34 et )5

.

3.12.21. Suppose that Y1, Y2, . . ., Yn is a random sample of size n from a normal distribution with mean μ and

standard deviation σ. Use moment-generating functions

to deduce the pdf of Ȳ = 1 n

n∑ i=1

Yi.

3.12.22. Suppose the moment-generating function for a random variable W is given by

MW (t) = e−3+3et · (

2 3

+ 1 3

et )4

Calculate P(W ≤ 1). (Hint: Write W as a sum.) 3.12.23. Suppose that X is a Poisson random variable, where pX (k) = e−λλk/k!, k = 0, 1, . . . . (a) Does the random variable W = 3X have a Poisson distribution? (b) Does the random variable W = 3X +1 have a Poisson distribution?

3.12.24. Suppose that Y is a normal variable, where fY (y) = (1/

√ 2πσ) exp

[ − 12

( y−μ σ

)2] , −∞ < y < ∞.

(a) Does the random variable W = 3Y have a normal distribution? (b) Does the random variable W = 3Y + 1 have a normal distribution?

3.13 Taking a Second Look at Statistics (Interpreting Means)

One of the most important ideas coming out of Chapter 3 is the notion of the expected value (or mean) of a random variable. Defined in Section 3.5 as a number that reflects the “center” of a pdf, the expected value (μ) was originally introduced for the ben- efit of gamblers. It spoke directly to one of their most fundamental questions—How much will I win or lose, on the average, if I play a certain game? (Actually, the real question they probably had in mind was “How much are you going to lose, on the average?”) Despite having had such a selfish, materialistic, gambling-oriented rai- son d’etre, the expected value was quickly embraced by (respectable) scientists and researchers of all persuasions as a preeminently useful descriptor of a distribution. Today, it would not be an exaggeration to claim that the majority of all statistical analyses focus on either (1) the expected value of a single random variable or (2) comparing the expected values of two or more random variables.

In the lingo of applied statistics, there are actually two fundamentally differ- ent types of “means”—population means and sample means. The term “popula- tion mean” is a synonym for what mathematical statisticians would call an expected value—that is, a population mean (μ) is a weighted average of the possible values

216 Chapter 3 Random Variables

associated with a theoretical probability model, either pX (k) or fY (y), depending on whether the underlying random variable is discrete or continuous. A sample mean is the arithmetic average of a set of measurements. If, for example, n observations— y1, y2, . . ., yn—are taken on a continuous random variable Y , the sample mean is denoted ȳ, where

ȳ = 1 n

n∑ i=1

yi

Conceptually, sample means are estimates of population means, where the “quality” of the estimation is a function of (1) the sample size and (2) the standard deviation (σ) associated with the individual measurements. Intuitively, as the sample size gets larger and/or the standard deviation gets smaller, the approximation will tend to get better.

Interpreting means (either ȳ or μ) is not always easy. To be sure, what they imply in principle is clear enough—both ȳ and μ are measuring the centers of their respec- tive distributions. Still, many a wrong conclusion can be traced directly to researchers misunderstanding the value of a mean. Why? Because the distributions that ȳ and/or μ are actually representing may be dramatically different from the distributions we think they are representing.

An interesting case in point arises in connection with SAT scores. Each fall the average SATs earned by students in each of the fifty states and the District of Columbia are released by the Educational Testing Service (ETS). At the state and

Table 3.13.1

State Average State Average SAT Score SAT Score

North Dakota 1816 Arizona 1547 Illinois 1802 Oregon 1544 Iowa 1794 Virginia 1530 South Dakota 1792 New Jersey 1526 Minnesota 1786 Connecticut 1525 Michigan 1784 West Virginia 1522 Wisconsin 1782 Washington 1519 Missouri 1771 California 1504 Wyoming 1762 Alaska 1485 Kansas 1753 North Carolina 1483 Kentucky 1746 Pennsylvania 1481 Nebraska 1745 Rhode Island 1480 Colorado 1735 Indiana 1474 Mississippi 1714 Maryland 1468 Tennessee 1714 New York 1468 Arkansas 1698 Hawaii 1460 Oklahoma 1697 Nevada 1458 Utah 1690 Florida 1448 Louisiana 1667 Georgia 1445 Ohio 1652 South Carolina 1443 Montana 1637 Texas 1432 Alabama 1617 Maine 1387 New Mexico 1617 Idaho 1364 New Hampshire 1566 Delaware 1359 Massachusetts 1556 DC 1309 Vermont 1554

Section 3.13 Taking a Second Look at Statistics (Interpreting Means) 217

federal level, these SAT averages are often used as indicators of educational suc- cess or failure. Does it make sense, though, to use SAT averages to characterize the quality of a state’s education system? Absolutely not! Averages of this sort refer to very different distributions from state to state. Any attempt to interpret them at face value will necessarily be misleading.

One such state-by-state SAT comparison that appeared in 2014 is reproduced in Table 3.13.1. Notice that North Dakota’s entry is 1816, which is the highest average listed. Does it follow that North Dakota’s educational system is among the best in the nation? Probably not. So, why did their students do so well on the SAT?

The answer to that question lies in the academic profiles of the students who take the SAT. North Dakota primarily uses the ACT for its college admission test. Only 2% of college-bound students took the SAT, approximately one hundred sixty seniors. The SAT is primarily used by private schools, where admissions tend to be more competitive. As a result, the students in North Dakota who take the SAT are not representative of the entire population of students in that state. A disproportion- ate number are exceptionally strong academically, those being the students who feel that they have the ability to be competitive at Ivy League–type schools. The number 1816, then, is the average of something (in this case, an elite subset of all students), but it does not correspond to the center of the SAT distribution for all students.

The moral here is that analyzing data effectively requires that we look beyond the obvious. What we have learned in Chapter 3 about random variables and prob- ability distributions and expected values will be helpful only if we take the time to learn about the context and the idiosyncrasies of the phenomenon being studied. To do otherwise is likely to lead to conclusions that are, at best, superficial and, at worst, incorrect.

Chapte r

4Special Distributions CHAPTER OUTLINE

4.1 Introduction 4.2 The Poisson

Distribution 4.3 The Normal

Distribution 4.4 The Geometric

Distribution 4.5 The Negative

Binomial Distribution 4.6 The Gamma

Distribution 4.7 Taking a Second

Look at Statistics (Monte Carlo Simulations)

Appendix 4.A.1 Proper- ties of Frequently- Used pdfs

Appendix 4.A.2 A Proof of the Central Limit Theorem

Although he maintained lifelong literary and artistic interests, Lambert Adolphe Jacques Quetelet (1796–1874) had mathematical talents that led him to a doctorate from the University of Ghent and from there to a college teaching position in Brussels. In 1833 he was appointed astronomer at the Brussels Royal Observatory after having been largely responsible for its founding. His work with the Belgian census marked the beginning of his pioneering efforts in what today would be called mathematical sociology. Quetelet was well known throughout Europe in scientific and literary circles: At the time of his death he was a member of more than one hundred learned societies.

4.1 INTRODUCTION Suppose for some unknown reason you feel the need to find out how many bug parts (eggs, legs, wings, etc.) you are eating in that package of crackers and cheese you just bought from a vending machine. For reasons we will learn later in this chapter, the distribution of bug parts in peanut butter is nicely modeled by the Poisson probability model

pX (k) = P(I have just eaten k bug parts) = e−λλk/k!, k = 0, 1, 2, . . . where, according to quality control studies done by the Food and Drug Administra- tion, the value of the parameter λ is 6.0 (184).

The equation pX (k) = e−λλk/k!, k = 0, 1, 2, . . . is one of the “special” distri- butions covered in Chapter 4. Recall from Chapter 3 that to qualify as a probability model, a function pX (k) or fY (y) defined over a sample space S needs to satisfy only two criteria: (1) It must be nonnegative for all outcomes in S and (2) It must sum or integrate to 1. Clearly, there are an infinite number of such functions. The only ones of interest for our purposes, though, are those that accurately describe the proba- bilistic behavior of real-world phenomena. Which ones are those? Common sense would tell us that the only pdfs with any connection to reality are those having im- plicit mathematical assumptions in pX (k) or fY (y) that mimic the physical factors influencing the observed values of X and Y .

So how many pX (k)’s and fY (y)’s are in this “useful” group? Given the incred- ible diversity of what researchers measure—ranging from the stars and planets of astronomers to the beasts and bugs of zoologists—we would expect it would take a boatload of different probability functions to meet all their needs. Not so. Not even close. Knowing the properties of just a dozen or so probability functions, some dis- crete and some continuous, provides sufficient background for addressing an amaz- ing diversity of statistical questions.

218

Section 4.2 The Poisson Distribution 219

A case in point is the discrete pdf just mentioned. In addition to its interesting (but not particularly important) ability to predict numbers of bug parts in peanut butter, the Poisson model has long been used by physicists to describe the nature and measurement of radioactivity, by traffic consultants to model the future occur- rences of fender-benders at a busy intersection, by insurance companies to predict how many of their policyholders will likely die next week, by engineers to monitor the number of defective products expected to come off an assembly line, and by the romantically-inclined when counting the number of shooting stars seen per minute during a meteor shower. At first blush, these phenomena seem very different. To a statistician they all look exactly the same, because in each case the data recorded would be generated by the same set of assumptions that underly the Poisson model.

Three of these particularly important probability functions have already been discussed: the binomial model (Theorem 3.2.1), the hypergeometric model (Theorem 3.2.2), and the uniform model (Example 3.4.1). Five other models (three discrete and two continuous) are the focus of Chapter 4.

By far the most important probability model introduced in Chapter 4 is the nor- mal curve (otherwise known as the Gaussian distribution or bell-shaped curve). The normal curve first appeared early in the eighteenth century as a numerical approxi- mation for the binomial distribution when n is large, but researchers early on realized that it had implications that were far more important than its utility as a binomial ap- proximation, and it became the subject of intense mathematical research for more than two hundred years. Today its present formulation is recognized as the single most important result in both applied and mathematical statistics. Sir Francis Galton once wrote that had the ancient Greeks known about the normal curve and all of its implications, they would have given it the name of a god and worshipped it.

Examples play an especially important role in this chapter, as they should. To spend a good deal of time learning about probability models as important as the ones covered in this chapter and never see them applied to real-world situations would be a wasted opportunity—much like teaching a visiting contingent of Martians the ins and outs of internal combustion engines but never taking them for a car ride. That mistake will not happen here. With seventeen examples, seven case studies, four data sets, and ninety-eight exercises, Chapter 4 is even more than just a car ride, it’s an all-out road trip. Buckle up.

4.2 The Poisson Distribution The binomial distribution problems that appeared in Section 3.2 all had relatively small values for n, so evaluating pX (k) = P(X = k) =

( n k

) pk(1 − p)n−k was not

particularly difficult. But suppose n were 1000 and k, 500. Evaluating pX (500) would be a formidable task for many handheld calculators, even today. Two hundred years ago, the prospect of doing cumbersome binomial calculations by hand was a catalyst for mathematicians to develop some easy-to-use approximations. One of the first such approximations was the Poisson limit, which eventually gave rise to the Poisson distribution. Both are described in Section 4.2.

Simeon Denis Poisson (1781–1840) was an eminent French mathematician and physicist, an academic administrator of some note, and, according to an 1826 let- ter from the mathematician Abel to a friend, a man who knew “how to behave with a great deal of dignity.” One of Poisson’s many interests was the application of probability to the law, and in 1837 he wrote Recherches sur la Probabilite de Juge- ments. Included in the latter is a limit for pX (k) =

( n k

) pk(1 − p)n−k that holds when

n approaches ∞, p approaches 0, and np remains constant. In practice, Poisson’s

220 Chapter 4 Special Distributions

limit is used to approximate hard-to-calculate binomial probabilities where the val- ues of n and p reflect the conditions of the limit—that is, when n is large and p is small.

THE POISSON LIMIT

Deriving an asymptotic expression for the binomial probability model is a straight- forward exercise in calculus, given that np is to remain fixed as n increases.

Theorem 4.2.1

Suppose X is a binomial random variable, where

P(X = k) = pX (k) = (n

k

) pk(1 − p)n−k, k = 0, 1, . . . , n

If n → ∞ and p → 0 in such a way that λ = np remains constant, then

lim n→∞ p→0

np= const.

P(X = k) = lim n→∞ p→0

np= const.

(n k

) pk(1 − p)n−k = e

−np(np)k

k!

Proof We begin by rewriting the binomial probability in terms of λ:

lim n→∞

(n k

) pk(1 − p)n−k = lim

n→∞

(n k

)(λ n

)k ( 1 − λ

n

)n−k

= lim n→∞

n! k!(n − k)!λ

k (

1 nk

)( 1 − λ

n

)−k ( 1 − λ

n

)n

= λ k

k! lim

n→∞ n!

(n − k)! 1

(n − λ)k (

1 − λ n

)n

But since [1 − (λ/n)]n → e−λ as n → ∞, we need show only that

n! (n − k)!(n − λ)k → 1

to prove the theorem. However, note that

n! (n − k)!(n − λ)k =

n(n − 1) · · · (n − k + 1) (n − λ)(n − λ) · · · (n − λ)

a quantity that, indeed, tends to 1 as n → ∞ (since λ remains constant).

Example 4.2.1

Theorem 4.2.1 is an asymptotic result. Left unanswered is the question of the rele- vance of the Poisson limit for finite n and p. That is, how large does n have to be and how small does p have to be before e−np(np)k/k! becomes a good approximation to the binomial probability, pX (k)?

Since “good approximation” is undefined, there is no way to answer that ques- tion in any completely specific way. Tables 4.2.1 and 4.2.2, though, offer a partial solution by comparing the closeness of the approximation for two particular sets of values for n and p.

Section 4.2 The Poisson Distribution 221

Table 4.2.1 Binomial Probabilities and Poisson Limits; n = 5 and p = 15 (λ = 1)

k (

5 k

) (0.2)k(0.8)5−k

e−1(1)k

k!

0 0.328 0.368 1 0.410 0.368 2 0.205 0.184 3 0.051 0.061 4 0.006 0.015 5 0.000 0.003 6+ 0 0.001

1.000 1.000

Table 4.2.2 Binomial Probabilities and Poisson Limits; n = 100 and p = 1100 (λ = 1)

k (

100 k

) (0.01)k(0.99)100−k

e−1(1)k

k!

0 0.366032 0.367879 1 0.369730 0.367879 2 0.184865 0.183940 3 0.060999 0.061313 4 0.014942 0.015328 5 0.002898 0.003066 6 0.000463 0.000511 7 0.000063 0.000073 8 0.000007 0.000009 9 0.000001 0.000001

10 0.000000 0.000000

1.000000 0.999999

In both cases λ = np is equal to 1, but in the former, n is set equal to 5—in the latter, to 100. We see in Table 4.2.1 (n = 5) that for some k the agreement between the binomial probability and Poisson’s limit is not very good. If n is as large as 100, though (Table 4.2.2), the agreement is remarkably good for all k.

Example 4.2.2

According to the IRS, 137.8 million individual tax returns were filed in 2008. Out of that total, 1.4 million taxpayers, or 1.0%, had the good fortune of being audited. Not everyone, though, had the same chance of getting caught in the IRS’s headlights: millionaires had the considerably higher audit rate of 5.6% (and maybe a bit higher still if the Feds found out about their offshore bank accounts in the Cayman Islands and those summer “cottages” in the Hamptons).

Criminal investigations were initiated against 3749 of all those indicted, and 1735 of that group were eventually convicted of tax fraud and sent to jail.

Suppose your hometown has 65,000 taxpayers, a cohort whose income profiles and proclivities for financial duplicity are much the same as those characteristic of the United States as a whole. What is the probability that at least three of your “neigh- bors” will be doing some Federal slammer-time next year?

222 Chapter 4 Special Distributions

Let X denote the number of your neighbors who will be incarcerated. Note that X is a binomial random variable based on a very large n (= 65,000) and a very small p (= 1735/137,800,000 = 0.0000126), so Poisson’s limit is clearly applicable (and helpful). Here, where λ = np = 65,000(0.0000126) = 0.819,

P(At least three neighbors go to jail) = P(X ≥ 3) = 1 − P(X ≤ 2)

= 1 − 2∑

k=0

( 65,000

k

) (0.0000126)k(0.9999874)65,000−k

=̇ 1 − 2∑

k=0 e−0.819

(0.819)k

k!

= 0.050

CASE STUDY 4.2.1

Leukemia is a rare form of cancer whose cause and mode of transmission remain largely unknown. While evidence abounds that excessive exposure to radiation can increase a person’s risk of contracting the disease, it is at the same time true that most cases occur among persons whose history contains no such overexpo- sure. A related issue, one maybe even more basic than the causality question, concerns the spread of the disease. It is safe to say that the prevailing medical opinion is that most forms of leukemia are not contagious—still, the hypothesis persists that some forms of the disease, particularly the childhood variety, may be. What continues to fuel this speculation are the discoveries of so-called “leukemia clusters,” aggregations in time and space of unusually large numbers of cases.

To date, one of the most frequently cited leukemia clusters in the medical literature occurred during the late 1950s and early 1960s in Niles, Illinois, a sub- urb of Chicago (81). In the 5 13 -year period from 1956 to the first four months of 1961, physicians in Niles reported a total of eight cases of leukemia among children less than fifteen years of age. The number at risk (that is, the number of residents in that age range) was 7076. To assess the likelihood of that many cases occurring in such a small population, it is necessary to look first at the leukemia incidence in neighboring towns. For all of Cook County, excluding Niles, there were 1,152,695 children less than fifteen years of age—and among those, 286 diagnosed cases of leukemia. That gives an average 5 13 -year leukemia rate of 24.8 cases per 100,000:

286 cases for 5 13 years

1,152,695 children × 100,000

100,000 = 24.8 cases/100,000 children in 5 13 years

Now, imagine the 7076 children in Niles to be a series of n = 7076 (indepen- dent) Bernoulli trials, each having a probability of p = 24.8/100,000 = 0.000248 of contracting leukemia. The question then becomes, given an n of 7076 and a p of 0.000248, how likely is it that eight “successes” would occur? (The expected number, of course, would be 7076 × 0.000248 = 1.75.) Actually, for reasons that will be elaborated on in Chapter 6, it will prove more meaningful to consider the related event, eight or more cases occurring in a 5 13 -year span. If the probability associated with the latter is very small, it could be argued that leukemia did not occur randomly in Niles and that, perhaps, contagion was a factor.

Section 4.2 The Poisson Distribution 223

Using the binomial distribution, we can express the probability of eight or more cases as

P(8 or more cases) = 7076∑ k=8

( 7076

k

) (0.000248)k(0.999752)7076−k (4.2.1)

Much of the computational unpleasantness implicit in Equation 4.2.1 can be avoided by appealing to Theorem 4.2.1. Given that np = 7076×0.000248 = 1.75,

P(X ≥ 8) = 1 − P(X ≤ 7)

=̇ 1 − 7∑

k=0

e−1.75(1.75)k

k!

= 1 − 0.99953 = 0.00047

How close can we expect 0.00047 to be to the “true” binomial sum? Very close. Considering the accuracy of the Poisson limit when n is as small as one hundred (recall Table 4.2.2), we should feel very confident here, where n is 7076.

Interpreting the 0.00047 probability is not nearly as easy as assessing its accuracy. The fact that the probability is so very small tends to denigrate the hypothesis that leukemia in Niles occurred at random. On the other hand, rare events, such as clusters, do happen by chance. The basic difficulty of putting the probability associated with a given cluster into any meaningful perspective is not knowing in how many similar communities leukemia did not exhibit a tendency to cluster. That there is no obvious way to do this is one reason the leukemia controversy is still with us.

About the Data Publication of the Niles cluster led to a number of research efforts on the part of biostatisticians to find quantitative methods capable of detecting clustering in space and time for diseases having low epidemicity. Several techniques were ultimately put forth, but the inherent “noise” in the data—variations in population densities, ethnicities, risk factors, and medical practices—often proved impossible to overcome.

Questions

4.2.1. If a typist averages one misspelling in every 3250 words, what are the chances that a 6000-word report is free of all such errors? Answer the question two ways— first, by using an exact binomial analysis, and second, by using a Poisson approximation. Does the similarity (or dissimilarity) of the two answers surprise you? Explain.

4.2.2. A medical study recently documented that 905 mis- takes were made among the 289,411 prescriptions written during one year at a large metropolitan teaching hospi- tal. Suppose a patient is admitted with a condition serious enough to warrant ten different prescriptions. Approxi- mate the probability that at least one will contain an error.

4.2.3. Five hundred people are attending the first annual “I was Hit by Lighting” Club. Approximate the proba- bility that at most one of the five hundred was born on Poisson’s birthday.

4.2.4. A chromosome mutation linked with colorblind- ness is known to occur, on the average, once in every ten thousand births. (a) Approximate the probability that exactly three of the next twenty thousand babies born will have the mutation. (b) How many babies out of the next twenty thousand would have to be born with the mutation to convince you that the “one in ten thousand” estimate is too low?

224 Chapter 4 Special Distributions

[Hint: Calculate P(X ≥ k) = 1 − P(X ≤ k − 1) for vari- ous k. (Recall Case Study 4.2.1.)]

4.2.5. Suppose that 1% of all items in a supermarket are not priced properly. A customer buys ten items. What is the probability that she will be delayed by the cashier because one or more of her items require a price check? Calculate both a binomial answer and a Poisson answer. Is the binomial model “exact” in this case? Explain.

4.2.6. A newly formed life insurance company has under- written term policies on 120 women between the ages of forty and forty-four. Suppose that each woman has a 1/150 probability of dying during the next calendar year, and that each death requires the company to pay out $50,000 in benefits. Approximate the probability that the company will have to pay at least $150,000 in benefits next year.

4.2.7. According to an airline industry report (189), roughly one piece of luggage out of every two hundred that are checked is lost. Suppose that a frequent-flying

businesswoman will be checking one hundred twenty bags over the course of the next year. Approximate the proba- bility that she will lose two or more pieces of luggage.

4.2.8. Electromagnetic fields generated by power trans- mission lines are suspected by some researchers to be a cause of cancer. Especially at risk would be telephone linemen because of their frequent proximity to high- voltage wires. According to one study, two cases of a rare form of cancer were detected among a group of 9500 line- men (185). In the general population, the incidence of that particular condition is on the order of one in a million. What would you conclude? (Hint: Recall the approach taken in Case Study 4.2.1.)

4.2.9. Astronomers estimate that as many as one hundred billion stars in the Milky Way galaxy are encircled by plan- ets. If so, we may have a plethora of cosmic neighbors. Let p denote the probability that any such solar system con- tains intelligent life. How small can p be and still give a fifty-fifty chance that we are not alone?

THE POISSON DISTRIBUTION

The real significance of Poisson’s limit theorem went unrecognized for more than fifty years. For most of the latter part of the nineteenth century, Theorem 4.2.1 was taken strictly at face value: It provides a convenient approximation for pX (k) when X is binomial, n is large, and p is small. But then in 1898 a German professor, Ladislaus von Bortkiewicz, published a monograph entitled Das Gesetz der Kleinen Zahlen (The Law of Small Numbers) that would quickly transform Poisson’s “limit” into Poisson’s “distribution.”

What is best remembered about Bortkiewicz’s monograph is the curious set of data described in Question 4.2.10. The measurements recorded were the numbers of Prussian cavalry soldiers who had been kicked to death by their horses. In an- alyzing those figures, Bortkiewicz was able to show that the function e−λλk/k! is a useful probability model in its own right, even when (1) no explicit binomial random variable is present and (2) values for n and p are unavailable. Other researchers were quick to follow Bortkiewicz’s lead, and a steady stream of Poisson distribution appli- cations began showing up in technical journals. Today the function pX (k) = e−λλk/k! is universally recognized as being among the three or four most important data mod- els in all of statistics.

Theorem 4.2.2

The random variable X is said to have a Poisson distribution if

pX (k) = P(X = k) = e −λλk

k! , k = 0, 1, 2, . . .

where λ is a positive constant. Also, for any Poisson random variable, E(X ) = λ and Var(X ) = λ. Proof To show that pX (k) qualifies as a probability function, note, first of all, that pX (k) ≥ 0 for all nonnegative integers k. Also, pX (k) sums to 1:

∞∑ k=0

pX (k) = ∞∑

k=0

e−λλk

k! = e−λ

∞∑ k=0

λk

k! = e−λ · eλ = 1

Section 4.2 The Poisson Distribution 225

since ∞∑

k=0 λk

k! is the Taylor series expansion of e λ. Verifying that E(X ) = λ and

Var(X ) = λ has already been done in Example 3.12.9, using moment-generating functions.

FITTING THE POISSON DISTRIBUTION TO DATA

Poisson data invariably refer to the numbers of times a certain event occurs during each of a series of “units” (often time or space). For example, X might be the weekly number of traffic accidents reported at a given intersection. If such records are kept for an entire year, the resulting data would be the sample k1, k2, . . . , k52, where each ki is a nonnegative integer.

Whether or not a set of ki’s can be viewed as Poisson data depends on whether the proportions of 0’s, 1’s, 2’s, and so on, in the sample are numerically similar to the probabilities that X = 0, 1, 2, and so on, as predicted by pX (k) = e−λλk/k!. The next two case studies show data sets where the variability in the observed ki’s is consistent with the probabilities predicted by the Poisson distribution. Notice in each case that

the λ in pX (k) is replaced by the sample mean of the ki’s, that is, by k̄ = (1/n) n∑

i=1 ki.

Why these phenomena are described by the Poisson distribution will be discussed later in this section; why λ is replaced by k̄ will be explained in Chapter 5.

CASE STUDY 4.2.2

Among the early research projects investigating the nature of radiation was a 1910 study of α-particle emission by Ernest Rutherford and Hans Geiger (161). For each of 2608 eighth-minute intervals, the two physicists recorded the num- ber of α particles emitted from a polonium source (as detected by what would eventually be called a Geiger counter). The numbers and proportions of times that k such particles were detected in a given eighth-minute (k = 0, 1, 2, . . .) are detailed in the first three columns of Table 4.2.3. Two α particles, for example, were detected in each of 383 eighth-minute intervals, meaning that X = 2 was the observation recorded 15% (= 383/2608 × 100) of the time.

Table 4.2.3

No. Detected, k Frequency Proportion pX (k) = e−3.87(3.87)k/k! 0 57 0.02 0.02 1 203 0.08 0.08 2 383 0.15 0.16 3 525 0.20 0.20 4 532 0.20 0.20 5 408 0.16 0.15 6 273 0.10 0.10 7 139 0.05 0.05 8 45 0.02 0.03 9 27 0.01 0.01

10 10 0.00 0.00 11+ 6 0.00 0.00

2608 1.0 1.0

(Continued on next page)

226 Chapter 4 Special Distributions

(Case Study 4.2.2 continued)

To see whether a probability function of the form pX (k) = e−λλk/k! can adequately model the observed proportions in the third column, we first need to replace λ with the sample’s average value for X . Suppose the six observations comprising the “11+” category are each assigned the value 11. Then

k̄ = 57(0) + 203(1) + 383(2) + · · · + 6(11) 2608

= 10,092 2608

= 3.87 and the presumed model is pX (k) = e−3.87(3.87)k/k!, k = 0, 1, 2, . . .. Notice how closely the entries in the fourth column (i.e., pX (0), pX (1), pX (2), . . .) agree with the sample proportions appearing in the third column. The conclusion here is inescapable: The phenomenon of radiation can be modeled very effectively by the Poisson distribution.

About the Data The most obvious (and frequent) application of the Pois- son/radioactivity relationship is to use the former to describe and predict the behavior of the latter. But the relationship is also routinely used in reverse. Workers responsible for inspecting areas where radioactive contamination is a potential hazard need to know that their monitoring equipment is functioning properly. How do they do that? A standard safety procedure before entering what might be a life-threatening “hot zone” is to take a series of readings on a known radioactive source (much like the Rutherford/Geiger experiment itself). If the resulting set of counts does not follow a Poisson distribution, the meter is assumed to be broken and must be repaired or replaced.

CASE STUDY 4.2.3

In the 432 years from 1500 to 1931, war broke out somewhere in the world a total of two hundred ninety-nine times. By definition, a military action was a war if it either was legally declared, involved over fifty thousand troops, or resulted in significant boundary realignments. To achieve greater uniformity from war to war, major confrontations were split into smaller “subwars”: World War I, for example, was treated as five separate conflicts (152).

Let X denote the number of wars starting in a given year. The first two columns in Table 4.2.4 show the distribution of X for the 432-year period in question. Here the average number of wars beginning in a given year was 0.69:

k̄ = 0(223) + 1(142) + 2(48) + 3(15) + 4(4) 432

= 0.69

Table 4.2.4

Number of Wars, k Frequency Proportion pX (k) = e−0.69 (0.69) k

k!

0 223 0.52 0.50 1 142 0.33 0.35 2 48 0.11 0.12 3 15 0.03 0.03 4+ 4 0.01 0.00

432 1.00 1.00

Section 4.2 The Poisson Distribution 227

The last two columns in Table 4.2.4 compare the observed proportions of years for which X = k with the proposed Poisson model

pX (k) = e−0.69 (0.69) k

k! , k = 0, 1, 2, . . .

Clearly, there is a very close agreement between the two—the number of wars beginning in a given year can be considered a Poisson random variable.

THE POISSON MODEL: THE LAW OF SMALL NUMBERS

Given that the expression e−λλk/k! models phenomena as diverse as α-radiation and outbreaks of war raises an obvious question: Why is that same pX (k) describing such different random variables? The answer is that the underlying physical conditions that produce those two sets of measurements are actually much the same, despite how superficially different the resulting data may seem to be. Both phenomena are examples of a set of mathematical assumptions known as the Poisson model. Any measurements that are derived from conditions that mirror those assumptions will necessarily vary in accordance with the Poisson distribution.

Suppose a series of events is occurring during a time interval of length T . Imagine dividing T into n nonoverlapping subintervals, each of length Tn , where n is large (see Figure 4.2.1). Furthermore, suppose that

T/n

T

5

Events occur

1 2 3 4 n

Figure 4.2.1

1. The probability that two or more events occur in any given subinterval is essentially 0.

2. The events occur independently.

3. The probability that an event occurs during a given subinterval is constant over the entire interval from 0 to T .

The n subintervals, then, are analogous to the n independent trials that form the backdrop for the “binomial model”: In each subinterval there will be either zero events or one event, where

pn = P(Event occurs in a given subinterval)

remains constant from subinterval to subinterval. Let the random variable X denote the total number of events occurring during

time T , and let λ denote the rate at which events occur (e.g., λ might be expressed as 2.5 events per minute). Then

E(X ) = λT = npn (why?)

228 Chapter 4 Special Distributions

which implies that pn = λTn . From Theorem 4.2.1, then,

pX (k) = P(X = k) = (n

k

)(λT n

)k ( 1 − λT

n

)n−k

=̇ e −n(λT/n) [n(λT/n)]k

k!

= e −λT (λT)k

k! (4.2.2)

Now we can see more clearly why Poisson’s “limit,” as given in Theorem 4.2.1, is so important. The three Poisson model assumptions are so unexceptional that they apply to countless real-world phenomena. Each time they do, the pdf pX (k) = e−λT (λT)k/k! finds another application.

Example 4.2.3

It is not surprising that the number of α particles emitted by a radioactive source in a given unit of time follows a Poisson distribution. Nuclear physicists have known for a long time that the phenomenon of radioactivity obeys the same assumptions that define the Poisson model. Each is a poster child for the other. Case Study 4.2.3, on the other hand, is a different matter altogether. It is not so obvious why the number of wars starting in a given year should have a Poisson distribution. Reconciling the data in Table 4.2.4 with the “picture” of the Poisson model in Figure 4.2.1 raises a number of questions that never came up in connection with radioactivity.

Imagine recording the data summarized in Table 4.2.4. For each year, new wars would appear as “occurrences” on a grid of cells, similar to the one pictured in Figure 4.2.2 for 1776. Civil wars would be entered along the diagonal and wars be- tween two countries, above the diagonal. Each cell would contain either a 0 (no war) or a 1 (war). The year 1776 saw the onset of only one major conflict, the Revolution- ary War between the United States and Britain. If the random variable

Xi = number of outbreaks of war in year i, i = 1500, 1501, . . . , 1931 then X1776 = 1.

0

1

00

0 0

0

0

0

0

0

00

0

00

00Austria

1776

A us

tri a

Br ita

in Fr

an ce

Sp ai

n

R us

sia

U ni

te d

St at

es

Rev olut

iona ry

War

Britain

France

Spain

Russia

United States

0

0

0

Figure 4.2.2

Section 4.2 The Poisson Distribution 229

What do we know, in general, about the random variable Xi? If each cell in the grid is thought of as a “trial,” then Xi is clearly the number of “successes” in those n trials. Does that make Xi a binomial random variable? Not necessarily. According to Theorem 3.2.1, Xi qualifies as a binomial random variable only if the trials are independent and the probability of success is the same from trial to trial.

At first glance, the independence assumption would seem to be problematic. There is no denying that some wars are linked to others. The timing of the French Revolution, for example, is widely thought to have been influenced by the success of the American Revolution. Does that make the two wars dependent? In a histor- ical sense, yes; in a statistical sense, no. The French Revolution began in 1789, thir- teen years after the onset of the American Revolution. The random variable X1776, though, focuses only on wars starting in 1776, so linkages that are years apart do not compromise the binomial’s independence assumption.

Not all wars identified in Case Study 4.2.3, though, can claim to be independent in the statistical sense. The last entry in column 2 of Table 4.2.4 shows that four or more wars erupted on four separate occasions; the Poisson model (column 4) pre- dicted that no years would experience that many new wars. Most likely, those four years had a decided excess of new wars because of political alliances that led to a cascade of new wars being declared simultaneously. Those wars definitely violated the binomial assumption of independent trials, but they accounted for only a very small fraction of the entire data set.

The other binomial assumption—that each trial has the same probability of success—holds fairly well. For the vast majority of years and the vast majority of countries, the probabilities of new wars will be very small and most likely similar. For almost every year, then, Xi can be considered a binomial random variable based on a very large n and a very small p. That being the case, it follows by Theorem 4.2.1 that each Xi, i = 1500, 1501, . . . , 1931 can be approximated by a Poisson distribution.

One other assumption needs to be addressed. Knowing that X1500, X1501, . . . , X1931—individually—are Poisson random variables does not guarantee that the distribution of all four hundred thirty-two Xi’s will have a Poisson distribution. Only if the Xi’s are independent observations having basically the same Poisson distribution—that is, the same value for λ—will their overall distribution be Pois- son. But Table 4.2.4 does have a Poisson distribution, implying that the set of Xi’s does, in fact, behave like a random sample. Along with that sweeping conclusion, though, comes the realization that, as a species, our levels of belligerence at the na- tional level [that is, the four hundred thirty-two values for λ = E(Xi)] have remained basically the same for the past five hundred years. Whether that should be viewed as a reason for celebration or a cause for alarm is a question best left to historians, not statisticians.

CALCULATING POISSON PROBABILITIES

Three formulas have appeared in connection with the Poisson distribution:

1. pX (k) .= e−np (np)kk!

2. pX (k) = e−λ λkk! 3. pX (k) = e−λT (λT)

k

k!

The first is the approximating Poisson limit, where the pX (k) on the left-hand side refers to the probability that a binomial random variable (with parameters n and p) is equal to k. Formulas (2) and (3) are sometimes confused because both presume to give the probability that a Poisson random variable equals k. Why are they different?

230 Chapter 4 Special Distributions

Actually, all three formulas are the same in the sense that the right-hand sides of each could be written as

4. e−E(X ) [E(X )] k

k!

In formula (1), X is binomial, so E(X ) = np. In formula (2), which comes from Theorem 4.2.2, λ is defined to be E(X ). Formula (3) covers all those situations where the units of X and λ are not consistent, in which case E(X ) �= λ. However, λ can always be multiplied by an appropriate constant T to make λT equal to E(X ).

For example, suppose a certain radioisotope is known to emit α particles at the rate of λ = 1.5 emissions/second. For whatever reason, though, the experimenter defines the Poisson random variable X to be the number of emissions counted in a given minute. Then T = 60 seconds and

E(X ) = 1.5 emissions/second × 60 seconds = λT = 90 emissions

Example 4.2.4

Entomologists estimate that an average person consumes almost a pound of bug parts each year (184). There are that many insect eggs, larvae, and miscellaneous body pieces in the foods we eat and the liquids we drink. The Food and Drug Ad- ministration (FDA) sets a Food Defect Action Level (FDAL) for each product: Bug- part concentrations below the FDAL are considered acceptable. The legal limit for peanut butter, for example, is thirty insect fragments per hundred grams. Suppose the crackers you just bought from a vending machine are spread with twenty grams of peanut butter. What are the chances that your snack will include at least five crunchy critters?

Let X denote the number of bug parts in twenty grams of peanut butter. As- suming the worst, suppose the contamination level equals the FDA limit—that is, thirty fragments per hundred grams (or 0.30 fragment/g). Notice that T in this case is twenty grams, making E(X ) = 6.0:

0.30 fragment g

× 20 g = 6.0 fragments

It follows, then, that the probability that your snack contains five or more bug parts is a disgusting 0.71:

P(X ≥ 5) = 1 − P(X ≤ 4) = 1 − 4∑

k=0

e−6.0(6.0)k

k!

= 1 − 0.29 = 0.71

Bon appetit!

Questions

4.2.10. During the latter part of the nineteenth century, Prussian officials gathered information relating to the haz- ards that horses posed to cavalry soldiers. A total of ten cavalry corps were monitored over a period of twenty years. Recorded for each year and each corps was X , the

annual number of fatalities due to kicks. Summarized in the following table are the two hundred values recorded for X (14). Show that these data can be modeled by a Poisson pdf. Follow the procedure illustrated in Case Studies 4.2.2 and 4.2.3.

Section 4.2 The Poisson Distribution 231

Observed Number of Corps-Years No. of Deaths, k in Which k Fatalities Occurred

0 109 1 65 2 22 3 3 4 1

200

4.2.11. A random sample of three hundred fifty-six seniors enrolled at the University of West Florida was categorized according to X , the number of times they had changed majors (117). Based on the summary of that information shown in the following table, would you conclude that X can be treated as a Poisson random variable?

Number of Major Changes Frequency

0 237 1 90 2 22 3 7

4.2.12. Midwestern Skies books ten commuter flights each week. Passenger totals are much the same from week to week, as are the numbers of pieces of luggage that are checked. Listed in the following table are the numbers of bags that were lost during each of the first forty weeks in 2009. Do these figures support the presumption that the number of bags lost by Midwestern during a typical week is a Poisson random variable?

Week Bags Lost Week Bags Lost Week Bags Lost

1 1 14 2 27 1 2 0 15 1 28 2 3 0 16 3 29 0 4 3 17 0 30 0 5 4 18 2 31 1 6 1 19 5 32 3 7 0 20 2 33 1 8 2 21 1 34 2 9 0 22 1 35 0

10 2 23 1 36 1 11 3 24 2 37 4 12 1 25 1 38 2 13 2 26 3 39 1

40 0

4.2.13. In 1893, New Zealand became the first country to permit women to vote. Scattered over the ensuing 113 years, various countries joined the movement to grant this

right to women. The table below (129) shows how many countries took this step in a given year. Do these data seem to follow a Poisson distribution?

Yearly Number of Countries Granting Women the Vote Frequency

0 82 1 25 2 4 3 0 4 2

4.2.14. The following are the daily numbers of death no- tices for women over the age of eighty that appeared in the London Times over a three-year period (80).

Number of Deaths Observed Frequency

0 162 1 267 2 271 3 185 4 111 5 61 6 27 7 8 8 3 9 1

1096

(a) Does the Poisson pdf provide a good description of the variability pattern evident in these data? (b) If your answer to part (a) is “no,” which of the Poisson model assumptions do you think might not be holding?

4.2.15. A certain species of European mite is capable of damaging the bark on orange trees. The following are the results of inspections done on one hundred saplings cho- sen at random from a large orchard. The measurement recorded, X , is the number of mite infestations found on the trunk of each tree. Is it reasonable to assume that X is a Poisson random variable? If not, which of the Poisson model assumptions is likely not to be true?

No. of Infestations, k No. of Trees

0 55 1 20 2 21 3 1 4 1 5 1 6 0 7 1

232 Chapter 4 Special Distributions

4.2.16. A tool and die press that stamps out cams used in small gasoline engines tends to break down once ev- ery five hours. The machine can be repaired and put back on line quickly, but each such incident costs $50. What is the probability that maintenance expenses for the press will be no more than $100 on a typical eight-hour workday?

4.2.17. In a new fiber-optic communication system, trans- mission errors occur at the rate of 1.5 per ten seconds. What is the probability that more than two errors will occur during the next half-minute?

4.2.18. Assume that the number of hits, X , that a base- ball team makes in a nine-inning game has a Poisson distribution. If the probability that a team makes zero hits is 13 , what are their chances of getting two or more hits?

4.2.19. Flaws in metal sheeting produced by a high- temperature roller occur at the rate of one per ten square feet. What is the probability that three or more flaws will appear in a five-by-eight-foot panel?

4.2.20. Suppose a radioactive source is metered for two hours, during which time the total number of alpha parti- cles counted is four hundred eighty-two. What is the prob- ability that exactly three particles will be counted in the next two minutes? Answer the question two ways—first,

by defining X to be the number of particles counted in two minutes, and second, by defining X to be the number of particles counted in one minute.

4.2.21. Suppose that on-the-job injuries in a textile mill occur at the rate of 0.1 per day. (a) What is the probability that two accidents will occur during the next (five-day) workweek? (b) Is the probability that four accidents will occur over the next two workweeks the square of your answer to part (a)? Explain.

4.2.22. Find P(X = 4) if the random variable X has a Poisson distribution such that P(X = 1) = P(X = 2). 4.2.23. Let X be a Poisson random variable with param- eter λ. Show that the probability that X is even is 12 (1 + e−2λ).

4.2.24. Let X and Y be independent Poisson random vari- ables with parameters λ and μ, respectively. Example 3.12.10 established that X +Y is also Poisson with param- eter λ + μ. Prove that same result using Theorem 3.8.3. 4.2.25. If X1 is a Poisson random variable for which E(X1) = λ and if the conditional pdf of X2 given that X1 = x1 is binomial with parameters x1 and p, show that the marginal pdf of X2 is Poisson with E(X2) = λp.

INTERVALS BETWEEN EVENTS: THE POISSON/EXPONENTIAL RELATIONSHIP

Situations sometimes arise where the time interval between consecutively occurring events is an important random variable. Imagine being responsible for the mainte- nance on a network of computers. Clearly, the number of technicians you would need to employ in order to be capable of responding to service calls in a timely fashion would be a function of the “waiting time” from one breakdown to another.

Figure 4.2.3 shows the relationship between the random variables X and Y , where X denotes the number of occurrences in a unit of time and Y denotes the interval between consecutive occurrences. Pictured are six intervals: X = 0 on one occasion, X = 1 on three occasions, X = 2 once, and X = 3 once. Resulting from those eight occurrences are seven measurements on the random variable Y . Obviously, the pdf for Y will depend on the pdf for X . One particularly important special case of that dependence is the Poisson/exponential relationship outlined in Theorem 4.2.3.

y1 y2 y3 y4 y5 y6 y7Y values:

Unit time

X = 1 X = 1 X = 0 X = 3X = 1 X = 2

Figure 4.2.3

Section 4.2 The Poisson Distribution 233

Theorem 4.2.3

Suppose a series of events satisfying the Poisson model are occurring at the rate of λ per unit time. Let the random variable Y denote the interval between consecutive events. Then Y has the exponential distribution

fY (y) = λe−λy, y > 0 Proof Suppose an event has occurred at time a. Consider the interval that ex- tends from a to a + y. Since the (Poisson) events are occurring at the rate of λ per unit time, the probability that no outcomes will occur in the interval (a, a + y) is e

−λy(λy)0

0! = e−λy. Define the random variable Y to denote the interval between consecutive occurrences. Notice that there will be no occurrences in the interval (a, a + y) only if Y > y. Therefore,

P(Y > y) = e−λy

or, equivalently,

FY (y) = P(Y ≤ y) = 1 − P(Y > y) = 1 − e−λy

Let fY (y) be the (unknown) pdf for Y . It must be true that

P(Y ≤ y) = ∫ y

0 fY (t) dt

Taking derivatives of the two expressions for FY (y) gives

d dy

∫ y 0

fY (t) dt = ddy (1 − e −λy)

which implies that

fY (y) = λe−λy, y > 0

CASE STUDY 4.2.4

The occurrences of earthquakes in a given area, after removing aftershocks, are believed to be Poisson events; that is, they are thought to occur independently and at a constant rate of λ per unit time. If so, the pdf describing the distribution of intervals between earthquakes should have the exponential form, fY (y) = λe−λy, for y > 0.

A source of many earthquakes is an area intersecting five states, running roughly south from New Madrid, Missouri, to the northwest corner of Ten- nessee. It is known as the New Madrid Seismic Zone. A recent study (117) of New Madrid earthquakes spanned the period from January 12, 1973, to December 29, 2013. There were two hundred nineteen moderate earthquakes during those 40 years. The average time between tremors was 65.91 days, so λ = 1/65.91 = 0.15. Is the exponential pdf with that particular value for λ consistent with the statement of Theorem 4.2.3?

To answer that question requires that a graph of the predicted exponential pdf be superimposed over a density-scaled histogram of the data (recall Case Study 3.4.1). Table 4.2.5 details the construction of the histogram. Notice in Figure 4.2.4 that the shape of that histogram is, indeed, entirely consistent with the theoretical model cited in Theorem 4.2.3— fY (y) = 0.015e−0.15y.

(Continued on next page)

234 Chapter 4 Special Distributions

(Case Study 4.2.4 continued)

Table 4.2.5

Interval (days), y Frequency Density

0 ≤ y < 30 86 0.0131 30 ≤ y < 60 53 0.0081 60 ≤ y < 90 29 0.0044 90 ≤ y < 120 13 0.0020

120 ≤ y < 150 11 0.0017 150 ≤ y < 180 11 0.0017 180 ≤ y < 210 2 0.0003 210 ≤ y < 240 7 0.0011 240 ≤ y < 270 6 0.0009

218

0.014

0.010

0.006

0.002

0

D en

si ty

60

Interval between earthquakes (days)

120 180 240

fY(y) = 0.015e –0.015y

y

Figure 4.2.4

About the Data Many pessimists believe that “Bad things come in threes.” Many optimists, not to be outdone, claim that “Good things come in threes.” Are they right? In a sense, yes, but not because of fate, bad karma, or good luck. Bad things (and good things and so-so things) sometimes seem to come in threes because of (1) our intuition’s inability to understand the nature of randomness and (2) the Poisson/exponential relationship. Case Study 4.2.4—specifically, the shape of the exponential pdf pictured in Figure 4.2.4—illustrates the statistics behind the superstition.

Suppose that bad things are, in fact, happening to us randomly in time at the rate of, say, twelve bad things per year. We mistakenly think that “randomness” means that those twelve bad things should occur roughly one month apart. The- orem 4.2.3, though, proves exactly the opposite—randomly occurring events will not be equally spaced. Look again at the shape of fY (y) in Figure 4.2.4. It would not be unlikely for the two separations between the next three earthquakes to both be very small compared to the average separation between the entire set of earthquakes. Every time that happens fuels the misconception that bad things (in this case, earthquakes) are occurring in threes.

Example 4.2.5

Among the most famous of all meteor showers are the Perseids, which occur each year in early August. In some areas the frequency of visible Perseids can be as high as

Section 4.3 The Normal Distribution 235

forty per hour. Given that such sightings are Poisson events, calculate the probability that an observer who has just seen a meteor will have to wait at least five minutes before seeing another one.

Let the random variable Y denote the interval (in minutes) between consecu- tive sightings. Expressed in the units of Y , the forty-per-hour rate of visible Perseids becomes 0.67 per minute. A straightforward integration, then, shows that the prob- ability is 0.035 that an observer will have to wait five minutes or more to see another meteor:

P(Y > 5) = ∫ ∞

5 0.67e −0.67y dy

= ∫ ∞

3.35 e−u du (where u = 0.67y)

= −e−u∣∣∞3.35 = e−3.35 = 0.035

Questions

4.2.26. Suppose that commercial airplane crashes in a cer- tain country occur at the rate of 2.5 per year. (a) Is it reasonable to assume that such crashes are Poisson events? Explain. (b) What is the probability that four or more crashes will occur next year? (c) What is the probability that the next two crashes will occur within three months of one another?

4.2.27. Records show that deaths occur at the rate of 0.1 per day among patients residing in a large nursing home. If someone dies today, what are the chances that a week or more will elapse before another death occurs?

4.2.28. Fifty spotlights have just been installed in an out- door security system. According to the manufacturer’s specifications, these particular lights are expected to burn out at the rate of 1.1 per one hundred hours. What is the expected number of bulbs that will fail to last for at least seventy-five hours?

4.2.29. Hawaii’s fourteen-thousand-foot Mauna Loa is one of the world’s most active volcanoes. Between 1832 and 1950, it erupted thirty-seven times, which translates to a rate of 0.027 times per month. Listed below are the thirty-six intervals (in months) that elapsed between those thirty-seven eruptions (113):

126 73 3 6 37 23 73 23 2 65 94 51 26 21 6 68 16 20 6 18 6 41 40 18

41 11 12 38 77 61 26 3 38 50 91 12

Can it be argued that the times Mauna Loa erupted are consistent with a series of events satisfying the Pois- son model? Answer the question by constructing and interpreting an appropriate graph.

4.3 The Normal Distribution The Poisson limit described in Section 4.2 was not the only formula—or even the first—developed for the purpose of approximating binomial probabilities when n is large. Early in the eighteenth century, Abraham De Moivre derived the unusual- looking result that

P

⎡ ⎣a ≤ X − n ( 12)√

n ( 1

2

) ( 1 2

) ≤ b ⎤ ⎦ =̇ 1√

2π

∫ b a

e−z 2/2dz

where X is a binomial random variable for which n is large and p = 1/2.

236 Chapter 4 Special Distributions

Figure 4.3.1 illustrates DeMoivre’s approximation. Shown is a bar graph of a binomial distribution with n = 20 and p = 1/2. That is,

pX (k) = (

20 k

)( 1 2

)k ( 1 − 1

2

)20−k , k = 0, 1, 2, . . . , 20

The superimposed dotted curve is the function fz(z) = 1√2πe−z 2/2,−∞ < z < ∞,

Theorem 4.3.1 integral approximation

22.46

5 6 7 8 9 10 11 12 13 14 15 X

(X 2 10) 5/ 22.0121.5621.12 2.67 2.22 0 .22 .67 1.12 1.56 2.01 2.46

0