RSH 901 Week 6, 7th and 8th $75

ISBN-13: ISBN-10:

978-0-13-449716-7 0-13-449716-3

9 7 8 0 1 3 4 4 9 7 1 6 7

9 0 0 0 0

Get the Most Out of MyStatLab®

R O B E R T S T I N E D E A N F O S T E R

S T I N E

F O S T E R

Statistics for Business

S tatistics fo

r B usiness

Decision Making and Analysis

D ecision M

aking and

A nalysis

Third Edition

Third Edition

MyStatLab is available for this textbook. To learn more, visit www.mystatlab.com

S Personalized and adaptive learning

S Interactive practice with immediate feedback

S Multimedia learning resources

S Complete eText

S Mobile-friendly design

S Full access to StatCrunch

MyStatLab is the leading online homework, tutorial, and assessment program designed to help you learn and understand statistics.

Statistics for Business DECISION MAKING AND ANALYSIS

ROBERT STINE Wharton School of the University of Pennsylvania

DEAN FOSTER Emeritus, Wharton School of the University of Pennsylvania

330 Hudson Street, NY NY 10013

THIRD EDITION

A01_STIN7167_03_SE_FM.indd 1 12/11/16 10:03 AM

For permission to use copyrighted material, grateful acknowledgment has been made to the copyright holders listed on p. C-1 and C-2, which is hereby made part of the copyright page.

Many of the designations by manufacturers and sellers to distinguish their products are claimed as trade- marks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed in initial caps or all caps.

Microsoft® and Windows® are registered trademarks of the Microsoft Corporation in the U.S.A. and other countries. Screen shots and icons reprinted with permission from the Microsoft Corporation. This book is not sponsored or endorsed by or affiliated with the Microsoft Corporation.

MICROSOFT AND/OR ITS RESPECTIVE SUPPLIERS MAKE NO REPRESENTATIONS ABOUT THE SUITABILITY OF THE INFORMATION CONTAINED IN THE DOCUMENTS AND RELATED GRAPHICS PUBLISHED AS PART OF THE SERVICES FOR ANY PURPOSE. ALL SUCH DOCUMENTS AND RELATED GRAPHICS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND. MICROSOFT AND/OR ITS RESPECTIVE SUPPLIERS HEREBY DISCLAIM ALL WARRANTIES AND CONDI- TIONS WITH REGARD TO THIS INFORMATION, INCLUDING ALL WARRANTIES AND CONDITIONS OF MERCHANTABIL- ITY, WHETHER EXPRESS, IMPLIED OR STATUTORY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-IN- FRINGEMENT. IN NO EVENT SHALL MICROSOFT AND/OR ITS RESPECTIVE SUPPLIERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF INFORMATION AVAILABLE FROM THE SERVICES. THE DOCUMENTS AND RELATED GRAPHICS CONTAINED HEREIN COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN. MICROSOFT AND/ OR ITS RESPECTIVE SUPPLIERS MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED HEREIN AT ANY TIME. PARTIAL SCREEN SHOTS MAY BE VIEWED IN FULL WITHIN THE SOFTWARE VERSION SPECIFIED.

The student edition of this title has been catalogued at the Library of Congress as follows:

Library of Congress Cataloging-in-Publication Data

Names: Stine, Robert A., author. | Foster, Dean P., author. Title: Statistics for business : decision making and analysis / ROBERT STINE, Wharton School of the University of Pennsylvania, DEAN FOSTER, Wharton School of the University of Pennsylvania. Description: Third Edition. | Boston : Pearson, 2016. | Revised edition of

the authors’ Statistics for business, 2013. | Includes index. Identifiers: LCCN 2016016748| ISBN 9780134497167 (hardcover) | ISBN 0134497163 (hardcover) Subjects: LCSH: Commercial statistics. | Statistics. Classification: LCC HF1017 .S74 2016 | DDC 519.502/465--dc23 LC record available at https://lccn.loc.gov/2016016748

Copyright © 2018, 2014, 2011 Pearson Education, Inc. All rights reserved. No part of this publication may be re- produced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, pho- tocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. For information regarding permissions, request forms and the appropriate contacts within the Pearson Education Global Rights & Permissions department, please visit www.pearsoned.com/permissions/.

1 16

Director, Portfolio Management, Math & Statistics: Deirdre Lynch

Portfolio Management Assistant: Justin Billing

Managing Producer, Math/Stats: Karen Wernholm

Content Producer: Peggy McMahon

Producer, Production & Digital Studio, Math & Statistics: Aimee Thorne

Manager, Courseware Content Develop- ment, Math & Stats: Robert Carroll

Manager, Courseware QA: Mary Durnwald

Director, Product Marketer, Math & Sta- tistics: Erin Kelly

Product Marketing Assistant: Jennifer Myers

Senior Author Support/Technology Spe- cialist: Joe Vetere

Manager, Rights and Permissions: Gina Cheselka

Manufacturing Buyer: Carol Melville/ LSC Communications

Production Coordination, Composition, Text Design, Illustrations: Cenveo® Publisher Services

Associate Director of Design: Blair Brown

Program Design Lead: Barbara Atkinson

Cover Design: Tamara Newnam

Cover Photo: Getty Images/BeholdingEye

Student Edition: ISBN-13: 978-0-13-449716-7 ISBN-10: 0-13-449716-3

A01_STIN7167_03_SE_FM.indd 2 12/11/16 10:03 AM

iii

Robert Stine holds a Ph.D. from Princeton University. He has taught at the Whar- ton School since 1983, during which time he has regularly taught business statis- tics. During his tenure, Bob has received a variety of teaching awards, including regularly winning the MBA Core Teaching Award, which is presented to faculty for outstanding teaching of the required curriculum at Wharton. He also received the David W. Hauck Award for Outstanding Teaching, awarded to the most highly rat- ed faculty member teaching in the Wharton undergraduate program. Bob active- ly consults for industry. His clients include the pharmaceutical firms Merck and Pfizer, and he regularly works with the Federal Reserve Bank of Philadelphia on models for retail credit risk. This collaboration has produced three well-received conferences held at Wharton. His areas of research include computer software, time series analysis and forecasting, and general problems related to model iden- tification and selection. Bob has published numerous articles in research journals, including the Journal of the American Statistical Association, Journal of the Royal Statistical Society, Biometrika, and The Annals of Statistics.

Dean Foster holds a Ph.D. from the University of Maryland. He has taught at the Wharton School since 1992 and previously taught at the University of Chicago. Dean has taught courses in introductory business statistics, probability and Markov chains, statistical computing, and advanced statistics for managers. Dean’s research areas are statistical inference for stochastic processes, game theory, ma- chine learning, and variable selection. He is published in a wide variety of jour- nals, including The Annals of Statistics, Operations Research, Games and Economic Behaviour, Journal of Theoretical Population Biology, and Econometrica. Currently Senior Principal Scientist at Amazon; Dean Foster is working on big data there, and his goal is to predict the sales of each and every product that Amazon sells.

Bob Stine and Dean Foster (along with Richard Waterman) have co-authored two casebooks: Basic Business Statistics (Springer-Verlag) and Business Analysis Using Regression (Springer-Verlag). These casebooks offer a collection of data analysis examples that motivate and illustrate key ideas of statistics, ranging from standard error to regression diagnostics and time series analysis. They also have collabo- rated on a number of research articles.

ABOUT THE AUTHORS

A01_STIN7167_03_SE_FM.indd 3 12/11/16 10:03 AM

561590_MILL_MICRO_FM_ppi-xxvi.indd 2 24/11/14 5:26 PM

This page intentionally left blank

v

Preface xi

Index of Application xxi

PART I Variation

1 Introduction 2 1.1 What Is Statistics? 2

1.2 Previews 4

2 Data 10 2.1 Data Tables 11

2.2 Categorical and Numerical Data 12

2.3 Recoding and Aggregation 14

2.4 Time Series 17

2.5 Further Attributes of Data 18

Chapter Summary 22

3 Describing Categorical Data 26 3.1 Looking at Data 27

3.2 Charts of Categorical Data 28

3.3 The Area Principle 33

3.4 Mode and Median 38

Chapter Summary 42

4 Describing Numerical Data 51 4.1 Summaries of Numerical Variables 52

4.2 Histograms 57

4.3 Boxplots 59

4.4 Shape of a Distribution 62

4.5 Epilog 68

Chapter Summary 72

5 Association between Categorical Variables 80 5.1 Contingency Tables 81

5.2 Lurking Variables and Simpson’s Paradox 89

CONTENTS

A01_STIN7167_03_SE_FM.indd 5 18/11/16 2:22 PM

vi CONTENTS

5.3 Strength of Association 92

Chapter Summary 100

6 Association between Quantitative Variables 109 6.1 Scatterplots 110

6.2 Association in Scatterplots 111

6.3 Measuring Association 114

6.4 Summarizing Association with a Line 120

6.5 Spurious Correlation 123

6.6 Correlation Matrix 126

Chapter Summary 129

CASE: STATISTICS IN ACTION Financial Time Series 140

CASE: STATISTICS IN ACTION Executive Compensation 148

PART II Probability

7 Probability 156 7.1 From Data to Probability 157

7.2 Rules for Probability 161

7.3 Independent Events 166

Chapter Summary 170

8 Conditional Probability 179 8.1 From Tables to Probabilities - 180

8.2 Dependent Events 184

8.3 Organizing Probabilities 187

8.4 Order in Conditional Probabilities 190

Chapter Summary 195

9 Random Variables 202 9.1 Random Variables 203

9.2 Properties of Random Variables 205

9.3 Properties of Expected Values 211

9.4 Comparing Random Variables 214

Chapter Summary 216

10 Association between Random Variables 224 10.1 Portfolios and Random Variables 225

10.2 Joint Probability Distribution 227

10.3 Sums of Random Variables 230

10.4 Dependence Between Random Variables 232

10.5 IID Random Variables 236

10.6 Weighted Sums 239

Chapter Summary 243

A01_STIN7167_03_SE_FM.indd 6 12/11/16 10:03 AM

viiCONTENTS

11 Probability Models for Counts 251 11.1 Random Variables for Counts 252

11.2 Binomial Model 254

11.3 Properties of Binomial Random Variables 255

11.4 Poisson Model 259

Chapter Summary 265

12 The Normal Probability Model 270 12.1 Normal Random Variable 271

12.2 The Normal Model 274

12.3 Percentiles 280

12.4 Departures from Normality 282

Chapter Summary 290

CASE: STATISTICS IN ACTION Managing Financial Risk 298

CASE: STATISTICS IN ACTION Modeling Sampling Variation 306

PART III Inference

13 Samples and Surveys 314 13.1 Two Surprising Properties of Samples 315

13.2 Variation 320

13.3 Alternative Sampling Methods 323

13.4 Questions to Ask 326

Chapter Summary 329

14 Sampling Variation and Quality 334 14.1 Sampling Distribution of the Mean 335

14.2 Control Limits 340

14.3 Using a Control Chart 344

14.4 Control Charts for Variation 347

Chapter Summary 354

15 Confidence Intervals 362 15.1 Ranges for Parameters 363

15.2 Confidence Interval for the Mean 368

15.3 Interpreting Confidence Intervals 372

15.4 Manipulating Confidence Intervals 373

15.5 Margin of Error 376

Chapter Summary 384

16 Statistical Tests 391 16.1 Concepts of Statistical Tests 392

16.2 Testing the Proportion 397

16.3 Testing the Mean 404

A01_STIN7167_03_SE_FM.indd 7 12/11/16 10:03 AM

viii

16.4 Significance versus Importance 408

16.5 Confidence Interval or Test? 409

Chapter Summary 413

17 Comparison 420 17.1 Types of Comparisons 421

17.2 Data for Comparisons 421

17.3 Two-Sample z-Test for Proportions 424

17.4 Two-Sample Confidence Interval for Proportions 425

17.5 two-Sample t-Test 429

17.6 Confidence Interval for the Difference Between Means 433

17.7 Paired Comparisons 436

Chapter Summary 446

18 Inference for Counts 453 18.1 Chi-Squared Tests 454

18.2 Test of Independence 454

18.3 General versus Specific Hypotheses 466

18.4 Tests of Goodness of Fit 467

Chapter Summary 477

CASE: STATISTICS IN ACTION Rare Events 484

CASE: STATISTICS IN ACTION Data Mining Using Chi-Squared 491

PART IV Regression Models

19 Linear Patterns 498 19.1 Fitting a Line to Data 499

19.2 Interpreting the Fitted Line 501

19.3 Properties of Residuals 506

19.4 Explaining Variation 508

19.5 Conditions for Simple Regression 510

Chapter Summary 520

20 Curved Patterns 528 20.1 Detecting Nonlinear Patterns 529

20.2 Transformations 531

20.3 Reciprocal Transformation 532

20.4 Logarithm Transformation 538

Chapter Summary 550

21 The Simple Regression Model 557 21.1 The Simple Regression Model 558

21.2 Conditions for the SRM 562

21.3 Inference in Regression 565

21.4 Prediction Intervals 573

Chapter Summary 587

CONTENTS

A01_STIN7167_03_SE_FM.indd 8 12/11/16 10:03 AM

ix

22 Regression Diagnostics 596 22.1 Changing Variation 597

22.2 Outliers 607

22.3 Dependent Errors and Time Series 611

Chapter Summary 622

23 Multiple Regression 630 23.1 The Multiple Regression Model 631

23.2 Interpreting Multiple Regression 632

23.3 Checking Conditions 640

23.4 Inference In Multiple Regression 642

23.5 Steps In Fitting A Multiple Regression 646

Chapter Summary 656

24 Building Regression Models 667 24.1 Identifying Explanatory Variables 668

24.2 Collinearity 673

24.3 Removing Explanatory Variables 678

Chapter Summary 694

25 Categorical Explanatory Variables 703 25.1 Two-Sample Comparisons 704

25.2 Analysis of Covariance 706

25.3 Checking Conditions 711

25.4 Interactions and Inference 712

25.5 Regression with Several Groups 719

Chapter Summary 726

26 Analysis of Variance 736 26.1 Comparing Several Groups 737

26.2 Inference in ANOVA Regression Models 744

26.3 Multiple Comparisons 748

26.4 Groups of Different Size 754

Chapter Summary 759

27 Time Series 768 27.1 Decomposing a Time Series 769

27.2 Regression Models 772

27.3 Checking the Model 782

Chapter Summary 797

CASE: STATISTICS IN ACTION Analyzing Experiments 807

CASE: STATISTICS IN ACTION Automated Modeling 815

CONTENTS

A01_STIN7167_03_SE_FM.indd 9 12/11/16 10:03 AM

x

Appendix: Tables 823

Answers A-1

Credits C-1

Index I-1

Supplementary Material (online-only) S1 Alternative Approaches to Inference S1-1

S2 Two-Way Analysis of Variance S2-1

S3 Regression with Big Data S3-1

CONTENTS

A01_STIN7167_03_SE_FM.indd 10 12/11/16 10:03 AM

xi

PREFACE

Knowledge of statistics is a great asset in business, but getting the most value from this asset requires knowing how to ask and answer the right questions. Choosing the right question and solving the problem correctly require an appreciation of business as well as the subtleties of statistics. Unless you understand the business issue from a finance, marketing, management, or accounting per- spective, you won’t see how statistics can help solve the problem. Performing the statistical analysis must wait until you have grasped the issue facing the business.

Solving Business Problems

This application-directed approach is key to business analytics and shapes our examples. We open each chap- ter with a business question that motivates the contents of the chapter. For extra practice, worked-out examples within each chapter follow our 4M (Motivation, Method, Mechanics, Message) problem-solving strategy. The mo- tivation sets up the problem and explains the relevance of the question at hand. We then identify the appropri- ate statistical method and work through the mechanics of its calculation. Finally, the message answers the ques- tion in language suitable for a business presentation or report. Through the 4Ms, we’ll show you how a business context guides the statistical procedure and how the re- sults determine a course of action. Motivation and Mes- sage are critical. The Motivation answers the question “Why am I doing this analysis?”. If you cannot answer that question, it’s hard to get the statistics correct. The Message has to express your answer in language that is used in the business world. Understand the business first, then use statistics to help formulate your conclu- sion. Notice that we said “help.” A statistical analysis by itself is not the final answer. You must frame that analy- sis in terms that others in the business will understand and find persuasive.

Our emphasis on the substantive use of statistics in business shapes our view that the ideal reader for this text is someone with an interest in learning how statis- tical thinking improves the ability of a manager to run or contribute to a business. Whether you’re an under- graduate with an interest in business, an MBA looking to improve your skills, or a business owner looking for

another way to get ahead of the competition, the key is a desire to learn how statistics can produce better de- cisions and insights from the growing amount of data generated in modern businesses.

We don’t assume that readers have mastered the do- mains of a business education, such as economics, fi- nance, marketing, or accounting. We do assume, though, that you care how ideas from these areas can improve a business. If you’re interested in these applications—and we think you will be—then our examples provide the background you will need to appreciate why we want to solve the challenges that we present in each chapter. Readers with more experience will discover that we’ve simplified the technical details of some applications, such as those in finance or marketing. Even so, we think that the examples offer those with substantive experience a new perspective on familiar problems. We hope that you will agree that the examples are realistic and get to the heart of quantitative applications of statistics in business.

Technology

You cannot do research in modern applied statistics without computing. Data sets have grown in size and complexity, making it impossible to work out the cal- culations by hand. Rather than dwell on routine cal- culations, we rely on software (often referred to as a statistics package) to compute the results. Although we emphasize the use of technology, we give the formulas and illustrate the calculations introduced in each chap- ter so that you will always know what the software is doing. It is essential to appreciate what happens in the calculations: You need to understand how the calcu- lations are done in order to recognize when they are appropriate and when they fail. That does not mean, however, that you need to spend hours doing routine calculations. Your time is precious, and there’s only so much of it to go around. We think it makes good eco- nomic sense to take advantage of modern technology in order to give us more time to think harder and more thoroughly about the motivating context for an applica- tion and to successfully present the business message.

When we present results obtained with a calcula- tor or computer, we typically round them. You don’t

A01_STIN7167_03_SE_FM.indd 11 12/11/16 10:03 AM

xii

need to know that the profits from a projected sale are $123,234.32529. It’s usually better to round such a number to $123 thousand. To let you know when we’ve rounded a calculation, we say about or approximately. In expressions, we denote rounding with the symbol < , as in 1/6 < 0.167.

To help you learn how to use software, each chapter includes hints on using Excel®, MinitabExpress®, and JMP® for calculations. These hints won’t replace the help provided by your software, but they will point you in the right direction so that you don’t spin your wheels figur- ing out how to get started with an analysis. Supplemental software study cards are available for specific packages.

Data

Statistical analysis uses data, and we’ve provided lots of data to give you the opportunity to have some real hands-on experience. As you read through the chap- ters, you’ll discover a variety of data sets that include real estate markets, stocks and bonds, technology, retail sales, human resource management, and fundamental economics. These data come from a range of sources, and each chapter includes a discussion about where we found the data used in examples. We hope you’ll use our suggestions and find more.

Prerequisite Knowledge

To appreciate the illustrative calculations and formulas, readers will need to be familiar with basic algebra. Por- tions of chapters that introduce a statistical method of- ten include some algebra to show where a formula comes from. Usually, we only use basic algebra (up through top- ics such as exponents and square roots). Several chapters make extensive use of the logarithm function. If you’re interested in business and economics, this is a function worth getting to know a lot better. The applications we’ve provided, such as modeling sales or finding the best price, show why the logarithm is so important. Occasion- ally, we give credit to calculus for solving a problem, but we don’t present derivations using calculus. You’ll do fine if you are willing to accept that calculus is a branch of more advanced mathematics that provides, among other things, the ability to derive formulas that have special properties. If you do know calculus, you’ll be able to see where these expressions come from.

WHAT’S NEW IN THIS EDITION This edition adds more of what readers have found re- ally useful:

■■ Business analytics relies on linking data to business decisions. Businesses ranging from traditional banks to the latest game developers are clamoring for em- ployees who can connect data and models to substan- tive business problems. This edition adds emphasis,

examples, and illustrations that stress the impor- tance of these connections. For example, previous editions introduced the 4M paradigm—motivation, method, mechanics, and message—that shows how to combine data and statistics to solve problems in business. This edition carries this metaphor further. By explicitly linking this paradigm to analytics, this edition shows that business analytics requires blend- ing substantive relevance with statistical analysis.

■■ Up-to-date applications explore problems related to “big data” and introduce hot topics such as A/B test- ing that are popular in today’s businesses. Although the methods behind these new topics are familiar within statistics, the names are new. This edition makes sure students know the new names so that they can link what they learn in the classroom to what they read online.

■■ This edition features more than 90 new and updated data sets. The changed data range from examples used within chapters to those underlying exercises. Important, highly visible changes include “through the cycle” finance and economic time series that span the 2008 recession.

■■ More than 100 enhanced exercises remove ambi- guities and capture nuances in revised data. Many of these changes address issues identified by tracking online student performance in completing related exercises in MyStatLab. Problems that were worded in a way that might confuse students were clarified.

■■ Excel is the workhorse tool of many businesses. This edition adds a section to every chapter that shows step by step how to complete analytic exercises with the latest version of Excel. Excel is the most popular software for introductory statistics, but some prefer the features offered by statistics packages such as Minitab or JMP. We’ve retained and updated hints in each chapter for these as well.

■■ It’s the little things. Hundreds of changes have been made throughout this edition to emphasize and clar- ify key points. For example, this edition highlights additional tips throughout the text that help readers recognize important points that might be overlooked. Clarified explanations, analogies, and examples in every chapter encourage students to delve deeper and learn for themselves.

COVERAGE AND ORGANIZATION We have organized the chapters of this book into four parts:

1. Variation 2. Probability 3. Inference 4. Regression Models

Part I. These chapters introduce summary statistics such as the mean and important graphical summaries, including bar charts, histograms, and scatterplots. Even

PREFACE

A01_STIN7167_03_SE_FM.indd 12 12/11/16 10:03 AM

xiii

if you are familiar with these methods, we encourage you to skim the examples in these chapters. These ex- amples introduce important terminology that appears in subsequent chapters. A quick review will introduce the notation that we use (which is rather standard) as well as give you a chance to look at some interesting data. If you do skip past these, take advantage of the index of Key Terms in each chapter to find definitions and examples.

Part II. Many courses in mathematics now include topics from probability. Even if you have seen basic probability, you might benefit from reviewing how methods, such as Bayes’ Rule, can be used to improve business processes (Chapter 8). If you plan to skip or move briskly through the rest of the chapters in Part 2, be sure that you’re famil- iar with the concept of a random variable (Chapter 9). Sta- tistical models use random variables to present an ideal- ized description of the data in applications. Unless you’re familiar with random variables, you won’t appreciate the important assumptions that come with their use in prac- tice. Chapter 11 describes special random variables used to model counts, and Chapter 12 defines normal random variables that appear so often in statistical models.

Part III. This part presents the foundations for statistical inference, the process of inferring properties of an entire population from those of a subset known as a sample. Even if you are not interested in quality control, we en- courage you to read Chapter 14. Chapter 14 uses quality control to introduce a fundamental concept of inferential statistics, the sampling distribution and standard error. You can get by in statistics with a basic understanding of the concept of a sampling distribution, but the more you know about sampling distributions, the better. Each in- ferential procedure comes with a checklist of conditions that tell you whether your data and situation match up to the various inferential techniques in these chapters.

Part IV. The chapters in Part 4 describe regression mod- eling. Regression modeling allows us to associate how differences in data that describe one phenomenon are related to differences in others. Regression models are among the most powerful ways to use statistics in busi- ness, providing methods for assessing profitability, setting prices, identifying anomalies, and generating forecasts. We encourage you to slow down and take your time study- ing these chapters. Even if you don’t see yourself doing statistics in your career in business, you can be sure that you will be presented with the results of regression models. Because the examples in these chapters allow us to describe the interconnectedness of several busi- ness processes at once, they become even more interest- ing than those in prior chapters. Be careful if you skip Chapter 20. The material in this chapter shows how to model a richer set of patterns and is less common in business textbooks, but we think these ideas are an es- sential component of every manager’s tool set.

Case Studies

Each of the four main parts of this book includes two supplemental case studies called Statistics in Action. Each case study provides an in-depth look at a business application of statistics. Every case uses real data and takes students through the details of using those data to address a business question. For example, a case study for Part 1 explains details of stock market data, such as how stock returns account for dividends, and elabo- rates the nuances of financial data beyond the coverage in the surrounding chapters.

We’ve found that it is easy to have a “chapter-centric” view of any subject; you know how to approach a prob- lem if the question identifies a chapter. Executing the right approach is more difficult without that sort of clue.

Case studies allow us to extend the statistical con- cepts introduced in the accompanying chapters in the context of a longer, more complex case. For example, the second case in Part 1 carefully explains how to in- terpret and use logarithms in the context of executive salaries. A case in Part 3 explores the use of many chi- squared tests in an operations management problem that resembles data mining. While logs, chi-squared tests, and issues of multiple testing all appear in the regular flow of the main chapters, case studies provide a means for us to cover these topics in more detail than we thought was appropriate for everyone.

Supplementary Chapters

For this edition, we’ve added a few supplementary chap- ters that are available online. These cover topics that are less common in the typical business stats course, but of- ten useful. One chapter covers methods that are needed when the usual approaches don’t apply. For example, suppose data are so skewed that one cannot use stan- dard methods for building a confidence interval for the mean. What are you to do? The supplemental chapter Alternative Approaches to Inference gives an answer. Two other supplemental chapters go deeper into regres- sion modeling. The chapter Two-Way Analysis of Vari- ance goes beyond Chapter 26 and looks at two-way (and higher) analysis of variance, including those with ran- domized blocking and interactions. The chapter Regres- sion Modeling with Big Data goes beyond Chapter 24 and the Statistics in Action cases with coverage of how to build regression models when confronted by “big-data” issues that have become more common in business.

FEATURES Motivating Examples. Each chapter opens with a business example that frames a question and motivates the contents of the chapter. We return to the example throughout the chapter, as we present the statistical

PREFACE

A01_STIN7167_03_SE_FM.indd 13 12/11/16 10:03 AM

xiv

methods that provide answers to the question posed in the opening example.

The 4M (Motivation, Method, Mechanics,

Message) problem-solving strategy gives students a clear outline for solving any business problem. Each 4M ex- ample first expresses a business question in context, then guides students to determine the best statistical method for working the problem using statistical soft- ware, and, finally, frames the analysis in terms that oth- ers in the business world will understand.

Short question sets throughout each chapter

give students the opportunity to check their under- standing of what they’ve just read. These questions are intended to be a quick check of key concepts and ideas presented in the chapter; most questions involve very little calculation. Answers are located in a footnote so that students can easily check their answers before moving on in the text.

Tips. We highlight useful hints for applying sta- tistical methods within the exposition so that

students don’t miss them.

Caution. You’ll see the caution icon next to material that might be confusing. You

should be extra careful to make sure you understand the material being discussed.

✓ Checklist. Some statistics presume that the informa- tion presented satisfies several conditions or assump- tions. For example, certain statistics only detect pat- terns that resemble lines. You would not want to use these if you were looking for a curve. To help you keep track of the assumptions, the conditions are collected in a checklist.

Best Practices. At the end of each chapter, we include a collection of tips for applying the chapter’s concepts successfully and ethically.

Pitfalls. Most of the unintentional mistakes people make when learning statistics are avoidable and usually come from using the wrong method for the situation or misinter- preting the results. This feature at the end of each chapter provides useful tips for avoiding common mistakes.

Data Analytics: The authors analyzed aggregated student usage and performance data from MyStatLab™ for the previous edition of this text. The results of this analysis helped improve the quality and quantity of exercises that matter the most to instructors and students.

Software Hints. Each chapter includes hints on using Excel, Minitab, and JMP for calculations. These hints give students a jumping off point for getting started doing statistical analysis with software. Supplemental study cards for these and other software packages are available from the publisher.

Behind the Math. At the end of most chapters, a Be- hind the Math section provides interesting technical details that explain important results, such as the jus- tification or interpretation for an underlying formula. If you are so inclined, they will help you appreciate the subtleties and logic behind the mechanics, but they are not necessary for using statistics.

Chapter Summary. These chapter-ending summaries provide a complete review of the content.

■■ Key Terms We provide an index of the chapter’s key terms at the end of each chapter to give students a quick and easy way to return to important definitions in the text.

■■ Objectives This new feature provides a list of what students should understand after having read a chapter.

■■ Formulas Important formulas introduced within the chapter are restated.

■■ About the Data This feature provides sources for the data used throughout the chapter with further back- ground.

Exercises. Each chapter contains a variety of exercises at escalating levels of difficulty in order to give students a full complement of practice in problem solving using the skills they’ve learned in the chapter. Types of ex- ercises include Matching, True/False, Think About It, You Do It, and 4M Exercises. You’ll find the data for the 4M and You Do It exercises on Pearson’s Math and Sta- tistics Resources Website: http://www.pearsonhighered .com/mathstats

■■ Matching and True/False exercises test students’ abil- ity to recognize the basic mathematical symbols and terminology they have learned in the chapter. We avoid unnecessary formulas, but certain symbols and terminology show up so often that students are well served to recognize them.

■■ Think About It exercises ask students to pull together the chapter’s concepts in order to solve conceptual problems. You don’t need a computer or calculator for most of these.

■■ You Do It exercises give students practice solv- ing problems that reinforce the mechanics they’ve learned in the chapter. These exercises apply the methods of the chapter to data related to a business

PREFACE

4M Analytics Examples

What Do You Think?

tip

caution

A01_STIN7167_03_SE_FM.indd 14 12/11/16 10:03 AM

xvPREFACE

ACKNOWLEDGMENTS

We didn’t develop our approach to business statistics in isolation. Our colleagues at Wharton have helped shape our approach to teaching statistics in business. Many of the ideas and examples that you’ll find here arose from suggestions made by colleagues, including Andreas Buja, Sasha Rakhlin, Paul Shaman, Richard Waterman, and Adi Wyner. Over the years, members of our department have come to share a common attitude toward the use of statistics in business, and this text re- flects that shared perspective. Most of the examples and many exercises from the text have been tried in other classes and improved using that feedback. We owe these friends a debt of gratitude for their willingness to talk about the fundamental use of statistics in business and to explore alternative explanations and examples.

Many thanks to the following reviewers for their comments and suggestions during the revision of this text.

Kunle Adamson, DeVry University

Elaine Allen, Babson College

Randy Anderson, California State University—Fresno

Djeto Assane, University of Nevada, Las Vegas

Rajesh K. Barnwal, Middle Tennessee State University

Dipankar Basu, Miami University

Mark Bloxom, Alfred State College SUNY College of Technology

Hannah Bolte, Indiana University Bloomington

David Booth, Kent State University, Main Campus

John E. Boyer, Jr., Kansas State University

Michael Braun, Southern Methodist University

Daniel G. Brick, University of St. Thomas

Nancy Burnett, University of Wisconsin—Oshkosh

Richard Cleary, Bentley College

Ismael Dambolena, Babson College

Anne Davey, Northeastern State University

Dr. Michael Deis, Clayton University

Frederick W. Derrick, Loyola University Maryland

Neil Desnoyers, Drexel University

Joan Donohue, University of South Carolina

Steve Erikson, Babson College

Nancy Freeman, Shelton State Community College

Daniel Friesen, Midwestern State University

Deborah J. Gougeon, University of Scranton

Christian Grandzol, Bloomsburg University

Betsy Greenberg, University of Texas—Austin

Ken Griffin, University of Central Arkansas

John Grout, Berry College

Warren Gulko, University Of North Carolina, Wilmington

Marie Halvorsen-Ganepola, Notre Dame University

Clifford B. Hawley, West Virginia University

Bob Hopfe, California State University—Sacramento

Max Houck, West Virginia University

David Hudgins, University of Oklahoma

Jeffrey Jarrett, University of Rhode Island

Chun Jin, Central Connecticut State University

Christopher K. Johnson, Ph.D. University of North Florida

Morgan Jones, University of North Carolina

Ronald K. Klimberg, Saint Joseph’s University

David Kopcso, Babson College

Supriya Lahiri, University of Massachusetts, Lowell

Mark T. Leung, University of Texas—San Antonio

Tony Lin, Ph.D., University of Southern California

John McKenzie, Babson College

Kay McKinzie, University of Central Arkansas

Mark R. Marino, Niagara University

Dennis Mathaisel, Babson College

Sherryl May, University of Pittsburgh

Bruce McCullough, Drexel University

Richard McGowan, Boston College

Constance McLaren, Indiana State University

Robert Meeks, Pima Community College

Jeffrey Michael, Towson University

application. Working through the steps of these ex- ercises helps you practice the mechanics. We expect you to use a statistics software package for many of these.

■■ 4M Analytics exercises are rich, challenging applica- tions rooted in real business situations. These ask students to apply the statistical knowledge they’ve developed in the chapter to a set of questions about a particular business problem.

A01_STIN7167_03_SE_FM.indd 15 12/11/16 10:03 AM

PREFACExvi

Prakash Mirchandani, University of Pittsburgh

Jason Molitierno, Sacred Heart University

Carolyn H. Monroe, Baylor University

Gourab Mukherjee, University of Southern California

Patricia Ann Mullins, University of Wisconsin—Madison

Quinton J. Nottingham, Virginia Polytechnic & State University

Keith Ord, Georgetown University

Michael Parzen, Emory University

M. Patterson, Midwestern State University

Robert Pred, Ph.D., Temple University

Leonard Presby, William Paterson University

Darrell Radson, Drexel University

Farhad Raiszadeh, University of Tennessee—Chattanooga

Ranga Ramasesh, Texas Christian University

Deborah Rumsey, The Ohio State University

John Saber, Babson College

Dr. Subarna Samanta, The College of New Jersey

Subarna Samanta, The College of New Jersey

Hedayeh Samavati, Indiana University—Purdue Fort Wayne

Rose Sebastianelli, University of Scranton

Omeed Selbe, University of Southern California

Gary Smith, Florida State University

Erl Sorensen, Bentley College

Bert Steece, University of Southern California

J. H. Sullivan, Mississippi State University

Dr. Kathryn A. Szabat, LaSalle University

Rajesh Tahiliani, University of Texas—El Paso

Patrick A. Thompson, University of Florida

Denise Sakai Troxell, Babson College

Bulent Uyar, University of Northern Iowa

John Wang, Montclair State University

Dr. William D. Warde, Oklahoma State University, Main Campus

Elizabeth Wark, Worcester State University

James Weber, University of Illinois—Chicago

Fred Wiseman, Northeastern University

Roman Wong, Barry University

Zhiwei Zhu, University of Louisiana at Lafayette

We would also like to thank our accuracy checkers Caroline Swift and Dirk Tempelaar. Thanks also to Lifland et al., and our Pearson Education team for their help and support, especially: Deirdre Lynch, Erin Kelly, Peggy McMahon, Justin Billing, Jennifer Myers, and Aimee Thorne.

A01_STIN7167_03_SE_FM.indd 16 12/11/16 10:03 AM

Student Resources Student’s Solutions Manual Student’s Solutions Manual, by Zhiwei Zhu, Univer- sity of Louisiana, Lafayette

This manual provides detailed, worked-out solu- tions to all odd-numbered text exercises. (ISBN-10: 0-13-449736-8; ISBN-13: 978-0-13-449736-5)

Study Cards for Business Statistics Software This series of study cards provides students with easy step-by-step instructions for using statistics software. Available for native Excel® 2016 (0-13- 457679-9) Excel 2016 with XLStat™ (0-13-457683-7), Minitab 17 (0-13-457681-0), Minitab Express for PC (0-13-457685-3), Minitab Express for Mac (0-13- 457691-8), JMP (0-321-64423-9), and StatCrunch™ (0-13-397513-4)

Instructor Resources Instructor’s Edition This version of the text contains short answers to all of the exercises within the exercise sets. (ISBN-10: 0-13-449738-4; ISBN-13: 978-0-13-449738-9)

Instructor’s Solutions Manual, Instructor’s Solutions Manual, by Zhiwei Zhu, Uni- versity of Louisiana, Lafayette

This manual provides detailed, worked-out solu- tions to all of the book’s exercises. The Instruc- tor’s Solutions Manual is available for download in MyStatLab and from Pearson Education’s online catalog (http://www.pearsonhighered.com/irc).

Business Insight Video Assessment Questions Written to accompany the Business Insight Videos, these video-specific questions and answers can be used for assessment or classroom discussion. These are available for download from MyStatLab or at www.pearsonhighered.com/irc.

Online Test Bank Online Test Bank, by Paul Lorczak

This test bank contains ready-to-use quizzes and tests that correlate to chapters in the text. The test- bank is available for download from Pearson Educa- tion’s online catalog (http://www.pearsonhighered .com/irc).

TestGen® TestGen® (www.pearsoned.com/testgen) enables instructors to build, edit, print, and administer tests using a computerized bank of questions developed to cover all the objectives of the text. TestGen is al- gorithmically based, allowing instructors to create multiple but equivalent versions of the same ques- tion or test with the click of a button. Instructors can also modify test bank questions or add new ques- tions. The software and testbank are available for download from Pearson Education’s online catalog (http://www.pearsonhighered.com/irc).

Technology Resources Data Sets Data sets formatted for Minitab, Excel, JMP, and text files can be downloaded from MyStatLab or www .pearsonhighered.com/mathstatsresources/.

Business Insight Videos This series of ten 5- to 7- minute videos, each about a well-known business and the challenges it faces, focuses on statistical concepts as they pertain to the r eal world. The videos can be downloaded from within MyStatLab. Contact your Pearson represen- tative for details.

MyStatLab™ Online Course (access code required) MyStatLab from Pearson is the world’s leading on- line resource for teaching and learning statistics; integrating interactive homework, assessment, and media in a flexible, easy-to-use format. MyStatLab is

Resources for Success

www.mystatlab.com

A01_STIN7167_03_SE_FM.indd 17 12/11/16 10:03 AM

a course management system that helps individual students succeed.

■■ MyStatLab can be implemented successfully in any environment—lab-based, traditional, fully online, or hybrid—and demonstrates the quan- tifiable difference that integrated usage has on student retention, subsequent success, and overall achievement.

■■ MyStatLab’s comprehensive gradebook automat- ically tracks students’ results on tests, quizzes, homework, and in the study plan. Instructors can use the gradebook to provide positive feedback or intervene if students have trouble. Gradebook data can be easily exported to a variety of spread- sheet programs, such as Microsoft Excel.

MyStatLab provides engaging experiences that per- sonalize, stimulate, and measure learning for each student. In addition to the resources below, each course includes a full interactive online version of the accompanying textbook.

■■ Personalized Learning: MyStatLab’s personal- ized homework, and adaptive and companion study plan features allow your students to work more efficiently spending time where they really need to.

■■ Tutorial Exercises with Multimedia Learning Aids: The homework and practice exercises in MyStat- Lab align with the exercises in the textbook, and most regenerate algorithmically to give students unlimited opportunity for practice and mastery. Exercises offer immediate helpful feedback, guided solutions, sample problems, animations, videos, statistical software tutorial videos and eText clips for extra help at point-of-use.

■■ Learning Catalytics™: MyStatLab now provides Learning Catalytics—an interactive student re- sponse tool that uses students’ smartphones, tablets, or laptops to engage them in more so- phisticated tasks and thinking.

■■ Videos tie statistics to the real world. ■■ StatTalk Videos: Fun-loving statistician An-

drew Vickers takes to the streets of Brooklyn,

NY, to demonstrate important statistical con- cepts through interesting stories and real-life events. This series of 24 fun and engaging vid- eos will help students actually understand sta- tistical concepts. Available with an instructor’s user guide and assessment questions.

■■ Business Insight Videos [for Business Statis- tics only]: 10 engaging videos show managers at top companies using statistics in their ev- eryday work. Assignable question encourage discussion.

■■ Additional Question Libraries: In addition to al- gorithmically regenerated questions that are aligned with your textbook, MyStatLab courses come with two additional question libraries:

■■ 450 exercises in Getting Ready for Statistics cover the developmental math topics stu- dents need for the course. These can be as- signed as a prerequisite to other assignments, if desired.

■■ 1000 exercises in the Conceptual Question Li- brary require students to apply their statistical understanding.

■■ StatCrunch™: MyStatLab integrates the web-based statistical software, StatCrunch, within the online assessment platform so that students can eas- ily analyze data sets from exercises and the text. In addition, MyStatLab includes access to www .statcrunch.com, a vibrant online community where users can access tens of thousands of shared data sets, create and conduct online surveys, perform complex analyses using the powerful statistical software, and generate compelling reports.

■■ Statistical Software, Support and Integration: We make it easy to copy our data sets, from both the eText and the MyStatLab questions, into soft- ware such as StatCrunch, Minitab®, Excel®, and more. Students have access to a variety of sup- port tools—Technology Tutorial Videos, Technol- ogy Study Cards, and Technology Manuals for select titles—to learn how to effectively use sta- tistical software.

www.mystatlab.com

A01_STIN7167_03_SE_FM.indd 18 12/11/16 10:03 AM

MathXL® for Statistics Online Course (access code required) MathXL® is the homework and assessment engine that runs MyStatLab. (MyStatLab is MathXL plus a learning management system.)

With MathXL for Statistics, instructors can:

■■ Create, edit, and assign online homework and tests using algorithmically generated exercises correlated at the objective level to the textbook.

■■ Create and assign their own online exercises and import TestGen tests for added flexibility.

■■ Maintain records of all student work, tracked in MathXL’s online gradebook.

With MathXL for Statistics, students can:

■■ Take chapter tests in MathXL and receive person- alized study plans and/or personalized home- work assignments based on their test results.

■■ Use the study plan and/or the homework to link directly to tutorial exercises for the objectives they need to study.

■■ Students can also access supplemental anima- tions directly from selected exercises.

■■ Knowing that students often use external statisti- cal software, we make it easy to copy our data sets, both from the eText and the MyStatLab questions, into software like StatCrunch™, Minitab, Excel and more.

MathXL for Statistics is available to qualified adopters. For more information, visit our Web site at www.mathxl.com, or contact your Pearson representative.

MyStatLab Accessibility ■■ MyStatLab is compatible with the JAWS screen

reader, and enables multiple-choice, fill-in-the- blank and free-response problem-types to be read, and interacted with via keyboard controls and math notation input. MyStatLab also works with screen enlargers, including ZoomText, MAGic®, and SuperNova. And all MyStatLab vid-

eos accompanying texts with copyright 2009 and later have closed captioning.

■■ More information on this functionality is avail- able at http://mystatlab.com/accessibility.

And, MyStatLab comes from an experienced partner with educational expertise and an eye on the future.

■■ Knowing that you are using a Pearson product means knowing that you are using quality content. That means that our eTexts are accurate and our assessment tools work. It means we are commit- ted to making MyStatLab as accessible as possible.

■■ Whether you are just getting started with MyStatLab, or have a question along the way, we’re here to help you learn about our technologies and how to incorporate them into your course.

To learn more about how MyStatLab combines prov- en learning applications with powerful assessment, visit www.mystatlab.com or contact your Pearson representative.

PowerPoint Lecture Slides PowerPoint Lecture Slides provide an outline for use in a lecture setting, presenting definitions, key concepts, and figures from the text. These slides are available within MyStatLab or at www.pearson- highered.com/irc.

StatCrunch StatCrunch is powerful Web-based statistical soft- ware that allows users to perform complex analyses, share data sets, and generate compelling reports of their data. The vibrant online community offers tens of thousands of data sets for students to analyze.

■■ Collect. Users can upload their own data to Stat- Crunch or search a large library of publicly shared data sets, spanning almost any topic of interest. Also, an online survey tool allows users to quickly collect data via Web-based surveys.

■■ Crunch. A full range of numerical and graphical methods allow users to analyze and gain insights from any data set. Interactive graphics help users understand statistical concepts, and are available

www.mystatlab.com

A01_STIN7167_03_SE_FM.indd 19 12/11/16 10:03 AM

for export to enrich reports with visual represen- tations of data.

■■ Communicate. Reporting options help users cre- ate a wide variety of visually appealing represen- tations of their data.

Full access to StatCrunch is available with a MyStatLab and StatCrunch is available by itself to qualified adopters. For more information, visit our Web site at www.StatCrunch.com, or contact your Pearson representative.

Minitab® 17 and Minitab Express™ Minitab 17 and Minitab Express make learning statistics easy and provide students with a skill-set that’s in demand in today’s data-driven workforce. Bundling Minitab ® software with educational materials ensures students have access to the soft- ware they need in the classroom, around campus, and at home. And having 12-month versions of Minitab 17

and Minitab Express available ensures students can use the software for the duration of their course. (ISBN-10: 0-13-445640-8; ISBN-13: 978-0-13-445640-9)

JMP Student Edition JMP Student Edition is an easy-to-use, streamlined version of JMP desktop statistical discovery soft- ware from SAS Institute, Inc., and is available for bundling with the text. Check with your Pearson sales representative for order information. (ISBN- 10: 0-13-467979-2; ISBN-13: 978-0-13-467979-2)

XLSTAT™ for Pearson Used by leading businesses and universities, XL- STAT is an Excel® add-in that offers a wide variety of functions to enhance the analytical capabilities of Microsoft Excel, making it the ideal tool for your ev- eryday data analysis and statistics requirements. XL- STAT is compatible with all Excel versions. Available for bundling with the text. (ISBN-10: 0-321-75940-0; ISBN-13: 978-0-75940-5)

www.mystatlab.com

A01_STIN7167_03_SE_FM.indd 20 12/11/16 10:03 AM

xxi

INDEX OF APPLICATIONS

Accounting

Accounting Firm Filing Tax Forms (TAI) 295 Accounting Procedures (TAI) 332 Auditing a Business (4M) 178 Auditor Checking Transactions (WT) 368 Auditor Comparing Billable Invoices (TAI) 728 Budget Allocation of a New Business (4M) 701–702 Cost Accounting (4M) 139 Research and Development Expenses (YDI) 525, 593, 626, 663,

699, 732

Advertising

Advertising Among Internet Hosts (CO) 83; (IE) 80–83, 180–185; (WT) 183

Advertising and Sales (IE) 559–561; (TAI) 591 Advertising Firm Renewing Contract (TAI) 175 App advertising 187–190 Direct Mail Advertising (4M) 418 Display Space (YDI) 553 Evaluating a Promotion (4M) 435–436 Financial Advisor (YDI) 804 Judging the Credibility of Advertisements (4M) 751–754; (AE)

756–758 Monthly Sales and Advertising (P) 513 New Advertising Program (TAI) 449 Pharmaceutical Advertising (CO) 251–251 Priming in Advertising (4M) 723–725 Promotion Response (4M) 389–390 Television Advertising (TAI) 102 Television Commercials (4M) 403; (AE) 411; (TAI) 416

Agriculture

Blood Sample from Cattle (WT) 319 Dairy Farming (YDI) 267 Food-Safety Inspectors Visiting Dairy Farms (TAI) 331 Grain Produced per Acre (CO) 736 Wheat Trials (IE) 737–742, 744–754; (TAI) 763

Automotive

Auto Dealer Attending Car Auctions (TAI) 592 Residual Car Values (4M) 665 Base Price and Horsepower of Cars (YDI) 525–526, 593, 627,

663–664, 699–700, 732–733 Buying Tires at an Auto Service Center (WT) 205; (IE) 212 Car Theft (4M) 86–88; (AE) 98 Cars in 1989 (4M) 555 Customer Options When Ordering a New Car (YDI) 198

Dealer Earnings per Day (TAI) 266 Door Seam of a Vehicle (YDI) 359 Fatal Roll-Over Accidents (4M) 35–36; (AE) 40–41 Favorite Car Color (TAI) 103 Fuel Consumption in Cars (CO) 528; (IE) 529–538 Leasing Cars (4M) 390 Male Drivers Involved in Serious Accidents (YDI) 296 Motor Shafts in Automobile Engines (YDI) 359 Predicting Sales of New Cars (4M) 775–778; (AE) 792–793 Price of New Cars (WT) 632 Pricing of a Car (IE) 7–9 Rated Highway Gasoline Mileage (YDI) 77 Stopping Distances (YDI) 765 Trade in Asian Models (TAI) 44 Trade in Domestic Models (TAI) 44 Used Cars (WT) 14; (TAI) 76; (YDI) 449, 553–554

Banking

Adjustable Rate Mortgage (TAI) 74 ATM (YDI) 221 Bank Collecting Data on Customers (TAI) 23, 24 Banks Compete by Adding Special Services (YDI) 418 Basel II Standards for Banking (TAI) 386–387 Check Fees (TAI) 23 Credit Card Offer (CO) 362; (IE) 363, 366–367, 370–371, 374, 409 Credit Card Profit Earned from a Customer (IE) 375–376 Credit Cards (IE) 372; (AE) 472–473; (4M) 527 Credit Risk (IE) 185 Direct Deposits for Employees (TAI) 331 Federal Regulators Requiring Bank to Maintain Cash Reserves

(IE) 373 Loan Approval (IE) 161; (WT) 166 Loan Balances (TAI) 74 Loan Defaults (YDI) 267–268 Manager Tracking Bank Transactions (YDI) 267 Mortgage Loan Defaults (WT) 161–161 On Time Loan Repayment (IE) 166–167 Profit for a Bank (IE) 374–375 Size of Credit Card Transactions (IE) 321–322 Subprime Mortgages (4M) 646–650; (AE) 651–653

Business (General)

Auto Dealer (TAI) 173 Bookstore (YD I) 804 Catalog Sales Companies (TAI) 387 Clothing Buyer for a Chain of Department Stores (TAI) 386 Company Free Giveaways (IE) 207–208 Company Stocking Shelves in Supermarkets (YDI) 416, 422

CO ∙ Chapter Opener; IE ∙ In-Text Example; WT ∙ What Do You Think?; 4M ∙ Motivation, Method, Mechanics, Message; P ∙ Pitfalls; BP ∙ Best Practices; AE ∙ Analytics in Excel; AD ∙ About the Data; BTM ∙ Behind the Math; TAI ∙ Think About It; YDI ∙ You Do It; SA ∙ Statistics in Action; QT ∙ Questions for Thought

A01_STIN7167_03_SE_FM.indd 21 12/11/16 10:03 AM

xxii INDEX OF APPLICATIONS

Customer Focus (4M) 19 Data-driven Culture (TAI) 45 Display Space (TAI) 765 Employee Absences (YDI) 48 Employee Drug Testing (TAI) 197–198 Fast-Food Restaurant Chains (YDI) 176, 177; (WT) 253 Forecasting Profits (4M) 787–790; (AE) 794–795 Gross Profit (YDI) 804 Growth Industries (4M) 49–50 Large Company Correlation (IE) 126–127 Mail-Order Catalog (TAI) 74 Multinational Retail Company (TAI) 523 Optimal Pricing (4M) 543–545; (AE) 546–548; (BTM) 550 Price and Weights of Diamonds (WT) 114; (TAI) 134, 591;

(CO) 498; (IE) 499–504, 507–509, 529, 576 Price Scanners at Check-Out Registers (TAI) 76 Reams of Paper Used in an Office (YDI) 220 Repairing an Office Machine (YDI) 221 Restaurant Chain Choosing a Location (CO) 630 Revenue Generated by Individual Sales Representatives (TAI)

727–728 Sales by Day of the Week (TAI) 74 Shopping Mall Environment (TAI) 74 Start-Up Company (TAI) 24, 48 Supermarket Scanner Data (4M) 200 Technology Businesses Moving Corporate Headquarters near a

Mall (WT) 639 Value of New Orders for Computers and Electronics (WT) 772 Women-Owned Businesses (YDI) 48

Company Names

Apple (TAI) 45 Amazon (CO) 26; (IE) 27, 32, 38; (P) 40 Bike Addicts (CO) 10; (IE) 12, 14, 17, 22 Dell Revenue and Inventories (TAI) 800–801 Facebook 176 Ford (WT) 326 Intel (IE) 772 L.L. Bean (TAI) 387 Levi Strauss (IE) 18 Lockheed Martin (IE) 35 Netflix (TAI) 44–45 Target (CO) 10; (4M) 125; (YDI) 552–553 Wal-Mart (CO) 10, 703; (4M) 451–452; (IE) 15, 17, 18, 125, 151,

320, 703, 715; (SA) 491; (YDI) 552, 804

Construction

City Building a New Public Parking Garage (YDI) 387 Construction Estimates (4M) 240–241; (AE) 242 Construction Firm Bidding on a Contract (YDI) 219, 248 Contractor Building Homes in a Suburban Development (TAI)

295 Contractor Replaces Windows and Siding in Suburban Homes

(WT) 639, 643 Cost of Building an Elementary School (TAI) 523 Housing Permits and Construction (YDI) 804–805 Kitchen Remodeling (TAI) 246

Consumers

Bargain on Blouses (YDI) 199 Buying a Laptop (YDI) 176 Buying Running Shoes (TAI) 174 Cell Phone Subscribers (4M) 614–615; (AE) 620–621; (YDI) 802 Choices for Paint Colors and Finishes at a Hardware Store (TAI)

103–104

Convenience Store Shopper Choosing Food (TAI) 173; (WT) 231–232

Cost of Diamonds (IE) 271–272, 499, 503–504; (YDI) 730 Customer Preferences of a New Product (TAI) 102 Customer Rating a Power Tool (TAI) 591 Customer Satisfaction with Calls to Customer Service (CO) 156;

(IE) 157–159; (BP) 169; (TAI) 172–173; (YDI) 175, 200 Diamond Ring Prices (YDI) 523–524, 592, 625 Drive Preferences (YDI) 135 Emerald Diamonds (YDI) 730 Estimating Consumption (4M) 504–506; (AE) 514–516 Gasoline Prices (4M) 805–806 Gasoline Sales (YDI) 104 Gold Chain Prices (YDI) 661, 698 Guest Satisfaction (4M) 332 Lease Costs (4M) 511–512; (AE) 517–518 Pant Choices at a Clothing Store (TAI) 173 Purchasing Habits (TAI) 45 Rating Hotel Chains (TAI) 23 Spending at a Convenience Store (TAI) 45; (YDI) 524, 592,

625–626, 661–662, 698, 730

Demographics

Ages of Shoppers (TAI) 75 Heights of Students (TAI) 74 Number of Children of Shoppers in a Toy Store (TAI) 75

Distribution and Operations Management

Assembly Line Production (TAI) 358 Book Shipments to University Bookstores (YDI) 388 Cost of Building Cars at Plants (TAI) 728 Customized Milling Operation (TAI) 523 Delivery Stops for a Freight Company (YDI) 247 Efficiency of Automated Factories (TAI) 728 Forecasting Inventory Levels at Wal-Mart (YDI) 803, 804 Importer of Electronic Goods (YDI) 417 Imports (YDI) 803–804 Maintenance Staff of a Large Office Building (YDI) 220 Managing Inventories (SA) 491–493 Number of Employees and Items Produced (TAI) 134 Operating Margin of a National Motel Chain (TAI) 661 Overnight Shipping Firm (YDI) 763–764 Package Delivery Service and Fuel Costs (TAI) 523 Packages Processed by Federal Express (TAI) 75 Packaging Types (SA) 491–495; (QT) 495–496 Performance of Two Shipping Services (IE) 89–90 Planning Operating Costs (4M) 249–250 Production Costs (YDI) 524–525, 592–593, 626, 662, 698, 731 Production Line Filling Bottles (TAI) 358 Production Time and Number of Units (WT) 505–506 Seasonal Component of Computer Shipments (IE) 770–771 Shipping Companies (YDI) 450 Shipping Computer Systems (TAI) 103 State of a Production Line (IE) 340–343, 345–346 Value of Shipments of Computers and Electronics (CO) 768; (IE)

770–771, 773–775, 778–784; (TAI) 800 Windows Shipped Daily (IE) 212–213 Wine Exports (IE) 33–34

E-Commerce

A/B Testing of Web Site Design (IE) 428–429 Click fraud (YDI) 389 Filtering Junk Mail (4M) 193–194; (CO) 391; (IE) 392–393, 402;

(TAI) 416 Internet Ad Spending (CO) 26

A01_STIN7167_03_SE_FM.indd 22 12/11/16 10:03 AM

xxiiiINDEX OF APPLICATIONS

Internet Browsers (YDI) 49 Internet Hosts (IE) 28–30, 32; (P) 40 Monitoring an E-mail system (4M) 360 Recipe Source (IE) 33 Web Purchases (4M) 50; (TAI) 266, 416 Web Hits (YDI) 360; (AE) 474–476 Web Site Monitoring the Number of Customer Visits (TAI)

762–763 Web Site of a Photo Processor (TAI) 359 Web Site to Take Photography Lessons (TAI) 799 Web Site Visitors Clicking on an Ad (TAI) 174

Economics

Consumer Sentiment and Inflation (TAI) 134 Dollar/Euro Exchange Rate (IE) 229, 374 Economic Time Series (4M) 24 Estimating the Rise of Prices (4M) 324–326 Exchange Rate (YDI) 220, 221; (4M) 229–230 Expectations for the Economy (IE) 485 Forecasting Unemployment (4M) 784–787; (AE) 793–794 Gross Domestic Product (IE) 394–395 Housing and Stocks (YDI) 135 Macro Economics (YDI) 137–138 Organization for Economic Cooperation and Development (TAI)

133; (YDI) 526, 593, 627; (IE) 394 Seasonally Adjusted Civilian Unemployment Rate (YDI) 802 U.S. Gross National Product (TAI) 799

Education

Annual Tuition of Undergraduate Business Schools (TAI) 76–77 College Attended by CEOs (TAI) 46 College Graduate Debt (TAI) 198 Education and Income (CO) 179 High/Scope Perry Preschool Project (YDI) 389 Letter Grades (IE) 39 Multiple Choice Quiz (YDI) 176 Public College Tuition (WT) 65 SAT and Normality (4M) 277–279; (AE) 288 SAT Scores (TAI) 294 Test Scores (IE) 66

Energy

Annual Use of Natural Gas per Household (CO) 109; (WT) 120; (IE) 120–121

Daylight Savings Time and Reducing Energy Consumption (TAI) 449

Electricity Supplied to Residences (TAI) 294 Energy Policy Act (TAI) 76 Heating Degree Days (IE) 110–117, 121–122 Lighting Efficiency (TAI) 44 Natural Gas and Electricity Use (YDI) 248 Steel Mill Monitoring Energy Costs (TAI) 697 U.S. Department of Energy (IE) 18, 130

Environment

Arctic Ice (YDI) 802 Carbon Dioxide Emissions (YDI) 138–139 Hurricane Bond (YDI) 296 Hurricane Katrina (IE) 4, 5, 6–7 January Average Temperatures (CO) 109 Polluting a Local River System (TAI) 415 Temperature (IE) 13 Weather at a Beachside Vacation Resort (TAI) 174 Weather Forecasts (4M) 628–629; (YDI) 664, 700

Finance and Investments

Apple Stock (YDI) 222, 526–527, 594, 628, 665, 701 Bond Ratings (TAI) 103 Calendar Effects on Stocks (4M) 767 Capital Asset Pricing Model (CO) 667; (IE) 668–669; (TAI) 697;

(4M) 594–595 Cash or Credit Card (YDI) 137, 268 Climate Change (4M) 577–580; (AE) 583–586 Comparing Returns on Investments (4M) 406–408 Continuous Compound Interest (BTM) 489 Credit Rating Agency (TAI) 173 Credit Risk (4M) 766–767 Credit Scores (4M) 79, 223 Day Trading (CO) 202; (WT) 214 Defaults on Corporate Bonds (SA) 485 Disney Stock (YDI) 222 Disposable Income and Household Credit Debt (TAI) 696 Enron Stock Prices (SA) 140–145 Exxon Stock (YDI) 802 Familiar Stock (YDI) 78 FICO Score (YDI) 105 Financial Ratios (4M) 78 Fraud Detection (4M) 201; (AE) 473–474 General Motors Stock Return (TAI) 294 Hedge Funds (YDI) 296 High-Frequency Data (4M) 595 Historical Monthly Gross Returns of Stocks and Treasury Bills

(SA) 304–305 Holdings of U.S. Treasury Bonds (TAI) 44 Household Credit Market Debt (TAI) 799–800 Household Incomes (TAI) 74, 75; (4M) 297 IBM Stock (IE) 206, 208–209, 225–227, 230–231, 233–234,

236–238; (AE) 411–412; (AD) 217, 244–245 Investing in Stock (YDI) 219 Investment Risk (SA) 298 IRA (TAI) 23 Loan Status (TAI) 102 McDonalds Stock (IE) 214 Microsoft Stock (IE) 225–227, 230–231, 233–234, 237, 239; (AD)

244–245 Monthly Prices of Shares in JCPenny (YDI) 802–803 Normality of Stock Returns (4M) 296–297 Pfizer Stock (QT) 146–147 Quality Control of Finance Data (4M) 360–361 Real Money (4M) 249 Sony Stock (IE) 668–678; (TAI) 697 Startup Technology Companies (YDI) 268 Stock Exchanges (CO) 224 Stock Market (IE) 66, 271–272, 275, 303; (4M) 138; (TAI) 591 Stock Returns (SA) 142–145 Student Budget (YDI) 247 Student Loan Debt (TAI) 198 Tech Stocks (YDI) 78 Value at Risk (4M) 281–282; (AE) 289; (YDI) 295–296; (SA) 145 Whole Stock Market and S&P 500 (TAI) 696

Food/Drink

Artificial Sweetener (YDI) 47–48, 763 Bread Volume (YDI) 764–765 Chocolate Snacks (YDI) 48–49 Fast Food Restaurant Customers (YDI) 247 Food and Drug Administration Vetoing Name Choices

(YDI) 268 Frozen Food Package Weight (IE) 347–348 Gourmet Steaks (TAI) 295 Low-Calorie Sports Drink (YDI) 417

A01_STIN7167_03_SE_FM.indd 23 12/11/16 10:03 AM

xxiv INDEX OF APPLICATIONS

M&Ms (IE) 58; (AE) 69; (TAI) 74; (YDI) 175; (SA) 306–309; (4M) 55–56

Package Weights of M&Ms (SA) 310–311; (QT) 312 Take-Out Food at a Local Pizzeria (YDI) 388 Taste Test (TAI) 267, 331; (IE) 437 Weights of Cereal Boxes (IE) 280 Wine (YDI) 449, 553, 733

Games

Arcade Game (TAI) 219 Dice Game (SA) 298–304; (QT) 305 Fair Coin (TAI) 415; (WT) 253 Fair Game (TAI) 219 Game Consoles (YDI) 49 Lotteries (TAI) 219 Lucky Ducks Carnival Game (WT) 208 Online Poker (YDI) 247 Slot Machine (QT) 305 Video Game Previewed by Teenagers (TAI) 24

Government

Political Candidate Anxious about the Outcome of an Election (TAI) 386

Political Poll (4M) 379–380 Property Taxes (4M) 378–379; (AE) 381–382 Sales Tax (TAI) 75, 133 Tax Audits (4M) 333 Top Government Research Priority (YDI) 47 Travel Expenses for Staff (WT) 637

Human Resource Management/Personnel

Average Age of MBAs Hired (TAI) 386 Businesses Planning to Hire Additional Employees

(TAI) 386 Calls Handled at a Corporate Call Center (BP) 68 Correlation between Employee Absent from Year to Year (TAI)

133 Dexterity Testing and Hiring People for a Factory Assembly Line

(YDI) 449 Direct Sales Team (YDI) 199 Discrimination in Hiring (4M) 107 Employee Experience 175 Employee Testing (YDI) 135 Employees Interested in Joining a Union (TAI) 697 Employment in Four Industries (YDI) 105 Evaluating the Performance of New Hires (TAI) 658, 664 Headhunters (YDI) 418 Hiring (YDI) 526, 593–594, 627, 664, 700–701, 733 Hiring Engineering Graduates Who Speak a Foreign Language

(TAI) 173 Home-Based Operator (YDI) 176 Outsourcing High-Level White Collar Jobs (YDI) 49 Personality Test (TAI) 416 Predicting Success of Candidates (TAI) 658 Reasons for Missing Work (TAI) 102 Reducing Turnover Rates (4M) 419; (YDI) 450 Sex Discrimination in the Workplace (4M) 451–452 Training Program (YDI) 248; (WT) 729–730; (TAI) 664

Insurance

Auto Insurance Premiums (TAI) 75 Comparing Average Sales of an Insurance Company (YDI) 764 Cost of Covering Auto Accidents (TAI) 246 Insurance Policies (TAI) 197, 198

Insurance Salesman (YDI) 221 Life Insurance Benefits (IE) 122–123 Selling Life and Auto Insurance (YDI) 247 Stock Market Insurance (CO) 270

Labor

Absent Employees (YDI) 199 Assembly Line Workers Missing Work (YDI) 102 Civilian Unemployment Rate (IE) 17 Days Employees Were Out Sick (WT) 56 Predicting Employment (IE) 4–7 Tornado insurance (WT) 253 Worker Productivity (TAI) 523 Workforce by Gender (YDI) 199

Law

Jury Trial (TAI) 415–416 Law Firm (YDI) 219–220 Law Suit against Wal-Mart (CO) 703–704; (4M) 451–452

Management

Analyzing the Performance of a Fast-Food Chain (TAI) 658 Average Amount of a Purchase Order (WT) 371 Business Offering Free Fitness Center Membership to Staff

(TAI) 448 Employee Confidence in Senior Management (YDI) 389 Management of a Chain of Hotels (YDI) 417 Management Presentation (WT) 33 Management Tracking Growth of Sales versus Number of Outlets

(TAI) 625 Manager Predicting Sales (TAI) 799 Managers with an MBA (YDI) 198 Managing a Process (4M) 167–168; (IE) 167–168 Project Management (4M) 222–223 Sales Force Comparison (4M) 438–440; (TAI) 449 Supervising Experimental Projects (TAI) 267 Supervisors Tracking the Output of a Plant (TAI) 625

Manufacturing

Appliance Assembly (YDI) 199 Assembly Line (YDI) 175 Canadian Paper Manufacturer (TAI) 23 Car Manufacturer (TAI) 23–24, 332 Making M&Ms (4M) 55–56 Printer Manufacturing (YDI) 220–221 Tire Manufacturer (YDI) 296

Marketing

Age, Income and Product Rating (TAI) 658–659, 697; (IE) 681–682

Analysis of Car Buyers (YDI) 418 Coupons expiring 765 Coupons Increasing Sales (TAI) 103 Launching a Product (IE) 214 Locating a New Store (4M) 125–126; (AE) 128 Loyalty Programs (YDI) 417 Mailing List (WT) 88, 94; (4M) 418 Market Analyst (TAI) 175 Market Segmentation (4M) 678–682; (AE) 688–691 Market Share for Artificial Sweeteners (YDI) 47–48 Marketing Courier Paks (TAI) 729 Marketing Team Designing a Promotional Web Page (WT) 431 Retailer Offering Scratch-Off Coupons (YDI) 176 Smartphone Sales (AE) 41; (YDI) 388–389

A01_STIN7167_03_SE_FM.indd 24 12/11/16 10:03 AM

xxvINDEX OF APPLICATIONS

Spending for Promoting a Cholesterol-Lowering Drug (YDI) 526, 594, 627, 664–665, 701, 733

Supermarket Mailing Coupons (TAI) 23 U.S. Wireless Telephone Market (TAI) 45

Media and Entertainment

Beatles (YDI) 77 Box-Office Gross of Movies (WT) 114 Flat-Screen Television Registration Card (TAI) 331 Flat-Screen TV and Surge Suppressors (YDI) 221 iTunes (YDI) 734 Junk Mail (YDI) 193–194, 388 Literary Digest (IE) 316–318 Movie Reviews (YDI) 766 Movie Schedule (IE) 165 News Report Summarizing a Poll of Voters (TAI) 386 Public Radio Station Soliciting Contributions (WT) 258 Song Lengths (YDI) 766 Textbooks (4M) 24–25 Types of Media in Children’s Bedrooms (YDI) 47

Pharmaceuticals, Medicine, and Health

Accidents at a Construction Site (WT) 161; (TAI) 175 Average Caloric Intake of Female Customers (IE) 377 Births by Day of the Week (IE) 160 Bone Density (IE) 271–272, 275, 291 Breathing Illness and Smoking (YDI) 200 Clinical Trials (SA) 484–485, 487 Comparing Diets (CO) 420; (IE) 420–423, 433–435; (4M) 431–433 Competing Promotions (4M) 665–666 Diagnostic Testing (4M) 191–193; (TAI) 198 Doctors Testing a New Contact Lens (TAI) 449 Drug Researchers (TAI) 415 Drug to Help Insomniacs Sleep (TAI) 247 Efficacy of a New Medication (TAI) 416 Flu (TAI) 103 Losing Weight (4M) 450–451 Lung Cancer (YDI) 176 Medical Researchers Receiving Money from Drug Manufacturers

(YDI) 106 Pharma Promotion (YDI) 296 Pharmaceutical Rep Meeting a Doctor (IE) 251–253, 255–257;

(BTM) 264; (TAI) 267 Picking a Hospital (4M) 107–108 Smoking and Lung Cancer (TAI) 103 Staffing a Maternity Ward (WT) 160–161 Therapies for Lower Back Pain (WT) 743–744, 746, 751 Therapy Lowering Blood Pressure (TAI) 415

Quality Control

Average Life of Light Bulbs (YDI) 388 Complaints Received at a Catalog Sales Center (TAI) 102 Computer Defects (YDI) 175–176, 267 Computer Shipments and Quality (4M) 209–210; (IE) 210 Damaged Appliances (YDI) 416 Defect Rate (YDI) 199–200 Defective Parts (YDI) 199; (IE) 253 Defects in Electronic Components (YDI) 416; (SA) 485 Defects in Semiconductors (4M) 261; (AE) 263 Failure Rate of Electronic Devices (YDI) 198 Financial Data (4M) 360–361 Highly Accelerated Life Test (CO) 334; (IE) 335–340, 344–346;

(WT) 340 Kitchen Appliance Breaking in the First Month (TAI) 415 Mistakes Made by Data Entry Clerks (YDI) 135–136

Safety Monitoring (4M) 269 Snack Food Quality (TAI) 331

Real Estate

Average Value of Home Sales per Agent (YDI) 764 Boston Housing (YDI) 137 Crime and Housing Prices in Philadelphia (4M) 555–556 Developer Choosing Heating Systems and Appliances (4M) 96–97 Do Fences Make Good Neighbors? (4M) 629 Home Prices and Square Feet (CO) 593; (AE) 617–619; (IE)

602–607; (YDI) 662, 698–699; (TAI) 697, 734–735 House Price Index (YDI) 802 Leasing Office Space (YDI) 525, 593, 626, 663, 699, 731–732 Locating a Franchise Outlet (4M) 571–573; (AE) 582–583 Philadelphia Housing (YDI) 136–137 Seattle Homes (YDI) 525, 593, 604–605, 626, 731; (IE) 597–599

Salary and Benefits

CEOs Average Total Compensation (P) 288; (WT) 570–571 Executive Compensation (4M) 64, 148–152; (AE) 69 Health Care Benefits (IE) 15; (TAI) 763, 765 Hourly Compensation in the U.S. Manufacturing Sector (YDI)

803 Salaries of Male and Female Managers (IE) 703–715 Salary and Years of Experience (IE) 705–715 Salary Data (TAI) 728–729 Top Paid CEOs (SA) 148–152 Weekly Salary (TAI) 294–295

Sales and Retail

Amount Purchased in a Supermarket Express Lane (WT) 340 Annual Sales at Clothing Stores (YDI) 248–249 Cash Taken During a Two-Hour Shift (TAI) 75 Cigarette Sales (TAI) 102 Clothing Purchases (IE) 14 Decoy Security Cameras (YDI) 450 Dress Shoe Sizes (TAI) 294 Focus on Sales (4M) 258–259; (AE) 262 Forecasting Change in Sales (TAI) 801–802 Hand Soap Sales (SA) 491–493 Hospital Supply Sales versus Number of Sales Representatives

(WT) 601 Item Produced during a Shift (TAI) 523 Local Supermarket Spending (TAI) 523 Missing Items from Storage Bins (TAI) 332 Net Sales (SA) 151–152; (QT) 153 Number of Customers and Sales (TAI) 133 Online Retailer (YDI) 388 Paper Towel Sales (SA) 492–493 Placing Orders on Outdoor Gear (YDI) 198 Purchases at a 24–Hour Supermarket (TAI) 103 Quarterly Report of Sales for a Department Store (TAI) 386 Retail Chain Store Sales in Three Markets (IE) 719–722 Retail Profits (4M) 682–687; (AE) 691–693 Retail Sales (TAI) 246; (YDI) 449–450 Sales and Price of Pet Food (IE) 538–543; (YDI) 554 Sales at a Convenience Store (TAI) 75, 524, 592, 625–626,

661–662, 730 Sales at Best Buy (4M) 787–789, 805 Sales by Company Representatives (TAI) 133 Sales, Income and Competitors (IE) 631–646; (BTM) 654–655 Shirt Sales at a Men’s Clothing Retailer (IE) 85–86; (YDI) 104 Shoppers Using Coupons versus Shoppers Who do Not (WT) 706,

709, 710; (YDI) 764 Snack Bars 47

A01_STIN7167_03_SE_FM.indd 25 12/11/16 10:03 AM

xxvi INDEX OF APPLICATIONS

Snack Counter at a Movie Theater (YDI) 198 Soft Drinks Sold in the United States (YDI) 46–47 Weekly Sales at a Home Improvement Center (WT) 343 Weekly Sales of Men’s Shirts (TAI) 763

Science

Diamond Cutting (YDI) 267 Insulation Damage to the Space Shuttle (IE) 3 Mosquito Breeding Areas (TAI) 332 Research Chemist Producing Synthetic Yarn (TAI) 763

Service Industries

Monitoring a Call Center (4M) 349–350; (AE) 351–352 Number of Calls at a Customer Help Line (IE) 259–260; (YDI)

358, 388 Repair Service Calls (YDI) 198, 221

Sports

Basketball (TAI) 175; (YDI) 177, 221–222, 247–248, 268 Betting on a Horse Race (4M) 177–178 Field Goals (YDI) 222 Fitness Center Scales (TAI) 133 Football Game (YDI) 268 Interval Training (TAI) 74 Kentucky Derby (TAI) 77 New York Giants (YDI) 248 Tour de France (YDI) 135

Surveys and Opinion Polls

Attitudes toward Sharing Copyrighted Music (IE) 92–93, 95–96

Banking Hours Survey (TAI) 331 Consumer Satisfaction Survey (TAI) 101–102 Dealership Survey (TAI) 332 Exit Surveys (4M) 322–323 Federal Reserve Survey of Consumer Finances (TAI) 76 Hotel Exit Survey (IE) 323, 409 Hotel Manager Surveying Customers (IE) 323 J.D. Power and Associates Questionnaire (CO) 314 Market Survey on Hybrid Cars (4M) 269 Marketing Survey of Car Owners (YDI) 104 Online Survey of Wealthy Homeowners about Their Insurance

Coverage (YDI) 389

Questionnaire to Human Resource Directors (TAI) 331 Shoppers’ Opinions (WT) 319 Stock Market Poll (YDI) 104–105 Survey of Brand Awareness (IE) 377 Survey on Trusting Different Figures in the News (YDI) 105–106

Technology

Android Devices (YDI) 450 Assembling Computer Systems (YDI) 199 Capacity of an Apple iPhone (CO) 51; (IE) 52, 68; (TAI) 74 Cellular Phones in Africa (YDI) 554 Cellular Phones in the United States (YDI) 554 Digital Recording (TAI) 219 Download (YDI) 524, 592, 626, 662, 698, 731 Downloading Songs (TAI) 74 GPS Chips (CO) 334, 335 Insulator Applied to Semiconductor Chip (YDI) 359–360 Internet Access (YDI) 200 Internet Connection (YDI) 48 Internet Search Engines (YDI) 450 MP3 Player (IE) 14 New Type of Cellular Telephone (YDI) 106 Sizes of Songs on a Computer (IE) 57–64; (TAI) 76 Software Project (TAI) 267

Transportation

Airline Arrivals (4M) 90–91; (AE) 98; (IE) 91–92 Airline Crash (TAI) 175 Airline On-Time Arrival Performance (YDI) 106–107 Airline Predicting Revenue from Feeder City Flights (TAI) 660,

661 Cars Sold in the United States (YDI) 77, 136, 525–526, 593, 627,

663–664, 699–700, 732–733 City’s Income from Parking Fees (YDI) 387–388 Commuting Time (TAI) 448 Flight Arrival and Luggage (YDI) 200 Flight Delays (YDI) 138 Frequent Flyers (4M) 734 Lost Luggage (WT) 253 Overbooking Flights (YDI) 267 Road Time of a Sales Representative (TAI) 74 Types of Vehicles in a Lot (TAI) 24 Weight of Passenger Luggage (TAI) 174 Weights of Commercial Trucks (YDI) 136

A01_STIN7167_03_SE_FM.indd 26 12/11/16 10:03 AM

PART I

Variation

M01_STIN7167_03_SE_C01_pp001-009.indd 1 25/10/16 12:10 PM

2

1 Introductionc h a p t e r

statistic A property of data.

1.1 ❘  WHAT IS STATISTICS? What do you think statistics is all about?

Statistics answers questions using data, or informa- tion about the situation. A statistic is a property of data, be it a number such as an average or a graph that displays information. Statistics—the discipline— is the science and art of extracting answers from data. Some of these answers do require putting numbers into formulas, but you can also do every statistical analysis with graphs and tables. It’s hard to be fooled when the answer stares you in the face.

Think of statistics as art. An artist must choose the right subject, and a good statistician finds the right picture. Rather than learning to paint, in this course you’ll learn how to use statistics to interpret data and answer interesting questions. Learning how to use data to solve problems is central to business analytics and data science.

Which questions are interesting? The answer is simple: those you care about. Of course, what inter- ests one person may be of no interest to someone else. In this text we apply statistics to a mix of topics, rang- ing from finance and marketing to personal choices. Most of the questions in this book concern business, but statistics applies more generally. We’ll help you appreciate the generality of statistics by solving prob- lems from health and science, too.

Variation

What kinds of questions can be answered with statistics? Let’s start with an example. In November 2011, Barnes and Noble debuted its entry into the market for tablet computers called the Nook Tablet™ and joined the field of challengers to the Apple iPad. A question fac- ing Barnes and Noble was, What’s the right price for the Nook Tablet?

That’s a hard question. To find an answer, you need to know basic economics, particularly the relationship among price, supply, and demand. A little finance and accounting determine the cost of development and production. Then come questions about the customers. Which customers are interested in a tablet computer? How much are they willing to pay? The Nook Tablet was smaller than an iPad with less technology, making it cheaper to produce. It could be sold at a profit for less than the $499 iPad, but how much less? Should it cost more than the competing $199 Kindle Fire?

Suddenly, the initial pricing question branches into several questions, and the answers depend on whom you ask. There’s variation among customers; customers react differently. One customer might be willing to pay $300, whereas another would pay only $200. Once you recognize these differences among customers, how are you going to set one price? Statistics

1.1 WHAT IS STATISTICS?

1.2 PREVIEWS

M01_STIN7167_03_SE_C01_pp001-009.indd 2 25/10/16 12:10 PM

shows how to use your data—what you know about your product and your customers—to set a price that will attract business and earn a profit. That you likely have not heard of the Nook tells you Barnes and Noble didn’t find the right combination of price and technology.

Here’s another interesting question: Why does a shopper choose a partic- ular box of cereal? Modern grocers have become information-rich retailers, tracking every item purchased by each patron. That’s why they give out per- sonalized shopping cards; they’re paying customers with discounts in return for tracking purchases. Customers keep retailers off balance because they don’t buy the same things every time they shop. Did the customer buy that box of cereal because it was conveniently positioned at the end of an aisle, because he or she had a discount coupon, or simply because a six-year-old just saw a commercial while watching Sponge Bob? Variation makes the question harder to answer.

Patterns and Models

Statistics helps you answer questions by providing methods designed to han- dle variation. These methods filter out the clutter by revealing patterns. A pattern in data is a systematic, predictable feature. If customers who receive coupons typically buy more cereal than customers without coupons, there’s a pattern.

Patterns form one part of a statistical model. A statistical model describes the variation in data as the combination of a pattern plus a background of remaining, unexplained variation. The pattern in a statistical model describes the variation that we claim to understand. The pattern tells us what we can anticipate in new data and thus goes beyond describing the data we observe. Often, an equation summarizes the pattern in a precise mathematical form. The remaining variation represents the effects of other factors we cannot explain because we lack enough information to do so. For instance, retail sales increase during holiday seasons. Retailers recognize this pattern and prepare by increasing inventories and hiring extra employees. It’s impossible, though, for retailers to know exactly which items customers will want and how much they will spend. The pattern does not explain everything.

Good statistical models simplify reality to help us answer questions. Indeed, the word model once meant the blueprints, the plans, for a building. Plans answer some questions about the building. How large is the building? Where are the bathrooms? The blueprint isn’t the building, but we can learn a lot from this model. A model of an airplane in a wind tunnel provides insights about flight even though it doesn’t mimic every detail of flight. Models of data provide answers to questions even though those answers may not be entirely right. A famous statistician, George Box, once said, “All models are wrong, but some are useful.”

A simple model that we understand is generally better than a complex model that we do not understand. A challenge in learning statistics is to recog- nize when a model can be trusted. Models based on physics and engineering often look impressively complex, but don’t confuse complexity with being cor- rect. Complex models fail when the science does not mimic reality. For exam- ple, NASA used the following elaborate equation to estimate the chance of foam insulation breaking off during take-off and damaging the space shuttle:

p = 0.01951L>d20.451d2rF0.271V - V *2 .67

ST .25rT

.16

The model represented by this equation failed to anticipate the risk of dam- age from faulty insulation. Damage from insulation caused the space shuttle Columbia to break apart on reentry in 2003.

variation Differences among individuals or items; also fluc- tuations over time.

pattern A systematic, predictable feature in data.

statistical model A breakdown of variation into a predictable pattern and the remaining variation.

3

M01_STIN7167_03_SE_C01_pp001-009.indd 3 25/10/16 12:10 PM

4 CHAPTER 1 Introduction

Models also fail if we mistake random variation for a pattern. People are great at finding patterns. Ancient people looked into the sky and found patterns among the stars. Psychiatrists use the Rorschach ink blot test to probe deep feelings. People even find patterns in clouds, imagining shapes or faces float- ing in the sky. The phenomenon of perceiving something familiar in a place where it does not occur is so common that it even has a name: pareidolia.1 A key task in statistics is deciding whether the pattern we have discovered is real or something that we’ve imagined. Finding a pattern allows us to anticipate what is most likely to happen next, to understand the data in order to plan for the future and make better decisions. But if we imagine a pattern when there is none, we become overconfident and make poor decisions.

1.2 ❘ PREVIEWS The following two examples preview the use of statistics to answer questions. Movie theaters show previews of coming attractions with lots of action and explosions, and save the character development for later. These examples in- troduce recurring themes and showcase several methods that are fully devel- oped in later chapters. The point is to advertise the types of analyses you will be able to do after you finish this book.

Each example begins with a question motivated by a story in the news, and then uses a statistical model to formulate an answer to the question. The first example uses a model to predict the future, and the second uses a model to fill in for an absence of data. These are previews, so we emphasize the results and skip the details.

Predicting Employment

In early November 2005, national broadcasts announced surprising and disturbing economic results. The big story was not a recession, but rather the U.S. economy’s slower than expected growth. The Labor Department reported that only 56,000 jobs had been created in October 2005, far short of the 100,000 additional jobs expected by Wall Street forecasters.

Financial markets react to surprises. If everyone on Wall Street expects the Labor Department to report large numbers of new jobs, the stock market can tumble if the report announces only modest growth. What should we have expected? What made Wall Street economists expect 100,000 jobs to be cre- ated in October? Surely they didn’t expect exactly 100,000 jobs to be created. Was the modest growth a fluke? These are serious questions. If the shortfall in jobs is the start of a downward trend, it could indicate the start of an eco- nomic recession. Businesses need to anticipate where the economy is headed in order to schedule production and supplies.

Was the weather responsible for the modest growth? On August 29, 2005, Hurricane Katrina slammed into Louisiana, devastating the Gulf Coast (see Figure 1.1). Packing sustained winds of 175 miles per hour, Katrina over- whelmed levees in New Orleans, flooded the city, and wrecked the local econ- omy. Estimates of damages reached $130 billion, the highest ever attributed to a hurricane in the United States, with more than 1,000 deaths. Katrina and the hurricanes that followed during the 2005 season devastated the oil industry concentrated along the Gulf of Mexico and disrupted energy supplies around the country. Did Katrina wipe out the missing jobs?

Let’s see if we can build our own forecast. Back in September 2005, how could you forecast employment in October?

1The New York Times article “Is That Jesus in Your Toast” (April 4, 2014) discusses the underlying psychol- ogy and offers several more examples.

M01_STIN7167_03_SE_C01_pp001-009.indd 4 25/10/16 12:10 PM

1.2 PREVIEWS 5

We need two things to get started: relevant data and a method for using these data to address the question at hand. Virtually every statistical analysis proceeds in this way. Let’s start with data. At a minimum, we need the number employed before October. For example, if the number of jobs had been steady from January through the summer of 2005, our task would be easy; it’s easy to forecast something that doesn’t change.

The problem is that employment does change. Table 1.1 shows the number of thousands employed each month since 2003. These are the data behind the story.

FIGURE 1.1 Hurricane Katrina on August 29, 2005.

TABLE 1.1 Nonfarm employ- ment in the United States, in thousands on payrolls.

2003 2004 2005

Jan 130,247 130,372 132,573

Feb 130,125 130,466 132,873

Mar 129,907 130,786 132,995

Apr 129,853 131,123 133,287

May 129,827 131,373 133,413

Jun 129,854 131,479 133,588

Jul 129,857 131,562 133,865

Aug 129,859 131,750 134,013

Sep 129,953 131,880 134,005

Oct 130,076 132,162 134,061

Nov 130,172 132,294

Dec 130,255 132,449

Each column gives the monthly counts for a year. The first number in the table represents 130,247,000 jobs on payrolls in January 2003. The following number shows that payrolls in February 2003 fell by 122,000. At the bottom of the third column, we can see that employment increased by 56,000 from September to October 2005, as reported by Reuters. This variation complicates the task of fore- casting. We’ve got to figure out how we expect employment to change next month.

We won’t replicate the elaborate models used by Wall Street economists, but we can go a long way toward understanding their models by plotting the data. Plots are among the most important tools of statistics. Once we see the plot, we can decide how to make a forecast.

The graph in Figure 1.2 charts employment over time, a common type of display that we’ll call a timeplot. To keep the vertical axis nicely scaled and avoid showing extraneous digits, we labeled the employment counts in millions

timeplot A chart of values ordered in time, usually with the values along the y-axis and time along the x-axis.

M01_STIN7167_03_SE_C01_pp001-009.indd 5 25/10/16 12:10 PM

6 CHAPTER 1 Introduction

rather than thousands. When displaying numerical information, showing fewer digits often produces a better presentation.

Once you’ve seen this timeplot, you can probably come up with your own forecast. Employment grew steadily during 2004 into 2005. This steady growth is a pattern. Employment varies from month to month, but the upward trend suggests a way to extrapolate the pattern into the future. The line drawn in the next figure summarizes the pattern we see in the data and suggests a forecast for October.

Models help us simplify complicated phenomena, but that does not make them correct. Good models convey the possibility of an error. The data on employment follow a line after 2004, but that does not mean that this pat- tern will continue. It’s not as though the economy knows about the line in Figure 1.3 and must stick to this trend. This line allows us to anticipate where employment is headed.

tip

129

131

132

133

134

135

2003 2004 2006

130

E m

p lo

ye e s

(m ill

io n s)

2005 Year

FIGURE 1.2 Timeplot of employment.

129

131

132

133

134

135

2003 2004 2006

130

E m

p lo

ye e s

(m ill

io n s)

2005 Year

FIGURE 1.3 Linear pattern and region of uncertainty.

Reality may not match our forecast, so we use ranges to convey the uncer- tainty associated with using a statistical model. Ranges are used in many fields to indicate uncertainty. For example, Figure 1.4 shows the uncertainty surrounding the projected path for Katrina before it made landfall. The wider the cone, the more doubt about the path of the storm.

Similarly, the shaded region around the line in Figure 1.3 indicates how closely the historic data stick to this pattern. In addition, this region suggests the amount of unexplained variation around the pattern. The more closely the data follow the pattern, the narrower this region becomes. The employment data track the line very closely during the period summarized by the line.

M01_STIN7167_03_SE_C01_pp001-009.indd 6 25/10/16 12:10 PM

To get a forecast, we extend the pattern. It is easy to extend the line beyond the data, as in Figure 1.3. The line passes above the count for October. This model predicts employment to be near 134,150,000 in October 2005, about 90,000 more than the reported count. That’s close to the value claimed in the news reports mentioned at the beginning of this subsection. We can also extend the region of uncertainty around the line. We should not expect counts of employment in the future to be closer to the line than those in the past. The line—our pattern—forecasts employment to be near that predicted by Wall Street economists. On the basis of this pattern, we would forecast employ- ment in October to lie between 134,130,000 and 134,533,000 jobs.

Our simple model confirms that the level of employment in October 2005 is surprising. Not only is the reported count of 134,061,000 for October less than expected, but it’s also outside the anticipated range. Even allowing for varia- tion around the pattern, employment is smaller than expected. Something happened in October that reduced the number of jobs. This is a large break from the pattern and demands our attention. Do we know why the employ- ment was less than expected?

Anticipating the impact of weather on employment is an ongoing concern. In the fall of 2008, Hurricanes Ike and Gustav were blamed for putting 50,000 Americans out of work, and the Department of Commerce estimated that Hurricane Sandy put 65,000 out of work in New Jersey and New York in 2012. The only way to arrive at these estimates is to predict what would have hap- pened had these hurricanes not struck. That’s a job for a statistical model.

Pricing a Car

For our second preview, let’s talk about cars. You may not be interested in cars, but, even so, you have almost certainly heard of the automaker BMW.

Writers for Car and Driver magazine love cars made by BMW, particularly the popular sporty sedans that go by numeric names like the 328 or the 335. According to these journalists, competitors have yet to figure out a way to beat the BMW 3-series. Competitors keep trying to knock BMW off the top of the pedestal but keep falling short in comparison. Year after year, Car and Driver has included BMW 3-series models in its “top 10” list of best cars of the year.

What’s it going to cost to get behind the wheel of a BMW 3-series? The manufacturer’s suggested retail price for a basic 2016 BMW 328i is $38,350, excluding options like leather seats that add $1,500 to the bottom line. That’s a lot to spend, so let’s see what a used BMW costs. For example, a search on the Web turned up a 2013 BMW 328i advertised for $27,000. The car has 40,000 miles, an automatic transmission, and a variety of options. On one hand, it sounds like a lot to spend $27,000 on a used car. On the other hand, its price

FIGURE 1.4 Projected path for Katrina.

1.2 PREVIEWS 7

M01_STIN7167_03_SE_C01_pp001-009.indd 7 25/10/16 12:10 PM

8 CHAPTER 1 Introduction

is $11,350 less than the cost of a stripped-down new car that lacks options like the heated leather seats found on this car.

Companies face similar decisions: Should a company buy new equipment, or should it look for a deal on used substitutes? As for cars, auto dealerships face these questions every day. What should the dealership charge for a three-year lease? The ultimate resale value has a big effect. If, when the car comes back, the dealership can sell it for a good price, then it can offer a better deal on a lease.

Once again, we need to identify data and decide how to use them. To get data on the resale value of used BMWs, students in a class downloaded prices for used cars offered by BMW dealers. They gathered information about 153 cars in the 328-series, from the 2011 through the 2016 model years. Table 1.2 summarizes the prices of these cars, broken out by model year. Eighty-four of these cars are 2013 models, like the car in the online ad. On average, these sell for $29,784. That’s $2,784 more than the car in the ad.

TABLE 1.2 Average prices vary by year.

Model Year Number Average Price

2011 16 $22,967

2012 3 $25,993

2013 84 $29,784

2014 15 $34,409

2015 34 $38,385

2016 1 $38,999

Before we decide that the used car in the ad is a bargain, we ought to con- sider other things that affect the price. You can probably name a few. We’ve taken into account the age (2013 model year) and style (328), but not the mile- age. We’d guess you’d be willing to pay more for a car with 10,000 miles than an otherwise similar car with 100,000 miles.

The scatterplot in Figure 1.5 shows that mileage is related to price. Each point shows the mileage and price of one of these 84 cars. None of these cars has exactly the same mileage as the advertised car. We can use a statistical model to compensate for our lack of cars with the same mileage. The plot in Figure 1.6 shows a line that relates mileage to price. The plot includes the region of uncertainty around the line. The region of uncertainty is wide because other factors, such as the condition of the car and its options, affect the price. There’s much more variation around this pattern than around the line in Figure 1.3.

scatterplot A graph that shows pairs of values (x, y) laid out on a two-dimensional grid.

Mileage

P ri ce

$25,000

$30,000

$35,000

$40,000

0 10,000 30,000 50,000FIGURE 1.5 Scatterplot of price versus mileage.

In our first example, we extrapolated a pattern in historic data. In this exam- ple, the pattern serves a different purpose. The line in Figure 1.6 allows us to

M01_STIN7167_03_SE_C01_pp001-009.indd 8 25/10/16 12:10 PM

borrow strength. Borrowing strength refers to using a model to glean more from data for answering a question. Rather than search for used cars with exactly 40,000 miles, this model—the line—allows us to estimate the price of a car with 40,000 miles, even though we haven’t seen a car with exactly this mileage. Even though the mileage varies among these cars, the pattern allows us to “borrow” some of the information about each car to estimate the price of a car with specific mileage. The estimated value, reading off the height of the line, is about $28,900. As an added bonus, the negative slope of the line in Figure 1.6 shows how higher mileage reduces the value of a car. On average, these cars lose about 13 cents of value per mile.

Having seen this analysis, do you think the car in the classified ad is a bar- gain? Its $27,000 price is less than the predicted price from this model, but well within the range for cars of this age and mileage. It might be worth a further look, particularly if you like the options that it includes!

The following chapters contain many more examples like these that illustrate the use of statistics in business. The only difference is that you will be doing the analysis, making the choices, and interpreting the results. The first step in learning these methods is to appreciate the importance of data, and that is the subject of the next chapter.

borrowing strength The use of a model to allow all of the data to answer a specific question.

Mileage

P ri ce

$25,000

$30,000

$35,000

$40,000

0 10,000 30,000 50,000FIGURE 1.6 A line relates mileage to price for these cars.

1.2 PREVIEWS 9

M01_STIN7167_03_SE_C01_pp001-009.indd 9 25/10/16 12:10 PM

10

2 Datac h a p t e r RUNNING A BUSINESS IS HARD WORK. One way to succeed in retail is to offer something missing from the big-box stores like Wal-Mart or Target, such as a personal touch. Store clerks once knew each customer, but few such stores remain because most of us prefer the lower prices and wider selections that we find at discount centers or on the Internet.

Paul Molino, owner of Bike Addicts in Philadelphia, loves to ride bicycles. What could be better than running his own bike shop? At first, it was easy to know his customers; he talked with them every weekend when the friends met to ride.

That got harder as the business grew. Without that personal touch, why would a customer looking for a $6,000 Italian bike come to him instead of shopping online? Perhaps because Paul knows what each customer wants—favorite brands, sizes, and styles—just like an old- fashioned clerk.

Databases allow modern businesses to keep track of vast amounts of information about customers. With a database, Paul can keep track of preferences for brands, sizes, and even colors.

A database of purchases also comes in handy at the end of the year when it’s time for inventory. After all, Paul would rather be riding than counting parts.

StatiSticS relieS on data, information that deScribeS the world around uS. If these data are to be useful, however, we need to organize them. That’s what this chapter is about: organizing data in a way that allows us to perform a statistical analysis. Most statistical data are organized into tables in which the rows and columns have specific meaning. Making sense of data requires knowing what those values mean.

2.1 DATA TABLES

2.2 CATEGORICAL AND NUMERICAL DATA

2.3 RECODING AND AGGREGATION

2.4 TIME SERIES

2.5 FURTHER ATTRIBUTES OF DATA

CHAPTER SUMMARY

M02_STIN7167_03_SE_C02_pp010-025.indd 10 25/10/16 12:11 PM

2.1 ❘ DATA TABLES Data are a collection of numbers, labels, or symbols and the context of those values. Often the data we have represent a subset of a much larger group. The data might consist of the purchases of a few customers, the amounts of several invoices, or the performance ratings of several employees. Data can also arrive sequentially, such as the closing price of a stock each day or the monthly sales at a Web site.

For the information in a database to be useful, we must know the context of the numbers and text it holds. For example, one value in Paul’s records is 61515, but without a frame of reference, a context for this value, this number is useless: 61515 might stand for $615.15 or $6,151.50; or it could stand for a zip code in Illinois. Figure 2.1 shows a few more values from Paul’s records.

11

Oscar 52215

58

0 27

Colnago

3810

BB

82015 Tu 4.5

Karen

1

1

6315

56 Bob

Ti

27 Michelin 4625

Conti

Karen

61515 Kestrel

31.05

1

FIGURE 2.1 Disorganized data hide their meaning.

When these numbers and names are scattered about with no structure, they mean little to us. They aren’t going to be useful for helping Paul run his business. For instance, he won’t be able to figure out which items are most popular until the data are organized.

When organized into a table with helpful column titles, as in Table 2.1, the values begin to take on meaning. This simple arrangement makes the data interpretable. The values first seen in Figure 2.1 describe four items bought at Paul’s shop. Each row in Table 2.1 describes a purchase, and each column collects a specific property of the purchases. The columns identify who made the purchase, what was bought, and when the purchase occurred. The first row, for example, tells us that Oscar bought a size 58 Colnago bike on May 22, 2015. Tables such as this are often called data tables, data sets, or data frames. Most statistical software and spreadsheets organize data into a table like this one.

data table A rectangular arrangement of data, as in a spreadsheet. Rows and columns carry specific meaning.

TABLE 2.1 Same data as in Figure 2.1 but in a table.

Customer

Club Member

Date

Type

Brand

Size

Amount

Oscar 0 52215 B Colnago 58 4625

Karen 1 6315 Tu Conti 27 4.50

Karen 1 61515 Ti Michelin 27 31.05

Bob 1 82015 B Kestrel 56 3810

This table is a huge improvement over the jumble in Figure 2.1, but we can improve it with better labeling. What does it mean that Oscar has a 0 in the second column? What do those letters mean in the column labeled Type? We can guess, but only Paul would know for sure. To remove the ambiguity, we should use more meaningful names, improve the formatting, and add units, as in Table 2.2.

M02_STIN7167_03_SE_C02_pp010-025.indd 11 25/10/16 12:11 PM

12 CHAPTER 2 Data

The meaning of the second column is clear in Table 2.2. A 0 in this col- umn means that the customer who made the purchase is not a club member; a 1 means that the customer is a club member. Similarly, a B in the fourth column of Table 2.1 stands for Bike. Table 2.2 also formats dates and dollar amounts. Presenting dates and dollars in a familiar format makes the values self-explanatory. Now that we’re looking at data, we can learn about these cus- tomers. For example, it seems that Karen is having problems with flat tires.

Rows and Columns

The rows of a data table describe a collection of things, with one row for each thing. Each row in the data for Bike Addicts denotes a purchase. The rows are often called observations (as in “observed items purchased at Bike Addicts”) or cases. The cases in Paul’s data are the four items bought by Bob, Oscar, and Karen. Other names for the rows signify special situations: People who participate in a survey are respondents; people in a medical study are subjects or participants. In a database, rows are called records—in this example, pur- chase records or transaction records.

Each column of a data table describes a common attribute shared by the items. Columns in a data table are called variables because we expect to find different values within a column. Consider the column labeled “Amount” in Table 2.2. Each row describes a purchase, and the amount paid varies from item to item. The name of a variable should describe the attribute. Avoid abbre- viated names like V1, V2, V3, and so forth. An acronym like DOLPTIS (for Dollars Per Transaction In Store) might make sense today but be puzzling in a month.

The number of rows in a data table is almost universally denoted by the symbol n. For Table 2.2, n = 4. There’s no consensus on a symbol for the number of columns, but you will see the symbol n for the number of rows in a data table over and over.

A universal convention among statistics packages that handle com- puting is that the observations form the rows and the variables form the columns. If your data table is arranged the reverse way, most software includes a func- tion called transpose that turns rows into columns and columns into rows.

2.2 ❘ CATEGORICAL AND NUMERICAL DATA Variables come in several types, depending on the data in the column. Most variables consist of either categorical or numerical data. Categorical vari- ables are sometimes called qualitative or nominal variables, and numerical variables are sometimes called quantitative or continuous variables. The type of a variable, numerical or categorical, is important because the type deter- mines how best to analyze the variable.

Categorical variables identify group membership. The labels within the col- umn identify the observations that belong to each group. Most of the columns in Table 2.2 are categorical variables. For example, the first column names the customer making the purchase, and the fourth column names the item.

tip

TABLE 2.2 Units and formatting make data self-explanatory.

Customer

Club Member

Date

Type

Brand

Size

Amount

Oscar No 5-22-15 Bike Colnago 58 cm $4,625.00

Karen Yes 6-3-15 Tube Conti 27 in $4.50

Karen Yes 6-15-15 Tire Michelin 27 in $31.05

Bob Yes 8-20-15 Bike Kestrel 56 cm $3,810.00

observation, case Names given to the rows in a data table.

variable A column in a data table that holds a common attribute of the cases.

tip

categorical variable Column of values in a data table that identifies cases with a common attribute.

n Number of rows in a data table.

M02_STIN7167_03_SE_C02_pp010-025.indd 12 25/10/16 12:11 PM

Numerical variables describe quantitative attributes of the observations. Familiar examples of numerical variables include incomes, heights, weights, ages, and counts. Numerical variables allow us to perform calculations that don’t make sense with categorical variables. Paul certainly wants to know the total amount of purchases made each day at his shop, but not the “average name” of his customers.

Numerical variables have measurement units that tell us how to interpret their values. The most important units in business denote currency, such as dollars $, euros €, or yen ¥. Lacking units, the values of a numerical variable have no meaning.

caution The data that make up a numerical variable in a data table must share a common unit.

While this convention seems routine, mixing numbers with different units can have disastrous consequences. The $200 million Mars Climate Orbiter crashed because some of the numbers it processed were in a metric scale but others were in English units. Paul is going to have a similar problem if he tries to average the sizes in Table 2.2 because some of these are measured in inches and others in centimeters.

Measurement Scales

The classical system for identifying the type of a variable refines categorical and numerical variables into four categories: nominal, ordinal, interval, and ratio. Nominal and ordinal variables are both categorical. Nominal variables name categories without implying an ordering, whereas the categories of an ordinal variable can be ordered. The brand of gasoline you recently bought defines a nominal variable; the grade of the gasoline (regular, plus, or premium) defines an ordinal variable. Interval and ratio variables are both numerical. It makes sense to add or subtract interval variables; ratio variables allow multiplication and division as well. For instance, temperature in degrees Fahrenheit is an in- terval variable. We can add and subtract temperatures, but ratios don’t make sense. Consider two days, one at 40° and the other at 80°. The difference in tem- perature is 40°, but it would not make sense to say that the warmer day is twice as hot as the cooler day. To see the problem, suppose we converted these tem- peratures to centigrade. Then the temperatures are 4.4°C and 26.7°C. On this scale, the warmer day is about six times as hot as the cooler day! In this text, we blur the distinction between interval and ratio scales, and call both numerical.

An ordinal variable defines categories, but the categories have a natural order. For example, a customer survey might ask, How would you rate our ser- vice? 5 = Great, 4 = Very good, 3 = Typical, 2 = Fair, and 1 = Poor . The ratings are ordered: Higher numbers indicate greater satisfaction. Manipulat- ing the numbers themselves might also make sense. You would probably pre- fer doing business with a firm that earns an average rating of 3.5 over another firm with an average rating of 2.5.

caution Even though it is common to treat ordinal variables as numerical and do things like find the average rating, be careful how you interpret the

results.

Does the difference between ratings of 4 and 5 mean the same thing as the difference between ratings of 2 and 3? It should if this variable is numerical. Is a hotel with average rating 4 twice as good as a hotel rated 2?

Many questions in surveys produce ordinal variables. Surveys frequently ask respondents to rate how much they agree with a statement. The result is an ordinal variable. For example, a survey question might look like this.

numerical variable Column of values in a data table that records numerical properties of cases (also called continuous variable).

measurement unit Scale that defines the meaning of nu- merical data, such as weights measured in kilograms or pur- chases measured in dollars.

ordinal variable A categorical variable whose labels have a natural order.

2.2 CATEGORICAL AND NUMERICAL DATA 13

M02_STIN7167_03_SE_C02_pp010-025.indd 13 25/10/16 12:11 PM

14 CHAPTER 2 Data

This MP3 player has all of the features that I want.

Strongly disagree Undecided

Strongly agree

1 2 3 4 5 6 7

The question usually offers five to seven choices. This approach to gauging attitudes was developed by Rensis Likert in 1932.1 A Likert scale produces an ordinal variable even though numbers label the choices.

Likert scale A measurement scale that produces ordinal data, typically with five to seven categories.

What Do You Think? The second preview example in Chapter 1 considered several variables that describe a used BMW. These variables include the price, mileage, model year, and model style (sedan, coupe, and convertible) of 153 used BMW cars in the 328 series.

a. If we organize these four characteristics into a data table, what are the rows and what are the columns?2

b. What’s n?3

c. Which variables are numerical, and which are categorical? Do any seem to be both numerical and categorical?4

d. What are the units of the numerical variables?5

3 n = 153, the number of cars.

1 “A Technique for the Measurement of Attitudes,” Archives of Psychology 140 (1932), 1–55. It is often necessary to add together several of these ordinal measures to form a numerical scale. 2 The rows identify cars, and the columns identify the four variables: price, mileage, model year, and style.

4 The price, mileage, and model year are numerical. The style is categorical. We might think of the model year as categorical if we use it to group the cars. BMW might treat the style as ordinal if some styles generate higher profits! 5 Price is measured in dollars, mileage is measured in miles, and model year is measured in years.

2.3 ❘ RECODING AND AGGREGATION Sometimes we have to rearrange data into a more convenient table. For in- stance, consider a data table that records every purchase of clothing at a retail store. A column that describes each item might be a detailed description, such as “Polo shirt, white, M,” “Dress shirt, red, M,” or “Polo shirt, red, L,” and so on. Such a column is a categorical variable, but one with hundreds of distinct categories, one for each combination of style, color, and size.

These distinctions are essential for managing inventory, but managers usu- ally prefer fewer categories. Rather than distinguish shirts by type, size, and color, for example, we could recode this column by using it to define a new column that only distinguishes sizes. The new column would record the three sizes: Small, Medium, and Large. For a categorical variable, recoding pro- duces a new column that consolidates some of the labels to reduce the num- ber of categories. This new column would show a manager which sizes are most popular.

Recoding can also be applied to numerical variables. Recoded numerical variables are typically ordinal and associate a range of values with a single label. Rather than look at the prices of individual items, for instance, Paul at Bike Addicts might want to distinguish expensive items from others. Starting from the purchase prices, he could define an ordinal variable that indicates if the price of the item is up to $10, up to $100, up to $1,000, and more than $1,000.

Modern retail check-out systems record every purchased item in order to keep track of inventory. Summaries that reduce the volume of transactions are essential for monitoring the overall health of a business. Inspecting every

recode To build a new variable from another, such as by combining several distinct values into one.

M02_STIN7167_03_SE_C02_pp010-025.indd 14 25/10/16 12:11 PM

transaction at a Wal-Mart store would be overwhelming. These “big-data” problems require that we summarize the data in order to reveal trends. To obtain a more compact summary, a manager can aggregate such data into a smaller table by summing values or counting cases within categories. Aggre- gation generates a new data table with fewer rows, whereas recoding adds more col- umns to the data table. A row in an aggregated table, for instance, could count the number of purchases and sum the total value of the purchases made day by day. Table 2.3 shows a portion of the original transaction data table and the aggregated data built from this portion (the actual data table of transac- tions has many more rows).

aggregate To reduce the number of rows in a data table by counting or summing values within categories. tip

TABLE 2.3 Aggregating the data table of transactions produces a new table with fewer rows that summarizes the transactions for each day.

Date Revenue

7/25/2016 $74.34

7/25/2016 $8.12

7/25/2016 $221.33

. . .

7/26/2016 $8.12

7/26/2016 $3.25

7/26/2016 $52.10

7/26/2016 $98.24

. . .

7/27/2016 $101.00

7/27/2016 $34.98

. . .

Date

Count

Total Revenue

7/25/2016 3 $303.79

7/26/2016 4 $161.71

7/27/2016 2 $135.98

4M ANALYTICS 2.1 MEDICAL ADVICE

MOTIVATION ▶ STATE THE QUESTION Private doctors often contract with health maintenance organizations (HMOs) that provide health care benefits to employees of large firms. Typically, HMO plans require a copayment from the patient at the doctor’s office. This copay- ment, along with fees paid by the insurer, keeps the doctor in business. To reduce paperwork, the insurer pays the doctor a fixed amount based on the number of patients in the care of that doctor, regardless of whether they visit the office.

This business model presents problems. To be profitable, the doctor (or office manager) needs to answer a few simple questions such as: Are patients from one HMO more likely to visit the office than those from another HMO? If so, the doctor may negotiate higher fees for covering patients from the plan with patients who are more likely to visit.

Many physicians still track patient records on paper. Our role in this exam- ple is to advise a doctor’s office on how to organize its data to make it easy for the office manager to compare patients from different HMOs. There’s not much that we can do with these data until we cover several more chapters, but we can think about the big picture so that we have the data ready. ◀

2.3 RECODING AND AGGREGATION 15

M02_STIN7167_03_SE_C02_pp010-025.indd 15 25/10/16 12:11 PM

16 CHAPTER 2 Data

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Let’s describe the data table. We start by conceiving a plan to collect and or- ganize the patient records to facilitate these comparisons. We may eventually need help from computer programmers, but we first have to choose which data to gather. To organize the records into useful data, we’ll put them into a table. The cases that make up the rows of the table will be office visits. The variables, or properties of these cases, will define the columns. ◀

MECHANICS ▶ DO THE ANALYSIS We cannot show all of the data, but we can illustrate how to organize portions of them. Regardless of what else gets added, the data table for this example has at least three columns: a patient identifier, the name of the HMO plan, and the length of a patient’s visit in minutes. The length of the visit is a numerical variable, and the other two variables are categorical.

The following table lists eight cases obtained from the data collection:

Patient ID HMO Plan Duration

287648723 Aetna 15

174635497 Blue Cross 20

173463524 Blue Cross 15

758547648 Aetna 25

837561423 Aetna 20

474739201 Blue Cross 20

463638200 AARP 30

444231452 AARP 25

Although the patient ID looks numerical, these numbers label the cases. This variable is categorical (a category identifies each patient). The HMO plan is also categorical. The durations in the third column are numerical but appear to have been rounded to multiples of five minutes. We might need to check that the staff recorded the actual duration rather than taking it from a pre- liminary schedule. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS The data table for answering questions about the length of time spent with patients should have at least three columns: one identifying the patient, one identifying the HMO, and one recording the length of the visit. Once these data become available, we can aggregate the counts to learn whether patients from one plan are consuming most of the doctor’s office time. We will see sev- eral ways to compare these numbers in the next few chapters.

It’s good to include concerns about your data in your summary message so that others are aware of possible limitations. We should contact the administrator to see how the durations of visits were measured. Our data seem to have been rounded. We should also find out how these records were chosen. Is the office going to supply us with data for every patient visit or just some? If they’re only going to give us data that describe a portion of the visits (called a sample), then we need to find out how they picked these cases. Otherwise, our results might turn out to be an artifact of how the subset of records was selected. The right way to identify a subset requires ideas that we’ll come to in Chapter 13. ◀

tip

M02_STIN7167_03_SE_C02_pp010-025.indd 16 25/10/16 12:11 PM

2.4 TIME SERIES 17

Gathering and organizing data is an important step of any statistical analy- sis. Don’t be hasty with this task. For many analyses, most of the time is spent collecting and organizing the data. A statistical analysis needs reliable data. You don’t want to spend hours analyzing data and writing a report only to dis- cover that the data do not measure what you thought.

2.4 ❘ TIME SERIES A time series is a sequence of data that records an attribute at different times. Examples of time series are the daily prices of Microsoft stock during the last year, the daily dollar value of purchases of customers at Wal-Mart, and the monthly employment counts reported by the Bureau of Labor Statistics. A timeplot graphs a time series by showing the values in chronological order. The timeplot in Figure 2.2 from the Federal Reserve Bank of St. Louis shows the percentage unemployed in the United States.

time series A sequence of data recorded over time.

timeplot A graph of a time series showing the values in chronological order.

Time series data are arranged in a table just like other types of data. Time series track stock prices, dollars in sales, employment, and other economic and social variables. The variables can be categorical or numerical.

The rows of time series data carry a different meaning from those in previ- ous examples. In place of different transactions, people, or businesses, a time series records the changing values of a variable over time. The rows measure the variable over and over at different points in time. Each day, for instance, we can record the price of Microsoft’s stock. Every row describes this same stock but at different points in time.

Some time series come from aggregating. For example, the purchases at Bike Addicts in Table 2.2 include the date. If we aggregate the data table by summing the purchased amounts for each day as done in Table 2.3, we obtain a new data table that has a time series of the daily total purchases. Typically, such aggregated time series are observed at a regular time interval known as the frequency of the time series. For instance, the unemployment time series in Figure 1.2 is observed every month, so the frequency of this time series is monthly. Each purchase at Bike Addicts happens at a unique time, and there’s no reason for purchases to happen regularly. Once aggregated to a daily total, the aggregated time series has a daily frequency.

Aggregation is essential when dealing with the vast amounts of data gen- erated by busy Websites such as Amazon or Google. Google, for example, records every one of the 3.5 billion searches made each day, watching to see if the user clicks on an advertisement. At some 40,000 queries per second, we couldn’t even read the data as fast as they accumulate. A little aggregation

2 3 4 5 6 7 8 9

10 11

C iv

ili an

U n e m

p lo

ym e n t

R at

e (%

)

1950 1960 1970 1980 1990 2000 2010 Date

Data from: U.S. Department of Labor, Bureau of Labor Statistics 2016

FIGURE 2.2 Timeplot of the monthly unemployment rate in the United States.

frequency The time spacing of data recorded in a time series.

M02_STIN7167_03_SE_C02_pp010-025.indd 17 25/10/16 12:11 PM

18 CHAPTER 2 Data

helps. By accumulating the total number of clicked ads per hour, we can see whether users click on ads more often at night than during the day. Convert- ing billions of clicks into an hourly time series—transforming big data into small data—can provide important insight and keep us from drowning in a tsunami of data.

If data measured over time are time series, do we have a special name for data not recorded over time? Not usually. If we want to be explicit and emphasize that data are not a time series, we can describe data observed at one point in time as cross-sectional data. For instance, retail sales at Wal-Mart stores around the United States in March 2016 are cross-sectional. These data vary because we have different stores, not because of the passage of time.

2.5 ❘ FURTHER ATTRIBUTES OF DATA We must understand the rows and columns of tables to analyze data. Other characteristics put this information into context. For example, it can be use- ful to know when and where the data were collected. A time or date can be relevant even if data are cross sectional. The prices of the used BMWs in the second preview example are not a time series, but it’s important to know when these data were collected. We would expect the asking price of a 2013 BMW offered for sale in 2016 to be a lot higher than the asking price of the same car in 2020. When and where often matter with Web data. Consider Google’s data on whether users click on ads during searches. In order to tell whether users click on ads more at night than during the day, we have to know the time for each user’s location. Midnight in New York is noon in Singapore!

The source of data may be useful to record as well, particularly if the data are online. Unless you gathered the measurements yourself, it’s good to give credit (or assign blame) to those who did. It is also useful to know why data are collected. The reason behind a survey might lead us, for example, to ques- tion the validity of the information. If we know that Levi Strauss conducted a survey, we may be skeptical when we hear that 95% of students expect Levi’s 501 jeans to be the most popular clothing item on campus.

It might even be important to know how data were collected. For exam- ple, the U.S. Department of Energy (DOE) keeps an eye on the supply and price of gasoline. At one point, DOE collected three time series, all claim- ing to measure the amount of gasoline used each month. When an analyst compared the three time series, he found that they occasionally differed by 600,000 gallons of gasoline a day! The explanation was simple: Each series defined gasoline differently. One originated in a survey of gas stations, another came from refiners, and the third was derived from state taxes on gasoline.

Sources of Data

Whether you’re searching for a new suit or the latest numbers on retail sales at Wal-Mart, it’s hard to beat Google. Common Web sites that we used in gath- ering the data that appear in this book include a data repository run by the Federal Reserve Bank in St. Louis (FRED) and sources at the Bureau of Labor Statistics. Look for files labeled “ascii” or “csv” for data that you can import into Excel or a statistics package. There’s also a project known as Data Ferret to simplify access to a variety of federal data sources. (You can download this software from the Census Web site.) You won’t even need to download the data; this application can build tables and compute various summaries.

cross-sectional data Data that measure attributes of different objects observed at the same time.

M02_STIN7167_03_SE_C02_pp010-025.indd 18 25/10/16 12:11 PM

2.5 FURTHER ATTRIBUTES OF DATA 19

Once you exhaust sites run by the government, the going gets harder, par- ticularly if you want to find data for a specific company. Data used for strate- gic decision making are closely held corporate secrets. To assemble data about a company, look online in Yahoo Finance or Hoovers. Unless you are prepared to pay for data, commercial sites might make you work to get all the data you want. After all, data are valuable!

4M ANALYTICS 2.2 CUSTOMER FOCUS

MOTIVATION ▶ STATE THE QUESTION Marketing studies use a focus group to test the reaction of consumers to a new product. A company shows its product to a few selected customers—the focus group—and measures their reactions. For example, an electron- ics company could ask a focus group to rate features of a prototype for a smartphone and ask how much they would pay for such a phone. How would you organize these data? Take a moment and think about the rows, columns, and anything else that is relevant.6

Our data record how customers in a focus group react to a new design. We plan to use this information to decide which type of advertising is most likely to reach interested customers, such as whether younger customers give it a higher rating. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Organize the data from the focus group as a table. Each row of this table will represent a participant of the focus group, and the columns will record the variables associated with a participant, particularly the rating given to the product. ◀

MECHANICS ▶ DO THE ANALYSIS Keep track of the details, particularly the units, of the variables in the data table. In addition to recording the rating assigned by each participant to the product, the columns in this data table should include characteristics of the participants, such as the name, age (in years), sex, and income (in dollars). The name and sex are categorical, and the age and income are numerical. Participants’ ratings of the product are likely going to be ordinal data, such as values from a Likert scale. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS Once we have the data, we will be able to find out if, for example, younger members of the focus group like the design or whether more affluent custom- ers like it more than those with less to spend. Once we know who likes the design, we will be able to pick the form of advertising that is most likely to appeal to that audience. ◀

6 Some companies avoid focus groups. Have a look at “Shoot the Focus Group” from the November 14, 2005 issue of Businessweek. A large group may not mean much if one or two personalities dominate the group.

M02_STIN7167_03_SE_C02_pp010-025.indd 19 25/10/16 12:11 PM

20 CHAPTER 2 Data

Best Practices

■ Provide a context for your data. At a minimum identify the rows and columns and include units for numerical variables. Verify that all of the cases (or rows) share the same units. Ide- ally, embellish the data with other information such as the name of the person who collected the data or the source of the data, when and where the data were gathered, and why the data were collected.

■ Use clear names for your variables. Use names that you will understand later and that will make sense to your audience. If you include units for numerical variables as part of the name, then the data become easier to use later on.

■ Distinguish numerical data from categorical data. Data do not have to be numerical to be useful. Information like names and addresses

can help us learn a lot about customers. Label categories with names rather than numbers.

■ Track down the details when you get the data. Even if you are told a context for the data, it may turn out that the true situation is a bit (or even a lot) different. The context colors our in- terpretation of the data, so those who want to influence what you think may slant the story. If you don’t learn these details as you gather the data, you probably won’t be able to fill them in later.

■ Keep track of the source of data. If you later dis- cover a strange value or question the meaning of a variable, it will be helpful to have a record of where the data came from. Also, some data change over time or become more extensive as time passes. It will be easier to update your data if you know where you got them.

Pitfalls

■ Do not assume that a list of numbers provides numerical data. Categories often masquerade as numbers. Don’t let that fool you into thinking they have quantitative meaning. Pay attention to the context. Zip codes and phone numbers are numbers, but they form categorical variables.

■ Don’t trust all of the data that you get from the In- ternet. The Internet is the most common place to get data, but that does not mean that every piece of information that you find online is accurate. If you can, find out how the data were collected and dig into the pedigree of that information be- fore you put too much trust in the numbers.

■ Don’t believe every claim based on survey data. A survey that claims to represent consumers

may in fact report just the opinions of those who visit a Web site. The question that respon- dents answered may be slanted to favor a par- ticular answer. Think about how people would respond to the question, Should the govern- ment raise taxes so that it can waste money? Not too many would favor raising taxes, would they? Now suppose the question were asked as, Should the government raise taxes to pro- tect our citizens and improve health care? We might expect this question to find that more are in favor of higher taxes than we thought. If you don’t know how the question was asked, try to get a copy of the questionnaire. We return to these issues in Chapter 13.

Software Hints

Most often we use computer software to handle the mechanics of calculating statistics from data. There are many different statistics packages that all do essentially the same things, but each speaks a differ- ent language. You have to figure out how to tell your package what you want it to do. That’s easy once you become familiar with your software.

The first step gets your data ready for analysis. No matter which software program you use, you need to get your data into the program. This typically re- quires that you type the data into the program (as in a spreadsheet) or tell the software (in a dialogue of some sort):

M02_STIN7167_03_SE_C02_pp010-025.indd 20 25/10/16 12:11 PM

SOFTWARE HINTS 21

■ Where to find the data. This usually means identi- fying the name of a file on your computer. Most software expects the data to be laid out as a data table with one line per case. The values for the variables in each line need to be separated, or delimited, by a common character known as the delimiter. Typical choices for the delimiter are spaces, commas (as in csv format), or tabs. The return character marks the end of a line (or case).

■ Where to put the data. After the package reads the data, it converts the data into an internal format that you can save as another file (usually identi- fied by a suffix such as .xls for Excel files). Save this file so you don’t have to read the original data each time you look at the data. This internal file will also hold changes you make such as recoding or aggregating cases.

■ Names for the variables. Most packages name variables automatically, but names like Column 1 or V1 aren’t helpful. Most packages allow you to include descriptive names for the variables in the first line of your external data file. If not, you’ll have to enter these manually within your package.

The following sections describe commands for specific packages: Excel, Minitab, and JMP. The sym- bol 7 denotes following a menu tree down to the next level. The software hints given here will get you started, but you’ll certainly want to read the docu- mentation for your package in order to exploit its full range of capabilities.

EXCEL To read data into Excel from a text file laid out like a data table, follow the sequence of menu commands

Data 7 Get External Data 7 Import text file

Then fill in the dialog boxes. Excel will show you how it interprets the items along the way so you can interactively change the way that it distinguishes col- umns, for example. As a convention, put the variable names in the first row of the worksheet. Then freeze these cells so that they remain visible if you scroll down the rows. (Use the Window 7 Freeze panes command after selecting the row below the labels. This option is only available in the normal view of the worksheet.)

Excel is particularly flexible for recoding and ag- gregation. For example, in the retail example, you want to recode a column of numerous labels such

as Polo shirt, red, M. For this task, follow the menu items

Data 7 Text to columns

and fill in the resulting dialog box. For aggregating data, use pivot tables. You can find details about the use of pivot tables in Excel’s online help.

MINITAB EXPRESS Minitab can read data from CSV files. Arrange the data as a table to begin with. Variables define the columns, and cases define each row. Identify the variables with names in the top row as described previously for Excel, and then add information on each obser- vation in the remaining rows. Be sure that each row describes the same variables. Follow the menu commands

File 7 Open worksheet g and then identify your file from the menu.

To transfer data from Excel, use the copy and paste commands. Lay out your data as described in the instructions for Excel. Then copy the entire spreadsheet. In Minitab, create a new data table us- ing the File 7 New menu command. Position your cursor in the top left cell and use the paste command to move your data.

Recoding operations, such as changing numerical data into categories, are handled by the Data 7 For- mula command.

JMP JMP will also read data directly from Excel if the data are arranged in the natural way with names of the variables in the first row and data for these vari- ables in following rows. To import data from a text file, use the

File 7 Open

menu command to get to a file dialog box. In this dialog box, enable the feature “All text documents,” pick the file, and open the file with the option Data (Using Preview). JMP also reads data from text files laid out in this format. It is usually able to recognize the layout of the file.

For recoding in JMP, you’ll need to master its for- mula calculator (for example, the Match command is very useful for grouping data). For aggregating a table (combining rows into a smaller table), use the Tables 7 Summary command and fill in the result- ing dialog box.

M02_STIN7167_03_SE_C02_pp010-025.indd 21 25/10/16 12:11 PM

22 CHAPTER 2 Data

CHAPTER SUMMARY

Data describe characteristics of a collection of cases or observations. Data for statistical analysis are typically arranged in a data table. The rows of the table identify the cases. The columns of the table denote variables, with each column containing the measured values of some attribute of the cases. Vari- ables may be numerical or categorical. Numerical variables have common measurement units, and it makes sense to average these values. Categorical variables identify cases with a shared label. Ordinal

variables, such as those from Likert scales, order the cases and may assign numerical labels; these are often treated as numerical data. Recoding a variable reduces the number of distinct values; aggregating a variable reduces the number of rows of data. Vari- ables measured over time are time series. The rows of a time series identify the times of the measure- ments, and the columns denote what was measured. Data measured at one point in time are called cross sectional.

■ Key Terms aggregate, 15 case, 12 categorical variable, 12 cross-sectional data, 18 data table, 11 frequency, 17

Likert scale, 14 measurement unit, 13 n (number of rows in data table), 12 numerical variable, 13 observation, 12

ordinal variable, 13 recode, 14 timeplot, 17 time series, 17 variable, 12

■ Objectives • Organize data into a table with multiple variables

(columns) and cases (rows). • Distinguish categorical from numerical variables.

Be aware that some categorical variables (ordinal) define an ordering of the cases.

• Recognize time series data. • Identify when recoding or aggregating data are

useful.

■ About the Data Bike Addicts is a privately held bike shop in Philadel- phia owned by Paul Molino. The customer data are real, but we altered the names. The increasing use of databases that track customers has made privacy and data security important to both customers and businesses.

The Federal Reserve Bank of St. Louis maintains a comprehensive collection of economic time series online called the Federal Reserve Economic Data (FRED).

Mix and Match

The following items describe a column in a data table. On the basis of the description, assign a one- or two-word name to the variable, indicate its type (categorical, ordi-

EXERCISES

nal, or numerical), and describe the cases. If the data are a time series, identify the frequency of the data. The first item is done as an example.

M02_STIN7167_03_SE_C02_pp010-025.indd 22 25/10/16 12:11 PM

EXERCISES 23

Variable Name

Type

CasesNom Ord Num

Model of cellular telephone used by customers Model ✓ Customers

Weights of people, measured in pounds Weight ✓ People

1. Brand of car owned by drivers

2. Income of households, measured in dollars

3. Color preference of consumers in a focus group

4. Counts of customers who shop at outlets of a chain of retail stores

5. Size of clothing items given as Small, Medium, Large, and Extra Large

6. Shipping costs in dollars for deliveries from a catalog warehouse

7. Prices of various stocks, in dollars per share

8. Number of employees absent from work each day

9. Sex (male or female) of respondents in a survey

10. Education of customer, recorded as High school, Some college, or College graduate

True/False

Mark each statement as True or False. If you believe that a statement is false, briefly say why you think it is false.

11. Zip codes are an example of numerical data.

12. Quantitative data consist of numbers, along with their units.

13. Cases is another name for the columns in a data table.

14. The number of rows in a data table is indicated by the symbol n.

15. The frequency of a time series refers to the time spac- ing between rows of data.

16. A column in a data table holds the values associated with an observation.

17. A Likert scale represents numerical data.

18. Recoding adds further columns to a data table.

19. Aggregation adds further rows to a data table.

20. Aggregation can convert cross-sectional data into a time series.

Think About It

For each situation described in Exercises 21–30, (a) Identify whether the data are cross sectional or a

time series. (b) Give a name to each variable and indicate if the

variable is categorical, ordinal, or numerical (if a variable is numerical, include its units if possible).

(c) List any concerns that you might have for the accuracy of the data.

21. The debate about the future of Social Security has renewed interest in the amount saved for retirement by employees at a company. While it is possible to open an Individual Retirement Account (IRA), many employees may not have done so. To learn about its staff, a company asked its 65 employees whether they have an IRA and how much they have contributed.

22. A bank is planning to raise the fees that it charges customers. It is concerned that the increase will cause some customers to move to other banks. It surveyed 300 customers to get their reaction to the possible increase.

23. A hotel chain sent 2,000 guests who visited the hotel within the last month an email asking them to rate the service in the hotel during their most recent visit. Of the 500 who replied, 450 rated the service as excellent.

24. A supermarket mailed 4,000 uniquely identifiable cou- pons to homes in local residential communities. Each day of the next week, it counted the number of coupons that were redeemed and the size of the purchase.

25. A Canadian paper manufacturer sells much of its paper in the United States. The manufacturer is paid in U.S. dollars but pays its employees in Canadian dollars. The manufacturer is interested in the fluc- tuating exchange rate between these two currencies. Each trading day in 2016 the exchange rate fluctuated by several basis points. (A basis point is 1/100 of a percent.) A year of data is collected.

26. Car manufacturers measure the length of time a car sits on the dealer’s lot prior to sale. Popular cars might only sit on average for 10 days, whereas

M02_STIN7167_03_SE_C02_pp010-025.indd 23 25/10/16 12:11 PM

24 CHAPTER 2 Data

overstocked models sit for months. This company collected data each month for the length of time cars sat on dealer lots for 10 models from 2002 to 2016.

27. A video game was previewed to a group of 30 teenag- ers. The teens were asked to rate the quality of the graphics and indicate if the game was too violent.

28. A major bank collected data on 100,000 of its customers (income, sex, location, number of cards, etc.) and then computed how much profit it made from the account of these customers during the past calendar year.

29. A start-up company built a database of customers and sales information. For each customer, it recorded the customer’s name, zip code, region of the country (East, South, Midwest, West), date of last purchase, amount of purchase, and item purchased.

30. A retail chain lines its parking lots depending on the types of vehicles driven by local shoppers. Large trucks and SUVs require bigger spots than compact cars. Each day, while collecting shopping carts, an employee records a list of the types of vehicles in the lot.

31. 4M ANALYTICS: Economic Time Series

Data don’t always arrive in the most convenient form. You may have the data, but they’re not in the same place or the same format. Unless you have data organized in a common data table, you’ll find it hard or impossible to get the answers you need.7

For this exercise, you must prepare the data that are needed to explore the relationship between the sales of a company, retailer Best Buy, and the health of the economy. Is there a relationship between the amount of money available to be spent, called the disposable income, and the net sales of retailer Best Buy?

The economic data come from the online repository hosted by the Federal Reserve Bank of St. Louis. The data are collected monthly, are reported in the number of bil- lions of dollars (at an annual rate), and date back to 1959. The data for 2010 are as shown in the following table:

Month Disp Income

2010-01-01 $11,041.1

2010-02-01 $11,023.0

2010-03-01 $11,060.3

2010-04-01 $11,141.1

2010-05-01 $11,220.6

2010-06-01 $11,231.2

2010-07-01 $11,253.9

2010-08-01 $11,304.7

2010-09-01 $11,301.3

2010-10-01 $11,355.5

2010-11-01 $11,407.2

2010-12-01 $11,514.5

The data for Best Buy come from another source. Compustat maintains a database of company informa- tion gleaned from reports that are required of all publicly traded firms. For a company to have its stock bought and sold, it must report data such as these quarterly gross profits, given in millions of dollars. The company data extend back to 2005 (only one year is shown).

Year Quarter Gross Profits

2010 1 $3,036.00

2010 2 $3,156.00

2010 3 $3,233.00

2010 4 $4,214.00

Motivation

(a) Explain why it would it be useful to merge these two data tables. What questions do you think would be interesting to answer based on the merged information?

Method

(b) Describe the difference in interpretation of a row in the two tables. Do the tables have a common frequency?

(c) The separate data tables each have a numerical column of gross profits or disposable income. Are the units of these comparable, or should they be expressed with common scales?

Mechanics

(d) What should you do if you want to arrange the data in a table that has a quarterly time frequency? Can you copy the data columns directly, or do you have to perform some aggregation or recod- ing first?

(e) Suggest improved names for the columns in the merged data table. How do you want to represent the information about the date?

(f) Show the merged data table for 2010.

Message

(g) With the data merged, what annual shopping ritual becomes apparent?

32. 4M ANALYTICS: Textbooks

This exercise requires research on how books are priced on the Internet. We will divide books into three broad categories: popular books, school books or textbooks, and recreational books. Some books may fall in two or even three of these categories.

Motivation

(a) If you were able to reduce the cost of the books that you buy each year by 5 or 10%, how much do you think you might be able to save in a year?

7 We’ll see these data again in Part 4 when we look at methods for forecasting.

M02_STIN7167_03_SE_C02_pp010-025.indd 24 25/10/16 12:11 PM

Method

(b) Create a data table of prices for the three types of books. You will compare prices of books at two Internet stores. Your data table will have four columns: one for the name of the book, one for the type of the book, and two more columns for the prices. (If you don’t have two Internet stores at which you usually shop, use amazon.com and barnesandnoble.com.) What types of data are these?

(c) Identify five books of each type. These form the rows of the data table.

Mechanics

(d) Fill in your table from prices on the Web. Did you find the prices of the books that you started

with, or did you have to change the choice of books to find the books at both stores?

(e) Do your prices include costs for shipping and taxes, if any? Should they?

Message

(f) Summarize the price difference that you found for each of the three types of books. Describe any clear patterns you found. For example, if one store is always cheaper, then you have an easy recommendation. Alternatively, if one store is cheaper for textbooks but more expensive for popular books, you have a slightly more complex story to tell.

EXERCISES 25

M02_STIN7167_03_SE_C02_pp010-025.indd 25 25/10/16 12:11 PM

26

3 CONSUMERS SPEND BILLIONS OF DOLLARS ONLINE. Amazon, one of the first e-tailers, remains a leading shopping destination on the Web. Every day, millions of shoppers visit its Web site, often by clicking through from ads hosted by another Web site.

Table 3.1 describes six visits to Amazon. Each row in the table lists five attributes of one of these sessions. These attributes appear in columns that define variables that indicate whether the customer made a purchase, the host, and demographics of the shopper (number of people in the household, geographic location, and income). If the customer typed amazon.com into a browser, as in the second row, then the host field is blank. The data are a mix of categorical and numerical variables. For instance, the name of the host is categorical, whereas the household size is numerical.

Which hosts generate the most business?

c h a p t e r Describing Categorical Data

3.1 LOOKING AT DATA

3.2 CHARTS OF CATEGORICAL DATA

3.3 THE AREA PRINCIPLE

3.4 MODE AND MEDIAN

CHAPTER SUMMARY

TABLE 3.1 Six visits to amazon.com.

Purchase Host Household

Size Region Income Range

No msn.com 5 North Central 35–50k

No 5 North Central 35–50k

No google.com 4 West 50–75k

Yes dealtime.com 3 South <15k

No mycoupons.com 2 West 50–75k

No yahoo.com 5 North Central 100k+

This chapTer shows how To describe a caTegorical variable using Two Types of charTs: bar charts and pie charts. You’ve probably seen these before, so this chapter concen- trates on the important choices that are made when building these graphs. The chapter also introduces principles that guide the construction of any graph that shows data.

M03_STIN7167_03_SE_C03_pp026-050.indd 26 25/10/16 12:13 PM

3.1 ❘ LOOKING AT DATA Let’s get specific about our motivation. You need a clear goal when you look at data, particularly a large data table. Our task in this chapter is to answer the following question:

Which hosts send the most visitors to Amazon’s Web site?

The data we have track a small percentage of activity at Amazon, but that still amounts to 188,996 visits from households in the United States. Table 3.1 lists six visits that we chose at random to illustrate the contents of each column. You would not want to read through the rest of these data, much less tabulate by hand the number of visits from various hosts.

That’s the problem with huge data tables: You cannot see what’s going on, and seeing is what we want to do. How else can we find patterns and answers to questions in data?

Variation

The difficulty in answering questions from data arises from the case-to-case variation. Variation in data appears in the form of differences among the values within each column of a data table. For example, if every visitor to Amazon came from Google, then every row in the host column would identify Google and there would be no variation. That’s not the case. The column that identifies the host in this data table has 11,142 different values.

Variation is also related to predictability. The larger the variation, the harder it is to predict the next observation. No variation would mean every one of these visitors came from the same host. If every prior visitor came from Google, we would expect the next to come from Google as well. Variation re- duces our ability to anticipate what will happen next. With 11,142 different hosts, we cannot be sure of the next host.

Frequency Table

To describe the variation in categorical data, statistics reduces to counting. Once we have counts, we’ll graph them. The trick is to decide what to count and how to graph the counts. Which variables in Table 3.1 should we focus on?

That’s the point of a clear motivation. Because we want to identify the hosts that send the most visitors, we know to count the number of visitors from each host rather than count the number of households of size 4. These counts define the distribution of the variable. The distribution of a categorical vari- able tells you two things about that variable: the possible values and the count or frequency of each. A frequency table presents the distribution of a cate- gorical variable as a table. Each row of a frequency table lists a category along with the number of cases in this category.

A statistics package can list the visits from every host, but we need to be choosy. These data include 11,142 different hosts, and we only want to identify the hosts that send the most visitors. In fact, among all of these hosts, only 20 delivered more than 500 visits. Table 3.2 lists these individually and combines hosts with fewer visits into a large category labeled Other. These counts were observed in 2004, and some of these sites are no longer active. Exercise 60 at the end of the chapter contrasts these with counts observed 10 years later.

This frequency table reveals several interesting characteristics of the hosts. First, the most popular way these users took to get to Amazon was to type amazon.com into a browser. You probably recognize the top three hosts: msn.com, yahoo.com, and google.com. More than 70% of Americans online visit these portals. The surprise is how many visits originate from less familiar hosts like recipesource.com. Quite a few visits also come from

variation in data Differences among the values within a column of a data table.

distribution The collection of values of a variable and how often each occurs. frequency table A tabular summary that shows the distribution of a variable.

27

M03_STIN7167_03_SE_C03_pp026-050.indd 27 25/10/16 12:13 PM

28 CHAPTER 3 Describing Categorical Data

Host Frequency Proportion

Typed “amazon.com” 89,919 0.47577

msn.com 7,258 0.03840

yahoo.com 6,078 0.03216

google.com 4,381 0.02318

recipesource.com 4,283 0.02266

aol.com 1,639 0.00867

iwon.com 1,573 0.00832

atwola.com 1,289 0.00682

bmezine.com 1,285 0.00680

daily-blessings.com 1,166 0.00617

imdb.com 886 0.00469

couponmountain.com 813 0.00430

earthlink.net 790 0.00418

popupad.net 589 0.00312

overture.com 586 0.00310

dotcomscoop.com 577 0.00305

netscape.com 544 0.00288

dealtime.com 543 0.00287

att.net 533 0.00282

postcards.org 532 0.00281

24hour-mall.com 503 0.00266

Other 63,229 0.33455

Total 188,996 1.00

TABLE 3.2 Frequency table of hosts.

less common hosts. The next-to-last row of Table 3.2 shows that other hosts that provide fewer visits supply about a third of all visitors to Amazon.

The last column in Table 3.2 shows the proportion of the visits from each host. A proportion is also known as a relative frequency, and a table that shows these is known as a relative frequency table. Proportions are sometimes multiplied by 100 and given as percentages. We prefer tables that show both counts and proportions side by side as in Table 3.2. With the column of pro- portions added, it is easy to see that nearly half of these visitors typed amazon. com and that a third came from small hosts. The choice between proportions and percentages is a matter of style. Just be clear when you label the table.

3.2 ❘ CHARTS OF CATEGORICAL DATA Let’s concentrate on the top 10 hosts. A frequency table is hard to beat as a summary of several counts, but it becomes hard to compare the counts as the table grows. Unless you need to know the exact counts, a chart or plot beats a table as a summary of the frequencies of more than five categories. The two most common displays of a categorical variable are a bar chart and a pie chart. Both describe a categorical variable by displaying its frequency table.

Bar Chart

A bar chart displays the distribution of a categorical variable using a sequence of bars. The bars are positioned along a common baseline, and the length of each bar is proportional to the count of the category. Bar charts should have

relative frequency The frequency of a category divided by the number of cases; a proportion or percentage.

tip

bar chart A display using horizontal or vertical bars to show the distribution of a categorical variable.

M03_STIN7167_03_SE_C03_pp026-050.indd 28 25/10/16 12:13 PM

3.2 CHARTS OF CATEGORICAL DATA 29

small gaps between the bars to indicate that the bars could be rearranged into any order. The bar chart in Figure 3.1 uses horizontal bars to show the counts for the largest 10 hosts, ordered from largest to smallest.

By showing the hosts in order of frequency, this bar chart emphasizes the bigger hosts. You can also orient the bars vertically, as in Figure 3.2.

We find it easier to compare the lengths of horizontal bars than vertical bars. Vertical bars also limit the number of categories in order to show the labels. That said, orientation is a matter of taste. Bar charts may be drawn either way.

When the categories in a bar chart are sorted by frequency, as in Figures 3.1 and 3.2, the bar chart is sometimes called a Pareto chart. Pareto charts are popular in quality control to identify the most common problems in a busi- ness process.

Pareto chart A bar chart with categories sorted by frequency.

FIGURE 3.1 Bar chart of top 10 hosts, with the bars drawn horizontally.

FIGURE 3.2 Bar chart of top 10 hosts, with the bars drawn vertically.

M03_STIN7167_03_SE_C03_pp026-050.indd 29 25/10/16 12:13 PM

30 CHAPTER 3 Describing Categorical Data

We could have shown the hosts in alphabetical order to help readers find a particular host, but with 10 labels, that’s not necessary. If the categorical variable is ordinal, however, you must preserve the ordering. To show a variable that mea- sures customer satisfaction, for instance, it would not make sense to arrange the bars alphabetically as Fair, Great, Poor, Typical, and Very good.

Bar charts become cluttered when drawn with too many categories. A bar chart that showed all 20 hosts in Table 3.2 would squish the labels, and the smallest bars would become invisible. There’s also a problem if the fre- quency of one category is much larger than those of the rest. The bar chart in Figure 3.3 adds a bar that counts the visits from every other host.

tip

The long bar that accumulates small hosts dwarfs those representing the most popular hosts. That might be a point worth making; most visits don’t come from the top 10 hosts. The prevalence of small hosts is an example of what’s called the long tail in online retailing; see About the Data for another example.

Which of these bar charts provides the best answer to the motivating question? We prefer the chart in Figure 3.1, or perhaps Figure 3.2. The ac- cumulated number of hits from the other hosts in Figure 3.3 obscures the differences among the top 10. It’s impossible to see that AOL generates almost twice as many visitors as imdb.com. The message from Figure 3.1 is clear: Among hosts, MSN sends the most visitors to Amazon, more than 7,000 dur- ing this period, followed by Yahoo, Google, and RecipeSource. A variety of other hosts make up the balance of the top 10.1

caution We use bar charts to show the frequencies of a categorical variable. You will often see bar charts used to graph other types of data, such as

a time series. For example, Figure 3.4 shows a chart created from data published in the Wall Street Journal. The figure looks like a bar chart, but it isn’t. This chart in- stead uses bars to graph a sequence of counts over time. It is fine to use bars in this way to emphasize the changing level of a time series, but these are not displays of a frequency table.2

1 Edward Tufte offers several elegant improvements to the bar chart, but most of these have been ignored by software packages. Look at his suggestions in his book The Visual Display of Quantitative Information (Cheshire, CT: Graphics Press, 1983), pp. 126–128. 2 The Wall Street Journal, “Ford’s Lincoln Brand Gets New . . . Life,” January 12, 2012.

FIGURE 3.3 One category dominates this bar chart. 0

msn.com

Other hosts

Number of Visits from Named Hosts

yahoo.com

google.com

recipesource.com

aol.com

iwon.com

atwola.com

bmezine.com

daily-blessings.com

imdb.com

10,000 20,000 30,000 40,000 50,000 60,000 70,000

M03_STIN7167_03_SE_C03_pp026-050.indd 30 25/10/16 12:13 PM

Pie Chart

Pie charts are another way to draw the distribution of a categorical variable. A pie chart shows the distribution of a categorical variable as wedges of a circle. It has also become popular in the business press to remove the center of the disk in a pie chart, leaving a doughnut chart. The area of each wedge is proportional to the count in a category. Large wedges indicate categories with large relative frequencies. These two charts are equivalent except for visual appearance, sharing strengths and weaknesses. We emphasize pie charts, but doughnut charts convey the same information similarly.

Pie charts convey immediately how the whole divides into shares, which makes these a common choice when illustrating, for example, market shares or sources of revenue within a company, as in Figure 3.5.

pie chart A display that uses wedges of a circle to show the distribution of a categorical variable.

FIGURE 3.4 Not every chart with bars shows the distribu- tion of a categorical vari- able. This chart tracks sales, in thousands of vehicles, at Ford’s Lincoln division.

150

175

125

100

75

50

25

0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Sales

Data from: The Wall Street Journal

FIGURE 3.5 Pie chart and doughnut chart of the annual earnings at Hewlett-Packard, in millions of dollars.

Pie charts are also good for showing whether the relative frequency of a category is near 1>2, 1>4, or 1>8 because we’re used to cutting pies into two, four, or eight slices. For example, in Figure 3.5, you can quickly see that about half of HP’s net revenue came from personal systems (computers) and printing. At the time, HP was preparing to split these divisions from the others that serve business customers.

Pie charts are less useful than bar charts if we want to compare the actual counts. People are better at comparing lengths of bars than comparing the angles

3.2 CHARTS OF CATEGORICAL DATA 31

Net Revenue at HP (in millions)

Software, $3,933Finance, $3,498

Printing, $22,979

Enterprise Systems, $27,814

Personal Systems, $34,303

Enterprise Services, $22,398

Data from: Hewlett-Packard’s 2014 Annual Report

Software 3%Finance 3%

Printing 20%

Enterprise Systems

24%

Enterprise Services

20%

Personal Systems

30%

doughnut chart A pie chart with the center of the disk removed.

M03_STIN7167_03_SE_C03_pp026-050.indd 31 25/10/16 12:14 PM

32 CHAPTER 3 Describing Categorical Data

of wedges.3 Unless the slices align conveniently, it’s hard to compare the sizes of the categories. Can you tell from Figure 3.6 whether RecipeSource or Google gen- erates more visitors? Comparisons such as this are easier to make in a bar chart.

3 See W. S. Cleveland and R. McGill, “Graphical Perception: Theory, Experimentation and Application to the Development of Graphical Methods,” Journal of the American Statistical Association (1984), 531–554.

Hits from Top 10 Hosts

msn.com

yahoo.com

google.com recipesource.com

aol.com

iwon.com

atwola.com

bmezine.com

daily-blessings.com imdb.com

FIGURE 3.6 Pie chart of the top 10 hosts.

caution Because they slice the whole thing into portions, pie charts make it easy to overlook what is being divided.

It is easy to look at the pie chart in Figure 3.6 and come away thinking that 25% of visitors to Amazon come from MSN. Sure, MSN makes up one-quarter of the pie. It’s just that this pie summarizes fewer than 16% of the visits. The pie chart looks very different if we add back the Other category or the typed-in hits.

In Figure 3.7 the chart on the left shows that the top 10 hosts generate slightly more than 25% of the host-generated visitors to Amazon. More come from small sites. A manager who forgets this fact might be in for a big sur- prise. The pie chart on the right makes you realize that any one of the large hosts generates only a small share of the traffic at Amazon.

other hosts

yahoo.com

google.com

recipesource.com

atwola.com

daily-blessings.com

aol.com iwon.com

bmezine.com

Visits from Hosts

msn.com

imdb.com

All Visits to Amazon

imdb.com

yahoo.com google.com

recipesource.com aol.com iwon.com atwola.com bmezine.com daily-blessings.com

other hosts

typed

msn.com

FIGURE 3.7 Pie charts with large categories.

Which of these pie charts is the best? As with choosing a bar chart, the mo- tivation determines the best choice. Figure 3.6 emphasizes the relative sizes of the major hosts; it’s best suited to our motivating question. What about the choice between a bar chart and a pie chart? For this analysis, we could use

M03_STIN7167_03_SE_C03_pp026-050.indd 32 25/10/16 12:14 PM

either the bar chart in Figure 3.1 or the pie chart in Figure 3.6. Because it’s easier to compare the sizes of categories using a bar chart, we prefer bar charts. That said, pie charts are popular, so make sure you can interpret these figures.

What Do You Think? Consider this situation: A manager has partitioned the company’s sales into six districts: North, East, South, Midwest, West, and international. What graph or table would you use to make these points in a presentation for management?

a. A figure that shows that slightly more than half of all sales are made in the West district

b. A figure that shows that sales topped $10 million in every district4

3.3 ❘ THE AREA PRINCIPLE There’s flexibility in the mechanics of a bar chart, from picking the categories to laying out the bars. One aspect of a bar chart, however, is not open to choice. Displays of data must obey a fundamental rule called the area principle. The area principle says that the area occupied by a part of the graph should corre- spond to the amount of data it represents. Violations of the area principle are a common way to mislead with statistics.

The bar charts we’ve shown obey the area principle. The bars have the same width, so their areas are proportional to the counts from each host. By presenting the relative sizes accurately, bar charts make comparisons easy and natural. Fig- ure 3.1 shows that MSN generates almost twice as many visits as RecipeSource and reminds you that RecipeSource generated almost as many hits as Google.

How do you violate the area principle? It is more common than you might think. News articles often decorate charts to attract attention. Often, the dec- oration sacrifices accuracy because the plot violates the area principle. For instance, Figure 3.8 shows the value of wine exports from the United States.5

area principle The area of a plot that shows data should be proportional to the amount of data.

4 Use a table (there are only two numbers) or a pie chart for part a. Pie charts emphasize the breakdown of the total into pieces and make it easy to see that more than half of the total sales are in the western region. For part b, use a bar chart because bar charts show the values rather than the relative sizes. Every bar would be long enough to reach past a grid line at $10 million. 5 Data from USA Today, November 1, 2006. With only five numbers to show, it might make more sense to list the values, but readers tend to skip over articles that are dense with numbers.

ITALY

42.5

NETHER LANDS

34.4JAPAN 82.8

CANADA

146.8 UNITED

KINGDOM

150.3

Where we send our best grapes Total U.S. wine exports (value in millions of dollars):

FIGURE 3.8 Decorated graphics may ignore the area principle.

3.3 THE AREA PRINCIPLE 33

M03_STIN7167_03_SE_C03_pp026-050.indd 33 25/10/16 12:14 PM

34 CHAPTER 3 Describing Categorical Data

Where’s the baseline for this chart? Is the amount proportional to the num- ber of wine bottles? The chart shows bottles on top of labeled boxes of various sizes and shapes.

The bar chart in Figure 3.9 shows the same data but obeys the area prin- ciple. It is less attractive than its counterpart in Figure 3.8, but it is accurate. For example, you can tell that exports to Italy are slightly larger than exports to the Netherlands.

$0

$20

$40

$60

$80

$100

$120

$140

$160

Wine Exports

Un ite

d Ki

ng do

m Ca

na da

Ja pa

n Ita

ly

N et

he rla

nd s

V al

u e (m

ill io

n s)

FIGURE 3.9 Bar charts must respect the area principle.

(a) (b) 65,000

60,000

55,000

50,000

45,000

40,000

35,000

30,000

Ce nt

ra l

No rth

ea st

So ut

h W

es t

Region

C o

u n t

C o

u n t

60,000

50,000

40,000

30,000

20,000

0

10,000

Ce nt

ra l

No rth

ea st

So ut

h W

es t

Region FIGURE 3.10 Region of visitors to Amazon.

Another common violation of the area principle occurs when the baseline of a bar chart is not at zero. Take a look at Figure 3.10(a). This bar chart iden- tifies where in the United States the 188,996 Amazon shoppers are from. At first glance, it appears as if the South sends several times as many shoppers to Amazon as the Central, Northeast, or West regions. That’s misleading. The baseline in this bar chart is not at zero; it is at 30,000. As a result, this chart violates the area principle because the sizes of the bars are not proportional to the counts. This scaling is helpful for seeing differences among the North- east, Central and West regions, but it exaggerates the number coming from the South. Figure 3.10(b) shows the same data with the baseline at zero. We can see that the number of visitors from each of the other regions is about two-thirds the number from the South.

M03_STIN7167_03_SE_C03_pp026-050.indd 34 25/10/16 12:14 PM

You can respect the area principle and still be artistic. Just because a chart is not a bar chart or a pie chart doesn’t mean that it’s dishonest. For example, the plots in Figure 3.11 divide net sales at Lockheed Martin into its main lines of business.

The chart on the left labels a grid with photos associated with the four main business lines of Lockheed. This chart obeys the area principle. Each box in the grid represents 1% of its sales. The adjacent pie chart shows the same data. Which grabs your attention?

4M ANALYTICS 3.1 ROLLING OVER

MOTIVATION ▶ STATE THE QUESTION Authorities enter a description of every fatal automobile ac- cident into the Fatality Analysis Reporting System (FARS). Once obscure, FARS became well known when reporters from the New York Times discovered something unusual. Their tools: a question, some data, and a bar chart.

The question arose anecdotally. Numerous accidents seemed associated with vehicles rolling over. Were these reports coincidental, or was something system- atic and dangerous happening on the highways? For this example, we’ll put you in the place of a curious reporter following up on a lead that has important im- plications for the businesses involved. We’ll use data for the year 2000 because that’s when this issue came to the public’s attention.

News reports suggested some types of cars were more prone to rollover accidents than others. Most of the reported incidents involved SUVs, but are all SUVs dangerous? If some types of cars are more prone to these dangerous accidents, the manufacturer is going to have a real problem. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Identify the data and come up with a plan for how you will use the data. For this example, we will extract from FARS those accidents for which the primary cause was a rollover. We’ll also stick to accidents on interstate highways.

The rows of the data table are accidents resulting in rollovers in 2000. The one column of interest is the model of the car involved in the accident. Once we have the data, we plan to use a frequency table that includes percentages and a bar chart to show the results. ◀

MECHANICS ▶ DO THE ANALYSIS FARS reports 1,024 fatal accidents on interstates in which the primary event was a rollover. The accidents include 189 different types of cars. Of these, 180 models were involved in fewer than 20 accidents each. We will combine these into a category called Other. Here’s the frequency table, with the names sorted

3.3 THE AREA PRINCIPLE 35

Aeronautics 31% Information 31% Electronics 20% Space 18%

Data from: Lockheed-Martin Annual Report in 2011

Aeronautics

Electronics

Information

Space

FIGURE 3.11 Alternative charts of earnings at Lockheed.

M03_STIN7167_03_SE_C03_pp026-050.indd 35 25/10/16 12:14 PM

36 CHAPTER 3 Describing Categorical Data

alphabetically. (As in the example of Web visits, most of the rows land in the Other category, and we have not shown it in this bar chart.)

Model Count Percentage

4-Runner 34 3.3

Bronco/Bronco II/Explorer 122 11.9

Cherokee 25 2.4

Chevrolet C, K, R, V 26 2.5

E-Series Van/Econoline 22 2.1

F-Series Pickup 47 4.6

Large Truck 36 3.5

Ranger 32 3.1

S-10 Blazer 40 3.9

Other 640 62.5

Total 1,024

0 20 40 60 80 100 120 140

Bronco/Bronco II/Explorer

Fatal Rollover Accidents in 2000

F-Series pickup

S-10 Blazer

Large Truck (CBE)

4-Runner

Ranger

Chevrolet C, K, R, V

Cherokee

E-Series Van/Econoline

The bar chart sorted by count from largest to smallest shows that the Ford Bronco (or Explorer) had the most fatal rollovers on interstates in 2000. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS Data from FARS in 2000 show that Ford Broncos (and the comparable Ex- plorer) were involved in more fatal rollover accidents on interstate highways than any other model. Ford Broncos were involved in more than twice as many rollovers as the next-closest model (the Ford F-series truck) and more than three times as many as the similar Chevy Blazer.

Be honest about any caveats or assumptions you have made. These results could also be explained by popularity. Perhaps more Broncos show up in rollover acci- dents simply because there are more Broncos being driven on interstate highways. Perhaps Broncos are driven more aggressively. We cannot resolve these ques- tions from the data in FARS alone, but this bar chart begs for an explanation. ◀

The explanation of the bar chart in Example 3.1 came several years later and caused a bitter dispute between Ford and Bridgestone, maker of the Fire- stone tires that were standard equipment on Ford Broncos and Explorers. Ford claimed that the Firestone tires caused the problem, not the vehicle. The dispute led to a massive recall of more than 13 million Firestone tires.

M03_STIN7167_03_SE_C03_pp026-050.indd 36 25/10/16 12:14 PM

4M ANALYTICS 3.2 SELLING SMARTPHONES

MOTIVATION ▶ STATE THE QUESTION Apple, Google, and Research in Motion (RIM) were the leading competitors in the smartphone market in 2012. Apple makes the iPhone, Google promotes vari- ous phones using the Android system, and RIM sells the Blackberry.

RIM has dominated the business market with its Blackberry line, but has that success held up to the intense competition from Apple and Google? ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH This task requires us to compare counts of a categorical variable observed at two points in time. The counts being compared measure the same thing at different points in time. The following table shows sales in millions of smartphones in the United States in 2012 and 2016.

Type 2012 2016

Android 60.7 109.9

iPhone 36.8 89.2

Blackberry 19.0 2.1

Microsoft 5.5 5.8

One categorical variable indicates the type of phone being used in 2012, and the second categorical variable indicates the type of phone being used in 2016. We plan to use both pie charts and bar charts to contrast the shares, and pick the plot that most clearly shows the differences in the two years. ◀

MECHANICS ▶ DO THE ANALYSIS You often need to consider several plots in order to decide which style presents the data in a way that best addresses the problem at hand. You might decide to show the data several ways.

2012 122 million

2016 207 million

Microsoft 4%

Blackberry 16%

iPhone 30%

Android 50%

Android 53%

iPhone 43%

Blackberry 1%

Microsoft 3%

3.3 THE AREA PRINCIPLE 37

M03_STIN7167_03_SE_C03_pp026-050.indd 37 25/10/16 12:14 PM

38 CHAPTER 3 Describing Categorical Data

For example, these two pie charts emphasize market shares within each year. We can see that Blackberry phones were popular in 2012, but most of these us- ers had switched to something else by 2016. The charts suggest that the number of iPhone subscribers grew substantially, whereas the number using Android phones did not increase very much. To show the growth of the market, the sec- ond pie chart is larger so that the combined chart obeys the area principle.

In contrast, the following paired bar chart combines two bar charts. The bars representing smartphone sales in the two years are next to each other for each type of phone, making it easier to see the change of each brand.

Smartphone Subscribers

M ill

io n s

S o

ld

Android iPhone Blackberry Microsoft 0.0

20.0

40.0

60.0

80.0

100.0

120.0 2012

2016

This plot tells a different story from these counts. Although the iPhone grabbed a larger share than it had in 2012, the number of Android subscribers grew almost as much as the number of iPhone subscribers.

Which plot is better? It depends. The two pie charts emphasize comparisons of shares within the years, whereas the bar chart emphasizes the growth of each type. Both are helpful. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS The number of Android and iPhone subscribers increased dramatically from 2012 to 2016. The once-popular (particularly in business) Blackberry has almost vanished. Microsoft retains a toehold in the market but has not been able to expand.

It is worth mentioning that this analysis only looks at these leading competi- tors. Other phones also compete in this market but have yet to attract many subscribers. ◀

3.4 ❘ MODE AND MEDIAN Plots are the best summaries of categorical data, but you may need a more compact summary. Compact summaries are handy when you need a quick way to compare categorical variables, such as variables measured at two points in time.

The mode of a categorical variable is the most common category, the cat- egory with the highest frequency. The mode labels the longest bar in a bar chart or the widest slice in a pie chart. In a Pareto chart, the mode is the first category shown. If two or more categories tie for the highest frequency, the data are said to be bimodal (in the case of two) or multimodal (more than two). For example, among the visitors to Amazon, the modal behavior (the most common) is to type amazon.com into the browser.

mode The mode of a categorical variable is the most common category.

M03_STIN7167_03_SE_C03_pp026-050.indd 38 25/10/16 12:14 PM

Ordinal data offer another summary, the median, that is not available unless the data can be put into order. The median of an ordinal variable is the label of the category of the middle observation when you sort the values. If you have an even number of items, choose the category on either side of the middle of the sorted list as the median. For instance, letter grades in courses are ordinal; someone who gets an A did better than someone who got a B. In a class with 13 students, the sorted grades might look like

A A A A A A B B B B C C C

The most common grade, the mode, is an A, but the median grade is a B. The median does not match the mode in this example. Most students got an A, with fewer getting a B and fewer still getting a C. The median lies in the middle of the list of grades, but that does not make it the most common.

The bar chart in Figure 3.12 shows another use of letter grades. These grades were assigned to summarize and contrast the performance of 80 real estate investment trusts (REITs) that own office properties.

median The median of an ordinal variable is the category of the middle observation of the sorted values.

0

5

10

15

20

25

30

35

C o

u n t

Performance Grade BA C D F

FIGURE 3.12 Ratings of the performance of 80 real estate investment trusts.

As published by Forbes, none of these REITs rated an A. The most com- mon grade, the mode, is a C. The median grade, in the middle of the ordered grades, is also a C. Investors in these trusts would certainly like for both the median and the mode of the performance ratings to increase.

Best Practices

■ Use a bar chart to show the frequencies of a cat- egorical variable. Order the categories either alphabetically or by size. The bars can be ori- ented either horizontally or vertically.

■ Use a pie chart to show the proportions of a cat- egorical variable. Arrange the slices (if you can) to make differences in the sizes more recogniz- able. A pie chart is a good way to show that one category makes up more than half of the total.

■ Keep the baseline of a bar chart at zero. Mov- ing it elsewhere leads to a plot that violates the area principle.

■ Preserve the ordering of an ordinal variable. Arrange the bars on the basis of the order im- plied by the categories, not the frequencies.

■ Respect the area principle. The relative size of a bar or slice should match the count of the asso- ciated category in the data relative to the total number of cases.

■ Show the best plots to answer the motivating question. You may have looked at several plots when you analyzed your data, but that does not mean you have to show them all to someone else. Choose the best plot to make your point.

■ Label your chart to show the categories and indi- cate whether some have been combined or omit- ted. Name the bars in a bar chart and slices in a pie chart. If you have omitted some of the cases, make sure the label of the plot defines the collection that is summarized.

BEST PRACTICES 39

M03_STIN7167_03_SE_C03_pp026-050.indd 39 25/10/16 12:14 PM

40 CHAPTER 3 Describing Categorical Data

Pitfalls

■ Avoid elaborate plots that may be deceptive. Modern software makes it easy to add an ex- tra dimension to punch up the impact of a plot. For instance, this 3-D pie chart shows the top six hosts from the Amazon example.

Referring Sites

msn.com

iwon.com aol.com

recipesource .com

google.com

yahoo.com

Showing the pie on a slant violates the area principle and makes it harder to compare the shares, the principal feature of data that a pie chart ought to show.

■ Do not show too many categories. A bar chart or pie chart with too many categories might conceal the more important categories. In these cases, group other categories together and be sure to let your audience know how you did this.

■ Do not put ordinal data in a pie chart. Because of the circular arrangement, pie charts are not well suited to ordinal data because the order gets lost. The pie chart shown at the top of the next column summarizes grades given to Ben Bernanke, former chairman of the Federal Reserve, in a survey of 45 econ- omists. The pie separates the big slices but hides the message.

A 33%

F 13%

B 40%

D 5%

C 9%

Grade for Ben Bernanke

Data from: "Economists Expect Shifting Work Force" The Wall Street Journal. February 11, 2010.

■ Do not carelessly round data. Do you see the problem with the following pie chart? Evi- dently, the author rounded the shares to inte- gers at some point but then forgot to check that the rounded values add up to 100%.

Samsung 20%

Apple 15%

Nokia 14%

Global Smartphone Shipments

HTC 11%

RIM 10%

Others 31%

Data from: "Samsung outshines Apple in Smartphone Shipments, Market Share," CNET News. November 4, 2011

3.1 Analytics in Excel: Rolling Over

Open the file 03_4m_rollover.csv in Excel. The file has two columns. The first column lists the model of the vehicle involved in a rollover accident. The second column gives the number of times that each model experienced a fatal rollover. The rows of the work- sheet are ordered alphabetically by the model name.

There are a variety of ways to identify the models with 20 or fewer accidents, but this can be done

by sorting the rows of the spreadsheet. Place the cursor in the data region and use the command Data + Sorts…. In the resulting dialog, instruct Excel to sort the rows using the counts in the sec- ond column from largest to smallest. After sorting, the first few lines of the worksheet should look like the following.

M03_STIN7167_03_SE_C03_pp026-050.indd 40 25/10/16 12:14 PM

3.2 ANALYTICS IN EXCEL: SELLING SMARTPHONES 41

3.2 Analytics in Excel: Selling Smartphones

Open the file 03_4m_smartphone.csv in Excel. The resulting worksheet is very small and appears as follows.

To construct the pie chart for 2012, select the range A2:C5. Use the menu sequence Insert + Chart and select the Pie option. To modify the chart, such as to show percentages or add a title, click on the chart and then select an option from the Quick Layout menu or Chart Design tab. You can also right-click directly on a chart to access a menu of tools that modify the chart.

To show the pie chart for 2016, click on the chart. Excel highlights the data in the worksheet like this.

To show the pie chart for 2016, drag one of the corners at the left of the blue data region to the right, so that just column C is selected. The pie chart then updates and shows the values for 2016.

To obtain the stacked bar chart that compares the frequencies in 2012 and 2016, select the range A1:C5. Use the menu commands Insert + Chart and select the Column layout for a clustered vertical bar chart or the Bar layout for horizontal bars.

To focus on models with 20 or more accidents, add two more columns to the worksheet. First copy the range A1:B10 to C1, and then add the two formulas shown below. The resulting table reproduces the fre- quencies shown in the example.

To produce the bar chart, select the range C1:D10, excluding the counts of the other models and the total count. Use the menu command Insert + Chart and pick the bar chart option. By default, Excel will or- der the bars in the opposite order from the bar chart shown in the text. To reorder the bars, sort the range C1:D9 so that the frequencies in D2:D9 are increasing.

M03_STIN7167_03_SE_C03_pp026-050.indd 41 25/10/16 12:14 PM

42 CHAPTER 3 Describing Categorical Data

Software Hints

EXCEL To build bar charts starting from the raw data, use pivot tables to create a frequency table, following the menu commands (see the Excel help for assistance with pivot tables)

Insert 7 Pivot Table (report)

Once you have the frequency distribution, use the Chart commands to build a bar chart or a pie chart. The categories shown in the bar chart appear in the order listed in the spreadsheet. If you’d like some other ar- rangement of the categories in the chart, rearrange the summary rows in the spreadsheet, and the chart will be redrawn. XLSTAT simplifies this process because it avoids the need to build a pivot table. Put the name of the variable in the first row of the data range; this name will then label the output. Then use the command

XLSTAT 7 Visualizing data 7 Univariate plots

to obtain a dialog for building the bar chart (or pie chart). If your data consist of text, fill the range for qualitative data. If you want a pie chart or relative frequencies, click the options tab and indicate your preferences.

MINITAB EXPRESS If your data use numerical codes for categories, you must first convert the numbers into text. That will keep you from averaging zip codes! Use the Data 7 Recode 7 To text . . . commands to convert the data in a column into labels that define a categorical variable.

The commands for bar charts and pie charts are next to each other on the Graphs menu.

For a bar chart, follow the commands

Graphs 7 Bar chart

and then fill the dialog with the name of the variable. (Notice that you can produce clustered versions of

bar charts using this dialog as well.) To make a pie chart, follow the commands

Graphs 7 Pie chart

The output for both bar charts and pie charts shows the chart and the associated frequencies.

JMP If you have used numbers to represent categori- cal data, you must tell JMP that these numbers are categories, not counts. A panel at the left of JMP’s spreadsheet lists the columns. If you click on the symbol next to the column name, a pop-up dialog allows you to change numerical data (called con- tinuous by JMP) to categorical data (either ordinal or nominal).

To get the frequency table of a categorical vari- able, use the

Analyze 7 Distribution

command and fill the dialog with the categorical variable of interest. You also will see a bar chart, but the bars are drawn next to each other without gaps.

To obtain a better bar chart and a pie chart, follow the menu items

Graph 7 Chart

and place the name of the variable in the Categories, X, Levels field, and click the OK button. That gets you a bar chart. To get a pie chart, use the pop-up menu (click on the red triangle above the bar chart) to change the type of plot. Other options let you mod- ify the chart by using another pop-up menu.

JMP includes an interactive table-building function. Access this command by following the menu items

Tables 7 Tabulate

This procedure becomes more useful when summa- rizing two categorical variables at once.

CHAPTER SUMMARY

Frequency tables display the distribution of a cate- gorical variable by showing the counts associated with each category. Relative frequencies show the propor- tions or percentages. Bar charts summarize graphi- cally the counts of the categories, and pie charts or doughnut charts summarize the proportions of data in the categories. Doughnut charts are pie charts with the center removed. These charts must obey the area principle. The area principle requires that the share of

the plot region devoted to a category is proportional to its frequency in the data. A bar chart arranged with the categories ordered by size is sometimes called a Pareto chart. When showing the bar chart for an ordinal vari- able, keep the labels sorted in their natural order rather than by frequency. The mode of a categorical variable is the most frequently occurring category. The median of an ordinal variable is the value in the middle of the sorted list of all the groups.

M03_STIN7167_03_SE_C03_pp026-050.indd 42 25/10/16 12:14 PM

EXERCISES 43

■ Key Terms area principle, 33 bar chart, 28 distribution, 27 doughnut chart, 31

frequency table, 27 median, 39 mode, 38 Pareto chart, 29

pie chart, 31 relative frequency, 28 variation in data, 27

■ Objectives • Create, describe, and interpret the distribution of

a categorical variable and link this distribution to variation. The distribution could be expressed in  frequencies (counts) or relative frequencies (percentages).

• Choose an appropriate plot (bar chart or pie chart) that shows the distribution of a categorical variable.

• Recognize the value of combining rare categories when summarizing a categorical variable.

• Follow the area principle as a guide when preparing displays of data or interpreting the graphs of others.

• Identify the mode (categorical variable) and me- dian (ordinal variable) from the distribution of a categorical variable.

■ About the Data The data on Internet shopping in this chapter come from ComScore, one of many companies that gather data on Web surfing. They get the data from computers that people receive for free in return for letting Com- Score monitor how they surf the Internet. These visits cover the time period from September through Decem- ber 2002. An interesting research project would be to see whether the distribution has changed since then.

An important aspect of these data is the prevalence of visits from small, specialized Web sites. Imagine how the bar charts of hosts shown in this chapter would look if we tried to show all 11,142 hosts! This

long tail (a term coined by Chris Anderson in 2004) is prominent in online sales of books, music, and video. For example, a typical Barnes & Noble car- ries 130,000 titles. That’s a lot, but more than half of Amazon’s book sales come from books that are not among its top 130,000 titles. A physical bookstore lacks the space to carry so many titles, but an online, virtual bookstore has plenty of room to stock them.

The data on real estate investment trusts (REITs) comprise a portion of a data table posted on forbes. com in 2008. Sources of other examples are noted in the text.

Mix and Match

Match these descriptions of variables to a bar chart, Pareto chart, pie chart, or frequency table. Some are counts and others are shares. If not given, indicate whether you plan to show the frequencies or relative frequencies.

1. Proportion of automobiles sold during the last quar- ter by eight major manufacturers

2. Number of different types of defects found in com- puter equipment

3. Number of cash-back coupons returned by purchas- ers of cameras, computers, and stereo equipment

4. Counts of the type of automobile involved in police traffic stops

5. Destinations for travelers leaving the United States and heading abroad

6. Reason for customers hanging up when calling a computer help desk

7. Excuses given for not finishing homework assign- ments on time

EXERCISES

8. Brand of computer chosen by new students of a large university

9. Share of software purchases devoted to games, office work, and design

10. Customer choices of compact, zoom, and high-end digital cameras

11. Number of cellular telephone customers rating their service as Poor, OK, Good, and Great

12. Share of bank loans that are in good standing, 30 days past due, and in default

True/False

Mark True or False. If you believe that a statement is false, briefly say why you think it is false.

13. Charts are better than frequency tables as summaries of the distribution of a categorical variable.

14. If all of the bars in a bar chart have the same length, then the categorical variable shown in the bar chart has no variation.

M03_STIN7167_03_SE_C03_pp026-050.indd 43 25/10/16 12:14 PM

44 CHAPTER 3 Describing Categorical Data

15. The frequency of a category is the dollar value of the observations in that group.

16. A relative frequency is the number of observations belonging to a category.

17. Use a bar chart to show frequencies and a pie chart to show shares of a categorical variable that is not ordinal.

18. The area principle says that the proportion of the area of a graph devoted to a category must match the number of cases found in the category.

19. Put the labels of the categories in order in a bar chart when showing the frequencies of an ordinal variable.

20. The bar chart of a recoded categorical variable has the same number of bars as the bar chart of the original categorical variable.

21. A Pareto chart puts the modal category first.

22. The median of a categorical variable is the category with the most items.

Think About It

23. These two pie charts show the origin of cars bought by customers who traded in domestic and Asian models. For example, among those trading in a car produced by a domestic manufacturer, 29% pur- chased a car made by an Asian manufacturer. What’s the message of these two pie charts?

Domestic Trade-in

European 2%

Asian 29%

Domestic 69%

Asian Trade-in

European 4%

Asian 78%

Domestic 18%

24. To reduce energy consumption, many governments around the world have considered legislation to phase out incandescent light bulbs. The goal is to move toward compact fluorescent bulbs and other more efficient sources of lighting. Can this really make much difference? The California Energy Commission reported the following summary of household electricity use. Based on this chart, can more efficient lighting have much impact on household energy use?

Household Uses of Electricity

Lighting 37%

Others 30%

Cooling 14%

Ventilation 11%

Refrigeration 8%

25. This plot shows the holdings, in billions of dollars, of U.S. Treasury bonds in five Asian countries. Is this a bar chart in the sense of this chapter, or is it a chart of a table of five numbers that uses bars?

$0

$100

$200

$300

$400

$500

$600

$700

$800

Holdings of U.S. Treasury Bonds

$900

$1000

$1100

$1200

Japan China South Korea

Country

(b ill

io n

s)

Taiwan Hong Kong

26. The Wall Street Journal on January 19, 2016, used a chart resembling the following one to illustrate the rising amounts that Netflix spends to provide streaming content online. Is this the bar chart of a categorical variable in the sense of this chapter, or is it a timeplot that uses bars to show the data?

M03_STIN7167_03_SE_C03_pp026-050.indd 44 25/10/16 12:14 PM

2011 2012 2013 2014 2015

$0.00

$2.00

$4.00

$6.00

$8.00

$10.00

$12.00 Costs for Streaming Content billions of dollars

27. This table summarizes results of a survey of the pur- chasing habits of 1,800 13- to 25-year-olds in the United States in 2006, a much sought-after group of consum- ers.6 Each row of the table indicates the percentage of participants in the survey who say the following:

Participants Say That . . . Percentage

They consider a company’s social commit- ment when deciding where to shop.

69

Given equal price and quality, they are likely to switch brands to support a cause.

89

They are more likely to pay attention to messages from companies deeply com- mitted to a cause.

74

They consider a company’s social commit- ment when recommending products to friends.

66

(a) Would it be appropriate to summarize these percentages in a single pie chart?

(b) What type of chart would you recommend for these four percentages?

28. The following table summarizes findings from a study done by Apple concerning working conditions in factories operated by its suppliers.7 Under pressure from consumers concerned about working conditions in China, Apple visited its suppliers and evaluated the labor conditions. Several problem areas surfaced.

Issue Percentage

Supplier does not limit working hours per week.

62

Supplier does not comply with rules for hazardous substances

32

Supplier does not follow Apple’s safety standards

35

(a) Would it be appropriate to summarize these responses together in a single bar chart?

(b) What sort of chart would you recommend to summarize these results?

29. These percentages summarize the responses of 530 senior executives when asked the question, “Which strategies have proved successful in promoting a data-driven culture in your organization?” Do these data belong together in a pie chart? State why or why not.

Strategy Percentage

Top-down guidance and/or mandates from executives

49

Promotion of data-sharing practices 48

Increased availability of training in data analytics

40

Communication of the benefits of data-driven decision making

40

Recruitment of additional data analysts

17

From the 2013 report Fostering a Data-Driven Culture prepared by the Economist Intelligence Unit.

30. This table summarizes shares of the U.S. wireless telephone market in 2011. To summarize these percentages together in a pie chart, what needs to be added?

Company Share (%)

AT&T 32

Verizon 31

Sprint 17

T-Mobile 11

31. What would happen if we were to generate a bar chart for a column that measures the amount spent by the last 200 customers at a convenience store?

32. Suppose that the purchase amounts in the previous question were coded as Small if less than $5, Typical if between $5 and $20, and Large if more than $20. Would a bar chart be useful? How should the categories be displayed?

33. Describe the bar chart of a categorical variable for which 900 of the 1,000 rows of data are in one cat- egory and the remaining 100 are distributed evenly among five categories.

34. Sketch by hand the pie chart of the data in the previ- ous question.

35. A data table with 100 rows includes a categorical variable that has five levels. Each level has 20 rows. Describe the bar chart.

36. Compare the bar chart of the variable described in the previous question with the pie chart of the same data. Which makes it easier to see that each category has the same number of cases?

37. A categorical variable has two values, male and female. Would you prefer to see a bar chart, a pie chart, or a frequency table? Why?

6 “Up Front: The Big Picture,” Business Week, November 6, 2006. 7 “Data from The Wall Street Journal, “Apple Navigates China Maze,” January 14, 2011.”

EXERCISES 45

M03_STIN7167_03_SE_C03_pp026-050.indd 45 25/10/16 12:14 PM

46 CHAPTER 3 Describing Categorical Data

38. A categorical variable denotes the state of residence for customers to a Web site. Would you prefer to see a bar chart, a pie chart, or a frequency table? Would it be useful to combine some of the categories? Why?

39. This pie chart shows the type of college attended by the CEOs of America’s 50 largest companies.8 Which category is the mode? Which category is the median?

Public 26

Other private

14

Ivy League 4

Other 6

40. The next two charts show other features of the col- leges attended by CEOs of the 50 largest companies

in the United States. Should the enrollment data be shown in a pie chart? What is the modal location? What is the median size?

41. Auto manufacturers are sensitive to the color preferences of consumers. Would they like to know the modal preference or the median preference?

42. A consumer group surveyed customers and asked them to rate the quality of service delivered by their wireless phone company. A publication of this group summarizes the rating provided by each customer as Excellent, Good, Fair, or Poor. What does it mean for a provider to get a modal rating of Excellent or a median rating of Excellent?

43. A Web designer wants to display the revenue earned in various market segments by Samsung Electronics. She plans to show the segments as stars in a night sky, with larger stars for segments with larger revenue. Clicking on a star would pop up details about that segment. This table gives the quarterly revenue of Samsung in four segments in 2009.

Segment Revenue

(billions of dollars)

Consumer electronics $10.5

Cell phones $ 9.1

Computer chips $ 6.3

LCD panels $ 5.7

In order to obey the area principle, how large should she make the stars for these segments? (Hint: the area of a circle is p times the square of the radius of the circle.)

44. A magazine wants a graphic that presents the sales of cars in the United States, broken down by the major manufacturers. To get the attention of readers, the graphic will show a car representing the sales of each of these manufacturers. How should the artist pick the size of the graphics for each manufacturer?

Manufacturer Vehicle Sales in Nov 2011

General Motors 180,402

Ford 166,441

Chrysler 107,172

Toyota 137,960

Honda 83,925

Nissan 85,182

You Do It

Bold names shown with a question identify the data table for the problem.

45. Soft drinks This table summarizes the number of cases (a case has 24 bottles or cans) of different types

8 Data from “Any College Will Do,” The Wall Street Journal, September 18, 2006.

By current enrollment size

N.A. - Not applicable

10,000 to 19,999

12

20,000 or more

20

2,000 to 9,999

10

N.A. 5

Less than 2,000

3

By geography

Midwest 13

East 15

South 13

West 4

N.A. 5

M03_STIN7167_03_SE_C03_pp026-050.indd 46 25/10/16 12:14 PM

of carbonated soft drinks sold in the United States in 2014. (From Beverage Digest, March 26, 2015.)

Brand Company Cases (millions)

Coke Coca-Cola 1548.8

Pepsi-Cola Pepsi 774.4

Diet Coke Coca-Cola 748

Mountain Dew Pepsi 607.2

Dr Pepper Pepsi 598.4

Sprite Coca-Cola 528

Diet Pepsi Pepsi 378.4

Fanta Coca-Cola 193.6

Diet Mountain Dew Pepsi 176

Coke Zero Coca-Cola 158.4

(a) Describe your impression of the underlying data table. Do you think that it has a row for every case of soft drinks that was sold?

(b) Prepare a chart of these data to be used in a pre- sentation. The plot should emphasize the share of the market held by the different brands.

(c) Prepare a chart of these data that compares the share of the market held by each of the two companies. Is this chart different from the chart used in part (b)? If not, explain how you could use the chart in part (a).

(d) Prepare a chart that contrasts the amounts of diet and regular soft drinks sold.

46. Snack bars This table summarizes the market shares of the top 10 leading brands of snack bars that were sold in 2012 in the United States.

Brand Name Manufacturer Market Share (%)

Kellogg’s Kellogg 19.1

Nature Valley General Mills 12.0

Clif Clif Bar 5.7

PowerBar Nestlé 5.3

Fiber One General Mills 4.8

Quaker Chewy PepsiCo Inc. 4.7

Betty Crocker General Mills 3.9

Luna Bar Clif Bar 3.5

ZonePerfect Abbott 3.3

Kashi Kellogg 2.7

(a) Prepare a chart that shows the shares of the snack-bar market by brand name. Be sure to account for the shares of other brands that are not shown in the table.

(b) Show the shares of the total market by the six manufacturers shown in the table. Again, account for the shares of other manufacturers. (Assume that smaller brands are produced by dif- ferent manufacturers.)

(c) This market totaled $6 billion in sales in 2012. Prepare a chart that contrasts the amounts sold by the different manufacturers.

47. Teen research These data summarize results of a survey that asked teens and adults to identify the single top government research priority. (From Business Week, December 6, 2004.)

Research Area Adults

(%) Teens (%)

Alternative energy 33 23

Stem-cell research 12 4

Water purification 11 20

Space exploration 3 15

(a) Compare the responses of adults to those of teens using two pie charts. Be sure to add a category for the Other responses. How does the presence of a large group of Other responses affect the use of these charts?

(b) Compare the responses using a bar chart. Do you have to show the other category when using the bar chart?

(c) If each respondent in the survey listed several areas for research, would it then be appropriate to form a pie chart of the values in a column of the table?

48. Media A report of the Kaiser Foundation in the New York Times shows the presence of different types of media in the bedrooms of children living at home.

Medium

Age (years)

2–4 (%)

5–7 (%)

8–13 (%)

14–18 (%)

TV 26 39 65 65

Radio 14 18 34 38

Video game 7 18 47 42

Cable TV 10 18 28 32

Computer 4 9 23 19

(a) Explain why it would not be appropriate to form a pie chart using the values in any row of this table. Would it be appropriate to use a pie chart to summarize the percentages within a column?

(b) Prepare a chart that compares the types of media available to children from 5 to 7 years old to those available to adolescents from 14 to 18 years old. What seems to be a key point made by the chart?

(c) For a company that makes video game consoles, what age range do these data suggest marketing should target for new sales?

49. Sweeteners These data summarize the market share for artificial sweeteners held by the major brands in 2005 and 2010. Prepare a chart that shows the shares each year and the brands that gained the most share from 2005 to 2010.

EXERCISES 47

M03_STIN7167_03_SE_C03_pp026-050.indd 47 25/10/16 12:14 PM

48 CHAPTER 3 Describing Categorical Data

Brand 2005 (%)

2010 (%)

Splenda 52 45

Sweet’n Low 15 11

Equal 18 7

Stevia/Truvia 0 14

Others 15 23

50. Internet This table summarizes results from a sur- vey of American households with Internet access that asks, “What kind of Internet connection do you use at home?” The named connection types are those that were widely available in 2007. These exclude choices that became available more recently. (The results for 2007 are from Forrester, and those for 2013 are from the American Community Survey from the U.S. Census.)

Connection Type 2007 (%)

2013 (%)

Cable modem 33 43

DSL 33 21

Fiber 1 8

Satellite 3 5

Dial-up 26 1

Other 4 22

(a) Prepare a chart that shows the shares in both years as well as identifies the technologies that are growing or fading.

(b) Does the growth in the share of satellite connec- tions mean that more households used satellite connections in 2013 than in 2007?

51. Women The U.S. Census Bureau occasionally surveys business owners and learns which firms are predomi- nantly owned by women. These data are a portion of the results reported in 2010 from the 2007 Survey of Business Owners. This table reports the number of women-owned businesses in 10 industries, along with the counts of all businesses in these industries.

Industry Women-Owned All

Agriculture 26,783 258,846

Arts and entertainment 376,259 1,236,352

Construction 268,809 3,414,399

Finance 200,473 1,019,055

Manufacturing 113,495 615,936

Mining, oil and gas 18,313 122,470

Retail trade 918,657 2,672,892

Services 1,096,422 3,790,845

Transportation 142,566 1,253,553

Wholesale trade 133,462 733,577

(a) Make a bar chart to display the counts of busi- nesses owned by women and a second bar chart that shows the counts for all businesses, and label them correctly. Do these plots suggest that women- owned businesses concentrate in some industries?

(b) Construct a single plot that directly shows which industries have the highest concentration of women-owned businesses.

52. Startups The Wall Street Journal (January 12, 2011) reported the results of a survey of reasons given by 18- to 34-year-olds for not starting their own businesses. The 872 respondents gave the following reasons.

Reason Percentage

Being unable to get a loan or credit 41%

Facing too much risk and uncertainty 31%

Not knowing how to run a business 19%

Lacking the skills to start and run a business

13%

Having no role models to show the way 12%

(a) Would it be appropriate to show these percent- ages in a pie chart?

(b) Do these responses define ordinal data? (c) What display would you recommend for showing

these data?

53. Absences A company reported the following summary of the days during the last year that were missed by its employees.

Reason Days

Illness 4,463

Planned medical leave 2,571

Vacation 15,632

Training 5,532

(a) The only absences that cannot be anticipated are those due to illness; the others are scheduled absences. Provide a figure, table, or chart that shows that most absences are anticipated.

(b) Show a Pareto chart of these data.

54. Chocolate In 2010, Kraft Foods bought the English chocolate maker Cadbury. The following table sum- marizes the global market share for chocolate snacks held by six companies in 2009, before the acquisition.

Company Market Share (%)

Mars 14.6

Nestlé 12.6

Kraft 8.3

Ferrero 7.3

Cadbury 6.9

Hershey 6.7

M03_STIN7167_03_SE_C03_pp026-050.indd 48 25/10/16 12:14 PM

(a) Use two pie charts to compare the market before and after the acquisition of Cadbury. Which choco- late maker is the modal brand after the acquisition?

(a) The market-share percentages are based on the dollar value of sales. If we instead counted the number of snack items sold, would it be possible for Hershey to be the modal brand?

55. Outsourcing A survey of 1,000 large U.S. compa- nies conducted by Forrester Research asked their plans for outsourcing high-level white-collar jobs to countries outside the United States. This table shows the percentage that indicated a certain action. The data for 2003 are real; the data for 2008 are speculative. (From Business Week, December 12, 2004.)

Plan to 2003 (%)

2008 (%)

Do virtually no white-collar offshoring

63 46

Offshore white-collar work to some extent

33 44

Offshore any white-collar work possible

4 10

(a) Summarize the responses to these questions using two pie charts. Are pie charts appropriate for summarizing these percentages?

(b) Summarize the data using a bar chart with side- by-side columns.

(c) Compare the impression conveyed by these plots of any anticipated trend in outsourcing of white- collar jobs.

(d) Does the mode differ from the median for either distribution?

56. Consoles This table summarizes the number of game consoles sold in three areas as of the end of 2010. The numbers are in millions.

Japan Europe US

Playstation 3 6.3 19.7 12

Wii 11.5 24.9 30

Xbox 360 1.4 13.7 18.6

(a) Present pie charts that show the market share within each geographic region. Size the pie charts to obey the area principle so that someone can see, for example, that the European market is larger than the market in Japan.

(b) Use a clustered bar chart to present these data, keeping the bars for each type of console in a group. What does this bar chart show that is not so evident in the pie charts?

57. Car Auction Each row of data indicates the make of vehicles sold at auction in the United States in 2010. The data table describes 1,884 vehicles.

(a) Using software, tabulate the frequencies of the makes of cars.

(b) What is the mode of the make of car? (c) Generate a pie chart that shows all of the relative

frequencies. What’s a problem with this chart? (d) Recode the variable that identifies the make

of the car so that the resulting pie chart distinguishes the shares of the top five makes. Draw the new pie chart.

58. Browser An online retailer identified the Web browser being used by a sample of 1,447 shoppers to its online site in 2012. Each row in the data table for this question identifies the browser being used by a shopper. Previously in 2010, 63% of shoppers used Internet Explorer, 24% Firefox, 5% Chrome, 4% Safari, and 4% Opera. (a) Using software, tabulate the frequency of the

choice of browser used by these shoppers. (b) Present a bar chart and a pie chart of these

frequencies. Which is more useful to compare the distribution of these frequencies to those observed in 2010?

(c) Do you see any changes in the distribution of the choice of browser?

59. 4M ANALYTICS: Growth Industries

The U.S. Department of Commerce tracks the number of workers employed in various industries in the United States. It summarizes these data in the Statistical Abstract of the United States. This regularly appearing volume con- tains a rich collection of tables and charts that describe the country.

The data shown in the following table appear in Table 620 in the 2012 edition, available online at www.census .gov. This table summarizes a categorical variable from each of two data tables. These categorical variables identify the type of industry that employs the worker. Each row in the underlying data tables represents an employee, either in 2000 or 2010.

Industry 2000 2010

Agriculture 2,464 2,206

Construction 9,931 9,077

Manufacturing 19,644 14,081

Transportation 7,380 7,134

Education 11,255 13,155

Health 14,933 18,907

Hospitality 11,186 12,530

Retail trade 15,763 15,934

Finance 9374 9350

Professional services 13,649 15,253

Each value shown in this frequency table is a rounded count given in thousands. For example, about 2,464,000 were em- ployed in agriculture and related industries in 2000.

EXERCISES 49

M03_STIN7167_03_SE_C03_pp026-050.indd 49 25/10/16 12:14 PM

50 CHAPTER 3 Describing Categorical Data

Motivation

(a) A firm that sells business insurance that covers the health of employees is interested in how it should allocate its sales force over the different industries. Why would this firm be interested in a table such as this?

Method

(b) What type of plot would you recommend to show the distribution of the workforce over industries in each year?

(c) Management is particularly interested in know- ing which industries have experienced the most growth in the number of employees. What graph do you recommend to show changes in the size of the workforce in the different industries?

Mechanics

(d) Prepare the chart that you chose in part (c). Ex- plain the order you chose to show the categories.

(e) How does the chart highlight industries that have smaller employment in 2000 than in 2010?

(f) Do you think you should show all 10 categories, or should some rows be combined or set aside?

Message

(g) What message does your chart convey about the changing nature of employment?

(h) By focusing on the growth and using a single chart, what’s hidden?

60. 4M ANALYTICS: Web purchases

The data for this exercise derive from a sample of 5,000 visits to amazon.com, and, as in the example in this chapter, the data for each visit list the host that linked to Amazon. The key difference from the example in the chapter is that these data are from the fall of 2014. The data omit sessions that did not come from a host (i.e., were directly typed). In 2014, about 60% of visits did not use a host.

Compare the hosts found in these data to those that are summarized in the example of this chapter. What important similarities and differences do you find, and explain briefly why you think these aspects of the comparison are important. Structure your answer us- ing the 4M paradigm, starting with a motivating ques- tion that you plan to answer, followed by your choice of approach, the mechanics, and then a concluding message.

M03_STIN7167_03_SE_C03_pp026-050.indd 50 25/10/16 12:14 PM

51

Describing Numerical Data

A STANDARD iPHONE 6 COMES WITH 16 GIGABYTES (GB) OF MEMORY. That sounds like a lot until you start saving pictures and adding apps. There might not be much room left for your favorite music. If you manage to set aside, say, 4 GB for music, how many songs will you be able to have with you all the time? Will you have to be picky, or is there room for thousands of songs?

Questions like these are easy to answer when there’s no variation. The answer is a simple yes or no. If every song requires the same amount of space, say 4 MB, then 1,000 will fit. If each song takes up 5 MB, then you will have to make do with fewer.

It is a lot harder to answer these questions with real songs. Digital recordings of songs don’t take the same amount of space. The longer the song, the more space it requires. There’s variation in the amount of space needed to store the songs.

The presence of variation affects business decisions as well. Customers with the same type of loan pay back different amounts each month. Packages labeled to weigh a certain amount in fact weigh different amounts. Dollar values of sales vary from day to day and from store to store. Managing loans, manufacturing, sales, and just about every other business activity requires that we understand variation.

This chapTer surveys meThods used To display and summarize numerical daTa. These methods characterize what is typical of data, as well as how data vary around this typical value. The description of a numerical variable offers more possibilities than were available for categorical data. This chapter considers graphical approaches and the use of numerical summaries based on averaging and sorting the data.

4c h a p t e r

4.1 SUMMARIES OF NUMERICAL VARIABLES

4.2 HISTOGRAMS

4.3 BOXPLOTS

4.4 SHAPE OF A DISTRIBUTION

4.5 EPILOG

CHAPTER SUMMARY

M04_STIN7167_03_SE_C04_pp051-079.indd 51 18/11/16 3:09 PM

4.1 ❘ SUMMARIES OF NUMERICAL VARIABLES

There’s a lot of variation in the size of files that store digital music. Table 4.1 describes five digital recordings of songs randomly picked from a friend’s computer. These songs vary in length, with longer songs taking up more room than shorter songs.

TABLE 4.1 Description of several digital recordings of songs.

Song Artist Genre Size (MB) Length (sec)

My Friends D. Williams Alternative 3.83 247

Up the Road E. Clapton Rock 5.62 378

Jericho k.d. lang Folk 3.48 225

Dirty Blvd. L. Reed Rock 3.22 209

Nothingman Pearl Jam Rock 4.25 275

To answer our question about fitting music files on an iPhone, we need to understand the typical size of the file for a song as well as the variation around this typical value. You can see that there is variation from reading the few rows of data in Table 4.1, but to understand this variation, we need to consider all of the data.

We’ll first summarize the sizes using just a few numbers. These summary num- bers that characterize a variable are known as summary statistics. The first type of summary statistic that we will describe comes from putting the data in order, and the second type comes from averaging. We’ll then relate these summary statistics to graphs that show a more complete view of a numerical variable.

Percentiles

Percentiles are found by sorting numerical data into ascending order. Several percentiles are commonly used as summary statistics. Of these, the median is the best-known. We met the median in Chapter 3 as a summary of an ordinal variable. It’s also useful as a summary of a numerical variable. The median is the 50th percentile, the value that falls in the middle when we sort the observations of a numerical variable. Half of the observed values are smaller than the median, and half are larger. If we have an even number of cases (n is even), the median is the average of the two values in the middle. The units of the median match those of the data.

Two other percentiles are also commonly used, and when combined these two provide a measure of variation. The 25th percentile and 75th percentile are known as the lower and upper quartiles. You can think of the lower and upper quartiles as medians of the lower half and the upper half of the data. One- quarter of the data are smaller than the lower quartile, and one-quarter of the data are larger than the upper quartile. Hence, half of the data fall between the quartiles. The distance from the lower quartile to the upper quartile is known as the interquartile range (IQR). Because it’s based on percentiles, the IQR is a natural summary of the amount of variation to accompany the median.

The minimum and the maximum of the data are also percentiles. The maxi- mum is the 100th percentile, and the minimum is the the 0th percentile. The range of a variable is the difference between its minimum and maximum val- ues. The range is another measure of variation derived from percentiles.

We can illustrate these percentiles using the lengths of the songs on our friend’s computer. The sorted sizes of all of these songs are:

0.148, 0.246, 0.396, … , 3.501, 3.501, 3.502, 3.502, … 17.878, 21.383, 21.622 1st 2d 3d 1991st 1992d 1993d 1994th 3,982d 3,983d 3,984th

summary statistic Numerical summary of a variable, such as its median or average.

median Value in the middle of a sorted list of numbers; the 50th percentile.

quartile The 25th or 75th percentile.

interquartile range (IQR) Distance between the 25th and 75th percentiles.

range Distance between the smallest and largest values.

52

M04_STIN7167_03_SE_C04_pp051-079.indd 52 18/11/16 3:09 PM

4.1 SUMMARIES OF NUMERICAL VARIABLES 53

Because n = 3,984 is even, the median is the average of the middle two sizes, 3.5015 MB. Half of these songs require less than 3.5015 MB, and half need more. The lower quartile of the sizes is 2.85 MB, and the upper quartile is 4.32 MB: 25% of the songs require less than 2.85 MB, and 25% of the songs require more than 4.32 MB. The middle half of the songs between the quartiles use between 2.85 and 4.32 MB. The difference between these quartiles is the interquartile range.

IQR = 75th percentile - 25th percentile = 4.32 MB - 2.85 MB = 1.47 MB

The range is considerably larger than the interquartile range. These songs range in size from 0.148 MB all the way up to 21.622 MB. The largest song is almost 150 times bigger than the smallest. Because the range depends on only the most extreme values of a numerical variable, it is less often used as a mea- sure of variation. The collection of five summary percentiles

(minimum, lower quartile, median, upper quartile, and maximum)

is known as the five-number summary of a numerical variable.

five-number summary The minimum, lower quartile, median, upper quartile, and maximum.

1 The median is $107,817,000 2 The upper quartile is $511,281,000, and the lower quartile is $60,157,000. 3 The interquartile range is +511,281,000 - +60,157,000 = +451,124,000 and the range is +862,909,000 - +883,000 = +862,026,000.

What Do You Think? The following table gives the amount of baggage fees charged to passengers in 2014 by the top 15 domestic airlines in the United States. The amounts are in thousands of dollars. The data are from the U.S. Department of Transportation.

Airline Baggage Fees

Delta $862,909

United 651,857

American 574,430

US Airways 511,281

Spirit 241,867

Frontier 144,853

Alaska 120,630

Allegiant 107,817

JetBlue 83,516

Hawaiian 76,097

Southwest 73,170

Virgin America 60,157

Sun Country 15,869

Island Air Hawaii 3,972

Mesa 883

a. Find the median amount charged in 2014. (Notice that the table orders air- lines by amount.)1

b. Find the upper and lower quartiles.2

c. Find the IQR and range.3

M04_STIN7167_03_SE_C04_pp051-079.indd 53 25/10/16 12:15 PM

54 CHAPTER 4 Describing Numerical Data

Averages

The most familiar summary statistic is the mean, or average, calculated by adding up the data and dividing by n, the number of values. The mean, like the median, has the same units as the data. Be sure to include the units when- ever you discuss the mean.

The mean gives us an opportunity to introduce notation that you will see on other occasions. Throughout this book, the symbol y stands for the variable that we are summarizing, the variable of interest. In this example, our inter- est lies in the sizes of songs, so the symbol y stands for the size of a song in megabytes. To distinguish the sizes of songs, we add subscripts. For instance, y1 denotes the size of the song in the first row of the data table, and y2 denotes the size of the song in the second row. Because n stands for the number of rows in the data table, yn denotes the size of the song in the last row. With this convention, the formula for the mean is

y = y1 + y2 + g + yn

n

Those three dots stand for the sizes of songs between the second and the last. The y with a line over it is pronounced “y bar.” In general, a bar over a symbol or name denotes the mean of a variable. The average size of the 3,984 songs works out to be

y = 3.0649 + 3.9017 + g + 3.5347

3,984 = 3.7794 MB

On average, each song needs a bit less than 3.8 MB. Notice that the average size of a song is larger than the median size 3.5 MB.

Averaging also produces a summary statistic that measures the amount of variation in a numerical variable. Consider how far each observation is from y. These differences from the mean are called deviations. Cases for which y

i

is less than y produce negative deviations ( yi - y 6 0). Cases for which yi is greater than y produce positive deviations ( yi - y 7 0). The average squared deviation from the mean is called the variance. We use the symbol s2 for the variance, with the exponent 2 to remind you that we square the deviations before averaging them.

s2 = ( y1 - y)2 + ( y2 - y)2 + g + ( yn - y)2

n - 1

If the data spread out far from y, then s2 is large. If the data pack tightly around y, then s2 is small. It’s natural to wonder why the divisor is n - 1 rather than n. The complete explanation appears in Chapter 15, but notice that dividing by n - 1 makes s2 a little bit bigger than dividing by n. That turns out to be just the right adjustment. For now, even though the divisor is not n, think of s2 as the average squared deviation from y. (With n = 3,984, the difference between dividing by n or n - 1 is small.)

It is important that we square the deviations before adding them. Regard- less of our data, the sum of the deviations is exactly zero. Positive and nega- tive deviations cancel. (In this chapter’s Behind the Math section, a unique property of the mean explains why.) To avoid the cancellation when finding the variance, we square each deviation before adding them. Squared devia- tions are greater than or equal to zero, so their sum won’t be zero unless all of the data are the same. Squaring also emphasizes larger deviations, so we’ll have to watch for the effects of unusually large or small values. (Using abso- lute values of the deviations rather than squares would also avoid the cancel- lation, but the standard is to square the deviations.)

tip

(p. 71)

mean The average, the ratio of the sum of the values to the number of values. Shown as a symbol with a line over it, as in y

variance The average of the squared deviations from the mean, denoted by the symbol s2.

M04_STIN7167_03_SE_C04_pp051-079.indd 54 25/10/16 12:15 PM

4.1 SUMMARIES OF NUMERICAL VARIABLES 55

We generally use software to compute s2, but a small example reviews the calculations and reinforces the point that you have to square the deviations before adding them. Table 4.2 shows the calculations needed to find s2 for a data table with n = 5 values.

TABLE 4.2 The calculation of the variance s2 of five values.

Value Deviation Squared

y1 4 4 - 7 = -3 9

y2 0 0 - 7 = -7 49

y3 8 8 - 7 = 1 1

y4 11 11 - 7 = 4 16

y5 12 12 - 7 = 5 25

Sum 35 0 100

The sum of the values is 35, so y = 35/5 = 7. Deviations from the mean fill the third column. The sum of the deviations equals 0, which is a handy way to check the calculations. The last column holds the squared deviations, which sum to 100. So, s2 = 100/(5 - 1) = 25.

Whatever the units of the original data, the variance has squared units: s2 is the average squared deviation from the mean. The variance of the size of the songs is 2.584 MB2, about 212 “squared megabytes.” How should we interpret that?

To obtain a summary we can interpret, we convert s2 into a statistic that has the same units as the data. To measure variation in the original units, take the square root of the variance. The result is called the standard deviation. In symbols, the formula for the standard deviation is

s = 2s2 The square root puts s on the scale of the data. The standard deviation of the songs is s = 22.584 MB2 < 1.607 MB. We use the standard deviation so often that we’ll frequently abbreviate it SD.

4M ANALYTICS 4.1 MAKING M&Ms

MOTIVATION ▶ STATE THE QUESTION Highly automated manufacturing processes often con- vey the impression that every item is the same. Think about the last time that you ate a handful of M&M choc- olate candies. They all seem to be the same size, but are they?

Mars, the manufacturer, would like to know how many pieces are needed to fill a bag that is labeled to weigh 1.6 ounces. M&Ms are packaged by a highly automated process. If every piece weighs the same amount, the sys- tem can count them out. If there’s variation, a different type of packaging is necessary. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH For this question, we need to know the amount of variation relative to the typical size. If the SD is tiny relative to the mean, then the candies will appear identical even though there’s actually some variation present. The ratio of the SD to the mean is known as the coefficient of variation.

cv = s y

standard deviation A measure of variability found by taking the square root of the variance. It is abbreviated as SD in text and identified by the symbol s in formulas.

coefficient of variation The ratio of the SD to the mean, s/y

_

M04_STIN7167_03_SE_C04_pp051-079.indd 55 25/10/16 12:15 PM

56 CHAPTER 4 Describing Numerical Data

The coefficient of variation cv has no units because the mean and SD share the same units. The units cancel. The coefficient of variation is more useful for data with a positive mean that’s not too close to 0. Values of cv larger than 1 suggest a distribution with considerable variation relative to the typical size. Values of cv near 0 indicate relatively small variation. In these cases, even though the SD may be large in absolute terms, it is small relative to the mean.

The data for this example are weights of 72 plain chocolate M&Ms taken from several packages. The data table has just one column. Each observation is one piece of candy, and the single data column is the weight (in grams). We are interested in the coefficient of variation cv of the weights. ◀

MECHANICS ▶ DO THE ANALYSIS The mean weight is y = 0.86 gram. The SD is s = 0.04 gram, so the coeffi- cient of variation is small.

cv = 0.04/0.86 < 0.0465

To determine how many M&Ms are needed to fill a package, we need to convert the package weight into grams. One ounce is equivalent to 28.35 grams, so a package that weighs 1.6 ounces has 1.6 * 28.35 = 45.36 grams. If every M&M were to weigh exactly 0.86 gram, it would take 45.36/0.86 < 52.74, about 53, pieces to obtain the sought package weight. The small size of cv suggests this rule ought to work pretty well. (See Statistics in Action: Modeling Sampling Variation to find out how well it actually does.) ◀

MESSAGE ▶ SUMMARIZE THE RESULTS The average weight of chocolate M&Ms in a sample of 72 is 0.86 gram, with SD 0.04 gram. The weights of M&Ms are not identical, but the SD is quite small compared to the mean, producing a coefficient of variation of about 5% (0.04/0.86 < 5%). (No wonder they look so similar when I’m eating them.)

Be sure to answer the question that motivated the analysis. These data suggest that 53 pieces are usually enough to fill the bag labeled to contain 1.6 ounces. Because of the variation, however, the packaging system may need to add a few more to allow for pieces that weigh less. As a result, the presence of variation adds to the cost of packaging. That’s typical: Variation generally adds to the cost of manu- facturing and explains why so many manufacturers work hard to remove varia- tion from their processes. (That’s the whole objective of Six Sigma manufacturing developed at Motorola and widely implemented at GE and other companies.) ◀

4 The unit of all of these summary statistics is that of the data: days. 5 The mean is 10 days and the median is 7. The large number of days missed by one employee makes the mean larger. 6 s = 1146.67 < 12.1 days and IQR = 12 - 2 = 10 days.

What Do You Think? A manager recorded the number of days employees were out sick from the office during the last year. For n = 7 employees, the counts are 0, 0, 4, 7, 11, 13, and 35 days.

a. What are the units of y and s? Are these different from the units of the median and interquartile range?4

b. Find y and the median. Which is larger? Why?5

c. Find s and the IQR. [Hint: The five-number summary is (0, 2, 7, 12, 35) and s2 = 146.67.]6

M04_STIN7167_03_SE_C04_pp051-079.indd 56 25/10/16 12:15 PM

4.2 HISTOGRAMS 57

4.2 ❘ HISTOGRAMS Summary statistics are concise but do not reveal other important properties of numerical variables. A plot can show more and help us appreciate what summary statistics like the mean and standard deviation tell us.

As a first step to building such a plot, we begin by grouping songs of similar size. We do this by partitioning the sizes into adjacent, equal-width intervals and then count the number in each interval. These intervals and counts sum- marize the distribution of the numerical variable. We could show the distribu- tion in a table as we did for categorical data, but it’s more useful to see a plot.

The most common plot of the distribution of a numerical variable is a histogram. A histogram shows the counts as the heights of bars that satisfy the area principle. The area of each bar is proportional to the number of cases that fall within its interval. The intervals slice up all possible values of a numerical variable. Unlike a bar chart, a histogram positions the bars next to each other, with no gaps. A relative frequency histogram displays the propor- tion or percentage of cases in each interval instead of the count.

The histogram in Figure 4.1 shows the distribution of the sizes of all 3,984 songs on our friend’s computer. Tall bars identify sizes that occur frequently; short bars identify sizes that seldom occur. We draw histograms so that cases that fall on the boundary between two intervals get counted as part of the interval on the right-hand side. Hence in Figure 4.1, the first interval includes songs from 1 MB up to but not including 2 MB (1 6 y 6 2).

distribution The distribution of a numerical variable is the collection of possible values and their frequencies (just as for categorical variables).

histogram A plot of the distribution of a numerical variable as counts of occurrences within adjacent intervals.

FIGURE 4.1 Histogram of song sizes.

The clump of tall bars tells us that most songs require 2 to 6 MB. The tallest bar indicates that nearly 1,500 of the 3,984 songs require between 3 and 4 MB. That’s about where we may have expected the songs to cluster because the mean y < 3.8 MB and the median is 3.5 MB.

Histograms versus Bar Charts

It is worthwhile to compare a histogram to a bar chart. The two are similar because they both display the distribution of a variable. Histograms describe numerical data, such as the sizes of these songs, that can take on any value. Bar charts show counts of discrete categories and are poorly suited for numerical data. A bar chart of the sizes of these songs would need to show thousands of bars, one for each size found in the data.

M04_STIN7167_03_SE_C04_pp051-079.indd 57 25/10/16 12:15 PM

58 CHAPTER 4 Describing Numerical Data

An explicit example clarifies the differences. The histogram on the left of Figure 4.2 gives the weights, in grams, of 48 packages of M&M candies. Five packages, for example, weigh between 48 and 49 grams (48 … Weight 6 49). Although the packages are all labeled to contain 1.69 ounces, about 48 grams, the weights of the packages vary. The bars of the histogram are adjacent because any weight within the limits of the intervals is possible. (The bar for the heaviest category is separated from the others since none of the packages weighs between 52 and 53 grams.)

47 48 49 50 Weight (grams)

51 52 53

C o

u n t 15

20

10

5

54

47 48 49 50 51 53

0

5

10

15

20

C o

u n t

Weight Group

FIGURE 4.2 Histogram and bar chart of M&M package weights.

The right side of Figure 4.2 shows a bar chart derived from the same data. To make this bar chart, we truncated the package weights to two digits, keep- ing only the integer weights and discarding any fraction of a gram. The bar chart describes the resulting ordinal variable.

The two descriptions of these data are clearly similar. Both show a peak (mode) around 50 grams, for instance. The differences, however, are impor- tant. The bars in the bar chart are not placed next to each other as in the histogram. Because of the way we constructed the ordinal variable, you can- not have a category between 49 and 50. Also notice that the bars are equally spaced, even though the values are not. The last two categories (labeled 51 and 53) are not so close as the others. For an ordinal variable, we only use the order of the categories, not the size of any differences.

The White Space Rule

We skipped over an important attribute of the recorded songs: A few songs are much larger than the others. The bars toward the far right of the histogram in Figure 4.1 become so short that you cannot see them. Two songs require between 21 and 22 MB, but you need to trust the x-axis to know they are here. (Length aside, these two songs are very different: a selection from Gershwin’s Porgy and Bess and “Whipping Post” by the Allman Brothers.)

The very long songs are outlying values, or outliers. Outliers affect every statistic. An outlier can be the most informative part of your data, or it can be an error. Because outliers are separated from the rest of the data, they often determine the appearance of a plot. The graph in Figure 4.1 devotes more than half of its area to show the 36 songs that are larger than 10 MB. More than half of the plot shows less than 1% of the songs.

The White Space Rule says that if your plot is mostly white space—like the histogram in Figure 4.1—then you may not be able to see important pat- terns in the data. When most of the data occupy a small part of a plot, there’s more going on than meets the eye. In this example, we want to get a better look at the distribution of the many songs below 10 MB. We cannot see what’s happening among these and see the outliers at the same time.

Width of Histogram Intervals

What might be happening that we cannot see in Figure 4.1? Figure 4.3 shows three histograms of songs smaller than 10 MB. The histograms use intervals of different lengths to group the songs.

outlier A data value that is isolated far away from the majority of cases.

White Space Rule If a plot has too much white space, refocus it.

tip

M04_STIN7167_03_SE_C04_pp051-079.indd 58 25/10/16 12:15 PM

4.3 BOXPLOTS 59

The intervals that group songs are 1 MB long in the histogram at the top of Figure 4.3 as shown in Figure 4.1. In the second histogram, the intervals are half as long, and in the third histogram, one-quarter as long.

Narrow intervals in a histogram expose details that are smoothed over by wider intervals. For example, consider songs between 2 and 3 MB. A histo- gram with 1-MB intervals piles these together and shows a flat rectangle. Nar- rower intervals reveal that these songs are not evenly spread from 2 to 3 MB; instead, they concentrate toward 3 MB. You can, however, take this process too far. Very narrow intervals that hold just a few songs produce a histogram that looks like a bar chart with too many categories.

What’s the right length for the intervals in a histogram? Most software packages have a built-in rule based on the range of the data and the number of observations that determines the length. In general, the larger the number of observations, the more intervals in the histogram. You should take such rules as suggestions and try several as shown in Figure 4.3.

4.3 ❘ BOXPLOTS Sometimes it’s handy to have a more compact summary of the distribution of a numerical variable, particularly when comparing several variables at once. A boxplot is a graphical summary that shows the five-number summary of a numerical variable in a graph. This schematic of a boxplot identifies the key features.

boxplot A graphic consisting of a box, whiskers, and points that summarize the distribution of a numerical variable using the median and quartiles.

0 1 2 3 4 5 6 7 8 9 10 1 MB

2 3

500

1,000

1,500

C o

u n t

0 1 2 3 4 5 6 7 8 9 10

250

500

750

1/2 MB 2 3

C o

u n t

100

200C o

u n t 300

400

0 1 2 3 4 5 6 7 8 9 10 1/4 MB2 3 FIGURE 4.3 Histograms on finer scales show more detail.

M04_STIN7167_03_SE_C04_pp051-079.indd 59 25/10/16 12:15 PM

60 CHAPTER 4 Describing Numerical Data

Vertical lines locate the median and quartiles. Joining these with horizontal lines forms a box. A vertical line within the box locates the median, and the left and right sides of the box mark the quartiles. Hence, the span of the box locates the middle half of the data, and the length of the box is equal to the IQR. Horizontal lines known as whiskers locate observations that are outside the box, but nearby. A whisker joins any value within 1.5 * IQR of the upper or lower quartile to the box. Distinct symbols identify outliers that are farther away. It is common to find some outliers in a boxplot. By design, most box- plots show roughly 1 to 5% of the observations outside the whiskers. Because we have almost 4,000 songs, we can expect to find quite a few points outside the whiskers in a boxplot of their sizes.

Combining Boxplots with Histograms

A boxplot is a helpful addition to a histogram. It locates the quartiles and median and highlights the presence of outliers. Figure 4.4 adds a boxplot to the histogram of the sizes.

Whisker

1.5 IQR 1.5 IQRIQR

Whisker Outlier

25% Lower Quartile

50% Median

75% Upper Quartile

0 1 2 3 4 5 6 7 8 9 10 11 12 13 Size in MB

14 15 16 17 18 19 20 21

C o

u n t

22

500

1,000

1,500

FIGURE 4.4 Histogram with boxplot of the sizes of songs.

The addition of the boxplot makes it easy to locate the median; the vertical line inside the box indicates that the median lies between 3 and 4 MB. (It is exactly 3.5015 MB.) The relatively short length of the box shows that the data concentrate near the median. The majority of the outliers (songs more than 1.5 * IQR away from the box and shown as individual points in the boxplot) lie to the right of the box. Whereas the short bars of the histogram make it easy to miss outliers, the boxplot draws our attention to them.

The combination of boxplot and histogram reinforces the connection between the median and the distribution of the data. The median is the 50th percentile, in the middle of the sorted values. That means that one-half of the area of the histogram is to the left of the median and one-half of the area is to the right, as shown in Figure 4.5. The shaded half of the histogram to the left of the median holds the smaller songs.

The mean has a different connection to the histogram. Imagine putting the histogram in Figure 4.5 on a balance beam. Where should you put the fulcrum

M04_STIN7167_03_SE_C04_pp051-079.indd 60 25/10/16 12:15 PM

4.3 BOXPLOTS 61

to make the histogram balance? The answer is at the mean. Think of two chil- dren on a seesaw. If one child is heavier than the other, then the heavier child has to sit near the fulcrum unless he wants to sit there with his friend lifted in the air. Values in the histogram that lie far from the center are like children sitting far from the fulcrum; they exert more leverage. The data cannot move, so the fulcrum—the mean—slides over to make the histogram balance.

Now that we have seen the connection of the mean and median to the his- togram, it is easier to appreciate the difference between these two summary statistics. Both provide a notion of the center of the data. The median is at the center of the data in the sense that half of the data lie on either side of it. In contrast, the mean takes account of how far it lies from the rest of the data. The mean is at the center in the sense of balancing. The sum of the deviations of the mean from larger values matches the sum of deviations of the mean from smaller values.

The big outliers in this distribution are long songs. To accommodate these outliers, the fulcrum shifts to the right. Consequently, the mean length 3.8 MB is larger than the median 3.5 MB. Outliers have less effect on the median. Had the Beatles let “Hey Jude” run on for several hours, imagine what such an out- lier would do to the mean. The mean would chase after this one song, growing larger and larger. The median, because it lies at the middle of the sorted list of sizes, would remain where it is, unaffected by the ever-longer song. As a result, outliers exert less influence on the median than the mean. This robustness of the median, or resistance to the effect of an outlier, becomes a liability when it comes to dollars. For variables measured in dollars, we will frequently want to know the mean regardless of outliers. Would you really want a summary statistic that ignores your most profitable customers or the source of your highest costs?

We can also visually connect the interquartile range to the histogram. The middle half of the data lie within the box of the boxplot, as shown in Figure 4.6.

tip

0 1 2 3 4 5 6 7 8 9 10 11 12 13 Size in MB

14 15 16 17 18 19 20 21

C o

u n

t

22

100

200

300

400

FIGURE 4.5 The median splits the area of the histogram in half.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 Size in MB

14 15 16 17 18 19 20 21

C o

u n t

22

100

200

300

400

FIGURE 4.6 The inner 50% of the values lie in the box.

Half of the area of the histogram lies within the span of the box.

M04_STIN7167_03_SE_C04_pp051-079.indd 61 25/10/16 12:15 PM

62 CHAPTER 4 Describing Numerical Data

There is a connection among means, standard deviations, and histograms, but that connection requires that we first understand the shape of the distri- bution of a numerical variable.

9 Five, adding counts in the third and fourth bins.

7 Use the boxplot to locate the median, and then use the length of the box to estimate the IQR. The me- dian offering is about $150 million, and the IQR < +200 million. 8 The largest offering is from $1,000 million (a billion) to $1,100 million. (It was $1 billion, for Zynga).

10 The mean (about $250 million) is larger than the median because of the outliers on the right.

What Do You Think? Twenty-six large companies made their first public stock offering during the last three months of 2011, including Zynga (makers of Farmville), Michael Kors, and Groupon. This histogram and boxplot summarize the value of the initial public offering of stock.

a. Estimate the median and IQR of these data.7

b. Estimate the size of the largest initial offering?8

c. How many of these offerings were valued $200 million up to, but not including, $400 million?9

d. Is the mean offering larger or smaller than the median offering?10

$0

2 4 6 8

10

$200 $400 $600 Millions of Dollars

$800 $1,000

C o

u n t

4.4 ❘ SHAPE OF A DISTRIBUTION Summary statistics describe the center of the distribution of a numerical vari- able (mean, median) and its variation (standard deviation, IQR). Additional statistics describe other features of the distribution, collectively known as the shape of a distribution.

Modes

Most data produce a histogram with a single, rounded peak. This peak identi- fies where the data cluster. The location of this peak is the mode of a numeri- cal variable. The bar at the center of the peak in the histogram in Figure 4.4 shows that the mode of these data lies between 3 and 4 MB. The data become sparser as you move away from a mode, and so the heights of adjacent bars in the histogram steadily decline.

A numerical variable can have several modes because the data may con- centrate in several locations. The histograms of such variables have several distinct peaks. A histogram such as the one shown in Figure 4.4 with a single peak is said to be unimodal; histograms with two distinct peaks are bimodal. Those with three or more are called multimodal. A histogram in which all the bars are approximately the same height is called uniform.

Modes in numerical data are tricky to identify because the number of modes depends on the width of the bars in the histogram. For example, look back at the histograms in Figure 4.3. With intervals that are 1 MB long, the distribution

mode Position of an isolated peak in the histogram. A histogram with one mode is unimodal, two is bimodal, and three or more is multimodal.

uniform A uniform histogram is a flat histogram with bars of roughly equal height.

M04_STIN7167_03_SE_C04_pp051-079.indd 62 25/10/16 12:15 PM

4.4 SHAPE OF A DISTRIBUTION 63

is unimodal. With one-quarter MB intervals, another mode emerges near 5 MB. Narrower intervals show even more. If the bins in a histogram are too narrow, you will find modes all over the place. In contrast, it is easy to identify the mode of a categorical variable. The mode of a categorical variable is the most com- mon label.

As an illustration, let’s consider the distribution of the year in which rock and roll songs in this collection were released. The histogram on the left in Figure 4.7 has narrow intervals and consequently shows many modes; the histogram on the right uses longer intervals and shows two well-defined modes (bimodal). The histogram on the right is smoother because it uses wider intervals. Its bimodal shape begs for an explanation: It seems this listener isn’t fond of music produced near 1980.

Symmetry and Skewness

Another aspect of the shape of the distribution of a numerical variable is sym- metry. A distribution is symmetric if you can fold the histogram along a verti- cal line through the median and have the edges match up, as if the two sides are mirror images as in Figure 4.8. It won’t be a perfect match, but to be sym- metric, the two sides should appear similar.

100

150

C o

u n

t

50

1960 1970 1980 1990 2000

200

300

100

1960 1970 1980 1990 2000

C o

u n t

FIGURE 4.7 Narrow intervals produce many modes, and wide intervals produce fewer modes.

symmetric The histogram of the distribution matches its mirror image.

60

40

20

-3.0 -2.0 -1.0 0.0 1.0 2.0 3.0

Fold along dotted line

A symmetric histogram . . .

60

40

20

-3.0 -2.0 -1.0 0.0 . . . can fold in the middle

so that the two sides almost match.

FIGURE 4.8 Checking for symmetry.

The extremes at the left and right of a histogram where the bars become short locate the tails of the distribution. If one tail of the distribution stretches out farther than the other, the distribution is skewed rather than symmetric.

The distribution of the sizes of songs in Figure 4.4 is right skewed. The right tail of the distribution reaches out farther from the single mode than the left tail. The boxplot confirms this skewness. The whisker and outliers extend more to the right than to the left of the box. The left side of the distribution is compact, with just a few outliers below the left whisker. The half of the songs from the minimum size to the median ranges from near 0 to 3.5 MB, but the right half extends from 3.5 MB all the way to 22 MB. The majority of outli- ers in the boxplot are on the right side when the data are right skewed (larger than typical) and on the left side of the boxplot when the data are left skewed.

tails The left and right sides of a histogram where the bars become short.

skewed An asymmetric histogram is skewed if the heights of the bins gradually decrease toward one side.

M04_STIN7167_03_SE_C04_pp051-079.indd 63 25/10/16 12:15 PM

64 CHAPTER 4 Describing Numerical Data

Data encountered in business are often right skewed. Variables measured in dollars, such as payrolls, income, costs, and sales, are frequently right skewed because the values cannot be negative and because a few are occasionally much larger than the others. Skewness can be more extreme than in the exam- ple of the sizes of songs, as in the example of executive compensation below.

4M ANALYTICS 4.2 EXECUTIVE COMPENSATION

MOTIVATION ▶ STATE THE QUESTION The press devotes a lot of attention to the compen- sation packages of chief executive officers (CEOs) of major companies. Total compensation includes much more than salary, such as bonuses tied to company performance. Let’s exclude those bonuses and consider salaries of CEOs in 2010. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH For data, we will examine the salaries of 1,766 CEOs reported (in thousands of dollars) by Compustat. (Compustat obtains these data from official finan- cial reports that include salary information.) Each row gives the salary of a CEO. We only need one column, the column of salaries.

The data are numerical and we’re interested in how they are distributed. A graph showing a histogram along with a boxplot gives a complete summary. ◀

MECHANICS ▶ DO THE ANALYSIS The distribution of salaries is right skewed and unimodal. The median salary is about $725,000. The average salary is a bit larger at $770,000. The middle half of salaries (those inside the box of the boxplot) ranges from $520,000 to $970,000. The data have several prominent outliers at the right. The largest salary, $3,510,000 (3.51 * $1,000,000), is that of Leslie Moonves, CEO of CBS. In com- parison, the salaries of 1,374 of the CEOs (about 78%) are less than $1 million. ◀

Salary (million)

C o

u n t

100

$0 $1 $2 $3

200

300

400

MESSAGE ▶ SUMMARIZE THE RESULTS When summarizing a distribution, describe its shape, center, and spread without using many technical terms. Also report the symmetry, number of modes, and any gaps or outliers. The annual base salary of CEOs ranges from less than $100,000 into the millions. The median salary is $725,000, with half of the salaries in the range from $520,000 to $970,000. The distribution of salaries is right skewed: Some CEOs make considerably more than the others. Even though more than three-fourths of CEOs have salaries below $1,000,000, a few salaries exceed $3,000,000. ◀

M04_STIN7167_03_SE_C04_pp051-079.indd 64 25/10/16 12:15 PM

4.4 SHAPE OF A DISTRIBUTION 65

11 The median is about $6,000 (centerline of the boxplot), and the IQR (the length of the box) is near 7,200 - 4,700 = +2,500. 12 The histogram is unimodal and a bit skewed to the right. There’s a large outlier at the right. 13 The claim is reasonable because the lower edge of the box, the lower quartile, is a bit less than $5,000.

What Do You Think? The histogram and boxplot presented here summarize tuition (in dollars) charged by 100 public colleges around the United States.

a. Estimate the median and the interquartile range.11

b. Describe the shape of the distribution of tuition. Is it symmetric? Unimodal? What about any outliers?12

c. If someone claims that 25% of these schools charge less than $5,000 for tuition, can that be right?13

C o

u n t

20

15

10

5

2,000 4,000 6,000 8,000 10,000 14,000 Tuition ($)

Bell-Shaped Distributions and the Empirical Rule

If data are symmetric and unimodal with a histogram that has a gently rounded peak at its center, we say that the distribution is bell shaped. For these, a special relationship known as the Empirical Rule connects the mean, standard deviation, and histogram.

The Empirical Rule, illustrated in Figure 4.9, uses the standard deviation s to describe how data that have a bell-shaped distribution cluster around the mean. According to the Empirical Rule, 68% (about two-thirds) of the data lie within one standard deviation of the mean, and 95% of the data lie within two standard deviations. Reaching out farther, the Empirical Rule says that almost all of the data (99.7%, to be precise) fall within three standard devia- tions of the mean.

Interval Percentage of Data

y 2 s to y 1 s 68

y 2 2s to y 1 2s 95

y 2 3s to y 1 3s 99.7

bell shaped A bell-shaped distribution represents data that are symmetric and unimodal.

Empirical Rule 68% of data within 1 SD of mean, 95% within 2 SDs, and almost all within 3 SDs.

68%

95%

99.7%

-3s -2s -1s 0 1s 2s 3s

Visits from Hosts FIGURE 4.9 The Empirical Rule.

M04_STIN7167_03_SE_C04_pp051-079.indd 65 25/10/16 12:15 PM

66 CHAPTER 4 Describing Numerical Data

caution The catch to using the Empirical Rule is that this relationship among y, s, and a histogram works well only when the distribution is bell shaped. In

comparison, the IQR always gives the length of an interval that covers half of the data, but the Empirical Rule is reliable only for bell-shaped data. Does the Empirical Rule describe the variation in the sizes of songs? We can anticipate that the match will be loose at best because the histogram is not exactly bell shaped; it’s skewed to the right. Even so, the range

y - 2s to y + 2s 3.8 - 2 * 1.6 = 0.6 MB 3.8 + 2 * 1.6 = 7 MB

holds 96% of the data. That’s close to what the Empirical Rule suggests. The Empirical Rule is less accurate for the shorter interval. According to the Em- pirical Rule, the sizes of 68% of the songs should fall between

y - s and y + s 3.8 - 1.6 = 2.2 MB 3.8 + 1.6 = 5.4 MB

In the data, however, 84% land in this interval. Although it’s not a perfect description of the distribution of the sizes of these

songs, consider what the Empirical Rule allows us to do with just a mean and standard deviation. We can describe the distribution of the sizes of songs like this: The distribution of the sizes of songs is right skewed with a bell-shaped distribution near the mean of the data. The mean size is 3.8 MB with standard deviation 1.6 MB. If someone knows the Empirical Rule, he or she can imagine what the histogram looks like from two summary statistics ( y and s) and from our hint about skewness. That’s a lot to get from two summary statistics.

The Empirical Rule explains why the IQR is usually bigger than the SD. The IQR is the length of the box in a boxplot, and this interval holds 50% of the data. If the data have a bell-shaped distribution, then the interval from y - s to y + s holds about 68% of the data. Let’s compare these intervals. One interquartile range holds 50% of the data, but it takes an interval of length 2s to hold 68% of the data. Consequently, the IQR is about 35% larger than the SD for bell-shaped distributions. The relationship between the IQR and SD changes for other shapes. If the distribution is skewed, for example, the SD may be larger than the IQR.

Standardizing

The Empirical Rule lets you judge the relative position of an observation in the distribution of a variable with a bell-shaped distribution without needing to see the histogram. For instance, what does it mean when the news reports, “The Dow Jones Industrial Average is down by 100 points today.” Is that 100- point fall in this index of the value of the stock market a big change? If we know that the SD of recent day-to-day changes in the Dow Jones index is 150, then we can see that a 100-point drop is not such a big change because it is only a fraction of a standard deviation. The Empirical Rule suggests that the Dow Jones index changes by 150 points or more on about one-third of days, so a change of 100 points today is not unusual. A similar use of the SD happens in courses that grade “on the curve.” For example, if you score 91 on a test, did you do well? If the scores on the test are bell shaped with mean y = 95 and s = 2 points, don’t expect an A if your professor grades on the curve. A score of 91 on that test is two standard deviations below the mean. In other words, only about 2.5% (1/2 of 5%) of the class scored less than you did.

This notion of measuring size relative to variation is so common that it has a name. A z-score is the ratio of a deviation from the mean divided by the standard deviation.

z = y - y

s

z-score The distance from the mean, counted as a number of standard deviations.

M04_STIN7167_03_SE_C04_pp051-079.indd 66 25/10/16 12:15 PM

4.4 SHAPE OF A DISTRIBUTION 67

Z-scores measure the distance of the cases from the mean in standard deviations; z-scores have no measurement units. Because the numerator and denominator in the ratio that defines a z-score have the same units, divid- ing one by the other cancels the units. Converting data to z-scores is known as standardizing the data. If z = 2, then the corresponding observation is two standard deviations above the mean. Data below the mean have negative z-scores; z = -1.6 means that y lies 1.6 standard deviations below the mean. Investors who own stocks or students in a class like positive z-scores, but golf- ers prefer negative z-scores.

Z-scores are particularly useful when we compare data that have been measured differently. For instance, a 100-point change in the Dow Jones index is not surprising since the magnitude of this change is less than one standard deviation. What about a change of 100 points in Standard & Poor’s 500 index? The Dow Jones index and the S&P index both measure the health of the stock market but are computed differently. We should not compare the two indices directly. We can, however, compare z-scores. The average day-to-day change in both indices is close to zero, but the standard deviations differ. The SD of the Dow Jones index is 150, but the SD of the S&P index is only 15. A 100-point drop in the Dow Jones index has z-score

z = 100 - 0

150 < 0.67

Because the SD of the S&P index is much smaller, a 100-point drop in the S&P index has a z-score that is 10 times larger.

z = 100 - 0

15 < 6.67

The first z-score tells us that a 100-point drop is not uncommon for the Dow Jones index, and the second z-score indicates that a 100-point drop is a huge shift in the S&P index.

standardizing Converting deviations into counts of standard deviations from the mean.

14 Not necessarily. Sales are steady (no trends), so about half of months have sales less than average if the data are bell shaped, and even more are less than the mean if the data are right skewed. 15 A timeplot to check for trends, and a histogram to check if the distribution is bell shaped. 16 z = 1150,000−90,0002>20,000 = -3. The negative sign is important; it tells the manager sales were less than usual. 17 Yes. If the data are bell shaped, the Empirical Rule says that sales in April are unusually low.

What Do You Think? The manager of a retail store gets a summary of the average daily sales for the prior month. The manager uses these data to monitor the performance of advertising and the employees. The store has been open for several years and has a steady history of solid business. In May, the manager learns that sales in April were below average.

a. Should it be a surprise to find that sales in a month are below average?14

b. Before the manager concludes there’s a problem, what plots would clarify the situation?15

c. Sales at this store average $150,000 daily with standard deviation $20,000. If sales were $90,000 in April, what is the z-score for sales in April?16

d. Should the manager be concerned about the level of sales in April?17

M04_STIN7167_03_SE_C04_pp051-079.indd 67 25/10/16 12:15 PM

68 CHAPTER 4 Describing Numerical Data

4.5 ❘ EPILOG What about the motivating question? Did our friend fit 1,000 of her songs into the 4 GB of space left on her iPhone?

Because of variation, not every collection of 1,000 songs will fit. The longest 1,000 songs she owns don’t fit, but those were not the songs she chose to download onto her iPhone. In fact, she squeezed 1,068 songs onto her iPhone, including that long song by the Allman Brothers!

Best Practices

■ Be sure that data are numerical when using histograms and summaries such as the mean and standard deviation. Your telephone num- ber and Social Security number are both com- posed of digits, but that doesn’t make them numerical data. How would you interpret the bars if you put these into a histogram? How would you interpret the average Social Secu- rity number?

■ Summarize the distribution of a numerical vari- able with a graph. A histogram is most com- mon, and a boxplot shows a more concise summary. If you have to use numerical summa- ries, be sure to note the shape of the histogram along with measures of its center and spread of the distribution. Also report on the symmetry, number of modes, and any gaps or outliers.

■ Choose interval widths appropriate to the data when preparing a histogram. Computer pro- grams do a pretty good job of automatically choosing histogram widths. Often, there’s an easy way to adjust the width, sometimes inter- actively. Since it’s easy to do, explore how the width of the bars affects the appearance of the shape of the histogram. If the shape seems consistent over several widths, choose the one you like best. If modes appear, you need to fig- ure out why the data cluster in two or more groups.

■ Scale your plots to show the data, not empty space. In other words, don’t forget the White Space Rule. You should not hide outliers, but it’s also not a good idea to show the extremes and ignore the details that might be hidden in the majority of your data.

■ Anticipate what you will see in a histogram. Think about what the histogram should look

like before you build the plot. If you discover something that you did not expect in the histo- gram, then confirm whether your data measure what you think they do and try to uncover the explanation behind the surprise.

■ Label clearly. Variables should be identified clearly and axes labeled so that a reader knows what the plot displays. Include units with sum- mary statistics like the mean, median, and standard deviation.

■ Check for gaps and multiple modes. Large gaps often mean that the data are not simple enough to be put into a histogram together. The his- togram in Figure 4.10 shows the number of calls handled daily at a call center operated by a bank. This histogram has two modes with a large gap between them.

You might be able to guess why there’s a gap: The histogram mixes data for weekdays with data for weekends. It’s not surprising to find two modes; fewer calls arrive over the weekend. We ought to separate the cases and show two histograms.

50,000 150,000 250,000 350,000

FIGURE 4.10 Number of calls handled daily at a corporate call center.

M04_STIN7167_03_SE_C04_pp051-079.indd 68 25/10/16 12:15 PM

4.2 ANALYTICS IN EXCEL: EXECUTIVE COMPENSATION 69

Pitfalls

■ Do not use the methods of this chapter for cat- egorical variables. This one is worth repeating. Some believe that it’s okay to use these meth- ods with ordinal data (such as from Likert scales), but be careful if you do this.

■ Do not assume that all numerical data have a bell-shaped distribution. If data are summa- rized by only a mean and standard deviation, you may not be seeing the whole picture. Don’t use the Empirical Rule until you verify that the data have a bell-shaped distribution.

■ Do not ignore the presence of outliers. If you do not look at your data and instead rely on summary statistics, particularly measures like the mean and standard deviation, you can be fooled. These statistics are very sensitive to the presence of extremely unusual observations.

■ Do not remove outliers unless you have a good reason. It is easy to get the impression that outliers are bad because of the way they pull the mean, inflate the SD, and make it hard to see the rest of the data. An outlier might be a coding error, but it might also represent your most valuable customer. Think hard before you exclude them.

■ Do not forget to take the square root of a variance. The units of the variance, or aver- age squared deviation around the mean, are the squares of the units of the data. These squared units make the variance hard to in- terpret. By taking the square root, we get the standard deviation, which is a more useful summary.

4.1 Analytics in Excel: M&Ms

Open the data file 04_4m_mandm.csv in Excel. The file has three columns as shown below. This anal- ysis concerns the weights (in grams) in the third column.

Excel offers many ways to obtain summary statis- tics such as those in this example. The direct method that works in all versions of Excel is to add formulas to the worksheet. For this example, we added three for- mulas to the worksheet, as shown below on the right in column F. When evaluated (as shown on the left), these formulas produce the values used in the example.

4.2 Analytics in Excel: Executive Compensation

Open the data file 04_4m_exec_comp_2010.csv. In addition to the salaries, the data file includes the

names of the CEOs and the companies that employ them.

M04_STIN7167_03_SE_C04_pp051-079.indd 69 25/10/16 12:15 PM

70 CHAPTER 4 Describing Numerical Data

You can obtain specific summary statistics such as the median or standard deviation by adding formulas to the worksheet. When several are needed, however, it is more convenient to use the Analysis Toolpak that comes with Excel. If you have not loaded this exten- sion to Excel, use the command Tools + Excel Add Ins… and follow the instructions. (Note that the Analysis ToolPak is only recently available for Mac versions of Excel.)

Use the command Tools + Data Analysis… and select the option Descriptive Statistics to open a dialog box. In the dialog, select the input range using the column option. Be sure the option “Labels in First Row” is checked (it is may not be checked by default). Select the Salary column and click the Summary Statistics item.

By default, Excel puts the resulting summary sta- tistics into a new worksheet.

For a histogram, use the menu command Tools + Data Analysis… + Histogram. Select the input range as done for summary statistics and check the box to have a chart in the output. Excel automati- cally chooses ranges for the bins in the histogram.

Next, to remove the gaps between the bars in the histogram, double click on the bars in the his- togram to open the Format Data Series dialog on the right of the screen. Under Series Options, set the Gap Width slider to 0%. If other ranges for the histogram bars are preferred, type the boundaries of the bin ranges into a separate column in the original worksheet with a label. Select these values in the Bin Range option when constructing the histogram.

Software Hints

Software handles most of the details of building a histogram. For example, what should be done for cases that fall on the boundary between two bins? (For the examples in this chapter, those on the boundary get put in the interval to the right.) Soft- ware also incorporates rules for determining the number of bins. As shown in this chapter, the num- ber of intervals used to build the histogram is impor- tant. You should generally let the software make the first plot, and then try some variations.

Histograms are common in statistics software, but each package does things a little differently. For example, some label the vertical scale with counts whereas others use relative frequencies. Some don’t label the axes at all! Software generally chooses the number and placement of the bins by default. You should explore several widths for the bins to see how this choice affects the appearance of the histogram and any conclusions that you might draw.

EXCEL Excel prefers to make bar charts. You have to work a bit harder to get a proper histogram. Once you have counts for the intervals, you can put these into a bar chart. To remove the gaps between the bars, some versions of Excel allow you to right-click on the plot and change the formatting to set the width between the bars to zero. XLSTAT simplifies this process. Put the name of the variable in the first row of your data range so that this name labels the output. Then fol- low the menu commands

XLSTAT 7 Visualizing data 7 Histograms

Then insert the range of your data in the dialog. To get a boxplot, follow the commands

XLSTAT 7 Describing data 7 Descriptive statistics

Enter the range of your data as before. XLSTAT dis- plays a boxplot of the variable along with summary

M04_STIN7167_03_SE_C04_pp051-079.indd 70 25/10/16 12:15 PM

BEHIND THE MATH 71

statistics that include the median, quartiles, mean, and standard deviation.

You can also obtain summary statistics directly from Excel. For example, the Excel formula

= average(A1:A10)

produces the mean value of the data in the indicated cell range. To find functions that are related to statis- tics, follow the menu commands

Formulas 7 Insert 7 Statistical

to get a list of the relevant functions. In addition to the function average, Excel includes

median, mode, max, min, percentile, quartile, and stdev

The last finds s, the standard deviation.

MINITAB EXPRESS Follow the menu commands

Graphs 7 Histogram 7 Simple

and fill in the dialog box with the name of a numeri- cal variable. Buttons adjacent to the graph allow you to modify the display.

To obtain a boxplot, follow the menu sequence

Graphs 7 Boxplot 7 Simple

and enter the name of the variable from the list shown in the dialog. The boxplot appears vertically oriented with percentiles shown beneath.

To obtain summary statistics, follow the menu items

Statistics 7 Summary Statistics 7 Descriptive Statistics

Enter the name of the variable and click the OK button.

JMP Follow the menu commands

Analyze 7 Distribution

and fill in the name of one or more numerical variables; then click OK. (This command produces a bar chart without spaces between the bars if you supply a cat- egorical variable.) The Hand tool allows you to inter- actively vary the width and position of the bins within the plot itself. Click the red triangle beside the name of the variable in the histogram and choose the item His- togram options to change other aspects of the display.

By default, the command that produces a histo- gram also shows a boxplot of the same data, aligned on the same scale as the histogram for easy compari- son. (You can easily find the median, for instance, in the histogram from the boxplot.) Your boxplot might contain a few extra bells and whistles, such as a dia- mond and red bar, like this one:

If you right-click on the boxplot and uncheck the options Mean Confid Diamond and Shortest Half, then the boxplot will look more like you expect.

A table that shows the basic summary statistics, including the median, quartiles, mean, and standard deviation, appear below the histogram.

BEHIND the MATH

A Unique Property of the Mean The mean y has a special property that explains why it is the balance point of the histogram. The devia- tions show how far each value lies from the mean.

y1 - y, y2 - y, g, yn - y

If you add these deviations from the mean, you get zero. To see that this property holds, rearrange the sum of the deviations like this.

( y1 - y) + ( y2 - y) + g + ( yn - y) = a n

i = 1 yi - n y

= n y - n y = 0

Deviations to the right of the mean (the y’s that are bigger than y) cancel those to the left (the y’s that are less than y). Because the sum is zero, the average deviation from y is also zero. The mean is the unique value with this property. By subtracting y from each value, we’ve converted our original data into a new set of values with mean zero.

M04_STIN7167_03_SE_C04_pp051-079.indd 71 25/10/16 12:15 PM

72 CHAPTER 4 Describing Numerical Data

CHAPTER SUMMARY

Histograms and boxplots display the distributions of numerical variables. The bars of the histogram parti- tion the range of the variable into disjoint intervals. The distribution can be either symmetric or skewed with a long tail reaching out to one side or the other. The histogram can be flat, called uniform; bell shaped with one mode; or split into two (bimodal) or more (multimodal) peaks. Stragglers on the edges of the distribution are known as outliers. The White Space Rule reminds you to think about the rest of your data when you find that an outlier dominates a graph.

Summary statistics quantify various attributes of the distribution. The median is the middle value of the distribution, and the mean is the average.

These define different ways to locate the center of a histogram. The interquartile range and standard deviation summarize the spread of the distribution. The coefficient of variation is the ratio of the stan- dard deviation to the mean. The boxplot shows the median and quartiles of the data.

The Empirical Rule connects the mean and standard deviation to the bell-shaped distributions. When the histogram is bell shaped, about two-thirds of the data fall in the range y - s to y + s, and 19/20 in the range y - 2s up to y + 2s. If you know the mean and SD, you can find the relative location of a value by computing its z-score, the number of SDs above or below the mean.

■ Formulas

■ Key Terms bell shaped, 65 boxplot, 59 coefficient of variation, 55 distribution, 57 Empirical Rule, 65 five-number summary, 53 histogram, 57 interquartile range (IQR), 52 mean, 54 median, 52 mode, 62 bimodal, 62

multimodal, 62 uniform, 62 unimodal, 62 outlier, 58 quartile, 52 range, 52 skewed, 63 standard deviation, 55 standardizing, 67 summary statistic, 52 summation notation, 72

symbol, 54 y (mean), 54 cv (coefficient of variation), 55 s (standard deviation), 55 s2 (variance), 54 (z-score), 66 symmetric, 63 tails, 63 variance, 54 White Space Rule, 58 z-score, 66

Many formulas in statistics involve averaging, as in the calculation of the mean and variance. To make these formulas more compact, it is common to use summation notation. Summation notation replaces a sum of similar terms by an expression starting with g followed by a generic summand. Subscripts and superscripts on g define a range that customizes

the generic summands. For example, the sum that defines the mean can be abbreviated as

y1 + y2 + g + yn = a n

i = 1 yi

Subscripts denote a row in the data table. The generic summand is yi. The subscript i stands for the cases to include in the sum, and the subscript and superscript on g indicate that the cases run from the first row 1i = 12 to the last row (n).

■ Objectives • Prepare, describe, and interpret a histogram

that summarizes the distribution of a numerical variable. Appreciate that the size of intervals in a histogram influences what this plot reveals.

• Interpret a boxplot and link it to the distribution (histogram).

• Calculate, interpret, and contrast the mean and the median.

• Calculate, interpret, and contrast the interquartile range (IQR) and the standard deviation (SD).

• Use the Empirical Rule to link the mean and stan- dard deviation to the concentration of data in a bell-shaped histogram (unimodal and symmetric).

• Use a z-score to locate the position of an observa- tion in a bell-shaped distribution.

• Distinguish bell-shaped distributions from bimod- al (multimodal) distributions and from skewed distributions.

summation notation Compact notation for a sum of similar terms.

M04_STIN7167_03_SE_C04_pp051-079.indd 72 25/10/16 12:15 PM

EXERCISES 73

Mean (Average)

y = y1 + y2 + g + yn

n =

a n

i = 1 yi

n

Variance

s2 = ( y1 - y)2 + ( y2 - y)2 + g + ( yn - y)2

n - 1

= a n

i = 1 ( yi - y)2

n - 1

Standard Deviation (SD)

s = 2s2 Z-Score

z = y - y

s

Coefficient of Variation

cv = s y

■ About the Data The data on digital recordings used in this chapter do indeed come from a friend’s computer. The applica- tion iTunes™ offers an option to export a summary of the stored music as a text file. We used that option to obtain these data. We should also note that all of these songs are recorded using a standard format (identified

as AAC in iTunes). The weights of M&M packages are from an experiment done by children of the authors for showing bar charts in school; their project was to identify the most common color. The data on executive compensation come from the Compustat ExecuComp database, by way of Wharton Research Data Services.

Mix and Match

Match the brief description to the item or value.

EXERCISES

1. Position of a peak in the histogram (a) median

2. Half of the cases are smaller than this value (b) mean

3. Length of the box in the boxplot (c) standard deviation

4. Proportion of cases lying within the box of the boxplot (d) variance

5. A histogram with a long right tail (e) z-score

6. The average of the values of a numerical variable (f ) two-thirds

7. The average squared deviation from the average (g) mode

8. The square root of the variance (h) interquartile range

9. The number of standard deviations from the mean (i) one-half

10. Proportion of a bell-shaped distribution within 1 SD of y ( j) skewed

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

11. The boxplot shows the mean plus or minus one stan- dard deviation of the data.

12. Any data outside the box of the boxplot are outliers and should be removed from the data.

13. Wider bins produce a histogram with fewer modes than would be found in a histogram with very narrow bins.

14. If a histogram is multimodal, the bin widths should be increased until only one bin remains.

15. If data are right skewed, the mean is larger than the median.

16. The histogram balances on the mean.

17. The Empirical Rule indicates that the range from y - s up to y + s holds two-thirds of the distribution of any numerical variable.

18. If a distribution is bell shaped, then about 5% of the z-scores are larger than 1 or less than -1.

19. The removal of an outlier with z = 3.3 causes both the mean and the SD of the data to decrease.

20. The interquartile range of a distribution is half the range from the smallest to largest value.

M04_STIN7167_03_SE_C04_pp051-079.indd 73 25/10/16 12:15 PM

74 CHAPTER 4 Describing Numerical Data

21. If the standard deviation of a variable is 0, then the mean is equal to the median.

22. The variance of a variable increases as the number of observations of the variable increases.

Think About It

23. If the median size used by 500 songs is 3.5 MB, will these all fit on an iPhone that has 2 GB of available storage? Can you tell?

24. (a) If the average size of 500 songs is 5.1 MB with standard deviation 0.5 MB, will these fit on an iPhone that has 2 GB of available storage?

(b) Does the size of the standard deviation matter in answering part (a)?

25. Suppose you were looking at the histogram of the incomes of all of the households in the United States. Do you think that the histogram would be bell shaped? Skewed to the left or right?

26. For the incomes of all of the households in the United States, which do you expect to be larger, the mean or the median?

27. An adjustable rate mortgage allows the rate of interest to fluctuate over the term of the loan, depending on economic conditions. A fixed rate mortgage holds the rate of interest constant. Which sequence of monthly payments has smaller variance, those on an adjustable rate mortgage or those on a fixed rate mortgage?

28. Many athletes use interval training to improve their strength and conditioning. Interval training requires working hard for short periods, say a minute, fol- lowed by longer intervals, say five minutes, that allow the body to recover. Does interval training increase or decrease the variation in effort?

29. The range measures the spread of a distribution. It is the distance from the smallest value to the largest value. Is the range sensitive to the presence of outliers?

30. What happens to the range as the number of observa- tions increases? If you have a data table that shows the balance of 100 loans, does the range get smaller, get larger, or stay the same if the number of loans is increased to 250?

31. Which has larger standard deviation, the distribution of weekly allowances to 12-year-olds or the distri- bution of monthly household mortgage payments? Which would have the larger coefficient of variation?

32. Which has larger variance, the prices of items in a grocery or the prices of cars at a new car dealer? Which has the larger coefficient of variation?

33. In a study of how the environment affects spending, researchers varied the presence of music and scents in a shopping mall. Interviews with shoppers who made unplanned purchases revealed the following average purchase amounts. Each group has about 100 shoppers.18

Condition Mean Spending ($)

None 64.00

Scent 55.34

Music 96.89

Scent and music 36.70

(a) If each group includes 100 shoppers, which group spent the most money in total?

(b) If medians summarized each group’s spending rather than means, would you be able to identify the group with the largest spending?

(c) Did every shopper who was exposed to scent and music spend less than every shopper who was only exposed to music?

34. The following table summarizes sales by day of the week at a convenience store operated by a major domestic gasoline retailer. The data cover about one year, with 52 values for each day of the week.

Mean ($) Std.Dev. ($)

Mon 2,824.00 575.587

Tue 2,768.85 314.924

Wed 3,181.32 494.785

Thur 3,086.00 712.135

Fri 3,100.82 415.291

Sat 4,199.33 632.983

Sun 3,807.27 865.858

(a) Which consecutive two-day period produces the highest total level of sales during the year?

(b) Do the distributions of the sales data grouped by day (as summarized here) overlap, or are the seven groups relatively distinct?

(c) These data summarize sales over 52 * 7 = 364 consecutive days. With that in mind, what impor- tant aspect of these data is hidden by this table?

35. Suppose sales representatives average 45 hours per week on the road with standard deviation 5. Would it be surprising to meet a representative who spent 51 hours on the road last week? Do you need to make any assumptions?

36. If a variable, such as income, is very right skewed, would you expect one-sixth of the values to be less than y - s, or would you expect more than one-sixth?

37. Would you expect distributions of these variables to be uniform, unimodal, or bimodal? Symmetric or skewed? Explain why. (a) The number of songs that each student in your

class has downloaded online (b) The cost of each order from a mail-order catalog

for clothing (c) Weights of bags of M&Ms that are labeled to

contain 6 ounces (d) Heights of students in your class

38. Would you expect the distributions of these variables to be uniform, unimodal, or bimodal? Symmetric or skewed? Explain why.

18 Adapted from M. Morrin and J.-C. Chebat, “Person-Place Congruency: The Interactive Effects of Shopper Style and Atmospherics on Con- sumer Expenditures,” Journal of Service Research 8(2005), 181–191.

M04_STIN7167_03_SE_C04_pp051-079.indd 74 25/10/16 12:15 PM

ExErcisEs 75

(a) Ages of shoppers in a convenience store near a university late Saturday night

(b) Number of children of shoppers in a toy store (c) Amount of cash taken in by retail cashiers during

a two-hour shift (d) Number of packages processed each day by Federal

Express in its hub location in Memphis, Tennessee, during August and the four weeks before Christmas

39. This histogram shows the distribution of the amount of sales tax (as percentages) in the 50 states and District of Columbia.

(a) About how many states charge between 4% and 5% sales tax?

(b) Estimate the mean from the histogram. Attach the appropriate unit to your estimate.

(c) Which do you think is larger, the mean or median? Why?

(d) Suggest an explanation for the multiple modes that are evident in the histogram.

(e) Is the SD of these data closer to 2, 5, or 10? (You don’t need to calculate the SD exactly to answer this question. Think about what the SD tells you about how data cluster around the mean.)

40. This histogram shows the average annual combined premium (in dollars) for auto insurance paid in the 50 states and District of Columbia.19

(a) Which holds more states, the interval from $600 to $700 or the interval from $700 to $800?

(b) Estimate the mean and median of the premiums from the histogram. Be sure to attach the appro- priate units to your estimates.

(c) Which do you think is larger, the mean or median? Why?

(d) Suggest an explanation for the multiple modes that are evident in the histogram. If the intervals in the histogram were wider, do you think there would still be two modes?

(e) Is the state with the highest rates slightly larger than the others or an outlier? Which state do you think earns this honor?

(f) Is the SD of these data closer to $200 or $600? (Use what you know about the relationship between the SD and the concentration of the data around the mean rather than trying to calculate the SD from the counts.)

41. The following histogram and boxplot summarize the distribution of income for 1,000 households in Colorado.

(a) Estimate the mean and median. How do you know that the mean is larger than the median for this distribution?

(b) Estimate the interquartile range from the figure.

(c) Which is larger, the IQR or the SD? (Hint: These data are very skewed.)

(d) How would the figure change if income were expressed in thousands of dollars rather than in dollars?

42. The following display summarizes the percentage of household income that goes to rent for 1,000 families that live in Denver, Colorado.

(a) Estimate the mean and median from the figure. (b) Estimate the IQR from the figure.

19 From the Auto Insurance Database Report produced by the National Association of Insurance Commissioners.

0 100,000 200,000 300,000 400,000

400 300 200 100

C o

u n t

Household Income ($)

150

100

50

C o

u n

t

0 10 20 30 40 50 60 70 80 90 100 Percentage of Income

M04_STIN7167_03_SE_C04_pp051-079.indd 75 06/02/17 9:31 AM

76 CHAPTER 4 Describing Numerical Data

(c) What’s the effect of the cluster of outliers on the right of the SD?

(d) What’s your explanation for the tall bin at the right side of the histogram?

43. Price scanners at the checkout register sometimes make mistakes, charging either more or less than the labeled price.20 Suppose that a scanner is wrong 4% of the time, with half of the errors in favor of the store and half in favor of the customer. Assume that an error in favor of the store means that the scanned price is 5% above the listed price and an error in favor of the customer means that the scanned price is 5% below the listed price. How would these errors affect the distribution of the prices of sold items?

44. The Energy Policy Act of 2005 mandates that gasoline sold in the United States must average at least 2.78% ethanol in 2006. Does this mean that every gallon of gas sold has to include ethanol?

45. The Census Bureau reported that the median house- hold net worth in the United States was $68,800 in 2011. In contrast, the mean household net worth was $339,000. How is it possible for the mean to be so much larger than the median?

46. The September 2014 issue of the Federal Reserve Bulletin results from the Survey of Consumer Finances, a national economic survey conducted by the Federal Reserve every three years. The survey found that from 2010 to 2013 overall average family income rose 4%, but median family income fell 5%. How is that possible?

47. (a) Convert the following summary values given in megabytes (MB) to seconds. Fill in the column for the times in seconds assuming that 1 MB of space provides 60 seconds of music.

Summary

File Size (MB)

Song Length (sec)

Mean 3.8

Median 3.5

IQR 1.5

Standard deviation 1.6

(b) If the size of each song were increased by 2 MB, how would the summary statistics given in MB change?

(c) If we were to add a very long song to this collection, one that ran on for 45 minutes, how would the median and interquartile range change?

48. (a) The following table summarizes the prices of a collection of used cars. What are the com- parable summary statistics when measured in euros?

Summary Price ($) Price €

Mean 30,300

Median 29,200

IQR 5,700

Standard deviation 4,500

Fill in the column for the prices in euros if the exchange rate is $1.20 per euro.

(b) If the price of each car were increased by $500, how would the summary statistics given in dollars change?

(c) If we were to include a very expensive BMW that sells for $125,000 to the collection considered in this exercise, how would the median and inter- quartile range change?

49. The data in this chapter concern the amount of space needed to hold various songs. What about the length, in minutes? Suppose you get 60 seconds of music from every megabyte of recorded songs. Describe the histogram that shows the number of seconds of each of the 3,984 songs in Figure 4.1.

50. This histogram shows the price in dollars of 218 used cars.

(a) Suppose we converted the prices of these cars from dollars to euros, with an exchange rate of 1.1 dollars per euro. Describe the histogram of the prices if denominated in euros.

(b) Cars are often sold with an additional dealer preparation fee that gets added on to the listed price. If the actual cost of each of the cars shown in this histogram goes up by $500, how does the histogram change?

51. This figure shows the histogram of the annual tuition at the top 20 undergraduate business schools in the United States, as rated by Bloomberg Businessweek.

20 “What, You Got a Problem Paying $102.13 for Tomatoes?” The New York Times, January 28, 2006.

60 50 40 30 20 10

C ou

nt

20,000 25,000 30,000 35,000 40,000 45,000 Dollars

1

3

5

7

C o

u n t

$5,000 $15,000 $25,000 $35,000 $45,000

M04_STIN7167_03_SE_C04_pp051-079.indd 76 25/10/16 12:15 PM

EXERCISES 77

(a) Estimate from the figure the center and spread of the data. Are the usual notions of center and spread useful for these data?

(b) Describe the shape of the histogram. (c) If you were only shown the boxplot, would you

be able to identify the shape of the distribution of these data?

(d) Can you think of an explanation for the shape of the histogram?

52. This histogram shows the winning times in seconds for the Kentucky Derby since 1875.

(a) Estimate the center and scale of these data from the histogram. Are the usual notions of center and spread useful summaries of these data?

(b) Describe the shape of the distribution of these data. (c) What do you think the boxplot of these data

would look like? (d) Can you think of an explanation for the shape

of the histogram? What plot of these data would help you resolve whether you have the right explanation?

You Do It

The title of each of the following exercises identifies the data table to be used for the problem.

53. Cars A column in this data file gives the engine dis- placement in liters of 413 vehicles sold in the United States. These vehicles are 2016 models, and exclude hybrids. (a) Produce a histogram of the engine displace-

ments, with the interval widths set to 1/2 liter and the first bin starting at 1/2 liter. Describe and interpret the histogram.

(b) Repeat (a) but with the width of the intervals set to 1 liter.

(c) Does the size of the interval width change your impression of the distribution? Explain briefly.

(d) Find the mean and standard deviation of the engine displacements. How are these related to the histogram?

(e) Find the coefficient of variation and briefly inter- pret its value.

(f ) Identify any unusual values (outliers). What would it mean to be an outlier?

54. Cars Another column in this data file gives the rated combined fuel economy (in miles per gallon) for 413 vehicles sold in the United States (see prior question).

(a) Produce a histogram of these data. Describe and interpret the histogram.

(b) Compare the histogram to the boxplot. What does the histogram tell you that the boxplot does not, and vice versa?

(c) Find the mean and standard deviation of the rated mileages. How are these related to the his- togram, if at all?

(d) Find the coefficient of variation and briefly inter- pret its value.

(e) Identify any unusual values (outliers). Do you think that these are coding errors?

(f ) Government standards call for cars to get 27.5 MPG. What percentage of these vehicles meet this goal? (Are all of these vehicles cars?)

55. Beatles Outliers have a dramatic effect on small data sets. For this exercise, the data consist of the sizes (in seconds and MB) of the 27 #1 hits on the Beatles’ album 1. (a) Generate the boxplot and histogram of the sizes

of these songs. (b) Identify any outliers. What is the size of this

song, in minutes and megabytes? (c) What is the effect of excluding this song on the

mean and median of the sizes of the songs? (d) Which summary, the mean or median, is the better

summary of the center of the distribution of sizes? (e) Which summary, the mean or median, is the

more useful summary if you want to know if you can fit this album on your iPod?

56. Beatles Outliers affect more than the statistics that measure the center of a distribution. Outliers also affect statistics that measure the spread of the distri- bution. Use the sizes of the songs in Exercise 55 for this exercise. (a) Find the standard deviation and interquartile

range of the sizes of the songs (in megabytes). (b) Which of these summary statistics is most

affected by the presence of the outlier? How do you know?

(c) Exclude the most extreme outlier from the data and find the mean and SD without this song. Which summary changes more when the outlier is excluded: the mean or SD?

57. Information Industry This data table includes several characteristics of 428 companies classified as being in the information industry in 2010. One column gives the total revenue of the company, in millions of dollars. (a) Find the median, mean, and standard deviation

of the total revenue of these companies. What units do these summary statistics share?

(b) Describe the shape of the histogram and boxplot. What does the White Space Rule have to say about the histogram?

(c) Do the data have any extreme outliers? Identify the company if there’s an extreme outlier.

(d) What do these graphs of the distribution of net sales tell you about this industry? Is the industry dominated by a few companies, or is there a level playing field with many comparable rivals?

120 130 140 150 160 170

50

40

30

20

10

C o

u n

t

Seconds

M04_STIN7167_03_SE_C04_pp051-079.indd 77 25/10/16 12:15 PM

78 CHAPTER 4 Describing Numerical Data

58. Information Industry Using the data from Exercise 57, consider the total revenue per employee. Revenue is in millions of dollars, and the count of employees is given in the data table. (a) Why are there fewer than 428 observations avail-

able for making a histogram? (b) Find the median, mean, and standard deviation

of the revenue per employee. Be sure to include units with these.

(c) Describe the shape of the histogram. Would the Empirical Rule be useful for describing how observations cluster near the mean of this ratio?

(d) Which company has the largest revenue per employee? Is it the revenue or the number of employees that makes the ratio so large for this company?

(e) A startup firm has 15 employees. What revenue does the company need to achieve in order to be comparable to other firms in this industry?

59. Tech Stocks These data give the monthly returns on stocks in three technology companies: HP, IBM, and Microsoft. For each month from January 1990 through the end of 2014 (300 months), the data give the return earned by owning a share of stock in each company. The return is the percentage change in the price, divided by 100. (a) Describe and contrast histograms of the returns

on the three stocks. Be sure to use a common scale for the data axes of the histograms to make the comparison easier and more reliable.

(b) Find the mean, SD, and coefficient of variation for each set of returns. Are means and SDs useful summaries of variables such as these?

(c) What does comparison of the coefficients of variation tell you about these three stocks?

(d) Investors prefer stocks that grow steadily. In that case, what values are ideal for the mean and SD of the returns? For the coefficient of variation?

(e) It is common to find that stocks that have higher average return also tend to be more volatile, with larger swings in price. Is that true for these three stocks?

60. Familiar Stocks These data give monthly returns for stocks of three familiar companies: Disney, Exxon- Mobil, and McDonald’s from January 1990 through December 2014. (a) Describe and contrast histograms of the three

companies. Be sure to use common axes to scale the histograms to make the comparison easier and more reliable.

(b) Find the mean, SD, and cv for each set of returns. Can these means and SDs be combined with the Empirical Rule to summarize the distributions of these returns?

(c) What does comparison of the coefficients of variation tell you about these three stocks?

(d) Typically, stocks that generate high average returns also tend to be volatile, with larger upward and downward swings in price. To get a higher average rate of return, an investor has to tolerate more volatility. Is that true for these three stocks?

61. Tech Stocks (See Exercise 59.) Some investors use the Sharpe ratio as a way of comparing the benefits of owning shares of stock in a company to the risks. The Sharpe ratio of a stock is defined as the ratio of the difference between the mean return on the stock and the mean return on government bonds (called the risk-free rate rf ) to the SD of the returns on the stock.

Sharpe ratio = y - rf

s

The mean return on government bonds during this period is rf = 0.0025 per month (that is, about 1>4 of 1% per month, or 3% annually). (a) Find the Sharpe ratio of stock in these three

companies. Which looks best from this investment point of view?

(b) Form a new column by subtracting rf from the return each month for HP. Then divide this col- umn of differences by the SD for HP. What’s the mean value for this column?

(c) How does the Sharpe ratio differ from the type of standardizing used to form z-scores?

62. Familiar Stocks See Exercise 60 regarding the data and Exercise 61 for the Sharpe ratio. (a) Find the Sharpe ratio of stock in these three

companies. Which looks best from this investment point of view?

(b) Form a new column by subtracting rf from the return each month on Disney. Next divide this column of differences by the SD for Disney. What is the mean value for this column?

(c) Look at the returns for October 2005. Do the returns in this month match up to the performance suggested by the Sharpe ratio? Explain briefly what happened.

63. 4M ANALYTICS: Financial Ratios

Companies vary greatly in size. This variation can hide how well a company is performing. A retailer, for instance, reports $500 million in profits for the past year. That would be a good year for Foot Locker stores but not for Wal-Mart whose profit was $16 billion in 2010. Rather than look at the raw profit numbers, analysts consider financial ratios that adjust for the size of the company.

A popular ratio is the return on assets, defined as

Return on Assets = Net Income>Total Assets Net income is another name for profits, and the total assets of a company is the value of everything it owns that is used to produce profits. The return on assets indicates how much profit the company generates relative to the amount that it invested to make that profit. It’s a bit like interest on a savings account. Five percent interest means that you make $5 for every $100 saved. Similarly, if the return on assets is 0.05, then the company profits $5 for each dollar of assets it owns. A company with losses rather than profits has a negative return on assets.

M04_STIN7167_03_SE_C04_pp051-079.indd 78 25/10/16 12:15 PM

EXERCISES 79

The data table for this exercise gives the total assets and net income reported by 179 retailers in the United States.

Motivation

(a) Why would it be helpful to describe manage- ment’s performance using return on assets rather than both net income and total assets?

(b) What are the units of the return on assets? (c) Describe how a firm might use the return on

assets as a yardstick for the performance of its management.

(d) Are there other ratios that might serve a similar purpose, or is this the only ratio that will be useful?

Method

(e) What plots will be useful for summarizing the distribution of return on assets and for showing if this distribution is bell-shaped?

(f) Why is it useful to have a bell-shaped distribution for the return on assets rather than one that is very skewed?

Mechanics

(g) Describe histograms of the variables Net Income and Total Assets. Are these skewed or bell shaped?

(h) Identify any exceptional outliers in these two distributions.

(i) Produce a histogram and boxplot for the return on assets. Describe the shape of this distribution.

(j) Is the company that is the major outlier on total assets also an outlier in terms of return on as- sets? Explain.

Message

(k) Summarize the distribution of the return on assets for these retailers.

(l) On the basis of the distribution of the return on assets, would a return on assets larger than 0.1 (10%) indicate that a retailer is having a good year in 2010?

64. 4M ANALYTICS: Credit Scores

When a customer asks to borrow money, many banks check their credit score before giving them a loan. The credit score estimates the risk associated with making a loan to the customer. Credit scores are based on how well an individual has handled past debts. Customers that have made regular payments on time on several loans get a high score, whereas those who have been late and perhaps defaulted on a loan have lower scores.

The data in this question give the credit score for 963 individuals who recently obtained small business loans from a lender. The average score is 580. The lender is con- sidering raising its standards for getting a loan to reduce its exposure to losses. An executive proposed raising the minimum credit score for these loans to 550, saying that such a score would not lose too much business as it was ‘below average’.

Motivation

(a) The average of these credit scores is 580. Why would it be useful for managers of the lender to examine other attributes of the data before deciding on new standards?

Method

(b) What other attributes of the data should the managers examine?

Mechanics

(c) Present a display that captures the relevant attributes of these credit scores. Include relevant summary statistics.

Message

(d) What do you think of the recommended new credit limit?

M04_STIN7167_03_SE_C04_pp051-079.indd 79 25/10/16 12:15 PM

80

5.1 CONTINGENCY TABLES

5.2 LURKING VARIABLES AND SIMPSON’S PARADOX

5.3 STRENGTH OF ASSOCIATION

CHAPTER SUMMARY

5 BUSY WEB SITES CHARGE FOR RUNNING ADS ON THEIR PAGES. If you are an online retailer, how are you to decide which sites to place your ads?

The answer comes from understanding the habits of on-line shoppers. Some shoppers buy, but others only browse. Suppose that every shopper from Yahoo was a buyer, but none of the shoppers from Google. Knowing which Web site delivers the buyers explains variation in behavior and reveals the better site for ads.

Let’s focus on the choices faced by an advertising manager at Amazon. She would like to find Web sites that deliver buyers. Her task is complicated because some referring Web sites (hosts) are more active than others. Table 5.1 summarizes 30,000 visits to Amazon from three hosts during 2015.

c h a p t e r Association between Categorical Variables

Host Visits

Comcast.net 300

Google.com 29,000

Nextag.com 700

Total 30,000

TABLE 5.1 Frequency table of the categorical variable that identifies the referring host.

It should come as no surprise that Google generated the most visits, but more visits do not necessarily mean more sales. More visits do imply higher costs, however. Amazon pays a fee for every visit, whether the visitor buys anything or not. Hosts that generate many visits but few sales are costly to Amazon. Should Amazon pay some hosts more than others for each visitor sent to Amazon?

Choosing the best host requires us to understand the assoCiation, or relationship, between two CategoriCal variables. In this example, the variables identify the host and whether a purchase was made. This chapter shows how to measure association be- tween variables by building tables. This chapter also presents methods that display the association in a graph.

80

M05_STIN7167_03_SE_C05_pp080-108.indd 80 26/10/16 12:57 PM

5.1 ❘ CONTINGENCY TABLES To discover whether some hosts yield more buyers than others, we have to consider the categorical variable that identifies those visits that result in a sale. Figure 5.1 shows bar charts of that variable and the referring host.

0 10,000 20,000 30,000

Google

H o

st

Nextag

Comcast

Visits 0 10,000 20,000 30,000

Yes

P u

rc h

as e

No

Visits

FIGURE 5.1 Bar charts of hosts and purchase actions.

The categorical variable Host identifies the referring Web site. The categori- cal variable Purchase indicates whether the session produced a sale. There’s precious little variation in Purchase. Only 1,065 visits, about 3.6%, result in a purchase. If all of these sales came from one of these three hosts, Amazon would know where to place its ads.

The bar charts in Figure 5.1 do not reveal the source of the purchases. These charts summarize each categorical variable separately, but we need to consider them simultaneously to discover how many purchases to associ- ate with each host. For instance, we need to count how many visitors from Google made a purchase.

The most common arrangement of counts based on two categorical vari- ables organizes them in a table. The rows of the table identify the categories of one variable, and the columns of the table identify the categories of the other. Such a table is called a contingency table. Table 5.2 shows the variable Purchase (along the rows) and the variable Host (columns).

contingency table A table that shows counts of the cases of one categorical variable contingent on the value of another.

Host

Comcast Google Nextag Total

Purchase

No 270 28,000 665 28,935

Yes 30 1,000 35 1,065

Total 300 29,000 700 30,000

TABLE 5.2 Contingency table of Web shopping.

The cells of this contingency table (which are lightly shaded in Table 5.2) count the visits for every combination of Host and Purchase. The cells of the contingency table are mutually exclusive; each visit appears in exactly one cell. For example, the column labeled Comcast shows that 30 of the 300 visits from Comcast generated a purchase.

Marginal and Conditional Distributions

The margins of Table 5.2 (shaded brown) give the total counts in each row and column. Because the cells of the table are mutually exclusive, the sum of the counts in the cells of the first column, for example, equals the total num- ber of visits from Comcast. The sum for each column appears in the bottom margin of the contingency table; these total counts match the distribution of

cell Intersection of a row and a column of a table.

mutually exclusive The cells of a contingency table are mutually exclusive if each case appears in only one cell.

81

M05_STIN7167_03_SE_C05_pp080-108.indd 81 26/10/16 12:57 PM

82 CHAPTER 5 Association between Categorical Variables

Host in Table 5.1. The right margin shows the distribution of Purchase. Be- cause these counts of the totals in each row and column are placed along the margins of a contingency table, the distributions of the variables in the table are also called marginal distributions. The bar charts in Figure 5.1 show the marginal distributions.

Percentages help us interpret a contingency table, but we’ve got to choose which percentage to show. For example, 30 shoppers from Comcast made a purchase. To show this count as a percentage, we have three choices.

30 is • 0.1% of all 30,000 visits. 10% of the 300 visits from Comcast. 2.82% of the 1,065 visits that made a purchase.

All are potentially interesting. Some statistics packages embellish a contin- gency table with every percentage, as in Table 5.3, leaving it up to the user to figure out which percentages are useful. Each cell lists the count, along with its percentage of the total, the column, and the row. Tables like this one show too many numbers and are confusing.

marginal distribution The frequency distribution of a variable in a contingency table given by counts of the total number of cases in rows (or columns).

To decide which percentage to show, choose the percentage that answers the relevant question. Because the account manager at Amazon is interested in which host produces the largest proportion of purchasers, a better table shows only the counts and column percentages. Let’s start with Comcast. For the moment, we’re interested in only the 300 visits from Comcast in the first column of cells in Table 5.2. The distribution of a variable that is restricted to cases that satisfy a condition is called a conditional distribution. In a table, a conditional distribution refers to counts within a row or column. By limit- ing our attention to visits from Comcast, we see the distribution of Purchase conditional on the host being Comcast.

Table 5.4 shows the counts and only the column percentages. The percent- ages within each column show the relative frequencies of the conditional dis- tribution of Purchase for each host.

conditional distribution The distribution of a variable restricted to cases that satisfy a condition, such as cases in a row or column of a table.

Count Total %

Column % Row %

Comcast

Host Google

Nextag

Total

Purchase

No 270 0.90%

90.00% 0.93%

28,000 93.33% 96.55% 96.77%

665 2.22%

95.00% 2.30%

28,935 96.45%

Yes 30 0.10%

10.00% 2.82%

1,000 3.33% 3.45%

93.90%

35 0.12% 5.00% 3.29%

1,065 3.55%

Total 300 1.00%

29,000 96.67%

700 2.33%

30,000

TABLE 5.3 Too many percentages clutter this contingency table.

Count Column % Comcast

Google

Nextag

Total

Purchase

No 270 90.00%

28,000 96.55%

665 95.00%

28,935

Yes 30 10.00%

1,000 3.45%

35 5.00%

1,065

Total 300 29,000 700 30,000

HostTABLE 5.4 Contingency table with relevant column percentages.

M05_STIN7167_03_SE_C05_pp080-108.indd 82 26/10/16 12:57 PM

5.1 CONTINGENCY TABLES 83

Compare this table with Table 5.3. Without the extraneous percentages, we can easily see that Comcast has the highest rate of purchases (10%), twice that of Nextag (5%). The counts of purchases from Comcast and Nextag are similar, but the rates are very different. And although visits from Google generate the most purchases, these purchases occur at the lowest rate (3.45%).

These differences among the column percentages imply that Host and Purchase are associated. Categorical variables are associated if the col- umn percentages vary from column to column (or if the row percentages vary from row to row). In this case, the proportion of visits that produce a purchase differs among hosts. The association between Host and Purchase means that knowing the host changes your impression of the chance of a purchase.

Variables can be associated to different degrees. Variables are not associ- ated if the column percentages are identical. Overall, 1,065>30,000 = 3.55% of the visits made a purchase. If 3.55% of visits from every host made a purchase, then the rate of purchases would not depend on the host. Each conditional distribution of Purchase given Host would match the marginal distribution of Purchase, and the percentage of purchases would be the same no matter which host was chosen. That’s not what happens here: Host and Purchase are associated. Visitors from some hosts are more likely to make a purchase.

The presence of association is important for the advertising manager at Amazon. Because Host and Purchase are associated, she might be willing to pay more to Comcast per visit than to Google or Nextag. The value of the visit depends on the host. (The manager would also be interested in the dollar amount of these purchases. We revisit that aspect of these data in Chapter 18.)

associated Two categorical variables are associated if the conditional distribution of one variable depends on the value of the other.

What Do You Think? The Human Resources (HR) department of a company would like to know the type of coverage preferred by its 1,648 employees. The following table shows the contingency table of the employees’ marital status versus the preferred type of medical coverage. POS plans are cheaper but require choosing a par- ticipating physician; the Co-Pay plan is more flexible but requires a copayment at each visit. Employees can also waive coverage.

Marital Status

Divorced Married Single

Medical Plan

Co-Pay 27 169 73

POS 36 372 138

Waive 34 699 100

a. What does it mean for this table to be a contingency table?1

b. Add the marginal distributions to the table.2

c. HR suspects that married employees are most likely to waive coverage, presumably because their spouse has family coverage. Choose the appro- priate conditional distributions and see if they are right.3

1 The cells are mutually exclusive; each employee belongs in one cell. 2 The row margins are 269, 546, and 833; column margins are 97, 1240, and 311. 3 They are right, based on the column percentages. Divorced prefer POS (37%), single prefer POS (44%), and married pick waive most often (56%).

M05_STIN7167_03_SE_C05_pp080-108.indd 83 26/10/16 12:57 PM

84 CHAPTER 5 Association between Categorical Variables

Stacked Bar Charts

Bar charts of the marginal distributions (Figure 5.1) don’t reveal association, but charts that reach inside the table do. For example, operations managers for Amazon locate warehouses near large concentrations of shoppers to reduce shipping costs. Being close makes it cheaper to offer free shipping. Table 5.5 shows the counts of Purchase by Location and the row percentages.

Count Row %

Region

North Central

North East South West Total

Purchase

No 5,790 20.01

4,825 16.68

11,585 40.04

6,735 23.28

28,935

Yes 210 19.72

175 16.43

415 38.97

265 24.88

1,065

Total 6,000 5,000 12,000 7,000 30,000

TABLE 5.5 Contingency table of purchases organized by region.

Because we’re interested in discovering where those who make a purchase live, this table shows row percentages. With four percentages in each condi- tional distribution, it becomes helpful to have a plot. A stacked bar chart divides the bars in a bar chart proportionally into segments corresponding to the percentage in each group. If the bars look identical, then the variables are not associated.

The stacked bar chart in Figure 5.2 shows the similarities in the row per- centages more clearly than Table 5.5. For example, shoppers from the North Central make up a roughly equal share of the browsers (16.7%, top bar) and the purchasers (16.4%, lower bar).

stacked bar chart A bar chart that divides the bars into shares based on a second cat- egorical variable.

0% 20% 40% 60% 80% 100%

Yes

No North Central North East South West FIGURE 5.2 A stacked bar

chart shows the presence or in this case absence of association.

caution Be careful interpreting a stacked bar chart. Figure 5.2 compares rel- ative frequencies of two conditional distributions. Because these are

relative frequencies rather than counts, the bars need not represent the same number of cases. The bar on the top in Figure 5.2 summarizes 28,935 cases, whereas the bar on the bottom summarizes 1065 purchases. The chart obeys the area principle, but the area is proportional to the percentages within each row of the table.

We can also draw a stacked bar chart using counts rather than percent- ages, but that choice does not work well with these data. Figure 5.3 shows the stacked bar chart based on counts. Because purchases make up such a small percentage of the visits, the bars have very different lengths. The dif- ference between the lengths of the bars makes it difficult to compare the regional shares.

M05_STIN7167_03_SE_C05_pp080-108.indd 84 26/10/16 12:57 PM

Mosaic Plots

A mosaic plot is an alternative to a stacked bar chart. A mosaic plot displays colored tiles, rectangular regions that represent the counts in each cell of a contingency table. The area of a tile is proportional to the count rather than the percentage within a row or column. The layout of the tiles matches the layout of the cells in a contingency table. The tiles within a column have the same width but possibly different heights. The widths of the columns are pro- portional to the marginal distribution of the variable that defines the columns. For example, Figure 5.4 shows the mosaic plot of Table 5.5. The tiny heights of the red tiles show the proportions making a purchase in each region; the tiny sizes remind you how rare it is to find a purchase. The tiny sizes also conceal that the chance of a purchase is slightly higher in the West than else- where (about 3.8% compared to 3.5% elsewhere).

mosaic plot A plot in which the area of each tile is propor- tional to the count in a cell of a contingency table.

0 5,000 10,000 15,000 20,000 Visits

25,000 30,000

No

Yes

North Central North East South West

FIGURE 5.3 A stacked bar chart using counts makes it hard to compare the shares.

P u rc

h as

e

0.00

0.25

0.50

0.75

1.00

N o

rt h e as

t

N o

rt h

C e n tr

al

S o

u th

W e st

Region

No

Yes

FIGURE 5.4 Mosaic plot of the purchases by region.

The widest tiles are in the column representing the South because most vis- its are from the South. Because purchases are so rare, you cannot see that the percentage of purchases is smallest in the North East region.

Mosaic plots are more useful for seeing association in data for which the relative frequencies do not get so small. As an example, Table 5.6 shows counts of sales of different styles of shirts in various sizes at a men’s cloth- ing retailer. Managers of this department need to know whether Style and Size are associated. If the two are not associated, managers should order the same proportion of sizes in every style. If the two are associated, the distribution of sizes varies from style to style and managers should order in different proportions.

5.1 CONTINGENCY TABLES 85

M05_STIN7167_03_SE_C05_pp080-108.indd 85 26/10/16 12:57 PM

86 CHAPTER 5 Association between Categorical Variables

It’s hard to see the association quickly in this contingency table, but the mosaic plot in Figure 5.5 makes the association very clear.

Style

Button-Down Polo Small Print Total

Size

Small 18 27 36 81

Medium 65 82 28 175

Large 103 65 22 190

Total 186 174 86 446

TABLE 5.6 Sales of shirts at a men’s clothing retailer.

0.00 Button-Down

Style Polo Print

0.25 Large

Medium

Small S iz

e

0.50

0.75

1.00

FIGURE 5.5 Mosaic plot of the shirt sales shows association between Size and Style.

The tiles would line up in the absence of association. In this example, the proportions of sizes vary across the styles, causing the tiles to vary in height. The irregular heights indicate that these variables are associated. Small sizes are much more prevalent among beach prints than button-down shirts. Because the mosaic plot respects the area principle, we can also see that the button- down style is the biggest seller overall (these tiles are wider than the others) and the beach-print style is the smallest seller. A segmented bar chart loses track of the marginal distribution, contrasting only the conditional distributions.

4M ANALYTICS 5.1 CAR THEFT

MOTIVATION ▶ STATE THE QUESTION Auto theft costs owners and insurance companies bil- lions of dollars. The FBI estimates that 690,000 cars worth more than $4.5 billion were stolen in 2014.

Should insurance companies charge the same premium for every car model for theft insurance, or should they vary the premium? Obviously, a policy that insures a $90,000 Porsche should cost more than a policy for a $15,000 Hyundai. But should the premium be a fixed percentage of the car’s value, or should the percentage vary from model to model? To answer this question, we need to know whether some cars are more likely to be stolen than others. It comes down to whether there is an association between car theft and car model.

For this example, it’s up to you to decide whether or not an insurance com- pany should charge a fixed percentage of the value of the car to insure against

M05_STIN7167_03_SE_C05_pp080-108.indd 86 26/10/16 12:57 PM

theft. The company can either charge a fixed percentage of the replacement cost or charge a variable percentage that is higher for cars that are stolen more often. Are there large differences in the rates of theft? ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Identify your data and verify that the categories are mutually exclusive. Then identify how you plan to use these data. The two categorical variables needed in this analysis identify the type of car and whether the car was stolen. The following data come from the National Highway Traffic Safety Administration (NHTSA). The data describe the 10 most common models made in 2014. We will organize these data into a table that shows the rates of theft for the differ- ent models. ◀

MECHANICS ▶ DO THE ANALYSIS Make an appropriate display or table to see whether there is a difference in the relative proportions. This table shows the counts along with the percentage of each model that was stolen. Among these models, the Toyota Corolla has the highest percentage stolen (0.181%), followed by the Nissan Altima (0.176%). The Honda CR-V has the lowest percentage stolen (0.037%).

Model Stolen Made % Stolen

Hyundai Elantra 469 411,249 0.114

Nissan Altima 693 393,800 0.176

Honda Accord 231 372,134 0.062

Honda Civic 189 361,723 0.052

Hyundai Sonata 388 313,346 0.124

Toyota Corolla 566 313,314 0.181

Ford Escape 265 310,054 0.085

Toyota Camry 353 280,399 0.126

Honda CR-V 102 278,583 0.037

Chevrolet Equinox 115 259,361 0.044

Notice that the table does not show the number that were not stolen. If you do that, you’ll see that the number made is the marginal (row) total. For example, for the Hyundai Elantra we have

Model Stolen Not Stolen Total

Elantra 469 410,780 411,249

We think that the table looks better, with more emphasis on the number stolen, without adding the additional column. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS Discuss patterns in the table and displays. Some models (e.g., Toyota Corolla) are more likely to be stolen than others (e.g., Honda Civic). About 0.20% of Corollas were stolen, compared to 0.05% of Civics.

A lot of Civics get stolen, but that count can be explained by the sheer num- ber of Civics sold each year. The following chart shows the percentage of cars within each category that were stolen. As in the example of Web shopping, a stacked bar chart or mosaic chart is less useful because the percentages stolen are so small.

5.1 CONTINGENCY TABLES 87

M05_STIN7167_03_SE_C05_pp080-108.indd 87 26/10/16 12:57 PM

88 CHAPTER 5 Association between Categorical Variables

Discuss the real-world consequences and address the motivating question. These data suggest that the insurer should charge higher premiums for theft insurance for models that are most likely to be stolen. For instance, a Nissan Altima is three times as likely to be stolen as a Honda Civic. Customers who buy an Altima (which costs about the same as a Civic) should pay a higher premium for theft insurance. ◀

0.00% 0.05% 0.10% 0.15% 0.20%

Chevrolet Equinox

Honda CR-V

Toyota Camry

Ford Escape

Toyota Corolla

Hyundai Sonata

Honda Civic

Honda Accord

Nissan Altima

Hyundai Elantra

Percent Stolen

4 The row totals determine the marginal distribution, 64 Yes who made a purchase and 4,063 No. 5 Among the 395 who joined, 52 made a purchase (13%). Among those who declined, 12 out of 3,732 made a purchase (0.32%). Customers who joined were more likely to make a purchase. The retailer could encourage customers who haven’t made a purchase to join. 6 Not really, because one percentage is so small. You could show a figure like that in the prior 4M. 7 The two are associated because the percentages within the rows (or within the columns) differ. The fraction of customers who make a purchase depends on whether they sign up.

What Do You Think? An online questionnaire asked visitors to a retail Web site if they would like to join a mailing list. This contingency table summarizes the counts of those who joined as well as those who made a purchase.

Mailing List

Join Decline

Purchase Yes 52 12

No 343 3,720

The columns indicate whether the visitor signed up (Mailing List = Join or Decline), and the rows indicate whether the visitor made a purchase (Purchase = Yes or No). For example, 52 visitors joined the mailing list and made a purchase.

a. Find the marginal distribution of Purchase.4

b. Find the conditional distributions of Purchase given whether or not the customer signed up. Do the conditional distributions differ? How can the retailer exploit this property of the data?5

c. Does a segmented bar chart provide a helpful plot for these data?6

d. Is the variable Purchase associated with the variable Mailing List?7

M05_STIN7167_03_SE_C05_pp080-108.indd 88 26/10/16 12:57 PM

5.2 ❘ LURKING VARIABLES AND SIMPSON’S PARADOX Association often gets confused with causation. This confusion can lead to serious errors in judgment. Consider Table 5.7, which compares the perfor- mance of two hypothetical shipping services.

Count Column %

Service

Orange Arrow

Brown Box Total

Status

Damaged 45 15%

66 33%

111 22.2%

OK 255 85%

134 67%

389 77.8%

Total 300 200 500

TABLE 5.7 Counts of damaged packages.

This contingency table shows the number of cartons that were damaged when shipped by two delivery services. The percentages in each cell are col- umn percentages. Overall, 22.2% of the 500 cartons that were shipped arrived with visible damage. Among these, 15% of cartons shipped via Orange Arrow arrived damaged compared to 33% for Brown Box. There’s definitely associa- tion; neither conditional distribution matches the marginal distribution of Status.

Table 5.7 suggests that Orange Arrow is the better shipper, and we might be tempted to believe that cartons are more likely to arrive okay because they are shipped on Orange Arrow. If we believe that, we might decide to ship ev- erything on Orange Arrow. Before we do that, however, we’d better make sure that this table offers a fair comparison. Maybe there’s another explanation for why packages shipped on Brown Box are more likely to arrive damaged.

To think of an alternative explanation, we have to know more about these packages. In this instance, the cartons hold car parts. Some cartons hold en- gine parts, whereas others hold plastic molding. Guess which cartons are heavier and more prone to damage. The two tables in Table 5.8 separate the counts in Table 5.7 into those for heavy cartons (left table) and those for light cartons (right table).

Count Column %

Heavy Light

Service Service

Orange Arrow

Brown Box Total

Orange Arrow

Brown Box Total

Status

Damaged 20 67%

60 40%

80 44.4%

25 9%

6 12%

31 9.7%

OK 10 33%

90 60%

100 55.6%

245 91%

44 88%

289 90.3%

Total 30 150 180 270 50 320

TABLE 5.8 Separate tables for heavy and light packages.

Orange Arrow no longer appears to be the better shipper. Among heavy packages, 67% of those shipped on Orange Arrow arrived damaged compared to 40% for Brown Box. For light cartons, 9% of those shipped by Orange Arrow arrived damaged compared to 12% for Brown Box.

The initial comparison favors Orange Arrow because it handles a higher share of light packages that are less prone to damage. Brown Box handles a greater proportion of heavy cartons. Heavy cartons more often arrive with

5.2 LURKING VARIABLES AND SIMPSON’S PARADOX 89

M05_STIN7167_03_SE_C05_pp080-108.indd 89 26/10/16 12:57 PM

90 CHAPTER 5 Association between Categorical Variables

some damage (44.4% versus 9.7%). Table 5.7 is misleading. It compares how well Orange Arrow ships light cartons to how well Brown Box ships heavy cartons. The weight of the cartons is a hidden, lurking variable. Table 5.8 adjusts for the lurking variable by presenting separate tables for heavy cartons and light cartons.

Such reversals often go by the name Simpson’s paradox. It can seem surprising—downright paradoxical—that one service looks better overall, but the other looks better when we restrict the comparison. The explanation lies in recognizing the presence of a lurking variable. Before you act on associa- tion (like sending all the business to Orange Arrow), be sure to consider the possible effects of lurking variables.

lurking variable A concealed variable that affects the as- sociation between two other variables.

Simpson’s paradox An abrupt change in the association be- tween two variables that oc- curs when data are separated into groups defined by a third variable.

4M ANALYTICS 5.2 AIRLINE ARRIVALS

MOTIVATION ▶ STATE THE QUESTION A corporate CEO regularly flies to meetings with branch managers in Boston, Orlando, Philadel- phia, and San Diego. The CEO is very pressed for time and must schedule flights that arrive just in time for meetings. A delay might cause the CEO to miss the meeting—a major headache.

The corporation has arranged deals with two airlines, Delta and American, to provide travel services. Does it matter which of these two air- lines the CEO chooses? ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH This analysis requires at least two categorical variables that describe flights into these four destinations. One variable identifies the airline (American or Delta), and the second variable identifies whether the flight arrived on time. We will use data from the U.S. Bureau of Transportation Statistics, the agency that monitors the status of all domestic flights within the United States. ◀

MECHANICS ▶ DO THE ANALYSIS The following table summarizes 10,906 arrivals on Delta and American at these four destinations during a recent month.

Arrivals to Four Destinations

Count Column %

Airline

Delta American Total

Arrival

Delayed 659 20%

1,685 22%

2,344

On Time 2,596 80%

5,966 78%

8,562

Total 3,255 7,651 10,906

This table suggests that the two airlines perform comparably, with a slight edge in favor of Delta. Eighty percent of flights on Delta arrived on time com- pared to 78% for American.

M05_STIN7167_03_SE_C05_pp080-108.indd 90 26/10/16 12:57 PM

Before we tell the CEO to book a flight on Delta, however, we should think about whether there’s a lurking variable. Identifying a lurking variable requires knowing a bit more about these flights.

Arrivals to Orlando

Count Col % Delta American Total

Delayed 228 19.5%

150 15.5%

378

On Time 940 80.5%

820 84.5%

1,760

Total 1,168 970 2,138

For example, consider the preceding contingency table, which isolates flights into Orlando. For flights to Orlando, American is the better choice. In fact, no matter which destination, American has a higher percentage of on-time arrivals.

On Time % Delta American

Boston 80.1% 81.7%

Orlando 80.5% 84.5%

Philadelphia 70.5% 74.3%

San Diego 84.2% 85.4%

That’s Simpson’s paradox. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS We recommend booking the flight for the CEO on American. No matter which destination, American is more likely to arrive on time. ◀

It is worthwhile to review why Delta appears better overall, even though American arrives on time more often for every destination. The initial 2-by-2 table in Example 5.2 masks a lurking variable: destination. The des- tination matters: Delays are more common at Philadelphia, as shown in Table 5.9.

Count Col %

Destination

Boston Orlando Philadelphia San Diego Total

Arrival

Delayed 615 19%

378 18%

1,230 26%

121 15%

2,344

On Time 2,620 81%

1,760 82%

3,505 74%

677 85%

8,562

Total 3,235 2,138 4,735 798 10,906

TABLE 5.9 Delayed arrivals by destinations.

In addition, most of these flights on American go to Philadelphia, whereas most on Delta go to Boston, as shown in Table 5.10.

5.2 LURKING VARIABLES AND SIMPSON’S PARADOX 91

M05_STIN7167_03_SE_C05_pp080-108.indd 91 26/10/16 12:57 PM

92 CHAPTER 5 Association between Categorical Variables

Table 5.9 answers an odd question, “Am I more likely to arrive on time fly- ing to Boston on Delta or to arrive on time flying to Philadelphia on Ameri- can?” The answer: Take Delta to Boston. There’s nothing wrong with that answer; it’s just an odd question. By focusing the analysis on flights to a specific destination, we control for this lurking variable and answer the right question.

Once you identify a lurking variable, you can remove its effects as we did in this example. But here’s the hard part. How do you know whether there is a lurking variable? It’s easy to imagine other possible lurking variables, too. Maybe it’s the type of airplane, the day of the week, or the time of day. You need to understand the context of your data to find a lurking variable.

5.3 ❘ STRENGTH OF ASSOCIATION In the first example in this chapter, we concluded that Purchase and Host are associated because the proportion of visitors who make purchases differs from host to host. Would you describe the association as weak or strong?

Chi-Squared

To answer this question, it is useful to have a statistic that quantifies the amount of association. Instead of saying “There’s some association” or “There’s a lot of association,” we can quantify the degree of association with the statistic called chi-squared (pronounced “ki squared”). (Some authors and software call the statistic chi-square, without the final “d.”) The larger chi- squared becomes, the greater the amount of association.

Chi-squared compares the observed contingency table to an artificial table with no association. If the tables are similar, then there’s not much association between the observed variables. The greater the difference between the tables becomes, the greater the association.

The following example illustrates the use and calculation of chi-squared. A recent poll asked 200 people at a university about their attitudes toward shar- ing copyrighted music. Half of the respondents were students, and half were staff (administrators or faculty). Table 5.11 summarizes the counts.

tip

chi-squared A statistic that measures association in a contingency table; larger values indicate more association.

tip

Count Row %

Destination

Boston Orlando Philadelphia San Diego Total

Airline

Delta 1,409 43%

1,168 36%

312 10%

366 11%

3,255

American 1,826 24%

970 13%

4,423 58%

432 6%

7,651

Total 3,235 2,138 4,735 798 10,906

TABLE 5.10 Airlines by destinations.

Attitude toward Sharing

OK Not OK Totals

Group Staff 30 70 100

Student 50 50 100

Total 80 120 200

TABLE 5.11 Attitudes toward sharing copyright materials.

Overall, 40% (80 of 200) of those questioned thought it was OK to share copyrighted music. That’s the marginal percentage. Each row determines a conditional distribution, one for staff and one for students. Only 30% of the

M05_STIN7167_03_SE_C05_pp080-108.indd 92 26/10/16 12:57 PM

staff thought it was OK to share, compared to 50% of students. Because the row percentages differ, Group and Attitude are associated.

To quantify the strength of association, we need a benchmark for compari- son, a point of reference. For that, consider what Table 5.11 would look like if there were no association. To figure this out, copy the marginal totals from Table 5.11, but not the counts within the table, as shown in Table 5.12.

Attitude toward Sharing

OK Not OK Total

Group Staff ? ? 100

Student ? ? 100

Total 80 120 200

TABLE 5.12 What goes in these cells if the variables are not associated?

Overall, one-half of the respondents are staff and one-half are students. If Group and Attitude were not associated, then one-half of the cases in each column would be staff and one-half would be students. The table would look like Table 5.13.

Attitude toward Sharing

OK Not OK Total

Group Staff 40 60 100

Students 40 60 100

Total 80 120 200

TABLE 5.13 Expected table if Group and Attitude are not associated.

These artificial counts are called the expected counts—what we expect to find in the cells if the variables are not associated. Chi-squared measures the difference between the counts in the real table and those in the table of ex- pected counts. To calculate chi-squared, first subtract the counts in the cells. The differences in the counts are shown in Table 5.14.

Real Data Expected Difference

30 70 - 40 60 = -10 10

50 50 40 60 10 -10

TABLE 5.14 Deviations from the original counts.

Next, we combine the differences. If we add them, we get zero because the negative and positive values cancel. We had a similar problem with cancel- lation when calculating the variance s2 in Chapter 4. We’ll solve the problem as we did then: square the differences before we add them. Chi-squared also requires another step before we add them.

Chi-squared assigns some of these differences a larger contribution to the total. Look at the differences in the first row. Both are 10 in absolute size, but the difference in the left column is larger relative to the expected count than the difference in the right column (10 out of 40 compared to 10 out of 60). Rather than treat these the same, chi-squared assigns more weight to the first. Saying 40 and finding 30 is a larger proportional error than saying 60 and finding 70. To give more weight to larger proportional deviations, chi-squared divides the squared deviations by the expected counts.

5.3 STRENGTH OF ASSOCIATION 93

M05_STIN7167_03_SE_C05_pp080-108.indd 93 26/10/16 12:57 PM

94 CHAPTER 5 Association between Categorical Variables

The chi-squared statistic is the sum of these weighted, squared differences. In this example, chi-squared, denoted in formulas as x2 is

x2 = 130 - 4022

40 + 170 - 6022

60 + 150 - 4022

40 + 150 - 6022

60

= 1-1022

40 + 11022

60 + 11022

40 + 1-1022

60

= 2.5 + 1.67 + 2.5 + 1.67

= 8.33

We generally use software to compute chi-squared, but it is useful to know what the software is doing in order to understand how it measures associa- tion. To summarize, here are the steps to follow in calculating chi-squared.

1. Make a table with the same margins as the original table, but insert the counts expected if there were no association.

2. Subtract the expected counts from those of the original table, and then square the differences.

3. Divide each squared difference by the corresponding expected count. 4. Sum the weighted, squared deviations over all of the cells.

Chi-squared has the same value if we exchange the rows and columns. It does not matter which variable defines the rows and which defines the columns.

8 If the two variables are not associated, then the percentage who make a purchase among those who join ought to be the same as the marginal percentage, which is 64/4,127, or about 1.55%. The expected count in the first cell is 395 * 64/4,127 < 6.126. 9 Subtract the expected count from the observed count in part a to get the deviation. Then square the deviation and divide by the expected count. The contribution is (52 - 6.126)2/6.126 < 343.5. 10 Yes. The largest contribution comes from the highlighted cell.

What Do You Think? Here’s the contingency table from the prior “What Do You Think?” section, including the marginal totals.

Mailing List

Join Decline Total

Purchase

Yes 52 12 64

No 343 3,720 4,063

Total 395 3,732 4,127

TABLE 5.15 Contingency table of Purchase by Mailing List

a. Chi-squared requires a table of expected counts. What count would be expected in the highlighted cell if Purchase and Mailing List are not associated?8

b. What is the contribution to chi-squared from the highlighted cell?9

c. The value of chi-squared for this table is x2 < 385.9. Does your answer in part b reveal which cell produces the largest contribution to chi-squared?10

Cramer’s V

Chi-squared is hard to interpret. Its value depends on the total number of cases and the size of the table. The larger the table and the larger the number of cases, the larger chi-squared becomes. A more interpretable measure of associa-tip

M05_STIN7167_03_SE_C05_pp080-108.indd 94 26/10/16 12:57 PM

tion allows comparison of the association across tables. Chi-squared for the example of music sharing is 8.33, whereas chi-squared for the mailing-list ex- ercise in Table 5.15 is 385.9. Is there that much more association in the exer- cise, or is chi-squared larger because n = 4,127 in the exercise is so much larger than n = 200 in the music-sharing example.

Cramer’s V adjusts chi-squared so that the resulting measure of associa- tion lies between 0 and 1. If V = 0 , the variables are not associated. If V = 1 , they are perfectly associated. If V 6 0.25, we will say that the association is weak. If V 7 0.75, we will say that the association is strong. In between, we will say there is moderate association.

The formula for Cramer’s V is simpler than writing out the definition. As usual, n stands for the total number of cases, and r is the number of rows and c is the number of columns. The formula for Cramer’s V is

V = B x2n min1r - 1, c - 12 To find Cramer’s V, divide x2 by the number of cases times the smaller of

the number of rows minus 1 or the number of columns minus 1, and take the square root. If V = 0, the two categorical variables are not associated. If V = 1, the two variables are perfectly associated. If variables are perfectly as- sociated, you know the value of one once you know the value of the other. For the survey of file sharing, x2 = 8.33 and both r and c are 2 with n = 200. Hence,

V = B x2200 min12 - 1, 2 - 12 = A8.33200 < 0.20 There’s association, but it’s weak. Staff and students have different attitudes

toward file sharing, but the differences are not very large. For the mailing-list example in Table 5.15, x2 = 385.9, n = 4,127, and r = c = 2. In this case,

V = B x24,127 min12 - 1, 2 - 12 = A385.94,127 < 0.31 There is indeed more association in this example than in the example of file sharing, but not much more. The huge difference between the values of chi- squared is a consequence of the difference in sample sizes, not the strength of the association.

What does a table look like when there is strong association? Strong as- sociation implies very large differences among row or column percentages of a table. Suppose the survey of attitudes toward file sharing had turned out as follows.

Cramer’s V A statistic derived from chi-squared that measures the association in a contingency table on a scale from 0 to 1.

tip

OK to Share Not OK Total

Staff 0 100 100

Students 80 20 100

Total 80 120 200

TABLE 5.16 A table with strong association.

No staff thought it was okay to share files, compared to 80% of the students. You’d expect arguments between staff and students about sharing materials on this campus. Let’s find x2 and Cramer’s V. The margins of Table 5.16 are

5.3 STRENGTH OF ASSOCIATION 95

M05_STIN7167_03_SE_C05_pp080-108.indd 95 26/10/16 12:57 PM

96 CHAPTER 5 Association between Categorical Variables

the same as those in Table 5.11, so the calculation of x2 is similar. We just need to replace the original counts by those in Table 5.16.

x2 = (0 - 40)2

40 +

(100 - 60)2

60 +

(80 - 40)2

40 +

(20 - 60)2

60

= 40 + 26.67 + 40 + 26.67

= 133.33

Cramer’s V indicates strong association between the variables.

V = A133.33200 = 0.816 The size of Cramer’s V implies that you can almost predict what a person will say

if you know whether that person is on the staff or is a student. If you know a person is on the staff, then you know that person thinks it is not okay to share files. Among students, however, you cannot be certain. Most—but not all—favor file sharing.

Checklist: Chi-squared and Cramer’s V. Chi-squared and Cramer’s V measure association between two categorical variables. Before you use these, verify that your data meet these prerequisites.

✓ Contingency table. Make sure that the cells of the table are mutually exclusive counts of two categorical variables.

✓ No obvious lurking variable. A lurking variable means that the associa- tion you’ve found is the result of some other variable that’s not shown.

4M ANALYTICS 5.3 REAL ESTATE

MOTIVATION ▶ STATE THE QUESTION A developer needs to pick heating systems and kitchen cooking appliances for newly built homes. If the home has electric heat, it’s cheaper to install electric appliances. If the home has gas heat, gas appliances make more sense. Should the developer always combine gas appli- ances with gas heat and electric appliances with electric heat? The developer wants to configure homes to suit the preferences of home buyers. Does everyone who heats with gas prefer to cook with gas as well? ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH The developer obtained the preferences of residents in 447 homes. For each, his data give the type of fuel used for cooking and the type used for heating. The rel- evant categorical variables are the preferred types of fuel for heating and for cook- ing. We will measure their association using Cramer’s V. Little or no association implies that we can summarize buyer preferences using marginal distributions. More association means that we need to use conditional distributions to convey the differences in preferences. In that case, we will need to tell the developer how to configure kitchens in homes with gas heat differently from those with electric heat.

MECHANICS ▶ DO THE ANALYSIS The contingency table below shows column percentages, which give the condi- tional distributions of cooking fuel given the type of fuel used for heating. Margin- ally, two-thirds heat with natural gas (298/447) and one-third with electricity.

M05_STIN7167_03_SE_C05_pp080-108.indd 96 26/10/16 12:57 PM

Best Practices

■■ Use contingency tables to find and summarize association between categorical variables. You cannot see the association in marginal bar charts. It only becomes evident when you look at the table and compare the conditional distri- butions with the marginal distributions.

■■ Be on the lookout for lurking variables. Before you interpret association, ask yourself whether there is some other variable that offers a dif- ferent explanation for the association. Are the data in the columns or rows of your table really comparable, or might some other factor explain the association that you see?

■■ Use plots to show association. Stacked bar charts and mosaic plots are useful for compar- ing relative frequencies in larger tables. Adja- cent pie charts are another choice, but these can make it hard to compare percentages un- less the differences are large.

■■ Exploit the absence of association. If the two categorical variables are not associated, the variation of each is self-contained. You do not need the complexity of a table to summarize the variables. You can study each, one at a time.

Pitfalls

■■ Don’t confuse association with causation. You might have found association, but that hardly means that you know why values fall in one category rather than another. Think about the possibility of lurking variables.

■■ Don’t display too many numbers in a table. Com- puters make it easy to decorate a table with too

many percentages. Choose just the ones that you need, those that help you answer the ques- tion at hand. Numerical tables with lots of rows and columns are also overwhelming; summa- rize these with a chart.

PITFALLS 97

The contingency table shows moderate association. Among homes with electric heat, 91% cook with electricity. Among homes with gas heat, 46% cook with elec- tricity. To quantify the strength of the association, we calculated that x2 = 98.62 and V = 21x2>n min1r - 1, c - 12=1198.62>447 * 1 < 0.47. ◀

Count Column %

Home Heat Fuel

Electricity Gas Total

Cooking Fuel

Electricity 136 91.28%

136 45.64%

272

Gas 10 6.71%

162 54.36%

172

Other 3 0.20%

0 0.00%

3

Total 149 298 447

MESSAGE ▶ SUMMARIZE THE RESULTS Homeowners prefer natural gas to electric heat by 2 to 1. These findings sug- gest building about two-thirds of the homes with gas heat and the rest with electric heat. Of those with electric heat, costs might dictate keeping it sim- ple and installing an electric kitchen. For those with gas heat, put an electric kitchen in half and gas in the other half.

Be up front about your concerns. If you have some reservations, mention them. This analysis assumes that buyers who are looking for new homes have the same preferences that these residents have—a big if. ◀

M05_STIN7167_03_SE_C05_pp080-108.indd 97 18/11/16 3:29 PM

98 CHAPTER 5 Association between Categorical Variables

5.1 Analytics in Excel: Car Thefts

the cells in column D to obtain the ratio for all rows of data.

To produce the bar chart, select the disjoint ranges A1:A12 and D1:D12. Then use the command Insert + Chart + Bar to generate a horizontal bar chart.

Open the data file 05_4m_car_theft.csv in Excel. The file has three columns: model name, number stolen, and number made. The data are in alpha- betical order by model name, so the first job is to sort the rows in descending order of the num- ber made. Select the range A1:C212, then use the menu command Data + Sort… In the resulting dialog, choose the column that shows the number made and indicate to sort the values from largest to smallest. The first few rows of the spreadsheet should then appear as shown to the right.

Now add a formula in column D that computes the percentage stolen, the ratio of column B to col- umn C. Label the added column “Pct Stolen” in D1 and then type the formula = B2/C2 into cell D2. Copy and paste this formula into the remainder of

5.2 Analytics in Excel: Airline Arrivals

When you click the OK button, Excel generates the pivot table in a new worksheet (by default). The Pivot Table appears as shown below, which is the first contingency table in the example.

To build a table of the column percentages, follow the instructions above to build a another pivot table with the same components. In the Pivot Table Builder, click the button labeled i next to “Count of Destination” in the Values box. Then click on the Show data as but- ton and select the option “% of column” from the menu.

Open the file 05_4M_arrivals.csv in Excel. The work- sheet has 10,907 rows with 4 columns: flight status, airline, destination, and origin. This analysis re- quires several contingency tables.

The first contingency table counts the number of delayed flights for each airline, combining all 4 des- tinations. We will form this contingency table using an Excel Pivot Table. Select the range A1:D10907, and then use menu command Insert + Pivot Table to open the Pivot Table Builder. In the Pivot Table Builder, select the three columns (Status, Airline, and Destination). Drag the icon for Airline to the Columns box, the icon for Status to the Rows box, and the icon for Destination to the Values box.

M05_STIN7167_03_SE_C05_pp080-108.indd 98 26/10/16 12:57 PM

SOFTWARE HINTS 99

The resulting table is shown below.

To separate the counts by destination, return to the Pivot Table Builder and add Origin to the anal- ysis. Drag the “Count of Destination” item to the Rows box and the Origin column to the Values box. The resulting dialog and table shown below gives the arrival status for both airlines at each destination.

Software Hints

EXCEL Excel has a powerful feature for producing contin- gency tables, but you need to master the concept of pivot tables. If you want to stay with Excel for all of your computing, then it’s worth the effort. Start by reading the Help files that come with Excel by search- ing for “pivot tables” from the Help menu. XLSTAT will build a contingency table and compute the chi- squared statistic at the same time. Use the command

XLSTAT 7 Preparing data 7 Create contingency table

In the dialog, identify the ranges of the two cat- egorical variables. Select the options tab and check the box for the chi-square test. Once you have chi- squared, use the formula to find V.

If you are working without XLSTAT, it is not too hard to compute chi-squared once you have the con- tingency table. Cramer’s V is easy to compute once you have chi-squared. We find it easiest to build a table of expected counts, then subtract this table from the observed table and square each cell. Adding up the squared deviations divided by the expected counts gives chi-squared.

MINITAB EXPRESS To obtain the contingency table, follow the menu items

Statistics 7 Tables 7 Cross@Tabulation and Chi@Square

and fill in the dialog box with the names of two cat- egorical variables. Pick one variable to identify the rows of the table and the other for the columns. It’s an easy calculation to convert chi-squared to Cramer’s V.

JMP Follow the menu commands

Analyze 7 Fit Y by X

and pick one categorical variable for the Y variable and one for X. The variable chosen for Y identifies the columns of the contingency table, and the vari- able chosen for X identifies the rows of the contin- gency table. By default, the output from JMP shows the mosaic plot. The pop-up menu produced by clicking on the red triangle beside the header Con- tingency Table in the output window allows you to

M05_STIN7167_03_SE_C05_pp080-108.indd 99 26/10/16 12:57 PM

100 CHAPTER 5 Association between Categorical Variables

CHAPTER SUMMARY

A contingency table displays counts and may include selected percentages. The totals for rows and columns of the table give the marginal distributions of the two variables. Individual rows and columns of the ta- ble show the conditional distribution of one variable given a label of the other. If the conditional distribu- tion of a variable differs from its marginal distribution, the two variables are associated. Stacked bar charts

and mosaic plots are useful for seeing association in a contingency table. A lurking variable offers an- other explanation for the association found in a table. A lurking variable can produce Simpson’s paradox; the association in a table might be the result of a lurk- ing variable rather than the variables that define the rows and columns. Chi-squared and Cramer’s V are statistics that quantify the degree of association.

■■ Key Terms associated, 83 chi-squared, 92 contingency table, 81 cell, 81 Cramer’s V, 95

distribution, conditional, 82 marginal, 82 lurking variable, 90 mosaic plot, 85 mutually exclusive, 81

Simpson’s paradox, 90 stacked bar chart, 84 symbol V (Cramer’s V), 95 x2 (chi-squared), 92

■■ Objectives • Form a contingency table from two categorical

variables. • Choose between row and column percentages

(conditional distributions) to illustrate the pres- ence of association between categorical variables.

• Connect marginal distributions of a contingency table to bar charts and distributions of a single categorical variable.

• Link conditional distributions in a table to stacked bar charts and mosaic plots.

• Recognize that a lurking variable produces Simp- son’s paradox and consequently explain that Simp- son’s paradox is not such a paradox after all.

• Calculate and interpret measures of associa- tion for categorical variables (chi-squared and Cramer’s V).

■■ Formulas

Chi-Squared Begin by forming a table with the same marginal counts as in the data, but no association. A formula shows how to compute the cells of the artificial table from the marginal counts. Let row

i denote the mar-

ginal frequency of the ith row (the total number of observations in this row), and let col

j denote the mar-

ginal frequency of the jth column (the number in this column). The count for the cell in row i and column j if there is no association is

expectedi, j = rowi * colj

n

A spreadsheet is helpful to organize the calculations. To find x2, sum the weighted, squared deviations

between expected i,j and observed

i,j . Using the summa-

tion notation introduced in Chapter 4, the formula for chi-squared is

x2 = a i,j

1observedi,j - expectedi,j22 expectedi,j

The sum extends over all the cells of the table.

Cramer’s V

V = B x2n min1r - 1, c - 12 for a table with r rows and c columns that summa- rizes n cases.

modify the table by removing, for instance, some of the shown percentages.

The value of chi-squared appears below the table in the section of the output labeled Tests. The value of chi-squared is in the output in the row labeled Pearson. (There are variations on how to compute the chi-squared statistic.)

Test Chi-Squared Prob + Chi-Sq

Likelihood Ratio 16.570 0.0054

Pearson 16.056 0.0067

Once you have chi-squared, use the formula given in the text to obtain Cramer’s V.

M05_STIN7167_03_SE_C05_pp080-108.indd 100 26/10/16 12:57 PM

■■ About the Data The Web shopping data in this chapter (and Chapter 3) are derived from surveys reported by ComScore, a firm that monitors the Web-browsing habits of a sample of consumers around the country. The data on airline arrivals in the 4M example of Simpson’s paradox are from the Bureau of Transportation Statis- tics. (From the main page, follow the links to data that summarize information about various types of travel

in the United States.) The data for kitchen preferences are a subset of the Residential Energy Consump- tion Survey (RECS), performed by the Department of Energy. The example of attitudes toward file sharing is from a story in the Daily Pennsylvanian, the stu- dent newspaper at the University of Pennsylvania. We rounded the counts to simplify the calculations in the example.

1. Table of cross-classified counts a. chi-squared

2. Counts cases that match values of two categorical variables b. mosaic plot

3. Shown in the bar chart of a categorical variable c. associated

4. Shown in a stacked bar chart d. Simpson’s paradox

5. Measure of association that grows with increased sample size e. marginal distribution

6. Measure of association that lies between 0 and 1 f. cell

7. Happens if the conditional distribution matches the marginal distribution g. conditional distribution

8. Identified when percentages within a row differ from marginal percentages h. Cramer’s V

9. Produced by a variable lurking behind a table i. not associated

10. Like a stacked bar chart but respecting the area principle. j. contingency table

Mix and Match

Match the description of the item in the first column to the term in the second column.

EXERCISES

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

11. We can fill in the cells of the contingency table from the marginal counts if the two categorical variables are not associated.

12. We can see association between two categorical vari- ables by comparing their bar charts.

13. The percentages of cases in the first column within each row of a contingency table (row percentages) are the same if the variables are not associated.

14. A large chi-squared tells us that there is strong asso- ciation between two categorical variables.

15. The value of chi-squared depends on the number of cases in a contingency table.

16. Cramer’s V is 0 if the categorical variables are not associated.

17. The value of chi-squared depends on which variable defines the rows and which defines the columns of the contingency table.

18. If variable X is associated with variable Y, then Y is caused by X.

19. If the categorical variable that identifies the supervis- ing manager is associated with the categorical vari- able that indicates a problem with processing orders, then the manager is causing the problems.

20. A small chi-squared statistic suggests that a lurking variable conceals the association between two cat- egorical variables.

21. If the percentage of female job candidates who are hired is larger than the percentage of male candidates who are hired, then there is association between the cat- egorical variables Sex (male, female) and Hire (yes, no).

22. If the percentage of defective items produced by a manufacturing process is about the same on Monday, Tuesday, Wednesday, Thursday, and Friday, then the day of the week is associated with defective items.

Think About It

23. This table shows counts from a consumer satisfaction survey of 2,000 customers who called a credit card company to dispute a charge. One thousand custom- ers were retired, and the remaining were employed. (a) What would it mean to find association between

these variables?

EXERCISES 101

M05_STIN7167_03_SE_C05_pp080-108.indd 101 26/10/16 12:57 PM

102 CHAPTER 5 Association between Categorical Variables

(b) Does the table show association? (You shouldn’t need to do any calculation.)

Employed Retired

Satisfied 697 715

Unsatisfied 303 285

24. This table summarizes the status of 1,000 loans made by a bank. Each loan either ended in default or was repaid. Loans were divided into large (more than $50,000) or small size. (a) What would it mean to find association between

these variables? (a) Does the table show association? (You shouldn’t

need to do much calculation.)

Repaid Default

Large 40 10

Small 930 20

25. A marketing study asked potential customers for their preferences for two attributes of a new prod- uct, the color and the packaging. The color of the final product is to be chosen by one manager and the packaging by another. Is this division of labor simpler if the two variables are associated or if they are not associated?

26. Executives at a retailer are comparing items cho- sen by shoppers who either saw or did not see its most recent television advertising. Do the manag- ers in charge of advertising hope to find association between viewing ads and the choice of items?

27. This chart summarizes explanations given for miss- ing work. The data are the explanations given for 100 absences by employees on the assembly line, adminis- tration, and supervising managers. The explanations are classified as medical, family emergency, or other. (a) For which group are absences due to family

emergencies most common? (b) Are the two variables associated? How can you tell?

28. The following chart summarizes the complaints received at a catalog sales call center by day of the week. The chart shows percentages for each day. The complaints are classified as related to delivery (the order has not arrived), fulfillment (an item was miss- ing from an order or was not what was ordered), size (an item did not fit), and damage. (a) Is the day of the week associated with the nature

of the complaint? Explain why you think these variables are or are not associated.

(b) What supplemental information that is not shown by the chart is important to managers in charge of dealing with complaints?

M on

da y

Tu es

da y

W ed

ne sd

ay

Th ur

sd ay

Fr id

ay

Problems with Orders

Fulfillment Size Damage

Delivery

0

20

40

60

80

100

29. The mosaic plot on the next page shows the sales of cigarettes by the three leading manufacturers, divided into six international regions. (Total sales in 2006 were approximately 2 trillion.) (a) In which region are total cigarette sales the largest? (b) In which two regions does British-American

Tobacco command more than half of the market? (c) Which share leads to higher sales: 80% of the

market in the United States and Canada (like that of Philip Morris) or 80% of the market in Asia and the Pacific?

(d) Can you easily tell what percentage of Philip Morris’s sales is in western Europe?

(e) Are brand and region associated?

M05_STIN7167_03_SE_C05_pp080-108.indd 102 26/10/16 12:58 PM

30. The following table compares market shares of sev- eral companies in the computer industry as reported by Gartner, an industry monitor. The table gives the number of computer systems shipped during each quarter, in thousands.

Company 4th Quarter, 2010 4th Quarter, 2011

HP 17,554,181 14,712,266

Lenovo 10,516,772 12,931,136

Dell 10,796,317 11,633,880

Acer 12,043,606 9,823,214

Asus 5,180,913 6,243,118

Other 37,358,786 36,827,666

(a) Summarize these data using both a stacked bar chart (of market share) and a mosaic plot. What message do these plots convey about the changes in market share from 2010 to 2011?

(b) Explain the differences between the two charts. One plot shows a feature of the data that the other does not show. What is it, and why is the difference subtle in this example?

31. A survey found that the favorite color of everyone who bought a vehicle overall is silver. In this case, should a new car dealer order most pick-up trucks to stock on its lot in silver?

32. A chain of stores suspects that coupons will increase sales to customers in its frequent shopper program. To test this theory, the chain mails some customers coupons and does not send these coupons to others. A manager builds a contingency table of two vari- ables: whether the customer was sent a coupon and whether the customer used the coupon in the store. Both variables have Yes/No values. Must the manager find association?

33. A study of purchases at a 24-hour supermarket recorded two categorical variables: the time of the purchase (8 a.m to 8 p.m vs. late night) and whether the purchase was made by someone with children present. Would you expect these variables to be associated?

34. Among other services, Standard and Poor’s (S&P) rates the risk of default for corporate bonds. AAA bonds are top rated (smallest chance of default), fol- lowed by AA, A, BBB, BB, B, CCC, CC, and last D. In a table of bond ratings crossed with the presence or absence of default, would you expect to see association?

35. Numerous epidemiological studies associate a his- tory of smoking with the presence of lung cancer. If a study finds association (cancer rates are higher among smokers), does this mean that smoking causes cancer?

36. Incidences of flu are much more prevalent in the win- ter months than in the summer. Does this association between respiratory illness and outside temperature imply that cold weather causes the flu?

37. (a) What is Cramer’s V for this contingency table, which shows choices for paint colors and finishes at a hardware store? (You should not need to do any calculations.)

Color

Red Green Blue

Gloss

Low 20 30 40

Medium 10 15 20

High 40 60 80

(b) What is the value of Cramer’s V for this con- tingency table, which reverses the order of the counts in the rows?

0.00

A si

a &

P ac

ifi c

E as

te rn

E u ro

p e

La ti

n

A m

e ri

ca

M id

E as

t &

A fr

ic a

U .S

. &

C an

ad a

W e st

e rn

E u ro

p e

Location

0.25

BA Tobacco

Japan Tobacco

Philip Morris B

ra n d

0.50

0.75

1.00

EXERCISES 103

M05_STIN7167_03_SE_C05_pp080-108.indd 103 26/10/16 12:58 PM

104 CHAPTER 5 Association between Categorical Variables

Color

Red Green Blue

Gloss

High 40 60 80

Medium 10 15 20

Low 20 30 40

(c) What is the implication of the association be- tween these variables for stocking paint at this hardware store?

38. (a) What is the value of Cramer’s V for this contin- gency table? (You should not need to do any calculations.)

Color

Red Green Blue

Gloss

Low 20 0 0

Medium 0 45 0

High 0 0 10

(b) What is the value of Cramer’s V for this contin- gency table, which reverses the position of the nonzero values?

Color

Red Green Blue

Gloss

Low 0 0 20

Medium 0 45 0

High 10 0 0

(c) What is the implication of the association be- tween these variables shown in (b) for stocking paint at this hardware store?

You Do It

Bold names shown with a question identify the data table for the problem.

39. Gasoline sales A service station near an inter- state highway sells three grades of gasoline: regular, plus, and premium. During the last week, the manager counted the number of cars that purchased these types of gasoline. He kept the counts separate for weekdays and the weekend. The data table has two categorical variables. One distinguishes weekdays from weekends, and the other indicates the type of gas (regular, plus, or premium). (a) Find the contingency table defined by the day of

the week and the type of gas. Include the mar- ginal distributions.

(b) Find the conditional distribution of purchase type for weekday purchases.

(c) Find the conditional distribution of premium purchases during the week and weekend.

(d) Does the fact that your answers to parts (b) and (c) are different indicate association?

(e) The owner of the station would like to develop better ties to customers who buy premium gas. (The owner expects these customers to be more affluent and likely to purchase other services from the station.) If the owner wants to meet more of these customers, when should the owner be around the station: on weekdays or weekends?

40. Mens Shirts A men’s clothing retailer stocks a vari- ety of men’s shirts. The data for this question record the type of sales during the last three months, indicat- ing shirt style and size. (You should recognize these data!) (a) Find the contingency table defined by the size

and type of shirt. Include the marginal distributions.

(b) Find the conditional distribution of sizes of polo shirts.

(c) Find the conditional distribution of the styles of large shirts.

(d) How should the manager in charge of stocking the store use this table?

41. The contingency table constructed in Exercise 39 shows counts of the types of gasoline bought during the week and during weekends. (a) Find the value of chi-squared and Cramer’s V for

this table. (b) Interpret these values. What do these tell you

about the association in the table?

42. The contingency table constructed in Exercise 40 shows counts of the sizes of shirts bought in three styles in a retail store. (a) Find the value of chi-squared and Cramer’s V for

this table. (b) Interpret these values. What do these tell you

about the association in the table?

43. Owner Satisfaction A marketing survey of car own- ers was presented in two ways.11 One wording asked the customer whether he was satisfied with his car. The other asked the customer whether he was dissat- isfied. The data table has two variables, one indicat- ing the wording of the question and one giving the level of reported satisfaction. (a) Find the contingency table defined by the level

of satisfaction and the wording of the question. Include the marginal distributions.

(b) Which group reported being more satisfied (either very satisfied or somewhat satisfied) with their cars?

(c) If a company wants to produce an ad citing this type of survey of its customers, how should it word the question?

44. Poll After the collapse of the stock market in October 1987, Business Week polled its readers and asked whether they expected another big drop in the market during the next 12 months.12 The data table for this question has two variables. One indicates whether the

11 R. A. Peterson and W. R. Wilson (1992), “Measuring Customer Satisfaction: Fact and Artifact,” Journal of the Academy of Marketing Science, 20, 61–71. 12 November 9, 1987, p. 36.

M05_STIN7167_03_SE_C05_pp080-108.indd 104 26/10/16 12:58 PM

reader owns stock, and the other gives the anticipated chance for another drop. (a) Find the contingency table defined by stock own-

ership and the anticipated chances for a big drop in the market. Include the marginal distributions.

(b) Did stockholders think that a drop was either somewhat likely or very likely?

(c) Did nonstockholders think that a drop was either somewhat likely or very likely?

(d) Do the differences between parts (b) and (c) make sense? Do these differences imply that the two variables summarized in the table are associated?

45. Exercise 43 compares the responses of customers in a survey depending on how a question about their satisfaction with their car was phrased. (a) Quantify the amount of association between the

type of question and the degree of satisfaction. (b) Combine the counts of very satisfied with some-

what satisfied, and very dissatisfied with some- what dissatisfied, so that the table has only two rows rather than four. What happens to the degree of association if the table merges the satisfied categories into only two rows rather than four?

46. Exercise 44 summarizes results from a survey of stockholders following the October 1987 stock market crash. (a) Quantify the amount of association between the

respondents’ stock ownership and expectation about the chance for another big drop in stock prices.

(b) Reduce the table by combining the counts of very likely and somewhat likely and the counts of not very likely and not likely at all, so that the table has three rows: likely, not likely, and unsure. Compare the amount of association in this table to that in the original table.

47. Employment This table shows percentages of men and women employed in four industries.13

Industry Men Women

Health care 21% 79%

Manufacturing 71% 29%

Retail trade 58% 42%

Book publishing 47% 53%

Legal services 46% 54%

(a) Is there association between the gender of the employee and the industry? How can you tell?

(b) Interpret the association (or lack thereof). (c) Find the chi-squared statistic and Cramer’s V if

these data were derived from n = 500 employees, with 100 in each industry. Repeat the process for data derived from n = 2,000 employees, with 400 in each industry. Which statistic changes? Which remains the same?

48. Credit Rating The best-known credit rating for con- sumers in the United States is the FICO score (named for the company Fair Isaac). The score ranges from 300 to 850, with higher scores indicating that the consumer is a better credit risk. For the following table, consumers with scores below 620 are labeled Risky. Those having scores between 620 and 660 are labeled Uncertain. Those between 660 and 720 have an Acceptable credit rating, and consumers with scores over 720 have Perfect credit. A department store kept track of loans (in the form of a store credit card) given to customers with various ratings. This table shows the proportion of customers within each risk category that did not repay the loan (defaulted).

Score Range Defaulted Repaid

Risky 30% 70%

Uncertain 22% 78%

Acceptable 2% 98%

Perfect 2% 98%

(a) Is the credit score (as defined by these four catego- ries) associated with default? How can you tell?

(b) What would it mean for the use of the FICO score as a tool for stores to rate consumers if there were no association between the category and default?

(c) If virtually all customers at this store have acceptable or perfect credit scores, will the as- sociation be strong or weak? Suppose that the store has 10,000 loans, with 50 made to risky customers, 100 to uncertain customers, 9,000 to acceptable customers, and the rest to perfect customers.

(d) If a higher proportion of the loans had been made to customers who were risky or uncertain, would the association have remained the same, increased, or decreased?

49. Trust Gallup reported the following results from a survey in 2015. The survey asked 1,527 adults to state how much they trust different institutions. This table summarizes the amount of trust placed in four groups.

Group

Great Deal of Trust

Some Trust

Hardly Any

Trust No

Opinion

Banks 28% 45% 26% 1%

Big business 21% 41% 37% 1%

Medical system 37% 37% 25% 1%

Military 72% 19% 7% 2%

Organized religion 42% 32% 23% 3%

(a) Does this table show any association between the group and the amount of trust placed in the group? Interpret the category No Opinion as

13 Bureau of Labor Statistics, 2014.

EXERCISES 105

M05_STIN7167_03_SE_C05_pp080-108.indd 105 26/10/16 12:58 PM

106 CHAPTER 5 Association between Categorical Variables

simply another category. (Hint: You might want to think about your answer to part (b) first.)

(b) Should these data be interpreted as a contingency table?

50. International Markets Business Week magazine asked a sample of investors in the United States and other nations which stock market they expected to produce the best percentage gains over the next 12 months.14 This table shows the percentage of American and foreign investors who picked each country as offering the best stock market. For instance, 31% of American investors thought the best market would be the U.S. market.

Nationality of Investor

U.S. Non-U.S.

Best Market

US 31% 10%

China 27% 22%

Brazil 13% 18%

India 8% 18%

Japan 2% 2%

(a) Does this table indicate association between the nationality of the investor and the choice of best performing market? What does the pres- ence or absence of association tell you about these investors?

(b) Is this a contingency table? (c) Quantify the amount of association present in

this table, if you can.

51. Does the fact that medical researchers get money from drug manufacturers affect their results? The New England Journal of Medicine published a paper that included the following results.15 The table below sum- marizes the tone of articles published by 69 authors during 1996–1997. The publication was labeled either supportive, neutral, or critical. For each author, the data show whether the researchers received support from a pharmaceutical company. (a) Do the data support the claim that by supporting

research, drug companies influence the results of scientists?

(b) Does the table show a plain relationship, or might another variable lurk in the background? What might be its impact? Explain.

Tone of Article

Supportive Neutral Critical

Supported by pharmaceutical

24 10 13

Not supported by pharmaceutical

0 5 17

52. To gauge the reactions of possible customers, the manufacturer of a new type of cellular telephone displayed the product at a kiosk in a busy shopping mall. The following table summarizes the results for the customers who stopped to look at the phone:

Reaction Male Female

Favorable 36 18

Ambivalent 42 7

Unfavorable 29 9

(a) Is the reaction to the new phone associated with the sex of the customer? How strong is the association?

(b) How should the company use the information from this study when marketing its new product?

(c) Can you think of an underlying lurking variable that might complicate the relationship shown here? Justify your answer.

53. These data compare the on-time arrival performance of American and US Airways shortly before their merger in 2015. The table shows the status of 9,350 arrivals during January 2015.

Status American US Airways

On time 3,525 4,083

Delayed 783 1,002

(a) On the basis of this initial summary, find the percentages (row or column) that are appropri- ate for comparing the on-time arrival rates of the two airlines. Which arrives on time more often?

(b) The next two tables organize these same flights by destination. The first also shows arrival time and the second shows airline. Does it appear that a lurking variable might be at work here? How can you tell?

Status Houston Las

Vegas Los

Angeles Philadelphia

On time 731 1,162 2,670 3,000

Delayed 189 227 536 835

Airline Houston Las

Vegas Los

Angeles Philadelphia

American 505 889 2,633 283

US Air 415 500 573 3,552

(c) Each cell of the following table shows the number of on-time arrivals for each airline at each destination. Is Destination a lurking factor behind the original 2 * 2 table?

Airline Houston Las

Vegas Los

Angeles Philadelphia

American 399 732 2,188 206

US Air 332 430 482 2,794

14 Bloomberg Business Week, “Investors Start to Loosen Up,” January 4, 2010. 15 H. T. Stelfox, G. Chua, K. O’Rourke, and A. S. Detsky, “Conflict of Interest in the Debate over Calcium Channel Antagonists,” New Eng- land Journal of Medicine, January 8, 1998, pp. 101–106.

M05_STIN7167_03_SE_C05_pp080-108.indd 106 26/10/16 12:58 PM

54. These data compare the on-time arrival performance of flights on American and Delta. The table shows the status of 17,064 arrivals during January 2006.

Status American Delta

On time 1,536 1,1769

Delayed 416 3,343

(a) On the basis of this initial summary, find the percentages (row or column) that are appropri- ate for comparing the on-time arrival rates of the two airlines. Which arrives on time more often?

(b) The next two tables organize these same flights by destination. The first shows arrival time and the second shows airline. Does it appear that a lurking variable might be at work here? How can you tell?

Status Atlanta Las Vegas San Diego

On time 11,512 1,007 786

Delayed 3,334 244 181

Airline Atlanta Las Vegas San Diego

American 653 698 601

Delta 14,193 553 366

(c) Each cell of the following table shows the number of on-time arrivals for each airline at each destination. Is Destination a lurking factor behind the original 2 * 2 table?

Airline Atlanta Las Vegas San Diego

American 497 561 478

Delta 11,015 446 308

55. 4M ANALYTICS: Discrimination in Hiring

Contingency tables appear frequently in legal cases, such as those that allege that a company has discriminated against a protected class. The following table gives the number of employees of different ages who were laid off when a company anticipated a decline in business:16

Laid Off Retained Total

Age

* 40 18 787 805

40–49 14 632 646

50–59 18 374 392

60 or more 18 107 125

Several long-time employees who were laid off filed a suit against the company for wrongful termination. All were over 50, and they claimed the company discriminated on the basis of age.

To use data like these in a trial requires a more com- plete analysis than we are ready to do. For now, we’ll settle for a descriptive analysis that lays the foundation.

Motivation

(a) From the employees’ point of view. On the basis of the claim of their lawsuit, would the laid- off employees expect to find association in this table?

(b) From the company’s point of view. If the company does not discriminate on the basis of age, would you expect to find association in this table?

Method

(c) From either point of view. Would it be more use- ful to find the percentages within the columns or the percentages within the rows?

(d) What statistic would you choose to represent the presence or absence of evidence of discrimination?

Mechanics

(e) Are age and employment status associated? Compute the appropriate summary statistic and interpret its value.

Message

(f) Summarize what you find in your analysis of the table.

(g) How would the presence of a lurking factor com- promise the use of data such as these in a legal case?

56. 4M ANALYTICS: Picking a Hospital

One effort to improve the efficiency of the health care system provides patients and doctors with more informa- tion about the quality of care delivered by hospitals. The idea is to help consumers find the best hospital to treat their illness.

These data describe patients with breast cancer. Breast cancer is the most frequently diagnosed cancer among women in the United States, with an estimated 207,090 new cases diagnosed and 39,840 deaths in 2010 alone. The data indicate outcomes at two hospitals, Community Hos- pital (CH) and University Hospital (UH), and the type of breast cancer. Early-stage cancers are more easily treated than later-stage cancers. Each cell of the table gives the number of deaths due to breast cancer and the number of surgeries. For example, at the Community Hospital (first row), there were 3 deaths among the 30 surgeries to treat early-stage cancer.

Stage of Cancer

Early Late Both

Hospital CH 3>30 12>21 15>51 UH 5>98 180>405 185>503

16 J. L. Gastwirth, “Statistical Evidence in Discrimination Cases,” Journal of the Royal Statistical Society, Series A 160(1997), 289–303.

EXERCISES 107

M05_STIN7167_03_SE_C05_pp080-108.indd 107 26/10/16 12:58 PM

108 CHAPTER 5 Association between Categorical Variables

Motivation

(a) In general, should patients be allowed to see only the marginal totals for each hospital rather than all of the information in this table? Give the advantages of releasing just the marginal infor- mation (the summary of both types of cancers) versus releasing the full table.

Method

(b) If a woman is diagnosed with breast cancer at an early stage, what probabilities should she focus on when comparing these two hospitals: mar- ginal or conditional?

Mechanics

(c) Create a table that shows the percentages of patients who die from breast cancer in each hos- pital at each stage.

(d) What proportion of cases at the community hospital are late stage? At the university hospital?

(e) Which hospital has the lower death rate for early- stage cancers? Late-stage cancers?

(f) Which hospital has the lower overall death rate from breast cancer?

Message

(g) How should a woman diagnosed with breast cancer interpret the results of these data? Write a few sentences giving guidance in how to interpret these results.

(h) In order to help patients choose a good hospital, is it sufficient to release marginal statistics like the overall death rate from breast cancer? Or is more information needed for patients to inter- pret health outcomes?

M05_STIN7167_03_SE_C05_pp080-108.indd 108 26/10/16 12:58 PM

109

A ssociation between Quantitative Variables

WHAT DOES IT COST TO HEAT A HOME FOR THE WINTER? You may not think about heating costs if you live in a dorm, but homeowners do.

Many factors influence these costs, such as the local climate. Homes in Minnesota need more heating than those in Florida. The reason? The average high temperature in Minnesota is 16 degrees in January compared to 75 degrees in Florida. Other factors influence costs as well. For example, bigger homes require more heat, and well-insulated homes need less. The setting on the thermostat matters, too.

This combination of factors produces a wide range in heating costs. For example, Figure 6.1 summarizes the annual consumption of natural gas of 300 homes in the United States (in thousands of cubic feet, abbreviated MCF).

The variation in consumption implies variation in costs. Natural gas sells for about $10 per MCF. The owner of the home with the highest consumption in Figure 6.1 used 223 MCF and so paid $2,230 for the year. By comparison, owners of homes at the far left spent less than $100. How much of this variation in consumption—and cost—can we anticipate from knowing the local climate?

6c h a p t e r

6.1 SCATTERPLOTS

6.2 ASSOCIATION IN SCATTERPLOTS

6.3 MEASURING ASSOCIATION

6.4 SUMMARIZING ASSOCIATION WITH A LINE

6.5 SPURIOUS CORRELATION

6.6 CORRELATION MATRIX

CHAPTER SUMMARY

This chapTer shows how To use The associaTion beTween numerical variables To undersTand and explain variaTion. Once we measure the amount of association, we can use this quantity to predict one variable from the other. For example, we can predict the consumption of natural gas from the local climate. The methods in this chapter are analogous to those described in Chapter 5 but take advantage of properties of numerical data.

FIGURE 6.1 Annual use of natural gas per household, in thousands of cubic feet (MCF).Natural Gas (MCF)

0 50 100 150 200

20

40

60

C o

u n

t

M06_STIN7167_03_SE_C06_pp109-139.indd 109 25/10/16 12:17 PM

6.1 ❘ SCATTERPLOTS The National Weather Service and Department of Energy use heating degree days (HDD) to measure the severity of winter weather. The larger the number of heating degree days becomes, the colder the weather. For example, a typical winter in Florida produces less than 1,000 HDD compared to 9,000 in Minne- sota. Figure 6.2 shows the distribution of the number of HDD experienced by the homes in Figure 6.1.1

1 To calculate HDD for a day, average the high and low temperatures. If the average is less than 65, then the number of heating degree days for that day is 65 minus the average. For example, if the high temperature is 60 and the low is 40, the average is 50. This day has 65 - 50 = 15 HDD. If the average of the high and low is above 65, HDD = 0. According to HDD, you don’t need heat on such days!

FIGURE 6.2 Heating degree days around the United States.

1,000 3,000 5,000 7,000

C o

u n t

10

20

30

Heating Degree Days

The homes on the left are evidently in Florida, whereas those at the right appear to be in Minnesota (or other locations as warm or cold, respectively).

For heating degree days to explain variation in the use of natural gas, these variables must be associated. Association between numerical vari- ables resembles association between categorical variables (Chapter 5). Two categorical variables are associated if the distribution of one depends on the value of the other. That’s also true for numerical variables, but the methods for seeing and quantifying association differ. In place of a contin- gency table, we use a scatterplot to look for association between numeri- cal variables.

A scatterplot displays pairs of numbers. In this example, we have 300 pairs, one for each home. One element of each pair is the number of heating degree days and the other is the amount of natural gas. The scales of these variables determine the axes of the scatterplot. The vertical axis of the scat- terplot is called the y-axis, and the horizontal axis is the x-axis. The variable that defines the x-axis specifies the horizontal location of a specific observa- tion, and the variable that defines the y-axis specifies the vertical location of the same observation. Together, a pair of values defines the coordinates of a point, which is usually written (x, y). To decide which variable to put on the x-axis and which to put on the y-axis, display the variable you would like to explain or predict along the y-axis. We will call the variable on the y-axis the response. The vari- able that we use to explain variation in the response is the explanatory variable; it goes on the x-axis. In this example, the amount of natural gas is the response, and the number of heating degree days is the explanatory variable.

Figure 6.3 shows the completed scatterplot. Each point in the plot represents one of the 300 households. For example, the point marked with an * represents a household for which (x, y) = (2092 HDD, 129 MCF).

xx

(x, y)

y

y tip

scatterplot A graph that displays pairs of values as points on a two-dimensional grid.

response Name for the vari- able that has the variation we’d like to understand, placed on the y-axis in scatterplots.

explanatory variable Name for a variable that is to be used to explain variation in the response; placed on the x-axis in scatterplots.

110

M06_STIN7167_03_SE_C06_pp109-139.indd 110 18/11/16 3:11 PM

6.2 ASSOCIATION IN SCATTERPLOTS 111

1,000 0

50N at

u ra

l G as

(M C

F )

100

150

200

2,000 3,000 4,000 Heating Degree Days

5,000 6,000 7,000 8,000FIGURE 6.3 Scatterplot with histograms along the axes.

Figure 6.3 adds the histogram of the response on the right and the his- togram of the explanatory variable at the top. These histograms show the marginal distributions of the variables in a scatterplot. These histograms are analogous to the marginal counts of the categorical variables in a con- tingency table. As with contingency tables, we cannot judge the presence of association from marginal distributions. For a contingency table, we look for patterns in the cells; for a scatterplot, we look for patterns in the display of points.

6.2 ❘ ASSOCIATION IN SCATTERPLOTS Is the amount of natural gas used associated with heating degree days? Can we explain differences in consumption by taking account of differences in cli- mate? It appears that we can: The points in Figure 6.3 generally shift upward as we move from left to right, from warmer to colder climates. This upward drift means that homes in colder climates typically use more natural gas for heating than homes in warmer climates.

Visual Test for Association

Before we conclude that there’s association between HDD and gas use, we need to decide whether this pattern we claim to see in Figure 6.3 is real. Is there a pattern that relates degree days to gas use, or are we imagining it?

Here’s a simple way to decide. A pattern in a scatterplot means that the value of the x coordinate of a pair tells us about the y coordinate. If we know x, we can use it to guess y. To see whether x and y are related, we compare the scatterplot of our data to one in which there is no pattern. Recall how we measure association in a table (Chapter 5). The chi-squared statistic compares a contingency table to an artificial table that forces the variables to be unrelated. The artificial table retains the marginal totals but removes any association between the variables. We do the same with scat- terplots: Compare the scatterplot that we see to a scatterplot that has the same marginal distributions but no association. It’s easy to do. Split up the observed (x, y) pairs, and then form new pairs at random. Pair each value of y with a randomly chosen value of x. If the scatterplot of the randomly paired data looks like the scatterplot of the original data, then there’s little or no association.

M06_STIN7167_03_SE_C06_pp109-139.indd 111 25/10/16 12:17 PM

112 CHAPTER 6 A ssociation between Quantitative Variables

Rather than measure the distance between two scatterplots, this proce- dure relies on a visual comparison. As an example, one of the scatterplots in Figure 6.4 is the scatterplot of gas use versus heating degree days (Figure 6.3). The other three scatterplots randomly pair the number of heating degree days with the amount of gas used. Do you recognize the original?

Heating Degree Days

0

50

100

N at

u ra

l G as

(M C

F )

150

200

1,000 3,000 5,000 7,000

Heating Degree Days

N at

u ra

l G as

(M C

F )

0

50

100

150

200

1,000 3,000 5,000 7,000

Heating Degree Days

N at

u ra

l G as

(M C

F )

0

50

100

150

200

1,000 3,000 5,000 7,000

Heating Degree Days

N at

u ra

l G as

(M C

F )

0

50

100

150

200

1,000 3,000 5,000 7,000

FIGURE 6.4 Do you recognize the original scatterplot?

The original scatterplot is at the lower right. In this frame, the amount of gas used is lower at the left (less in warm climates) and higher at the right (more in cold climates). In the other plots, the distribution of gas use is the same regardless of the climate. This comparison of the original plot to several artificial plots in which the variables are unrelated is the visual test for association. If the original plot stands out, there’s association. Otherwise, the vari- ability in the response is not related to the explanatory variable. Because the original plot stands out in Figure 6.4, there’s association between HDD and the amount of natural gas.

tip

visual test for association A method for identifying association between numerical variables. Compare the original scatterplot to others that randomly match the coordinates.

What Do You Think? The Internal Revenue Service (IRS) audits tax returns to ensure that the tax paid is reasonable for the amount of income. The following three plots display the amount of taxes paid versus the reported total income (both in thousands of dollars) for 45 taxpayers with incomes between $100,000 and $200,000. One plot shows the actual (income, tax) pairs; the other two scramble the pairs.

M06_STIN7167_03_SE_C06_pp109-139.indd 112 25/10/16 12:17 PM

a. Does the IRS need total income to be associated with the taxes paid if such data are to help it identify tax cheats?2

b. Are the taxes paid associated with the total income?3

2 Yes. If associated, the IRS can anticipate the amount of taxes from the income. 3 Yes. The middle plot is the real plot and is distinct from the other two.

Total Income

$25

$20

T ax

$15

$10

$100 $120 $140 $160 $180 $200

Total Income

$25

$20

T ax

$15

$10

$100 $120 $140 $160 $180 $200

Total Income

$25

$20

T ax

$15

$10

$100 $120 $140 $160 $180 $200

6.2 ASSOCIATION IN SCATTERPLOTS 113

small variation large variation

an outlier

Describing Association in a Scatterplot

Once you decide that a scatterplot shows association, then you need to de- scribe the association. To describe the association, start with its direction. In this example, the colder the winter, the larger the gas consumption tends to be. This pattern has a positive direction because the points in the scatterplot tend to concentrate in the lower left and upper right corners. As the explana- tory variable increases, so does the response. A pattern running the other way has a negative direction. As x increases, y tends to decrease.

Another property of the association is its curvature. Does the pattern resemble a line, or does it bend? The scatterplot of the natural gas use versus HDD appears linear. The points roughly concentrate along an upward-sloping line that runs from the lower left corner to the upper right. Linear patterns have a consistent direction. Linear patterns with positive direction follow a line with a constant positive slope; those with negative direction follow a line with a negative slope. Curved relationships are harder to describe because the direction changes.

The third property of the association is the amount of variation around the pattern. In the case of insulation, there’s quite a bit of variation among homes in similar climates. For instance, among homes in a climate with 6,000 HDD, some use 40 MCF of gas compared to others that use five times as much. In ad- dition, some homes in cold climates use less gas than others in warm climates. The variation around the linear pattern in gas use also appears to increase with the amount used. The points stick closer to the linear pattern in the warm cli- mates at the left of Figure 6.3 than in the colder climates on the right.

Finally, look for outliers and other surprises. Often the most interesting as- pect of a scatterplot is something unexpected. An outlying point is almost always interesting and deserves special attention. Clusters of several outliers raise ques- tions about what makes the group so different. In Figure 6.3, we don’t see outliers so much as an increase in the variation that comes with higher con- sumption of natural gas. A household that consumed 400 MCF of gas would be an outlier.

Let’s review the questions to answer when describing the association that you see in a scatterplot.

1. Direction. Does the pattern trend up, down, or both? 2. Curvature. Does the pattern appear to be linear, or does it curve? 3. Variation. Are the points tightly clustered along the pattern? Strong asso-

ciation means little variation around the trend. 4. Outliers and surprises. Did you find an isolated point or something

unexpected?

+ positive

- negative

tip

nonlinear

M06_STIN7167_03_SE_C06_pp109-139.indd 113 25/10/16 12:17 PM

114 CHAPTER 6 A ssociation between Quantitative Variables

Don’t worry about memorizing this list. The key is to look at the scatterplot and think. We’ll repeat these steps over and over again, and soon the sequence will become automatic.

4 The diamonds are on the left, with positive association. The pattern is vaguely linear, with quite a bit of variation around the trend. The movies are on the right with little or no association. The rating assigned by the critic has little to do with popularity at the box office.

What Do You Think? One of the following scatterplots shows the prices of diamonds versus their weights. The other shows the box-office gross in dollars of movies versus a critic’s rating on a 5-point Likert scale. Identify which is which and describe any association.4

(a) (b)

6.3 ❘ MEASURING ASSOCIATION Two statistics quantify the amount of association. The first statistic, the cova- riance, does most of the work but is difficult to interpret. The second statistic, the correlation, converts the covariance into an easily interpreted value. This conversion is analogous to using Cramer’s V to make chi-squared more inter- pretable in Chapter 5.

Covariance

Covariance quantifies the strength of the linear association between two nu- merical variables. It measures the degree to which data concentrate along either a positive or a negative diagonal line in a scatterplot. To see how covari- ance works, let’s continue with the energy data. Figure 6.5 divides the scatter- plot of gas consumption versus HDD into four colored quadrants.

covariance A statistic that measures the amount of linear association between two numerical variables.

1,000 3,000 5,000 7,000 9,000 Heating Degree Days

200

250

150

100

50

0

N at

ur al

G as

(M C

F)

+ -

- +

FIGURE 6.5 Scatterplot of the amount of natural gas versus HDD with reference lines at the means.

M06_STIN7167_03_SE_C06_pp109-139.indd 114 25/10/16 12:17 PM

The horizontal and vertical gray lines locate the means of the two variables and divide the plot into four quadrants. Green points in the upper right and lower left quadrants identify households for which both energy use and heating de- gree days are either both larger or both smaller than average. For instance, a household in the upper right quadrant is in a colder than average climate and uses more than the average amount of gas. Red points in the other two quad- rants identify households for which one variable is larger than its mean and the other is smaller. For instance, homes in relatively warm locations that use more gas than average produce the red points in the upper left quadrant.

Green points indicate positive association. Both variables are simultaneously larger or smaller than average. Above-average HDD comes with above-average gas use, and below-average HDD comes with below-average gas use. Red points in the other two quadrants suggest negative association: One variable is relatively large, whereas the other is relatively small. The prevalence of green points in Fig- ure 6.5 suggests that gas use and heating degree days are positively associated.

Covariance does more than count. It also takes into account how far each point lies from the pair of means. Points that are far from (x, y) contribute more to the covariance than those that are close to (x, y). To see how the co- variance determines the contribution of each point, look at Figure 6.6, which zooms in on one point with coordinates (x, y) in Figure 6.5. The rectangle in this figure has one corner at (x, y), and the opposite corner at (x, y). The area of this rectangle is (x - x) (y - y).

x - x

y - y

(x , y )

(x , y)

FIGURE 6.6 Deviations from the means in the scatterplot.

Both deviations x - x and y - y in Figure 6.6 are positive. Hence, their product (x - x) (y - y) is positive. That’s true for all of the points in the two green quad- rants in Figure 6.5 because both deviations of the green points have the same sign.

For the red points in two quadrants, the product is negative. One variable is above its mean (positive deviation), and the other is below its mean (negative de- viation). Hence, the product (x - x) (y - y) is positive for green points indicating positive association and negative for red points indicating negative association. Points with either deviation equal to zero don’t contribute to the covariance.

The covariance is (almost) the average of the signed areas of the rectangles defined by the data. The formula for the covariance is

cov1x, y2 = 1x1 - x2 1y1 - y2 + 1x2 - x2 1y2 - y2 + g + 1xn - x2 1yn - y2

n - 1

As in Chapter 4, subscripts identify rows of the data table; x1 - x denotes the deviation of the heating degree days of the home in the first row of the data table (x

1 ) and the mean HDD (x). Similarly, y1 - y denotes the deviation of

the gas used by the first home (y 1 ) and the mean gas use (y). The divisor n − 1

matches the divisor in the variance s2, defined in Chapter 4. Because the energy example has a large number of cases (n = 300), we

used a computer to compute the covariance between heating degree days and gas use. To illustrate how the data enter into the formula, Table 6.1 lays out the calculation of the covariance as if we had only 6 homes rather than 300.

The fourth and fifth columns of Table 6.1 subtract means from the observed number of heating degree days and the amount of gas used. These two columns

6.3 MEASURING ASSOCIATION 115

M06_STIN7167_03_SE_C06_pp109-139.indd 115 25/10/16 12:17 PM

116 CHAPTER 6 A ssociation between Quantitative Variables

Case

HDD Gas Use Deviation from Mean Product of Deviations

x y x - x y - y (x - x)(y - y)

1 2068 35.6 - 1943.17 - 60.65 117853.0583

2 3641 114.4 - 370.17 18.15 - 6718.5250

3 1848 51.5 - 2163.17 - 44.75 96801.7083

4 6862 183.9 2850.83 87.65 249875.5417

5 7386 131.7 3374.83 35.45 119637.8417

6 2262 60.4 - 1749.17 - 35.85 62707.6250

Sum 24067 577.5 0 0 640157.2500

x = 4011.167 y = 96.25 Cov(x, y = 128031.45

TABLE 6.1 Calculating the covariance for six homes.

each sum to zero, as explained in Chapter 4. The last column multiplies the devia- tions. This column gives the contribution of each home to the covariance. The covariance between HDD and the amount of gas used for these six homes is then

cov1HDD, Gas2 = 640,157.25 5

= 128,031.45

The same calculation, but with the full collection of n = 300 homes, gives a smaller value: cov(HDD, Gas) = 56,308.9.

The covariance confirms the presence of positive linear association, but it is difficult to interpret. Positive covariance is consistent with our intuition. On average, homes in colder climates use more than the average amount of natural gas. The size of the covariance, however, is difficult to interpret be- cause the covariance has units. The units of the covariance are those of the x-variable times those of the y-variable. In this case, the covariance is 56,308.9 HDD * MCF. That’s a large number, but that does not imply a lot of associa- tion. Changes in the scales of the data make the covariance larger or smaller. For example, the covariance would be 10 times larger if gas were measured in hundreds rather than thousands of cubic feet.

Correlation

Correlation is an easily interpreted measure of linear association derived from the covariance. The correlation is the covariance divided by the product of the standard deviations.

corr1x, y2 = cov1x, y2 sx sy

Because the standard deviations of two variables appear in this calculation, we use subscripts to distinguish them. The correlation between HDD and gas usage is

corr1HDD, Gas2 = cov1HDD, Gas2 sHDD sGas

= 56,308.9 HDD * MCF

2,243.8 HDD * 43.41 MCF < 0.58

The units of the two standard deviations cancel the units of the covariance. The resulting statistic, the correlation, does not have units and can be shown always to lie between -1 and +1,

-1 … corr1x, y2 … +1 The correlation is usually denoted by the letter r.

correlation (r) A standardized measure of linear association between two numerical variables; the correlation is always between –1 and +1.

tip

M06_STIN7167_03_SE_C06_pp109-139.indd 116 25/10/16 12:17 PM

Because r = 0.58 is much less than 1, the data for gas consumption show considerable variation around the linear pattern. Homes in the same climate vary considerably in how much gas they use. Other factors beyond climate, such as the size of the home and the thermostat settings, affect consumption.

Because the correlation has no units, it is unaffected by the scale of mea- surement. For instance, suppose the data measured gas use in cubic me- ters and climate in degrees Celsius. The correlation would still be r = 0.58. Figure 6.7 shows why the correlation is the same. The scatterplot on the left has the original units, and the one on the right has metric units.

Heating Degree Days

N at

u ra

l G as

(M C

F )

0

50

100

150

200

1,000 3,000 5,000 7,000

Heating Degree Days (Celsius)

N at

ur al

G as

(M C

M )

0

1

2

3

4

6

5

1,000 2,000 3,000 4,000

FIGURE 6.7 Scales affect the axes but not the content of a scatterplot.

If you hide the axes, these scatterplots are the same. Only the labels on the axes differ. These differences don’t affect the direction, curvature, or variation in the scatterplot, so they don’t change the strength of the association. The correlation is the same.

The correlation can reach -1.0 or +1.0, but these extremes are unusual. They happen only if all the data fall on a diagonal line. For example, Figure 6.8 shows the temperature at 50 locations on both Fahrenheit and Celsius scales.

6.3 MEASURING ASSOCIATION 117

0

10

20

30

40

50

60

70

80

90

100

-20 -10 0 10 20 30 40

Fa hr

en he

it

Celsius

FIGURE 6.8 Celsius temperatures are perfectly associated with Fahrenheit temperatures.

M06_STIN7167_03_SE_C06_pp109-139.indd 117 25/10/16 12:17 PM

118 CHAPTER 6 A ssociation between Quantitative Variables

r = 0.95 r = -0.95

r = 0.75 r = -0.75

r = 0.50 r = -0.50

r = 0.25 r = -0.25

r = 0

FIGURE 6.9 Correlation measures the tendency of data to concentrate along a diagonal of the scatterplot.

Strong Correlation

Moderate Correlation

Weak Correlation

M06_STIN7167_03_SE_C06_pp109-139.indd 118 25/10/16 12:17 PM

The correlation r = 1 because a line expresses one temperature exactly in terms of the other 1Fahrenheit = 32 + 1.8 * Celsius2. Data on a line with negative slope imply r = -1.

Keep these properties of the correlation in mind:

1. r measures the strength of linear association. 2. r is always between -1 and +1, -1 … r … 1. 3. r does not have units.

To help you appreciate the relationship between r and the strength of lin- ear association, Figure 6.9 shows nine scatterplots of 200 observations and their correlations. The values of r run from r = {0.95 in the first row down to r = 0 at the bottom. The larger the magnitude of r, the tighter the points cluster along a diagonal line. Roughly speaking, plots like those with r < {0.95 display strong association, those with r < {0.50 or r < {0.75 display moderate association, and those with r < {0.25 have weak asso- ciation. The pattern in a scatterplot tilts up when the correlation is pos- itive (plots on the left) and tilts down when the correlation is negative (right).

The linear pattern is subtle if r is close to 0. Indeed, you may not see any pattern in the two scatterplots with r = {0.25. That’s because when r is close to zero, the variation around the pattern obscures the pattern. In these cases, most of the variation in y is unrelated to x.

The size of a typical correlation depends on the context. Economic vari- ables that track the output of a nation or company over time produce cor- relations approaching 1. The correlation is large because all of the variables measure the same thing from different points of view. Figure 6.10(a), for ex- ample, graphs the total household credit debt on disposable income in the United States, measured quarterly in billions of dollars since 1960. The cor- relation is 0.98. Both series grow with the U.S. economy. Correlations based on the behavior of individuals tend to be small; people are less consistent in their behavior. Figure 6.10(b) graphs the rating assigned to a smartphone by consumers of various ages. The correlation is about 0.60, which is large for a consumer survey but nothing like the typical correlation between macroeco- nomic series.

Disposable Income ($B)

H o

us eh

o ld

C re

d it D

eb t

($ B

)

0 2,000 4,000 6,000 8,000 10,000 12,000 0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

Raters Age

P ro

d uc

t R

at in

g

20 30 5040 60 70 80 0

1

2

3

4

5

6

7

8

9

FIGURE 6.10 Correlations between macroeconomic variables often approach 1 (left), but smaller correlations are typical for behavioral data (right).

6.3 MEASURING ASSOCIATION 119

M06_STIN7167_03_SE_C06_pp109-139.indd 119 25/10/16 12:17 PM

120 CHAPTER 6 A ssociation between Quantitative Variables

5 There’s weak positive association, vaguely linear. It looks like the variation around the pattern increases with the size of the home. 6 The units of the covariance are those of x times those of y, so that’s 17,848 (square feet * MCF). 7 The correlation is 17,848/(869.5 * 43.41) < 0.473. 8 The correlation confirms that there is some linear association, but it’s not as strong as in the text example. These homes are in different climates; a large house does not require much heating if it’s in Florida.

What Do You Think? The following scatterplot shows the amount of natural gas used versus an- other variable, the size of the home measured in the number of square feet:

0 2,000 4,000 6,000 0

50

100

150

200

250

Square Feet

N at

ur al

G as

(M C

F)

a. Describe the association between the two variables.5

b. The covariance between these variables is 17,848. What are the units of the covariance?6

c. The SD of natural gas is sy = 43.41 MCF, and the SD of the size is sx = 869.5 sq ft. What is the correlation between the use of natural gas and the size of the home?7

d. Explain the size of the correlation. Is there a reason that the correlation is not larger?8

6.4 ❘ SUMMARIZING ASSOCIATION WITH A LINE Correlation measures the strength of linear association between two variables. The larger u r u becomes, the more closely the data cluster along a line. We can use r to find the equation of this line. Once we have this equation, it is easy to predict the response from the explanatory variable.

The simplest expression for this equation uses z-scores. A z-score (Chap- ter 4) is a deviation from the mean divided by the standard deviation. The correlation converts a z-score of one variable (say, heating degree days) into a z-score of the other (gas use). If we know that a home is, for instance, lo- cated in a climate 1 SD above the mean of HDD, then we expect to find its use of natural gas r SDs above the mean of gas consumption. For homes in a climate 1 SD below the mean, we expect their gas use to average r SDs below the mean. Since u r u … 1, the line associated with the correlation pulls values toward the mean.

M06_STIN7167_03_SE_C06_pp109-139.indd 120 25/10/16 12:17 PM

We can express this relationship as a simple equation. Consider a home in a climate with x degree days that uses y MCF of natural gas. Let zx denote the z-score for its climate, and let zy denote the z-score for its use of natural gas:

zx = 1x - x2>sx zy = 1y - y2>sy The line determined by the correlation is

zny = r zx

The ^ over zy distinguishes what this formula predicts from the real thing. Un- less r = {1, the data are not exactly on a single line. If r = 1, this line would match the data. If r = 0, this line is flat; it predicts zny = 0 for all values of x.

Slope-Intercept Form

The equation zny = r zx is compact, but it is common to see this equation ex- pressed directly in terms of x and y. The converted equation is easier to add to a plot. A bit of algebra shows that the line associated with the correlation can be written as

yn = a + bx

with a = y - bx and b = r sy>sx The term a is known as the y-intercept and b is the slope. The y-intercept a shows where the line intersects the y-axis, and the slope b determines how rap- idly the line rises or falls. This way of writing the line is called slope-intercept form. Once we have a and b, we simply plug in a value for x and the equation predicts y.

As an example, let’s find the line that describes the association between gas use ( y) and HDD (x). The underlying statistics for gas usage are y = 95.9 MCF and sy = 43.41 MCF; those for HDD are x = 4735.2 HDD and sx = 2243.8 HDD. Hence the slope of the line is

b = r sy>sx = 0.58143.41>2243.82 < 0.0112 MCF>HDD The slope has units: those of y (MCF from sy) divided by those of x (HDD from sx). The intercept is

a = y - b x = 95.9 - 0.0112 * 4735.2 < 42.9 MCF

The intercept takes its units from y (b converts the units of x into the units of y). Figure 6.11 adds the correlation line to the scatterplot of natural gas usage on heating degree days.

y-intercept The location at which a line intersects the y-axis in a graph.

slope Rate of change of a line; steepness of the line.

FIGURE 6.11 The correlation defines a line that summarizes the association between HDD and usage of natural gas.

1,000 3,000 5,000 7,000 Heating Degree Days

N at

ur al

G as

(M C

F)

200

100

0

6.4 SUMMARIZING ASSOCIATION WITH A LINE 121

M06_STIN7167_03_SE_C06_pp109-139.indd 121 25/10/16 12:17 PM

122 CHAPTER 6 A ssociation between Quantitative Variables

Pay attention to the scales of a and b when interpreting these statistics. The intercept indicates that on average homes with no need for heat (zero heat- ing degree days) evidently use some gas (a = 42.9 MCF) for other purposes, such as heating water or cooking. The slope indicates that the amount of gas used goes up by about 11.2 MCF (1,000 times b = 0.0112 MCF/HDD) when we compare the amount of gas used by homes in a climate with, say, 2,000 HDD to those in a colder climate with 3,000 HDD.

Lines and Prediction

Once we know where someone lives, we can use the equation of this line to guess his or her gas usage and the cost of that gas. (Few homeowners know offhand how much they spend annually for heating.) For example, we could offer an interactive Web ad: “Click to find typical savings from insulating!”

Suppose that a home is located in a cold climate with 8,800 HDD. Let’s esti- mate the amount of gas used by this home using both forms of the correlation line. This climate is about 1.8 SDs above the mean number of heating degree days:

zx = x - x

sx =

8,800 - 4,735.2 2,243.8

< 1.8

The positive association between heating degree days and gas usage suggests that gas use at this home is about r * 1.8 < 1.04 SDs above the mean of nat- ural gas used, at

y + rsy = 95.9 + 1.04143.412 < 141 MCF Using the line in the slope-intercept form gets the same answer more directly. Gas use in this climate averages yn = a + bx = 42.9 + 0.011218,8002 < 141 MCF. At $10 per MCF, this gas would cost $1,410. If the savings from insulating were 30% of the predicted costs, then the homeowner would save 0.3 * $1,410 = $423.

Let’s present another example, this time for a warmer climate. For a home that experiences 2,500 HDD, the line predicts the gas usage to be yn = 42.9 + 0.0112 (2,500) < 70.9 MCF. Because we expect less to be used, the predicted savings are smaller, only 0.3 * $709 < $213.

Nonlinear Patterns

caution If the association between numerical variables is not linear, a line may be a poor summary of the pattern. Linear patterns are common but do

not apply in every situation.

Chi-squared (Chapter 5) measures any type of association between categori- cal variables. Covariance and correlation measure only linear association. If the pattern in a scatterplot bends, then covariance and correlation miss some of the association. Be sure that you inspect the scatterplot before relying on these statistics to measure association.

For example, the scatterplot in Figure 6.12 shows data on employees of a small firm. The x-axis gives the age of each employee, and the y-axis shows the cost in dollars to the employer to provide life insurance benefits to each employee.

You can see a pattern, but it’s not linear. The direction of the pattern changes as suggested by the curve in the figure. The cost of providing life in- surance grows as young employees pass through their family years and opt for more insurance. As employees age, fewer want life insurance, and the cost to provide this benefit shrinks.

Correlation does not identify this association because the pattern is not lin- ear. In fact, the correlation between age and cost in Figure 6.12 is near zero (r = 0.09). When you think about the correlation and its associated line, this

M06_STIN7167_03_SE_C06_pp109-139.indd 122 25/10/16 12:17 PM

value for r makes sense. In a way, the overall slope is zero; the positive slope on the left cancels the negative slope on the right.

A problem occurs if benefit managers compute a correlation without look- ing at the scatterplot. They might conclude from the small value of r that these two variables are unrelated. That mistake could produce poor decisions when it comes time to anticipate the cost to the firm to provide this benefit.

6.5 ❘ SPURIOUS CORRELATION A scatterplot of the damage (in dollars) caused to homes by fire would show a strong correlation with the number of firefighters who tried to put out the blaze. Does that mean that firefighters cause damage? No, and a lurking vari- able, the size of the blaze, explains the superficial association. A correlation that results from an underlying, lurking variable rather than the shown x- and y-variables is often called a spurious correlation.

Scatterplots and correlation reveal association, not causation. You have to be cautious interpreting association in a contingency table (recall Simpson’s paradox), and the same warning applies to scatterplots. Lurking variables can affect the relationship in a scatterplot in the same way that they affect the re- lationship between categorical variables in a contingency table.

Interpreting the association in a scatterplot requires deeper knowledge about the variables. For the example concerning energy use in this chapter, it seems sensible to interpret the scatterplot of gas usage and heating degree days as meaning that “colder weather causes households to use more natu- ral gas.” We feel pretty good about this interpretation because we’ve lived in homes and had to turn up the heat when it got cold outside. This interpreta- tion makes sense, but it’s not proof.

What other factors might affect the relationship between heating degree days and the amount of gas that gets used? We already mentioned two: ther- mostat settings and the size of the home. How might these factors affect the relationship between heating degree days and gas use? Maybe homes in colder climates are bigger than those in warmer climates. That might explain the larger amount of gas used for heating. It’s not the climate; it’s the size of the home. These alternatives may not seem plausible, but we need to rule out other explanations before we make decisions on the basis of association.

spurious correlation Cor- relation between variables due to the effects of a lurking variable.

FIGURE 6.12 The cost of insurance does not continue to increase.

6.5 SPURIOUS CORRELATION 123

M06_STIN7167_03_SE_C06_pp109-139.indd 123 25/10/16 12:17 PM

124 CHAPTER 6 A ssociation between Quantitative Variables

Checklist: Covariance and Correlation. Covariance and correlation measure the strength of linear association between numerical variables. Before you use correlation (and covariance), verify that your data meet the prerequisites in this checklist. You need to understand the context of your data to check the first condition. We put conditions that require knowing something that you cannot verify from the data itself at the start of the list. The other conditions are easy to check if you look at the scatterplot.

✓ No obvious lurking variable. There’s a spurious correlation between math skills and shoe size for kids in an elementary school, but there are better ways to convey the effect of age.

✓ Numerical variables. Correlation applies only to quantitative vari- ables. Don’t apply correlation to categorical data.

✓ Linear. Correlation measures the strength of linear association. A correlation near zero implies no linear association, but there could be a nonlinear pattern.

✓ Outliers. Outliers can distort the correlation dramatically. An outlier can enlarge an otherwise small correlation or conceal a large correla- tion. If you see an outlier, it’s often helpful to report the correlation with and without the point.

9 S. Jacobsen, D. King, and R. Yuan, “A Note on the Relationship between Obesity and Driving,” Transport Policy, (2011), pages 772–776. 10 Strong, positive, linear association. The correlation is above 0.99. 11 No. Association is not causation. 12 Timeplots of the trends in these variables. Both trend upward.

What Do You Think? Policy makers and health professionals are concerned about the growing rates of obesity in the United States. In 2012, about one-third of adults were classified as obese (a body mass index above 30), raising concerns about the risk of dia- betes and other diseases associated with obesity. Analysts studying these trends prepared a scatterplot like that shown in Figure 6.13.9 The plot graphs the annual obesity rate versus the number of miles driven per licensed driver six years before.

11,500 12,000 12,500 Miles Driven per Licensed Driver

O b

es ity

(% )

13,000 13,500 14

16

18

20

22

24

26

28

FIGURE 6.13 Driving mileage is highly correlated with obesity six years later.

a. Describe the association between the percentage obese and the number of miles driven per licensed driver. What’s your guess of the correlation?10

b. Does the association imply that the decrease in driving mileage that is typi- cal during a recession implies that obesity rates will fall six years later?11

c. What other plots would be useful to see to understand this association?12

M06_STIN7167_03_SE_C06_pp109-139.indd 124 25/10/16 12:17 PM

caution Be cautious if a report gives a correlation without the associated scatterplot. The concerns in this checklist are important even if you

don’t see the plot.

4M ANALYTICS 6.1 LOCATING A NEW STORE

MOTIVATION ▶ STATE THE QUESTION When retailers like Target look for locations for opening a new store, they take into account how far the location is from the nearest com- petition. It seems foolish to open a new store right next to a Wal-Mart, or does it? Is it really the case that it’s better to locate farther from the competition? We’ll consider data from a regional chain of retail stores. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Plan your approach before you get wrapped up in the details. Our interest lies in the association between sales at the retail outlets and the distance to the near- est competitor. Do stores located far from the competition have higher sales? If there’s positive association between distance and sales, then we have sup- port for maintaining some separation from rivals.

Summarize how you’ll approach the analysis. This chain operates 55 stores. For each store, our data include the total retail sales over the prior year (in dollars). The data also include a numerical variable that gives the distance in miles from the nearest competitor. The first step of the analysis is to inspect the scatterplot of sales versus distance. Once we see the scatterplot, we will be able to check the conditions for using a correlation to quantify the strength of the association. If the association is linear, we can use the correlation line to summarize the relationship. ◀

MECHANICS ▶ DO THE ANALYSIS The scatterplot of sales versus distance for these stores shows positive associ- ation. Visually, we’d say this was linear and moderately strong. The scatterplot includes the line determined by the correlation.

0 1,500,000

2,000,000

2,500,000

3,000,000

1 2 3 4 Distance

Sa le

s

5 6 7 8 9 10

Let’s run through the checklist for the correlation.

✓ No obvious lurking variable. Without more background, we cannot be sure. This chain operates some superstores that combine a tradi- tional department store with a grocery market. If these large stores tend to be farther from the competition, the association may be due to size, not distance.

✓ Numerical variables. Both variables are numerical.

6.5 SPURIOUS CORRELATION 125

M06_STIN7167_03_SE_C06_pp109-139.indd 125 25/10/16 12:17 PM

126 CHAPTER 6 A ssociation between Quantitative Variables

✓ Linear. The pattern in the scatterplot seems linear, with substantial variation around the pattern.

✓ Outliers. Some stores sell a lot and others a little, but that’s to be expected. One store seems a bit of an outlier; this store is about 9 miles from the nearest competition and has the largest annual sales. Though separated from the others, this store falls in line with the others.

Using software, we computed the correlation between sales and distance to be r = 0.741. To draw the line in the scatterplot, express the equation in slope-intercept form. The relevant summary statistics are y = +2.308 million and sy = $340,000 for sales and x = 3.427 miles and sx = 1.582 miles for distance. Hence, the slope and intercept of the line are

b = rsy>sx = 0.7411340,000>1.5822 = +159,254.1>mile a = y - bx = 2,308,000 - 159,254.1 * 3.427 = +1,762,237 < +1.76 million

Though not part of the motivating task, this line anticipates sales levels at new locations. For example, sales at a location adjacent to a competitor (Distance = 0) are estimated to be yn = a + b102 = a < +1.76 million. If the distance were, say, 5 miles, then the expected sales grow by five times the slope, yn = a + b152 =1,762,237 + 159,254.1 * 5 = 2,558,508 <+2.56 million. It’s a good idea when doing these calculations to look back at the plot to make sure that your estimates are consistent with the graph. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS Sales and distance are positively related. The data show a strong, positive linear association between distance to the nearest competitor and annual sales. The correlation between sales and distance is 0.74. As distance grows, the data suggest sales grow on average as well, rising by about $160,000 per mile of added separation. There is moderate variation around the linear pattern.

If you are unsure or suspect a possible lurking variable, mention your concerns in the summary message. In this example, we have lingering concerns that there may be other lurking variables such as the size of the store. If stores that are farther from competitors are larger, we may be mistaking distance from the competition for size. ◀

6.6 ❘ CORRELATION MATRIX It is common in some fields to report the correlation between every avail- able pair of numerical variables and arrange these in a table. The rows and columns of the table name the variables, and the cells of the table hold the correlations.

A table that shows all of the correlations among a collection of numerical variables is known as a correlation matrix. Correlation matrices are com- pact and convey a lot of information at a glance. They can be an efficient way to explore a large set of variables because they summarize the pairwise relationships.

caution Before relying on a large table of correlations, be sure to review the checklist for correlations. Because a correlation matrix does not show

plots, you cannot tell whether the results conceal outliers or miss bending patterns.

As an example, Table 6.2 shows the correlation matrix for several charac- teristics of large companies.

tip

correlation matrix A table showing all of the correlations among a set of numerical variables.

M06_STIN7167_03_SE_C06_pp109-139.indd 126 25/10/16 12:17 PM

For example, the correlation between profits and cash flow is quite high, 0.989 (shown in boldface). Notice that this correlation appears twice in the table, once above the diagonal and once below the diagonal. The table is sym- metric about the diagonal that runs from the top left cell to the bottom right cell. The correlation between profits and cash flow is the same as the cor- relation between cash flow and profits, so the values match on either side of the diagonal. The correlation (and covariance) is the same regardless of which variable you call x and which you call y. The diagonal cells of a cor- relation table are exactly 1. (Can you see why? Think about where the points lie in a scatterplot of assets on assets, for instance. The plot would resemble Figure 6.8 and look like a single line.)

Assets Sales Market Value Profits

Cash Flow Employees

Assets 1.000 0.746 0.682 0.602 0.641 0.594

Sales 0.746 1.000 0.879 0.814 0.855 0.924

Market value 0.682 0.879 1.000 0.968 0.970 0.818

Profits 0.602 0.814 0.968 1.000 0.989 0.762

Cash flow 0.641 0.855 0.970 0.989 1.000 0.787

Employees 0.594 0.924 0.818 0.762 0.787 1.000

TABLE 6.2 Correlation matrix of the characteristics of large companies.

Best Practices

■■ To understand the relationship between two nu- merical variables, start with a scatterplot. If you see the plot, you won’t be fooled by outliers, bending patterns, or other features that can mislead the correlation.

■■ Look at the plot, look at the plot, look at the plot. It’s the most important thing to do. Be wary of a correlation if you are not familiar with the underlying data.

■■ Use clear labels for the scatterplot. Many people don’t like to read; they only look at the pictures. Use labels for the axes in your scatterplots that are self-explanatory.

■■ Describe a relationship completely. If you do not show a scatterplot, make sure to convey the

direction, curvature, variation, and any unusual features.

■■ Consider the possibility of lurking variables. Correlation shows association, not causation. There might be another factor lurking in the background that’s a better explanation for the pattern that you have found.

■■ Use a correlation to quantify the association be- tween two numerical variables that are linearly related. Don’t use correlation to summarize the association unless you are sure that the pattern is linear. Verify the correlation checklist before you report the correlations.

Pitfalls

■■ Don’t use the correlation if the data are categori- cal. Keep categorical data in tables where they belong.

■■ Don’t treat association and correlation as causa- tion. The presence of association does not mean that changing one variable causes a change in the other. The apparent relationship may in fact be due to some other, lurking variable.

■■ Don’t assume that a correlation of zero means that the variables are not associated. If the re-

lationship bends, the correlation can miss the pattern. A single outlier can also hide an oth- erwise strong linear pattern or give the impres- sion of a pattern in data that have little linear association.

■■ Don’t assume that a correlation near -1 or +1 means near perfect association. Unless you see the plot, you’ll never know whether all that you have done is find an outlier.

PITFALLS 127

M06_STIN7167_03_SE_C06_pp109-139.indd 127 18/11/16 3:31 PM

128 CHAPTER 6 A ssociation between Quantitative Variables

6.1 Analytics in Excel: Location

Open the data file 06_4m_location.csv in Excel. The worksheet is laid out as follows.

To scatterplot Sales on Distance, it is simplest to reverse the order of the first two columns of the ta- ble. Excel always places the leftmost of two columns on the x-axis in scatterplots. Highlight the entire Distance column by placing the mouse near the let- ter B above the column (the cursor will look like: ) and click. Then hold down the shift key and place the cursor at the left margin of cell B1 until the cur- sor becomes a hand. Then click and drag column B to the left of the Sales column. The resulting work- sheet will look like this.

Now select columns A and B and use the command Insert + Chart + Scatter. The default y-axis of the scatterplot extends to zero; to focus on the data, double-click the y-axis in the chart and change the minimum value in the Axis Options to 1.5 million. A variety of other options, such as those for label- ing the axes and adding the least squares line, can be found in the Chart Design menu that appears by clicking anywhere on the chart. Double clicking anywhere on the chart will open the Format Chart Area dialog.

To add the least squares line, the equation of the line, and the square of the correlation to the chart, click anywhere on the chart and choose Add New Element in the Chart Design menu.

If you would like to put the values of the least squares slope and intercept into cells of the work- sheet, use the formula SLOPE(B2:B56,A2:A56) t o f i n d t h e s l o p e a n d t h e s i m i l a r f o r m u l a INTERCEPT(B2:B56,A2:A56) to find the intercept. The formula CORREL(A2:A56,B2:B56) computes the correlation.

Software Hints

Statistics packages generally make it easy to look at a scatterplot to check whether the correlation is appropriate. Some packages make this easier than others.

Many packages allow you to modify or enhance a scatterplot, altering the axis labels, the axis num- bering, the plot symbols, or the colors used. Some options, such as color and symbol choice, can be  used to display additional information on the scatterplot.

EXCEL Use the Insert 7 Chart . . . command to produce a scatterplot of two columns. The process is simplest if you select the two columns prior to starting this command, with the x column adjacent and to the left of the y column. (If the columns are not adjacent or in this order, you’ll need to fill in a data dialog later.) Select the icon that looks like a scatterplot without connected dots. Click the button labeled Next and

M06_STIN7167_03_SE_C06_pp109-139.indd 128 25/10/16 12:18 PM

you’re on your way. Subsequent dialogs allow you to modify the scales and labels of the axes. The chart is interactive. If you change the data in the spread- sheet, the change is transferred to the chart.

To compute a correlation, select the function CORREL from the menu of statistical functions. En- ter ranges for two numerical variables in the dialog.

To compute the correlation using XLSTAT, use the menu command Correlation/Association tests 7 Correlation tests. In the resulting dialog, in- put a range from the spreadsheet that has two vari- ables, with the names of the variables in the first row of the range. To get a scatterplot in XLSTAT, use the command Visualizing data 7 Scatter plots. Fill in the ranges for the two variables, again with variable names in the first row of each range.

MINITAB EXPRESS To make a scatterplot, follow the menu commands

Graphs 7 Scatterplot Select the simple plot, then identify the numerical variable for the y-axis and the numerical variable for the x-axis.

To compute a correlation, follow the menu items

Statistics 7 Regression 7 Correlation

Fill the dialog with at least two numerical variables. If you choose more than two, you get a table of cor-

relations. The sequence Statistics 7 Regression 7 Covariance computes the covariance.

JMP To make a scatterplot, follow the menu commands

Analyze 7 Fit Y by X

In the dialog, assign numerical variables from the available list for the y-axis and the x-axis.

Once you have formed the scatterplot, to obtain a correlation use the pop-up menu identified by the red triangle in the upper left corner of the output window holding the scatterplot. Select the item la- beled Density ellipse and choose a level of coverage (such as 0.95). The ellipse shown by JMP is thinner as the correlation becomes larger. A table below the scatterplot shows the correlation.

To obtain a correlation table, follow the menu commands

Analyze 7 Multivariate Methods 7 Multivariate

and select variables from the list shown in the dialog. By default, JMP produces the correlation table and the scatterplot matrix (a table of scatterplots, one for each correlation). Clicking on the red triangle but- ton in the output window produces other options, including the table of covariances.

CHAPTER SUMMARY

Scatterplots show association between two numeri- cal variables. The response goes on the y-axis, and the explanatory variable goes on the x-axis. The visual test for association compares the observed scatterplot to artificial plots that remove any pattern. To describe the pattern in a scatterplot, indicate its direction, curvature, variation, and other surprising features such as outliers. Covariance measures the amount of linear association between two numerical

variables. The correlation r scales the covariance so that the resulting measure of association is always between - 1 and + 1. The correlation is also the slope of a line that relates the standardized deviations from the mean on the x-axis to the standardized deviations from the mean on the y-axis. Spurious correlation results from the presence of a lurking variable, and curvature or outliers can cause correla- tion to miss the pattern.

■ Key Terms covariance, 114 correlation matrix, 126 explanatory variable (x), 110 r (correlation), 116

response (y), 110 scatterplot, 110 slope, 121 spurious correlation, 123

visual test for association, 112 y-intercept, 121

■ Objectives • Recognize and describe the strength and direction

of association between two numerical variables from a scatterplot, as well as tell whether there is any association between the variables.

• Calculate and interpret the amount of linear asso- ciation using covariance and correlation.

CHAPTER SUMMARY 129

M06_STIN7167_03_SE_C06_pp109-139.indd 129 25/10/16 12:18 PM

130 CHAPTER 6 A ssociation between Quantitative Variables

■ Formulas

Covariance cov1x, y2

= 1x1 - x21y1 - y2 + 1x2 - x21y2 - y2 + g + 1xn - x21yn - y2

n - 1

= a n

i = 1 1xi - x21yi - y2

n - 1

Correlation

r = corr1x, y2 = cov1x, y2 sxsy

When calculations are done by hand, the following formula is also used.

r = a n

i = 1 1xi - x21yi - y2A ani = 11xi - x22 ani = 11yi - y22

Review this checklist when using a correlation.

✓ No obvious lurking variable

✓ Numerical variables

✓ Linear pattern

✓ No extreme outliers

Expressions Relating Covariance and Correlation

cov1x, y2 = sxsy corr1x, y2 corr1x, y2 = cov1x, y2>1sxsy2

Correlation Line

Using z-scores zny = rzx

Using x and y

yn = a + bx, a = y - b x, b = rsy>sx

■ About the Data The data on household energy consumption was col- lected by the Energy Information Agency of the U.S. Department of Energy (DoE) as part of its survey of residential energy consumption around the United States. How does the government use these data? For one, it uses the data to provide guidance to home- owners who are thinking about making their homes

more energy efficient. The Home Energy Saver at DoE’s Web site, hes.lbl.gov, provides a calculator that compares the energy costs for homes in your area. The calculator uses data like those in this chapter to estimate the value of modernizing appliances and adding insulation.

Mix and Match

1. Match the description to the scatterplot. (a) Negative direction, linear, moderate variation around line (b) Positive direction, linear, small variation around line (c) No pattern. A plot of scrambled pairs would look the same (d) Negative then positive direction, valley-shaped curve, moderate variation around curve

EXERCISES

• Recognize that correlation, unlike measures of as- sociation for categorical variables, only measures linear association.

• Find the line that summarizes the linear associa- tion that is measured by the correlation and use it to predict one numerical variable from another.

• Distinguish association from causation.

M06_STIN7167_03_SE_C06_pp109-139.indd 130 25/10/16 12:18 PM

I. II.

III. IV.

2. Match the description to the scatterplot. (a) No pattern. A plot of scrambled pairs would look the same (b) Negative direction, linear, small variation around line (c) Positive direction, but bending with small variation around curving pattern (d) Positive direction, linear, large variation around line

I. II.

III. IV.

3. Match the value of the correlation to the data in the scatterplot. (a) r = 0 (b) r = 0.5 (c) r = 0.8 (d) r = −0.6

I. II.

III. IV.

EXERCISES 131

M06_STIN7167_03_SE_C06_pp109-139.indd 131 25/10/16 12:18 PM

132 CHAPTER 6 A ssociation between Quantitative Variables

4. Match the value of the correlation to the data in the scatterplot. (a) r = 0 (b) r = −0.9 (c) r = 1 (d) r = 0.4

I. II.

III. IV.

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

5. The explanatory variable defines the x-axis in a scatterplot.

6. In a plot of income (the response) versus education (the explanatory variable) for managers, managers with the lowest levels of education are at the right- hand side of the figure.

7. The presence of a pattern in a scatterplot means that values of the response increase with the values of the explanatory variable.

8. The visual test for association is used to decide whether a plot has a real or imagined pattern.

9. A line with positive slope describes a linear pattern with a positive direction.

10. A company has learned that the relationship between its advertising and sales shows diminishing marginal returns. That is, as it saturates consumers with ads, the benefits of increased advertising diminish. The company should expect to find linear association between its advertising and sales.

11. If net revenue at a firm is about 10% of gross sales, then a scatterplot of net revenue versus gross sales would show a nonlinear pattern rather than a line.

12. If two variables are linearly associated, the peak in the histogram of the explanatory variable will line up with the peak in the histogram of the other variable.

13. If the correlation between the growth of a stock and the growth of the economy as a whole is close to 1, then this would be a good stock to hold during a recession when the economy shrinks.

14. If the covariance between x and y is 0, then the correlation between x and y is 0 as well.

15. If the correlation between x and y is 1, then the points in the scatterplot of y on x all lie on a single line with slope sy>sx.

16. An accountant at a retail shopping chain acciden- tally calculated the correlation between the phone number of customers and their outstanding debt. He should expect to find a substantial positive correlation.

17. A report graphs the number of daily employees on the x-axis and the number of items produced on the y-axis. The correlation would be larger if the plot reversed the x- and y-axes.

18. A retailer calculated the correlation line between the price of an item (x) and the amount sold (y). The cor- relation line is the same if the x and y variables are exchanged.

19. The correlation between sales and advertising when both are measured in millions of dollars is 0.65. The correlation remains the same if we convert these variables into thousands of dollars.

20. Half of the numbers in a correlation matrix are redundant, ignoring the diagonal.

Think About It

21. Suppose you collect data for each of the following pairs of variables. You want to make a scatterplot. Identify the response and explanatory variables. What would you expect to see in the scatterplot? Discuss the direction, curvature, and variation. (a) Sales receipts: number of items, total cost (b) Productivity: hours worked, items produced (c) Football players: weight, time to run 40 yards (d) Fuel: number of miles since filling up, gallons left

in your tank (e) Investing: number of analysts recommending a

stock, its subsequent price change

M06_STIN7167_03_SE_C06_pp109-139.indd 132 25/10/16 12:18 PM

22. Suppose you collect data for each of the following pairs of variables. You want to make a scatterplot. Identify the response and explanatory variables. What would you expect to see in the scatterplot? Discuss the likely direction, curvature, and variation. (a) Winter electricity: kilowatts, temperature (b) Long-distance calls: time (minutes), cost (c) Air freight: weight of load, fuel consumption (d) Advertising: length of commercial, number who

remember ad (e) Health: amount of exercise per week (hours),

percentage body fat

23. States in the United States are allowed to set their own rates for sales taxes as well as taxes on services, such as telephone calls. The scatterplot below graphs the state and local taxes charged for wireless phone calls (as a percentage) versus the state sales tax (also as a percentage).

0-1 1 2 3 4 5 6 7 8 Sales Tax (%)

15

10

5

0

St at

e &

L o

ca l

W ir el

es s

T ax

(% )

(a) Describe the association, if any, that you find in the scatterplot.

(b) Estimate the correlation between the two variables from the plot. Is it positive, negative, or zero? Is it closer to zero or to{0.5?

(c) The cluster of states with no sales tax in the lower left corner of the plot includes Alaska, Delaware, Montana, New Hampshire, and Oregon. What is the effect of this cluster on the association? If these were excluded from the analysis, would the correlation change?

(d) Would it be appropriate to conclude that states that have high sales tax charge more for services like wireless telephone use?

24. The Organization for Economic Cooperation and Development (OECD) is a loose affiliation of 30 nations. The data in this scatterplot show the per- centage of disposable income for a single citizen that remains after paying taxes versus the top personal income tax rate (also a percentage). (a) Describe the association in the scatterplot. (b) Does the association have the direction that

you’d expect? (c) Estimate the correlation between the two

variables.

(d) The data for which country appears most unusual relative to the others? How does the correlation change if this country is set aside and excluded from the calculation? (It may help to think about the line associated with the correlation.)

25. If the correlation between number of customers and sales in dollars in retail stores is r = 0.6, then what would be the correlation if the sales were measured in thousands of dollars? In euros? (1 euro is worth about 1.2 to 1.5 dollars.)

26. A fitness center weighed 53 male athletes and then measured the weight that they were able to lift. The correlation was found to be r = 0.75. Can you interpret this correlation without knowing whether the weights were in pounds or kilograms or a mixture of the two?

27. To inflate the correlation, a conniving manager added $200,000 to the actual sales amount for each store in Exercise 25. Would this trickery make the correlation larger?

28. After calculating the correlation, the analyst at the fitness center in Exercise 26 discovered that the scale used to weigh the athletes was off by 5 pounds; each athlete’s weight was measured as 5 pounds more than it actually was. How does this error affect the reported correlation?

29. Management of a sales force has noticed that there is a positive linear association (r = 0.6) between sales produced by company representatives in adjacent months. If a sales representative does really well this month (say, her z-score is + 2), why should we expect her z-score to be lower next month?

30. The human resources department at a company discovered linear association between the number of days that an employee is absent from year to year, with correlation 0.4. Should you expect the employee who is absent the least this year to have the fewest absences next year?

Top Personal Income Tax Rate (%)

D is

p . I

nc o

m e,

L es

s T

ax (%

, s in

g le

)

55

60

65

70

75

80

85

90

95

100

25 30 35 40 45 50 55 60

EXERCISES 133

M06_STIN7167_03_SE_C06_pp109-139.indd 133 25/10/16 12:18 PM

134 CHAPTER 6 A ssociation between Quantitative Variables

31. The following timeplot shows the values of two indices of the economy in the United States: inflation (left axis, in red, measured as the year-over-year per- centage change in the Consumer Price Index) and the Survey of Consumer Sentiment (right axis, in blue, from the University of Michigan). Both series are monthly and span January 2004 through April 2011.

0

60

Date

70

80

90

100

U M

ic h C

o n su

m e r

S e n ti

m e n t

-1 -2

20 04

/0 1/

01

20 05

/0 1/

01

20 06

/0 1/

01

20 07

/0 1/

01

20 08

/0 1/

01

20 09

/0 1/

01

20 10

/0 1/

01

20 11

/0 1/

01

1 2 3 4 5 6

In fla

ti o

n (Y

e ar

- o

ve r-

ye ar

% c

h an

g e )

(a) From the chart, do you think that the two sequences are associated?

(b) The scatterplot shown above displays the same time series, with consumer sentiment plotted versus inflation. Does this scatterplot change your impression of the association between the two?

(c) Visually estimate the correlation between these two series.

(d) For looking at the relationship between two time series, what are the advantages of these two plots? Each shows some things but hides others. Which helps you visually estimate the correla- tion? Which tells you the timing of the extreme lows and highs of each series?

(e) Does either plot prove that inflation causes changes in consumer sentiment?

32. The scatterplot that follows shows monthly values of consumer sentiment plotted versus inflation (see Exercise 31 for the definition of these variables). For this plot, we’ve shown the full history of the consumer survey back to January 1978. The subset of the points in black shows data since 2004.

Sheet 3Sheet 2 Ready

1

A B C D E F G H

2

3

4

5

6

7

8

9

10

11

12

13

14

15 Sheet 1

Microsoft Excel

110

100

90

80

70

60

0 5 10 15

U M

ic h

C o

n su

m e

r S

e n

ti m

e n

t

Inflation (Year-over-year % change)

(a) Describe the association between these two vari- ables over the full time period. Is the association strong? Is the relationship linear?

(b) Does the addition of the older data change your impression of the association between these two variables? To see a scatterplot of only the data since 2004, see Exercise 31.

(c) Do you think that the addition of the data prior to 2004 increases or decreases the size of the correlation?

33. Which do you think produces a larger correlation between the weight and the price of diamonds: using a collection of gems of various cuts, colors, and clarities or a collection of stones that have the same cut, color, and clarity?

34. To measure the association between the number of employees and the number of items produced on a small assembly line, a company recorded the number of employees each day (from 5 to 10) and the number of items produced. To make a nicer plot, a manager found the average number of items produced when 5 employees worked, the average number when 6 worked, and so forth. He then plotted these average levels of production versus the number of employees. How does this averaging affect the correlation?

35. Cramer’s V (Chapter 5) measures dependence and varies from 0 to 1. The correlation also measures dependence but varies from - 1 to + 1. Why doesn’t it make sense for V also to take on negative values?

36. The correlation measures the strength of linear dependence. Does Cramer’s V (Chapter 5) only measure linear dependence?

37. The visual test for association can be used to help distinguish whether the pattern in a plot is real or imagined. The original ought to stand out clearly. If you look at four plots, one the original and three with scrambled coordinates, and guess, what is the chance that you’ll guess the original even if there is no pattern?

38. The visual test for association requires you to compare several plots. One has the original data,

M06_STIN7167_03_SE_C06_pp109-139.indd 134 25/10/16 12:18 PM

and the others show the data with scrambled coor- dinates. Would it help in finding the original if these scatterplots include the marginal histograms along the edges of the plot?

You Do It

Bold names shown with a question identify the data table for the problem. The title of each of the follow- ing exercises identifies the data table to be used for the problem.

39. Housing and Stocks Each month, the Department of Commerce releases its estimate of construction activity. A key measure is the percentage change in the number of new homes under construction. Are changes in home construction associated with changes in the stock market? These data show the percentage change in the number of new, privately owned housing units started each month, as reported by the Department of Commerce. The data also include the percentage change in the S&P 500 index of the stock market. (a) Before looking at these data, do you expect these

variables to be associated? If so, what do you anticipate the correlation to be?

(b) Draw the scatterplot of the percentage change in the S&P 500 index versus the percentage change in the number of new housing units started. Describe the association.

(c) Find the correlation between the two variables in the scatterplot. What does the size of the correla- tion suggest about the strength of the association between these variables?

(d) Suppose you know that there was a 5% increase in the number of new homes. From what you’ve seen, can you anticipate movements in the stock market?

40. Drive Preferences These data give the percentage of new vehicles bought with four-wheel drive, state by state in the continental United States in 2014. The data include the average temperature in that state in January. (a) Do you expect the correlation between these

variables to be positive or negative? Explain your choice.

(b) Draw the associated scatterplot. Does the direc- tion of the association match what you expected to find?

(c) In which states is the percentage choosing four- wheel drive the highest? Lowest?

(d) Find the correlation between the variables. Is it weaker or stronger than you expected?

41. Employee Testing The slope for the correlation line when expressed in z-scores is r. Hence, when formu- lated in z-scores, the absolute value of the slope is less than 1 unless the data lie along a single line. The data in this example are productivity measures used to grade 15 employees who work on an assembly line. Each employee was tested once, then again a month later.

First Test Second Test

50 58

35 46

15 40

64 76

53 62

18 39

40 57

24 41

16 31

67 75

46 62

64 64

32 54

71 65

16 51

(a) Would you expect the scores to be associated? (b) Make a scatterplot of these data, with the first

test along the x-axis. Describe the relationship, if any.

(c) On the basis of the correlation, if an employee scores two SDs above the mean on the first test, do you expect him or her to be so far above the mean on the second test?

(d) Find the employee with the highest score on the first test. The score for this employee is not the best on the second test. Does this relative decline mean the employee has become less productive, or can you offer an alternative reason for the relatively lower score?

42. Tour de France These data give the times in minutes for cyclists who competed in the 2010 Tour de France and raced in both time trial events. In a time trial, each cyclist individually rides a set course. The rider with the shortest time wins the event. The courses in the 2010 tour varied in length. The first time trial at the start of the month- long race was 8.9 kilometers long, and the second time trial near the end of the race was 52 kilome- ters long. (a) Would you expect the times to be associated? (b) Make a scatterplot of these data, with the first set

of times along the x-axis. Do the variables appear to be associated? Describe the relationship, if any.

(c) Find the correlation between these variables, if appropriate.

(d) Identify the times for Alberto Contador (who won the overall race) in the scatterplot. Did he perform exceptionally well in these events?

43. Data Entry How many mistakes get made during data entry? The following table gives the number

EXERCISES 135

M06_STIN7167_03_SE_C06_pp109-139.indd 135 25/10/16 12:18 PM

136 CHAPTER 6 A ssociation between Quantitative Variables

of mistakes made by 15 data entry clerks who enter medical data from case report forms. These forms are submitted by doctors who participate in stud- ies of the performance of drugs for treating various illnesses. The column Entered indicates the number of values entered, and the column Errors gives the number of coding errors that were detected among these.

Entered Errors

4,434 35

4,841 42

6,280 15

1,958 28

7,749 36

2,829 42

4,239 18

3,303 54

5,706 34

3,770 40

3,363 36

1,740 23

3,404 27

1,640 26

3,803 56

1,529 20

(a) Make a scatterplot of these data. Which did you choose for the response and which for the explanatory variable? Describe any patterns.

(b) Find the correlation for these data. (c) Suppose we were to record the counts in the

table in hundreds, so 4,434 became 44.34. How would the correlation change? Why?

(d) Write a sentence or two that interprets the value of this correlation. Use language that would be understood by someone familiar with data entry rather than correlations.

(e) One analyst concluded, “It is clear from this cor- relation that clerks who enter more values make more mistakes. Evidently they become tired as they enter more values.” Explain why this expla- nation is not an appropriate conclusion.

44. Truck Weight The Minnesota Department of Transportation would like to measure the weights of commercial trucks on its highways without the expense of maintaining weigh stations along the highway. Weigh stations are expensive to operate, delay shipments, and aggravate truckers. To see if a proposed remote weight system was accurate, transportation officials conducted a test. They weighed trucks when stopped on a regular scale. They then used the remote system to estimate the weights. Here are the data.

Weight of a Truck (thousands of pounds)

New System Scale Weight

26 27.9

29.9 29.1

39.5 38

25.1 27

31.6 30.3

36.2 34.5

25.1 27.8

31 29.6

35.6 33.1

40.2 35.5

(a) Make a scatterplot of these data and describe the direction, form, and scatter of the plot.

(b) Find the correlation. (c) If the trucks were weighed in kilograms,

how would this change the correlation? (1 kilogram = 2.2 pounds)

(d) Interpret the correlation in the context of trucks and weights. What does the correlation tell the Minnesota Department of Transportation?

45. Cars These data report characteristics of 413 types of cars sold in the United States in 2016. One col- umn gives the official mileage (combined MPG), and another gives the rated horsepower. (a) Make a scatterplot of these two columns.

Which variable makes the most sense to put on the x-axis, and which belongs on the y-axis?

(b) Describe any pattern that you see in the plot. Be sure to identify any outliers.

(c) Find the correlation between these two variables. (d) Interpret the correlation in the context of these

data. Does the correlation provide a good sum- mary of the strength of the relationship?

(e) Use the correlation line to estimate the mileage of a car with 200 horsepower. Does this seem like a sensible procedure?

46. Cars These data are described in Exercise 45. (a) Make a scatterplot of weight and city mileage.

Which variable do you think makes the most sense to put on the x-axis, and which belongs on the y-axis?

(b) Describe any pattern that you see in the plot. Be sure to identify any outliers.

(c) Find the correlation between these two variables. (d) Use the correlation line to estimate the mileage

of a car that weighs 4,000 pounds. Does this seem like a sensible procedure?

47. Philadelphia Housing These data describe housing prices in the Philadelphia area. Each of the 110 rows of this data table describes a region of the metropolitan area. (Several make up the city of Philadelphia.) One column, labeled Selling Price, gives the median price for homes sold in that area during 1999 in thousands of dollars. Another, labeled Crime Rate, gives the number of crimes committed in that area, per 100,000 residents.

M06_STIN7167_03_SE_C06_pp109-139.indd 136 25/10/16 12:18 PM

(a) Make a scatterplot of the selling price on the crime rate. Which observation stands out from the others? Is this outlier unusual in terms of either marginal distribution?

(b) Find the correlation using all of the data as shown in the prior scatterplot.

(c) Exclude the distinct outlier and redraw the scat- terplot focused on the rest of the data. Does your impression of the relationship between the crime rate and selling price change?

(d) Compute the correlation without the outlier. Does it change much?

(e) Can we conclude from the correlation that crimes in the Philadelphia area cause a rise or fall in the value of real estate?

48. Boston Housing These data describe 506 census tracts in the Boston area. These data were assembled in the 1970s and used to build a famous model in economics, known as a hedonic pricing model.13 The column Median Value is the median value of owner- occupied homes in the tract (in thousands of dollars back in the 1970s, so the amounts will seem quite small), and the column Crime Rate gives the number of crimes per 100,000 population in that tract. (a) Make a scatterplot of the median value on the

crime rate. Do any features of this plot strike you as peculiar?

(b) Find the correlation using all of the data as shown in the scatterplot produced in part (a).

(c) Tracts in 1970 with median home values larger than $50,000 were truncated so that the larg- est value reported for this column was 50. The column Larger Value replaces the value 50 for these tracts with a randomly chosen value above 50. Use this variable for the response, make the scatterplot, and find the correlation with Crime Rate. Does the correlation change?

(d) What can we conclude about the effects of crime on housing values in Boston in the 1970s?

49. Cash Counts There’s a special situation in which you can measure dependence using either the correlation or Cramer’s V (Chapter 5). Suppose both variables indicate group membership. One variable might dis- tinguish male from female customers, and the other whether the customer uses a credit card, as in this example. These variables define a contingency table for a sample of 150 customers at a department store.

Sex

TotalMale Female

Pay with cash 50 10 60

Use a credit card 55 35 90

Total 105 45 150

(a) Compute the value of Cramer’s V for this table. You may need to look back at Chapter 5.

(b) Find the correlation between the numerical variables Sex (coded as 1 for male, 0 for female) and Cash (coded 1 for those paying with cash, 0 otherwise). Use the counts in the table to manufacture these two columns.

(c) What’s the relationship between Cramer’s V and the correlation?

50. Cash Counts A scatterplot is not so useful for numerical data when the data include many copies. In Exercise 49, there are 50 cases of male customers who paid with cash. That’s 50 pairs of values with Sex = 1 and Cash = 1. (a) What does the scatterplot of Cash versus Sex look

like? (b) Suggest a method for improving the scatterplot

that helps you see the association. (c) Cramer’s V measures any type of association,

but correlation only measures linear association. When you consider the association between the two 0/1 variables in this example, can there be anything other than linear association?

51. Macro The Bureau of Labor Statistics monitors characteristics of the United States economy. Sev- eral of these time series are in the data table for this question. These time series are quarterly and span the time period from 1980 through the third quarter of 2015. The variable Compensation is an index of the hourly pay of all persons in the business sector. The variable Output is an index of the output of goods and services per hour.. (a) A timeplot is another name for a scatterplot of a

sequence of values versus a variable that mea- sures the passage of time. Generate timeplots of Compensation and Output. Do these variables seem associated?

(b) Make a scatterplot of Output versus Compensa- tion. Does this plot suggest that the two series are associated? What does the White Space Rule suggest about this plot?

(c) Find the correlation between Output and Compensation.

(d) Does the correlation between Output and Compensation prove that managers can get more output by increasing the pay to their employees?

52. Macro The government wants consumers to have money to spend to keep the economy moving, so it watches the national level of personal disposable income. The Federal Reserve wants to make sure that these same consumers do not get into too much debt. The variable Disposable Income tracks the money available to consumers to spend (in billions of dollars), and the variable Debt tracks the money owed by households in the retail credit market (not includ- ing mortgages, also in billions of dollars). Both series are quarterly and span the time period from 1980 through the third quarter of 2015. (a) A timeplot is a scatterplot of the values in a

sequence in chronological order. Generate time- plots of these two series. Do the series appear to be related?

13 D. Harrison and D. L. Rubinfeld “Hedonic Housing Prices and the Demand for Clean Air,” Journal of Environmental Economics and Management, 5(1978), 81–102. The data are rather infamous. See the discussion in O. W. Gilley and R. Kelley Pace, “On the Harrison and Rubinfeld Data,” Journal of Environmental Economics and Manage- ment, 31(1996), 403–405.

EXERCISES 137

M06_STIN7167_03_SE_C06_pp109-139.indd 137 25/10/16 12:18 PM

138 CHAPTER 6 A ssociation between Quantitative Variables

(b) Make a scatterplot of Disposable Income on Debt. Does this plot reveal things that you could not tell from the timeplots? Does the White Space Rule suggest a need for care when looking at this relationship?

(c) Find the correlation between Disposable Income and Debt. Is the correlation a good summary of the relationship between these two series?

(d) Does the correlation imply that the government should try to lower the level of disposable income to get consumers to save more?

53. Flight Delays If the airline flight that you are on is 20 minutes late departing, can you expect the pilot to make these minutes up by flying faster than usual? These data summarize the status of a sample of 1,050 flights during August 2015. (Like the data used to illustrate Simpson’s paradox in Chapter 5, these data come from the Bureau of Transportation Statistics.) (a) Do you expect the number of minutes that the

flight is delayed departing to be associated with the arrival delay?

(b) Make a scatterplot of the arrival delay (in min- utes) on the departure delay (also in minutes). Summarize the association present in the scat- terplot, if any.

(c) Find the correlation between arrival delay and departure delay.

(d) How is the correlation affected by the evident outlier, a flight with very long delays?

(e) How would the correlation change if delays were measured in hours rather than minutes?

54. CO2 These data give the amount of CO2 produced in 40 nations along with the level of economic activity, both in 2009. CO

2 emissions are given in millions of

metric tons. Economic activity is given by the gross domestic product (GDP), a summary of overall eco- nomic output. Both variables are from the Interna- tional Energy Agency. (a) Make a scatterplot of CO

2 emissions and GDP.

Which variable have you used as the response and which as the explanatory variable?

(b) Describe any association between CO 2 emissions

and GDP. (c) Find the correlation between CO

2 emissions and

GDP. (d) Which cases are outliers? How does the correla-

tion change if outliers are removed?

55. 4M ANALYTICS: Correlation in the Stock Market

Apple, HP, IBM, and Microsoft are well known in the computer industry. If the computer industry is doing well, then we might expect the stocks of all four to increase in value. If the industry goes down, we’d expect all four to go down as well. How strong is the association among these companies? After all, they compete in some areas. For example, HP and IBM both sell powerful computer systems designed to power Web sites like Amazon or weather.com.

This data set has monthly returns for Apple, HP, IBM, and Microsoft for January 1990 through December 2014. The returns are calculated as

Rt = Pt - Pt - 1

Pt - 1

In this fraction, Pt is the price of the stock at the end of a month, and Pt - 1 denotes the price at the end of the prior month. If multiplied by 100, the return is the percentage change in the value of the stock during the month. The returns given here have been adjusted for accounting activities (such as dividend payments) that would other- wise produce misleading results. The data are from the Center for Research in Security Prices (CRSP).

Motivation

(a) Investors who buy stocks often buy several to avoid putting all their eggs into one basket. Why would someone who buys stocks care whether the returns for these stocks were related to each other?

(b) Would investors who are concerned about put- ting all their eggs into one basket prefer to buy stocks that are positively related, unrelated, or negatively related?

Method

(c) How can an investor use correlations to deter- mine whether these four stocks are related to each other? How many correlations are needed?

(d) Correlations can be fooled by patterns that are not linear or distorted by outliers that do not conform to the usual pattern. Before using cor- relations, how can the investor check the condi- tions needed for using a correlation?

(e) A key lurking variable anytime we look at a scat- terplot of two time series is time itself. How can an investor check whether time is a lurking fac- tor when looking at stock returns?

Mechanics

(f) Obtain the scatterplots needed to see whether there are patterns that relate the returns of these stocks. Does it matter which stock return goes on the x-axis and which goes on the y-axis? Do you find that the returns are associated? Is the association linear?

(g) Obtain the correlation matrix that has all of the correlations among these four stocks.

(h) Look at timeplots of the returns on each stock. Why are these important when looking at time series?

Message

(i) Summarize your analysis of the association among these returns for an investor who is think- ing of buying these stocks. Be sure to talk about stocks, not correlations.

M06_STIN7167_03_SE_C06_pp109-139.indd 138 25/10/16 12:18 PM

56. 4M ANALYTICS: Cost Accounting

Companies that manufacture products that conform to a unique specification for each order need help in esti- mating their own costs. Because each order is unique, it can be hard to know for sure just how much it will cost to manufacture. Plus, the company often needs to quote a price when the customer calls rather than after the job is done. The company in this exercise manufactures customized metal blanks that are used for computer-aided machining. The customer sends a design via computer (a 3-D blueprint), and the manufacturer replies with a price per unit. Currently, managers quote prices on the basis of experience.

What factors are related to the order’s cost? It’s easy to think of a few. The process starts with raw metal blocks. Each block is then cut to size using a computer-guided tool. All of this work requires a person to keep an eye on the process.

The data for the analysis were taken from the account- ing records of 200 orders that were filled during the last three months. The data have four variables of interest. These are the final cost (in dollars per unit manufac- tured), the number of milling operations required to make each block, the cost of raw materials (in dollars per unit), and the total amount of labor (hours per unit).

Motivation

(a) If the manufacturer can find a variable that is highly correlated with the final cost, how can it use this knowledge to improve the process that is used to quote a price?

Method

(b) What variable is the response? Which are the possible explanatory variables?

(c) How can the manufacturer use the correlation to identify variables that are related to this re- sponse?

(d) Explain why it is important to consider scatter- plots as well.

Mechanics

(e) Obtain all of the scatterplots needed to under- stand the relationship between the response and the three explanatory variables. Briefly describe the association in each.

(f) Obtain all of the correlations among these four variables. (These are most often displayed in a correlation matrix.) Which explanatory variable is most highly correlated with the response?

(g) Check the conditions for the correlation of the response with the three predictors. Do the scatterplots suggest any problems that make these three correlations appear unreliable?

(h) Adjust for any problems noted in part (g) and recompute the correlations.

Message

(i) Which variable is most related to the response? (j) Explain in the context of this manufacturing situ-

ation why the correlation is not perfect and what this lack of perfection means for predicting the cost of an order.

EXERCISES 139

M06_STIN7167_03_SE_C06_pp109-139.indd 139 25/10/16 12:18 PM

140

STATISTICS IN ACTION

Time series, data that flow over time, are common in busi- ness. Among these, time series that track the flow of dollars are the most studied. The financial time series considered in this case tracks the value of stock in a company, Enron.

This case reviews many of the essential concepts covered in the chapters of this section: the importance of under- standing the data, numerical summaries and distributions, plots to find patterns, and the use of models to describe those patterns. It is important to appreciate several quirks of stock prices in order to make sense of these data. The emphasis is on data that track a stock over time, but the concerns and methods are familiar.

STOCK PRICES At the end of the 20th century, Enron was one of the world’s largest corporations. It began in 1986 as a re- gional supplier of natural gas and grew into a domi- nant player in the energy trading business. All of that came to an end when it abruptly collapsed in 2001.

The price of a share of stock in Enron mirrors the spectacular rise and fall of the company. In April

1986, each of the 45 million shares of stock in the newly formed Enron Corporation sold for +37.50. By the end of 2000, there were 740 million shares valued at +80 apiece. At that point, Enron’s market value was +60 billion. Had you bought and held one of those first shares that cost +37.50, it would have grown into eight shares worth +650 by the end of 2000. Not bad. To match that, you’d need a savings account that paid 21% interest each year!

Timing in the stock market, however, is impor- tant. Had you held onto that stock in Enron a little longer, it would have been worth less than 650 cents! The price collapsed to +0.26 a share by the end of 2001.

Were there hints of trouble before the bottom dropped out? Books that describe Enron’s wheeling and dealing, such as The Smartest Guys in the Room (written by two reporters from Fortune magazine), convey the impression that a lot of people should have known something was going to happen. But what about regular investors? Could they have rec- ognized the risks of investing in Enron? Let’s see how investors could have used statistics to appreciate the risks in owning stock in Enron.

Stock Prices

To measure the risk of owning a stock, we start with data that record the price of one share. The data in this analysis record the price of Enron stock at the end of each month from April 1986 through Decem- ber 2001. These prices form a time series. Each row in the data table refers to a specific month, and the variable Price records the price of one share of En- ron stock at the end of the month. The rows are in order, so that running down the column of prices tracks Enron’s history. Table 1 lists the first four months of data.

Case: Financial Time Series

STOCK PRICES

STOCK RETURNS

VALUE AT RISK

CASE SUMMARY

TABLE 1 Prices of Enron stock in 1986.

Date Price

April 1986 $37.50

May 1986 $41.38

June 1986 $44.00

July 1986 $40.00

f f

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 140 26/10/16 12:41 PM

141FINANCIAL TIME SERIES

We begin our analysis as usual: with a plot. We’ve only got one time series and the dates, but we’ll begin with a scatterplot. How are we going to build a scat- terplot from one column of numbers? Easy. The time order defines the x-axis for the scatterplot, and the prices define the y-axis forming the timeplot (Chap- ter 1) shown in Figure 1.

The capitalized value rises steadily, without the sudden falls seen in the value of individual shares. The explanation is that the price of stock tracks only one component of the value of Enron. The price leaves out the number of shares. When the price of a stock increases, it is common for companies to “split” the stock, converting each outstanding share into two or three. So, if we held one share of XYZ Corporation that was valued at +100 before a split, then we’d have two shares after the split, each worth +50. Our investment is still worth +100, only split be- tween two shares. We’ll leave it for a finance class to explain the logic for splitting stocks, but the common explanation is that lower prices attract more buyers.

Stock splits explain the rapid drops in the value of Enron stock seen in Figure 1. The market did not abruptly devalue Enron. Rather, these rapid drops were stock splits. For example, at the end of Novem- ber 1991, Enron sold for about +70 per share. The stock split “2 for 1” in December, and not surpris- ingly, the price at the end of December was about +35 per share.

Dividend payments also add variation to stock prices. Let’s go back to our +100 investment in one share of stock. If that’s the price today and the com- pany pays a dividend of +5 per share tomorrow, then the value of each share drops to +95. We were paid the other $5 in cash. Like splits, dividends cause prices to change but don’t change the value of our investment.

We have to account for splits and dividends in or- der to build a useful plot. Splits and dividend pay- ments add variation to the prices, but these do not materially affect the value of the investment. We would not have lost money in 1991 when Enron’s stock split; we’d simply have had twice as many shares, each worth half of the original price. A more informative plot removes these accounting artifacts. We have to adjust for the splits and track the wealth of an investor who started with one share. Finance pros would recognize the splits right away.

P ri

ce ($

)

0

10

20

30

40

50

60

70

80

90

1986 1988 1990 1992 Year 1994 1996 1998 2000 2002

FIGURE 1 Timeplot of the price of Enron stock.

The timeplot begins in April 1986 when Enron sold for $37.50 per share. The price bounces around after that. Several times it runs up only to fall suddenly. Up until the end, the price recovered after each fall.

Once we’ve seen Figure 1, it’s possible to wonder why anyone would be surprised by the collapse of Enron. The price of a share fell 50% three times in the 1990s, and it steadily recovered after the first two. Perhaps the drop at the end of 1999 was the time to buy shares while the price was low, hoping for another steady recovery. That might seem like a risky bet. Wasn’t it inevitable that one of those drops would be deadly?

Some Details: Stock Splits

We have to understand the data in order to use sta- tistics. In this case, a simple accounting practice explains the sudden collapses in the price of Enron stock. To understand these data, we have to learn more about how stocks are priced.

Figure 1 doesn’t present the history of Enron in a way that’s helpful to an investor. It displays the prices of a share of stock in Enron at the end of each month. Had we gone to the New York Stock Exchange back then, this plot shows what we would have paid for a share. To appreciate that Figure 1 hides something, take a look at Figure 2. This timeplot tracks the capi- talized value of Enron, in millions of dollars, through the end of 1999 before the collapse. (Capitalized value, or market cap, is the price of a share times the num- ber of outstanding shares. It’s the total value of all of the stock in a company.)

1986 1988 1990 1992 Year

1994 1996 1998 2000

M ar

ke t

C ap

($ M

)

0

5000

10000

15000

20000

25000

30000

35000

FIGURE 2 Market capitalization of Enron, in millions of dollars.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 141 26/10/16 12:41 PM

142 PART I Statistics in Action

Figure 3 shows the timeplot of the wealth of an in- vestor who bought one share of Enron stock in 1986 and held onto it and all of its “children” (the stocks obtained when that original share split). These data are also corrected for dividend payments.

Let’s think about what happened in October 1987. To construct a more revealing plot, we need to think more like gamblers. If we had put +100 into almost any stock on October 1, 1987, we would have had less than +100 by the end of the month. All the vari- ous “bets” on the market lost money. Bets on IBM lost money. Bets on Disney lost money. So did GM, Enron, and most others. If we were to bet on a card game, we’d probably look at the game one play at a time. It’s too hard to think about how one round fits into the whole game, so most gamblers think about games of chance one play at a time.

We can think of the prices of Enron stock as telling us what happens in a sequence of bets, one wager at a time. Imagine that we start each month with +100 invested in Enron, regardless of what happened last month. Each month is a fresh bet.

We can get a plot that tracks our winnings each month by converting these prices into percentage changes. Let’s do one calculation by hand. On Janu- ary 1, 1997, Enron’s stock sold for +43.125. (Weird prices like this one come from the old convention of pricing stocks in eighths of a dollar; nowadays, stocks are priced to the nearest penny.) Our +100 would buy 100>43.125 = 2.32 shares. At the end of January, the price per share had dropped to +41.25. So, each of our shares lost 43.125 - 41.25 = +1.875. Taking into account the fraction of a share that we owned, we lost +4.35. Here’s the calculation:

a +100 +43.125>shareb * 1+41.25 - +43.1252>share

= -+4.35

The first expression inside the parentheses tells us how many shares our +100 buys. The second term tells us what happened to the price of each. The end result tells how much we win or lose by investing +100 in Enron for one month. The amount won or lost is equivalent to the percentage change:

Percentage change = 100 * aChange in Price Starting Price

b

= 100 * a+41.25 - +43.125 +43.125

b

= -4.35%

The ratio of the change in price to the starting price is known as the return on the stock. We convert these into percentage changes by multiplying the re- turn by 100.

You’ll often see the return described in a for- mula that uses subscripts. Subscripts distinguish

1986 1988 1990 1992 Year

1994 1996 1998 2000

S p

lit P

ri ce

($ )

0

50

100

150

200

250

300

350

400

FIGURE 3 Value of investment in Enron.

This timeplot looks very different from the time- plot of prices in Figure 1. The stock soars. Without the jumps caused by splits, we can appreciate the appeal of Enron. The soaring growth pulled in investors.

STOCK RETURNS We ended the timeplots in Figure 2 and Figure 3 at the end of 1999. We want to pretend that we’re in- vestors at the end of 1999 who are considering stock in Enron. We don’t know what’s coming, just what’s happened so far. Figure 3 certainly makes the stock look attractive, but do other views of these data ex- pose a different, less inviting perspective? What could we have seen in the history up to then that might have warned us to stay away?

caution There’s a problem with plots that track the cumulative value of an investment.

Plots like Figure 3 appear throughout the financial sec- tion of the newspaper—such as in ads for mutual funds—but these plots conceal certain risks. We get a hint of the problem from the White Space Rule. Most of the area of these timeplots does not show data. We cannot see the variation in the first half of the data that are squeezed down near the x-axis.

A substantive insight suggests a problem with Fig- ure 3. The stock market fell more than 20% in October 1987. Now look at the history of Enron in Figure 3. There’s hardly a blip for October 1987. It could be that the drop in markets in 1987 didn’t affect Enron, but that’s wishful thinking. The problem lies with this view of the data. A plot that can hide October 1987 is capable of hiding other things as well.

return Ratio of the change in the value of an investment during a time period to the initial value; the product of 100 times the return is the percentage change.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 142 26/10/16 12:41 PM

143FINANCIAL TIME SERIES

sense does not mean “constant.” All of that irregular, zigzagging variation in Figure 4 looks complicated, but that’s common in simple data. We say that these data are simple because it is easy to summarize this sort of variation. There’s variation, but the ap- pearance of the variation is consistent. Some would call this variation “random,” but we will see in later chapters that random variation often has patterns. Because there’s no pattern that would be hidden, we can summarize simple variation with a histogram. Simple variation is “histogrammable.”

It can be hard to decide if a timeplot shows simple variation because people find it easy to imagine pat- terns. The visual test for association works nicely in this context. In Chapter 6, we showed several scram- bled scatterplots of energy prices. If the original can easily be identified, there’s a pattern. If not, the varia- tion is simple and can be summarized with a histo- gram. To use the visual test for association with time series, mix the order of the data as though shuffling a deck of cards. If the timeplot of shuffled data looks different from the original timeplot, then there’s a pattern and the variation is not simple.

Figure 5 compares the timeplot of the price of Enron stock to the plot of the data after shuffling.

time periods and make it easy to express the per- centage change in a compact equation. For exam- ple, we can identify the price of a stock in the tth row of the data table by Pt. The symbol Pt - 1 rep- resents the price in the prior row (the end of the previous month). The change in price during this month is Pt - Pt - 1. The return on the stock is then

Rt = Pt - Pt - 1

Pt - 1

= +41.25 - +43.125

+43.125

= -0.0435

The percentage change multiplies the return by 100, a loss of 4.35%.

Simple Time Series

The timeplot of the monthly percentage changes in Figure 4 conveys a different impression of the risks associated with investing in Enron.

1986 1988 1990 1992 Year

1994 1996 1998 2000

P e

rc e

n ta

g e

C h

an g

e ,

E n

ro n

-30

-20

-10

0

10

20

30

FIGURE 4 Monthly percentage changes in price of Enron stock.

These hardly look like the same data. What hap- pened to the smooth upward growth evident in Figure 2 and Figure 3? Where did all the rough, irregular ups and downs come from?

By thinking of investing as a series of bets, we re- move the compounding that happens to investments and concentrate on what happens each month anew. Most of these monthly bets made money. After all, the price went up on average. But we can also see quite a few losing bets. Bets on Enron were hardly a lock to make money. We can also appreciate the magnitude of what happened in October 1987: Stock in Enron fell nearly 30% that month. That wasn’t the only big drop either. During August 1998, stock in Enron fell 20%.

There’s another key difference between the timeplot of the prices (Figure 3) and the timeplot of percentage changes (Figure 4). Which plot shows a pattern? We would say that the timeplot of percentage changes in Enron stock is simpler than the timeplot of the prices because there’s no trend in the returns. Simple in this

FIGURE 5 It’s easy to recognize the timeplot of prices.

simple Variation in simple data lacks a pattern, making it appropriate to be summarized in a histogram.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 143 26/10/16 12:41 PM

144 PART I Statistics in Action

timing of the ups and downs isn’t crucial unless you think you can predict the timing of these events.

What do we expect the distribution of the per

tip centage changes to look like? It’s often a good idea to imagine the distribution before asking soft- ware to make the picture for us. That way we’ll be

ready if the plot shows something unexpected. Figure 7 shows what we are doing when we make a histogram of a time series.

It’s easy to distinguish the actual prices from the scrambled prices. The prices of Enron stock are not simple and definitely show a pattern.

Now, let’s do the same thing with the percentage changes. Which of the timeplots in Figure 6 is the timeplot of the percentage changes?

FIGURE 6 It’s not so easy to recognize the timeplot of returns.

Unless we remember precisely where a big drop hap- pens, either of these could be the original. This simi- larity leads us to conclude that the returns on Enron are “simple enough” to summarize with a histogram.

tip Simple data that lack a pattern allow for a simple summary, namely, a histogram. Time series that show

a pattern require a description of the pattern.

Histograms

Simple time series free us to concentrate on the variation and ignore the timing. To focus on the variation, we collapse the plot into a histogram that ignores the time order. For an investor, it’s crucial to know that the price fell during some months in the past. The fact that the price of Enron stock bounced around a lot over its history is crucial. The precise

1985 1990 Year

1995 2000

P e rc

e n ta

g e C

h an

g e

E n ro

n

-30 -20

-10

0

10

20

30

FIGURE 7 Histogram and timeplot of the percentage changes.

We did not connect the dots in the timeplot so that the separate points can be seen more easily. Since time order is by and large irrelevant with a simple time series, there’s no point connecting the dots.

Instead of plotting the returns as a long sequence of dots, the histogram shows how many land in each interval of the y-axis. The histogram ignores the or- der of the data and counts the number of months in which the return falls in each interval. For instance, the highlighted bar of the histogram in Figure 7 counts the number of months in which the percent- age change in Enron stock was between 0% and 5%. If there were a pattern in the time series, the histo- gram would hide it.

Figure 8 rotates the histogram to the more famil- iar horizontal layout and adds a boxplot to locate the center of the data.

-30 -20 -10 0 10 20 30

10

20

30

40

50

60

C o

u n

t

FIGURE 8 Histogram of percentage changes in the price of Enron stock.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 144 26/10/16 12:41 PM

145FINANCIAL TIME SERIES

returns, VaR converts these into a dollar amount that conveys possible consequences of variation. The idea is to use the Empirical Rule (or other more elaborate rules) to make a statement about how much money might be lost.

For instance, suppose we invest +1,000 in Enron at the start of a month. If we take the Empirical Rule seriously (maybe too seriously), then there’s a 95% chance that the return during the month lies between -12.9% and 16.7%. Because the bell-shaped curve is symmetric, that means there’s a 2.5% chance that the percentage change could be less than -12.9%. If that were to happen, the stock would drop by more than 12.9% by the end of the month. For VaR, this invest- ment puts +1,000 * 0.129 = +129 “at risk.” Notice that saying that the VaR is +129 doesn’t mean that we cannot lose more. Instead, it means that this is the most that we would lose if we exclude the worst 2.5% of events. (VaR calculations vary in the percent- age excluded. We’re using 2.5% since it’s easy to find with the Empirical Rule.)

This calculation of VaR takes the Empirical Rule at face value. The Empirical Rule is a powerful tech- nique that allows us to summarize a histogram with the mean and standard deviation. Two summary sta- tistics and the Empirical Rule describe the chances for all sorts of events—as long as the data really are bell shaped. (The match seems pretty good. Stock in Enron fell by 12.9% or more during 4 of the 165 months prior to 2000. That works out to about 2.4%, close to the 2.5% anticipated by the Empirical Rule.)

For comparison, let’s figure out the VaR for a +1,000 investment spread over the whole stock mar- ket. The stock market as a whole grew on average 1.3% per month over this period with standard de- viation 4.4%. The market as a whole has less growth and less variation. The Empirical Rule says there’s a 95% chance that the return on the market lies in the range 1.3% { 2 * 4.4% = -7.5% to 10.1%. The VaR for this investment is +1,000 * 0.075 = +75 compared to the VaR of +129 for Enron.

What’s an investor to do? An investment in Enron puts more value at risk, but it offers a higher aver- age return than investing in the market as a whole. To choose between these requires trading risk for re- turn. Part 2 of this book offers several suggestions based on models for the data.

Once we have the histogram (and boxplot), it’s easy to see that the percentage changes for about one-half of the months lie between -5% and 5%. We can also see that more months show an increase of 10% or higher than show a decrease of 10% or lower. The first bin on the left, which counts months in which the stock lost between 25% and 30%, has only one month, October 1987.

VALUE AT RISK The histogram of stock returns reveals a great deal of risk. The average return during these 165 months is 1.9%. On average, an investor made almost 2% per month by owning stock in Enron. That sounds great compared to a savings account. Owning a stock, however, isn’t quite the same because the stock can go down in value. We need to think about the stan- dard deviation of the percentage changes as well. The standard deviation of the percentage changes in Enron stock is s = 7.4%.

The bell-shaped distribution of percentage changes in Figure 8 suggests that we can use the Empiri- cal Rule. Recall from Chapter 4 that the Empirical Rule connects the mean and SD to the concentra- tion of data around the mean. Of all the various ap- plications of the Empirical Rule in business, one of the most common is in the analysis of stock re- turns. The Empirical Rule predicts that in about two-thirds of the months, Enron stock changes by x# { s = 1.9 { 7.4% from -5.5% to 9.3%. That’s a wide interval. Reaching out further, the Empirical Rule suggests that returns in 19>20 months lie in the range x# { 2s = 1.9 { 2 * 7.4%, or -12.9% to 16.7%. With so much variation, a 2% average return per month seems less comforting.

In finance, risk is variation in the percentage change of the value of an investment. Investors con- sidering Enron in December 1999 should have been prepared for the possibility of losing 5% or 10% of their investment in a single month. Of course, the histogram also shows that the investment is more likely to increase by 10%. This possibility was evi- dently enough to lure investors.

Finance professionals often quantify the risk of an investment using a concept known as the value at risk (VaR). Rather than simply quote the mean and SD of

risk Variation in the returns (or percentage changes) on an investment. value at risk (VaR) An estimate of the maximum potential losses of an investment at a chosen level of chance.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 145 26/10/16 12:41 PM

146 PART I Statistics in Action

CASE SUMMARY

in an investment over a chosen time horizon if we rule out a fraction of the possibilities. When viewed as a time series, a sequence of returns typically has simpler structure than the underlying prices. Re- turns on most stocks are simple time series that can be usefully summarized by a histogram.

Returns summarize the performance of an asset such as stock in a firm. The calculation for returns on stock requires careful accounting for stock splits and dividend payments. Variation in returns defines the risk of the asset and allows one to anticipate the chance for the asset to rise or fall in value. Value at risk (VaR) measures the amount that might be lost

■■ Formula Return on an Investment

If the price of the investment at the end of the current period is Pt and the price at the end of the prior period is Pt - 1, then the return is

Rt = Pt - Pt - 1

Pt - 1

One hundred times the return is the percentage change in the price of the asset. Returns are computed for various time periods, such as daily, monthly, or annually.

■■ About the Data The stock data used in this chapter come from the Center for Research in Security Prices (CRSP) by way of Whar- ton Research Data Services. Analysts at CRSP carefully adjust their data for accounting adjustments such as dividend payments and stock splits.

■■ Questions for Thought These questions consider stock in Pfizer, the pharmaceuti- cal manufacturer. The data are monthly and span 1944 through November 2015.

1. The timeplot in Figure 9 shows the price at the end of the month of one share of stock in Pfizer.

What seems to be happening? How could you verify your answer?

2. Figure 10 tracks the value of an initial +100 investment made in Pfizer in 1944. For each month, the value given is the amount invested in Pfizer at the end of the previous month multiplied by the percentage change during the month. The value of the initial $100 investment is approaching $3 million.

■■ Key Terms return, 142 risk, 145

simple, 143 value at risk (VaR), 145

$20.00

$40.00

$60.00

$80.00

$100.00

$120.00

$140.00

P fiz

e r

P ri

ce

1950 1960 1970 1980 1990 2000 2010 Year

FIGURE 9 Timeplot of the price of stock in Pfizer.

$0

$500,000

$1,000,000

$1,500,000

$2,000,000

$2,500,000

$3,000,000

C o

m p

o u n d

e d

V al

u e o

f $ 1 0 0 In

ve st

m e n t

1950 1960 1970 1980 1990 2000 2010 Year

FIGURE 10 Compounded value of $100 investment made in Pfizer in 1944.

a. What does the White Space Rule say about this figure?

b. Would it be appropriate to summarize these data using a histogram?

3. Figure 11 shows the timeplot of month-to- month percentage changes in the value of Pfizer, adjusting for dividend payments and stock splits. Would it be appropriate to summarize these data using a histogram?

4. Explain in your own words how it is that the percentage changes (Figure 11) look so random, whereas the cumulative value of investing in Pfizer (Figure 10) grows so smoothly until the last 20 years.

5. a. Describe the distribution of percentage changes in the value of Pfizer stock and find the mean and standard deviation.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 146 26/10/16 12:41 PM

147FINANCIAL TIME SERIES

b. Using the mean and standard deviation, how might you estimate the chance that stock in Pfizer would fall 6% or more next month? How does this compare to the fraction of times that the stock fell by 6% or more in the data?

c. If we invest +1,000 in Pfizer for one month, what is the Value at Risk based on the Em- pirical Rule? Interpret the VaR in your own words.

-20

-10

0

10

20

30

40 P

e rc

e n ta

g e C

h an

g e

1950 1960 1970 1980 1990 2000 2010 Year

FIGURE 11 Monthly percentage change on Pfizer stock.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 147 26/10/16 12:41 PM

148

This case study explores variation among the salaries of 1,800 business leaders. Although the business press announces the stratospheric salaries of a few, most ex- ecutives earn quite a bit less than the superstars. To under- stand this variation requires that we transform the data to a new scale. Otherwise, histograms and summary statistics show us only the most extreme observations, ignoring the majority of the data. The transformation used in this example is the most common and important in business statistics, the logarithm.

INCOME AND SKEWNESS The salaries of the chief executive officer, or CEO, of large corporations have become regular news items. Most of these stories describe the salaries of top execu- tives and ignore everyone else. It turns out that there’s a surprising amount of variation in salaries across dif- ferent firms, if you know where and how to look.

The data table for this case study has 1,835 rows, one for each chief executive officer at 1,835 compa- nies in the United States. The data table has several columns, but we begin with the most interesting: total compensation. Total compensation combines salary

with other bonuses paid to reward the CEO for achievements during the year. Some bonuses are paid as cash, and others come as stock in the com- pany. The histogram in Figure 1 shows the distribu- tion of total compensation, in millions of dollars, in 2010. Does it look like what you expect for data that have a median of +4 million with interquartile range of +6 million?

Case: Executive Compensation

INCOME AND SKEWNESS

LOG TRANSFORMATION

ASSOCIATION AND TRANSFORMATIONS

CASE SUMMARY

STATISTICS IN ACTION

Millions of Dollars

$0 $10 $20 $30 $40 $50 $60 $70 $80

100 200C

o u n t

300 400 500

FIGURE 1 Histogram of the total compensation of CEOs.

TABLE 1 Top-paid CEOs in 2010.

CEO Company Business Compensation

Philippe P. Dauman

VIACOM Cable $84.5 million

Lawrence J. Ellison

Oracle Business software

$77.6 million

Ray R. Irani, Ph.D.

Occidental Petroleum

Petroleum and gas

$76.1 million

Leslie Moonves

CBS Television network

$57.7 million

Adam S. Metz

General Growth

Commercial leasing

$53.3 million

This distribution is very skewed. A few outlying CEOs earned extraordinarily high compensation: 17 earned more than +30 million during this year. These 17 are evident in the boxplot, though virtually invisible in the histogram. The histogram leaves so much of the area blank because these outliers span from +30 million to more than +80 million. It’s hard to see the tiny bars in the histogram that count these. You probably rec- ognize the names of several of these high-paid execu- tives. The highest is Philippe Dauman of VIACOM, a media conglomerate that includes MTV, Paramount, and Comedy Central. He earned close to +85 million in 2010. Table 1 lists the top five CEOs.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 148 26/10/16 12:41 PM

149EXECUTIVE COMPENSATION

When compared to the earnings of this elite group, the earnings of the remaining CEOs appear downright reasonable. The tallest bar at the far left of the histo- gram represents CEOs who made less than +2.5 mil- lion. That group accounts for about one-third of the cases shown in Figure 1, but we can barely see them in the boxplot because they are squeezed together at the left margin.

The skewness in the histogram affects numerical

tip summaries as well. When data are highly skewed, we have to be very careful interpreting summary sta-

tistics such as the mean and standard deviation. Skewed distributions are hard to summarize with one or two statistics. Neither the mean nor the median is a good summary. The boxplot gives the median, but it seems so far to the left. The mean is larger but still to the left of Figure 1 because so many cases accu- mulate near zero. The median compensation is +4.01 million. The mean is almost half again as large, +6.01 million. The skewness pulls the mean to the right in order to balance the histogram (see Chapter 4).

For some purposes, the mean remains useful. Suppose you had to pay all of these salaries! Which do you need to figure out the total amount of money that it would take to pay all of these: the mean or the median? You need the mean. Because the mean is the sum of all of the compensations divided by the count, +6.01 * 1833 = +11,030 million, about +11 billion, would be needed to pay them all.

Skewness also affects measures of variation. The interquartile range, the length of the box in the boxplot, is +5.9 million. This length is quite a bit smaller than the wide range in salaries. The standard deviation of these data is larger at +6.68 million but still small compared to the range of the histogram. (For data with a bell-shaped distribu- tion, the IQR is usually larger than the standard deviation.)

We should not even think of using the Empiri- cal Rule (Chapter 4). The Empirical Rule describes bell-shaped distributions and gives silly answers for data so skewed as these. When the distribution is bell shaped, we can sketch the histogram if we know the mean and standard deviation. For instance, the interval given by the mean plus or minus one stan- dard deviation holds about two-thirds of the data when the distribution is bell shaped. Imagine us- ing that approach here. The interval from x# - s to x# + s reaches from 6.01 - 6.68 = -+0.67 million to 6.01 + 6.68 = +12.69 million. The lower endpoint of this interval is negative, but negative compensa- tions don’t happen! Skewness ruins any attempt to use the Empirical Rule to connect x# and s to the histogram.

LOG TRANSFORMATION Visually, the histogram itself seems a poor summary. While complete (it does obey the area principle, Chapter 3), the histogram confines most of the data to a small part of the plot. The x-axis of the histo- gram in Figure 1 ranges from zero to more than +80 million. As a result, it crams about one-third of the cases into one bin that is squeezed into a corner of the plot. The large expanse of white space on the right sends a warning that the data use only a small part of the figure. The White Space Rule (Chapter 4) hints that we should consider other displays that show more data and less empty space.

To see the variation among the less well-paid ex- ecutives, we need to magnify the left side of the his- togram and condense the right. If we could stretch the x-axis on the left and squeeze it on the right, we might be able to see the variation among all of the amounts rather than emphasize the outliers. As an added benefit, the resulting histogram would be more bell shaped, making it sensible to summarize using its mean and standard deviation.

There’s a simple way to spread out the lower com- pensations and pull together the larger compensa- tions. We will do this with a transformation. The most useful transformation of business data is a log transformation. Logs allow us to re-express highly skewed data in a manner that can be summarized with a mean and a standard deviation. We will use base 10, or common, logs.1 The base-10 log of a number, abbreviated log10, is the power to which you must raise 10 in order to find the number. This relationship is easier to express with symbols. The log10 of 100 is 2 because 10

2 = 100, and the log10 of 1,000 is 3 because 103 = 1,000. Logs do not have to be integers. For instance, log10 250 = 2.4 because 102.4 = 250.

Notice that we can interpret base-10 logs as telling us the number of digits in someone’s paycheck (if we add 1). So, Philippe Dauman is an eight-figure CEO and Richard Anderson CEO of Delta Airlines, who made 10 times less (about $8 million), is a seven- digit CEO.

Logs change how we look at compensation. Table 2 shows several salaries with the base-10 logs. The two columns labeled “Gap to Next Salary” list differ- ences between adjacent rows.

transformation In data analysis, the use of a function such as the log to re-express the values of a numerical variable. 1 There’s another common choice for logs known as natural logs, often abbreviated as ln rather than log. The base for a natural log is a special number e = 2.71828 c. There’s a reason for the popularity of natural logs, but that comes later. For dollar amounts, base-10 logs are easier to think about.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 149 26/10/16 12:41 PM

150 PART I Statistics in Action

As we read down the rows of Table 2, the salaries increase by a factor of 10. The gaps between the sala- ries grow wider and wider. Compare these gaps to the fixed gaps between the logs of the salaries. In terms of dollars, there’s a much bigger gap between +10,000 and +100,000 than between +100 and +1,000. As far as logs are concerned, however, these gaps are the same because each represents a 10-fold increase.

Utility for Wealth

Relative change is natural when describing differ- ences among salaries. Think back to the last time that you heard of a salary increase, either your own or in the news. Chances are that the salary increase was described in percentage terms, not dollars. When you read about unions negotiating with com- panies, you hear reports of, say, a 5% increase, not

tip a +500 increase. Putting data on a log scale em- phasizes relative differences (or percentage change)

rather than absolute differences. When describing the value of money, economists

often use an important concept known as utility. Is every dollar that you earn as important as every other dollar? To most of us, earning another +1,000 a year has a lot of “utility.” The extra money means that we can afford to do and buy more than we would oth- erwise. But does the extra money affect us all in the same way? Would another +1,000 change the sort of things Phillipe Dauman can afford to do? Probably not. The formal way to convey the impact of the added +1,000 is through utilities.

Consider the impact of a +1,000 bonus. For Phillippe Dauman, going from +85,000,000 to +85,001,000 amounts to a 0.0012% increase—hardly noticeable. This bonus is a tiny fraction of a huge salary. For an engineer making +100,000, the bonus adds 1% more. In the language of utility, we’d say that the utility of the added +1,000 to the engineer

is 1>0.0012 = 833 times larger than the utility to Phillippe Dauman.

Logs capture this notion. Differences between logs measure percentage differences rather than actual differences. The difference between log10 101,000 and log10 100,000 is 1recall that log a - log b = log a>b2

log10 101,000 - log10 100,000

=log10 101,000 100,000

< 0.0043

That’s small and suggests that if we make +100,000, a 1% bonus is not going to put us in a new house. For Phillippe Dauman, the difference on a log scale is nearly zero:

log10 85,001,000 - log10 85,000,000

= log10 85,001,000 85,000,000

< 0.0000051

Logs change the way we think about differences. Dif- ferences between logs are relative differences. So, when we look at variation on a log scale, we’re think- ing in terms of relative comparisons.

Logs and Histograms

Let’s get back to executive compensation and trans- form compensation to a log scale. Transformation using logs does more than change the labels on the histogram; it also changes the shape of the histo- gram. On a log scale, the distribution of total com- pensation is bell shaped instead of right skewed.2

Figure 2 labels the x-axis in both log10 units and

tip dollar units. When using an unusual scale, help the reader by showing familiar units. For instance,

rather than only show 6, the plot shows +1,000,000.

TABLE 2 Illustration of salaries in dollars and on a log scale.

Salary

Gap to Next Salary

Log10 Salary

Gap to Next Log Salary

$100 $900 2 1

$1,000 $9,000 3 1

$10,000 $90,000 4 1

$100,000 $900,000 5 1

$1,000,000 $9,000,000 6 1

$10,000,000 7

utility A measure of the desire for or satisfaction gained from acquiring a quantity.

2Indeed, the histogram is slightly left skewed! Four CEOs, including Vikram Pandit of Citigroup, were paid $1 for the year. Four other CEOs were paid $0. Figure 2 excludes these eight outliers to show more detail in the histogram. Data are full of surprises if we know where to look.

100

4 $10,000

5 6 $1,000,000

7 8 $100,000,000

200C o

u n t 300

400

FIGURE 2 Histogram of total compensation on a log scale.

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 150 26/10/16 12:41 PM

151

We’d be hard pressed, however, to guess the strength of the association in Figure 3. Intuitively, there ought to be some association between the compensation of the CEO and net sales of the company. Most of these data, however, hide in the lower left corner of the plot because both variables are right skewed. To see the skewness, we can add histograms to the axes.

The histogram of net sales along the x-axis at the top of Figure 4 is even more skewed than the his- togram of executive compensation! The net sales of about two-thirds of the companies fall in the interval at the far left of the histogram. The skewness makes it impossible to see how variation in one variable is associated with variation in the other.

If all we want to know is how compensation and size are related for the superstars, then we might want to use the correlation between these variables.

When labeled with the corresponding amounts, read- ers can figure out right away what the histogram shows. For example, we can see from Figure 2 that a typical compensation lies between +1 million and +10 million (6 to 7 on the log10 scale). We can also see outliers, but this time on both sides of the boxplot.

Because the histogram of the logs of the to- tal CEO compensations is bell shaped, it is easy to summarize. The mean and median are almost identical and in the middle of the histogram. We can guess them from the plot. The average log10 compensation is 6.55, and the median is 6.60. The standard deviation on the log scale is 0.62, and the range x# { 2s = 6.55 { 2 * 0.62 holds about 95% of the data.

caution The use of a logarithm transformation, however, does lose some information.

For instance, when we say that the mean of log compensation is 6.55, what happens to the units? Should we say 6.55 log-dollars? Units are hard to define when data are transformed by logs, and we’ll avoid doing so for the time being. In general, avoid converting summary statistics on the log scale back to the original units. It might be tempting, for exam- ple, to convert 6.55 log-dollars back to dollars by “unlogging” the mean of the logs, getting 106.55 5 $3.55 million. But this is not the mean of the data and instead is close to the median compensation. The mean of the logs is not equal to the log of the mean; it’s much smaller.

Association and Transformations

If summaries like the mean are hard to interpret when the data are transformed, then why trans- form? The key reason to transform a skewed vari- able is not only to see details in its distribution that are concealed by skewness, but also to see how this variable is associated with other variables. With the variation exposed, it becomes easier to see how rela- tive differences in compensation are related to other variables.

Let’s look for association in a scatterplot using the original units. It seems reasonable that CEOs of com- panies with large sales should make more than CEOs of companies with small sales. To measure sales, our data include the total revenue, or sales, of each com- pany. Figure 3 shows a scatterplot of total compensa- tion versus total revenue for the 1,783 companies for which the data record both variables.

What do we learn from this plot? Certainly, we see the outliers. Those at the right are companies with large sales. Wal-Mart and ExxonMobil had the larg- est sales in 2010.

0 0

To ta

l C o

m p

e n sa

ti o

n ($

M ill

io n )

Total Revenue ($ Billion)

10

20

30

40

50

60

70

80

100 200 300 400

FIGURE 3 Scatterplot of total compensation and net sales, both in millions of dollars.

0

0 100 200 300 400

10

20

30

40

To ta

l C o

m p

e n sa

ti o

n ($

M ill

io n )

50

60

70

80

Total Revenue ($ Billion)

FIGURE  4 Histograms along the margins show the skew- ness of both variables.

EXECUTIVE COMPENSATION

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 151 26/10/16 12:41 PM

152 PART I Statistics in Action

a bit larger than correlation between compensation and net sales. More importantly, the correlation is a useful summary of the association between the logs. If we told you the correlation between two variables is 0.6, you ought to be thinking of a plot that looks like the data in Figure 5.

Even so, it is hard to guess the correlation between sales and compensation; we might not even be sure of the direction. It turns out that the correlation be- tween sales and compensation is r = 0.36. The direc- tion is positive, but the size might be surprising.

We get a much different impression of the asso- ciation between compensation and net sales if we

tip re-express both variables as logs. If variables are skewed, we often get a clearer plot of the asso-

ciation between them after using logs to reduce skewness. Figure 5 shows the scatterplot with both variables transformed using log10. Notice the log scale on both axes.

The association is more apparent: This plot of the logs shows that percentage differences in net sales are associated with percentage differences in total compensation. We can imagine a line running along the diagonal of this plot. The direction of the associa- tion is clearly positive, and the pattern is linear. (Fig- ure 5 excludes the eight CEOs who were paid either nothing or $1.)

The correlation between the log10 of total com- pensation and the log10 of net sales is r = 0.69, quite

100

10

To ta

l C o

m p

e n sa

ti o

n ($

M ill

io n )

1

0.1

0.01

0.01 0.1 Total Revenue ($ Billion)

1 10 100

FIGURE 5 Transformation to a log scale shows evident association.

CASE SUMMARY

because it shows variation on a relative scale. Eco- nomics sometimes use logs to measure utility. Vari- ation on a log scale is equivalent to variation among the percentage differences. The log transformation also allows us to measure association between in- come and other factors using the correlation.

Variables that measure income are typically right skewed. This skewness makes the mean and SD less useful as summaries. A transformation changes the scale of measurement in a fundamental way that al- ters the distribution of data. The log transformation is the most useful transformation for business data

■■ About the Data These data come from Standard & Poor’s ExecuComp da- tabase. This database tracks the salaries of top executives at large firms whose stock is publicly traded. Publicly traded firms are required by law to report executive sala-

ries. Privately owned companies are not. We accessed this database using Wharton Research Data Services (WRDS).

■■ Key Terms transformation, 149 utility, 150

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 152 26/10/16 12:41 PM

153

logs. These are more similar than you might expect. In particular, what is the association be- tween log10 compensation and loge compensa- tion? What does the strength of the association tell you?

4. Calculate the log10 of net revenue for these com- panies and find the mean of these values. Then convert the mean back to the scale of dollars by raising 10 to the power of the mean of the logs (take the antilog of the mean of the logs). Is the value you obtain closer to the mean or the median of net sales?

■■ Questions for Thought 1. If someone tells you the mean and SD of a vari-

able, what had you better find out before trying to use the Empirical Rule?

2. Another approach to working with skewed data is to remove the extreme values and work with those that remain. Does this approach work? Remove the top 5 or 10 compensations from these salaries. Does the skewness persist? What if you remove more?

3. You will often run into natural logs (logs to base e, sometimes written ln) rather than base-10

EXECUTIVE COMPENSATION

M06_STIN7167_03_SE_C06_SIA1_pp140-154.indd 153 26/10/16 12:41 PM

561590_MILL_MICRO_FM_ppi-xxvi.indd 2 24/11/14 5:26 PM

This page intentionally left blank

PART II

Probability

M07_STIN7167_03_SE_C07_pp155-178.indd 155 25/10/16 12:19 PM

7 Probabilityc h a p t e r

7.1 FROM DATA TO PROBABILITY

7.2 RULES FOR PROBABILITY

7.3 INDEPENDENT EVENTS

CHAPTER SUMMARY

THOUSANDS OF CUSTOMERS DIAL TOLL-FREE TELEPHONE NUMBERS TO GET HELP. The customer service agent who answers the phone in Bangalore, India, can solve most problems but not every one. The agent forwards hard calls to experienced agents back in the United States. Some days, every caller seems to need more help; on other days, every call seems to get an instant reply.

What’s going on? The number, timing, and questions of calls vary from day to day. This randomness complicates management’s decisions. Managers must decide how many agents to have available to respond to calls, particularly those who reach the second tier.

This decision requires two probabilities: the probability that a call arrives in the next few minutes and the probability that the agent who answers the call resolves the problem. What are probabilities? How are they useful?

This chapTer builds on our inTuiTive sense of prob- abiliTy, providing a framework ThaT allows us To use probabiliTy in making decisions. Part of this framework distinguishes probabilities from percentages observed in data. Probability and data are related, but the rela- tionship requires important assumptions. Probabilities allow us to speculate about what might happen in the future rather than describe what happened in the past. To anticipate what might happen, we still work with counts, but counts of possibilities rather than data.

156

M07_STIN7167_03_SE_C07_pp155-178.indd 156 25/10/16 12:19 PM

7.1 ❘ FROM DATA TO PROBABILITY Most of us have an intuitive sense for what it means to say, “The probability of a tossed coin landing heads is 1>2.” A coin has two sides, and either side is equally likely to be on top when it lands. With two equally likely possibilities, the probability of one or the other is 1>2. The same logic applies to tossing a six-sided die. The six outcomes are equally likely, so we’d say that the prob- ability for each is 1>6.

Away from games of chance, however, the notion of probability becomes subtle. Suppose a phone starts ringing at a call center in India. If the first-tier agent handles the call, it’s an easy call. Calls that need further help are hard calls. What is the probability that the next call is an easy call?

This situation has one characteristic in common with tossing a coin: There are only two possible outcomes, easy calls and hard calls. There’s also a big difference: Easy calls may not arrive at the same rate as hard calls. It’s not as though the caller tosses a coin to decide whether to pose an easy question or a hard question. If the probability of an easy call is not one-half, then what is it? We will look to data for an answer.

Let’s track a sequence of calls. As each call arrives, record a 1 for an easy call and a 0 for a hard call. Figure 7.1 shows the timeplot and histogram for 100 consecutive calls handled by an agent.

80 90 100706050403020100 20 40 60 Call Order Count

H an

d le

d C

al l

0

0.25

0.5

0.75

1.0

0

0.25

0.5

0.75

1.0

FIGURE 7.1 Timeplot with histogram of the type of 100 calls (1 for easy, 0 for hard) to a call-center agent.

There does not appear to be any association between the order of arrival and the outcomes. Scramble the time order, and the plot appears the same. The visual test for association (Chapter 6) finds no pattern in this sequence of calls. The absence of a pattern in the timeplot implies that the histogram is a useful summary of these data.

The mean of these data is the number of ones (easy calls) divided by 100, which is the relative frequency of easy calls. In this example, the mean is 0.64. That is, the agent handled 64% of the incoming calls. Is this the probability that the agent handles the next call? Maybe not. It’s a busy call center, so we can get more data. Figure 7.2 shows the timeplot of the outcomes of the next 100 calls. Of these 100 calls, the agent handled 72, not 64. So, what’s the prob- ability that the agent handles the next call: 0.64, 0.72, or the average of these?

None of these is quite right. Relative frequencies describe past data rather than what will happen next, and the relative frequency depends on which part of the past we observe. As we accumulate more and more calls, however, the relative frequency converges to a constant as shown in Figure 7.3.

tip

157

M07_STIN7167_03_SE_C07_pp155-178.indd 157 25/10/16 12:19 PM

158 CHAPTER 7 Probability

Initially, the relative frequencies bounce around. The graph starts at 1 (for 100%) because the agent handled the first two calls. The agent forwarded the third call, so the accumulated relative frequency of easy calls drops to 2>3. The agent forwarded the fourth and fifth calls as well, so the accumulated relative frequency drops to 2>4 and then to 2>5. Because each new call makes up a smaller and smaller portion of the accumulated percentage, the relative frequency changes less and less as each call arrives. By the 1,000th call—more than shown here—the relative frequency settles down to 70%.

We define the probability of an outcome to be its long-run relative frequency. An outcome is the result of a random experiment or process, whether tossing a coin or picking up the phone in a call center. By long-run relative frequency, we mean the accumulated proportion of times the outcome occurs in a never- ending sequence of trials. As in Figure 7.3, the relative frequency approaches this long-run proportion. (The question of how fast will have to wait until Chap- ter 14.) The probability of an easy call is the relative frequency in the long run as the agent handles more and more calls. If we were asked for the probability that the next call is easy, we would say 0.7 or 70%. That’s based on knowing that over a long run of calls in the past, 70% of the incoming calls were easy.

The Law of Large Numbers

Knowing the probability is 0.7 does not mean that we know what will happen for any particular call. It only means that 70% of the calls are easy in the long run. The same thinking applies to tossing coins. We do not know whether the next toss of a coin will land on heads or tails. Our experience tells us that if we keep tossing the coin, the proportion of tosses that are heads will eventually be close to one-half.

180 190 200170160150140130120110100 Call Order

H an

d le

d C

al l

0

0.25

0.5

0.75

1.0

20 40 60 80 Count

0

0.25

0.5

0.75

1.0

FIGURE 7.2 Timeplot with histogram of the next 100 calls to a call-center agent.

80 90 120110100706050403020100 Call Order

R el

at iv

e Fr

eq ue

nc y

0

0.25

0.5

0.75

1.0

FIGURE 7.3 Accumulated relative frequency of easy calls.

probability The long-run rela- tive frequency of an outcome.

outcome The result of a ran- dom process.

M07_STIN7167_03_SE_C07_pp155-178.indd 158 25/10/16 12:19 PM

7.1 FROM DATA TO PROBABILITY 159

The Law of Large Numbers (LLN) guarantees that this intuition is correct in these ideal examples.

Law of Large Numbers (LLN) The relative frequency of an outcome converges to a number, the probability of the outcome, as the number of observed outcomes increases.

For calls to the agent, the relative frequency of calls that are easily resolved settles down to 70%. We say that the probability of an easy call is 0.7.

The LLN does not apply to every situation. The LLN applies to data like those for the call center in Figures 7.1 and 7.2 because these data don’t have a pattern. The variation is simple as defined in the example of financial time series (p. 769). Weird things can happen for data with patterns. In some cases, the accumulated relative frequency does not settle down. For instance, suppose an agent handles the first 2 calls, forwards the next 4, handles the next 8, forwards the next 16, and so forth. The timeplot of this sequence has a very strong pattern, as shown in Figure 7.4.

80 90 150140130120110100706050403020100 Call Order

R el

at iv

e Fr

eq ue

nc y

0

0.25

0.5

0.75

1.0

FIGURE 7.4 The relative frequency does not settle down if the data have a strong pattern.

This pattern causes the accumulated relative frequency to wander endlessly between 1>3 and 2>3, no matter how long we accumulate the results. We can- not learn a probability from such data.

caution The Law of Large Numbers is often misunderstood because we forget that it only applies in the long run. Consider a contractor who bids

on construction projects. Over the past few years, the contractor won about one-third of the projects that he bid on. Recently, however, the contractor lost four bids in a row. It would be a mistake for him to think that these losses mean that he’s due to win the next contract. The LLN doesn’t describe what happens next. The connection to probabilities emerges only in the long run. A random process does not compensate for what recently happened.

If you toss a fair coin and get five heads in a row, is the next toss more likely to be tails because the coin owes you a tail? No. The coin doesn’t remember what’s hap- pened and balance the outcomes. In fact, if you flipped a fair coin several thousand times, you would find long streaks of heads and tails. Even so, the next toss is just as likely to be heads or tails. The LLN promises that the results ultimately over- whelm any drift away from what is expected, just not necessarily on the next toss.

The Law of Large Numbers establishes a strong connection between data and probabilities. For our purposes, probabilities are linked to data. You might believe that there’s a 50% chance for rain tomorrow, but a business such as AccuWeather that sells weather forecasts builds its probabilities from vast amounts of meteorological data.

M07_STIN7167_03_SE_C07_pp155-178.indd 159 18/11/16 3:27 PM

160 CHAPTER 7 Probability

Beliefs often masquerade as probabilities. For example, what is the prob- ability that the economy expands by more than 3% next year? Figure 7.5 sum- marizes the rate of growth of GDP in the United States in late 2016 anticipated by 71 economists at the start of the year.

Growth in GDP

5

10

15

C o

u n t

0 1 2 3 4

1 Data from: The Wall Street Journal, economic forecasting survey, January 2016. GDP stands for gross domestic product, a measure of the total output of goods and services in the economy.

FIGURE 7.5 Anticipated rates of growth in U.S. gross domestic product (GDP) in 2016.1

It would not be appropriate to conclude from this figure that the probability that GDP will grow by less than 2% is 5>71 < 0.07. This histogram summa- rizes opinions. Opinions such as these are often called subjective probabili- ties. In this book, we do not use probabilities in this sense. A probability must be connected to data as described by the LLN.

This link to data is essential because chance does not always produce the results we expect. Managers of a hospital, for instance, might be inclined to assign nurses in the maternity ward evenly over the week. It’s “obvious” that babies are equally likely to be born on any day. Data, however, tell a different story in Figure 7.6. Babies may not care whether they are born on a weekend, but it appears that pregnant mothers and doctors who deliver the babies do!

FIGURE 7.6 Births in Canada by day of week.2

2 Data from “Risks of Stillbirth and Early Neonatal Death by Day of Week,” by Zhong-Cheng Luo, Shiliang Liu, Russell Wilkins, and Michael S. Kramer, for the Fetal and Infant Health Study Group of the Canadian Perinatal Surveillance System. They collected data on 3,239,972 births in Canada between 1985 and 1998. The reported percentages do not add to 100% due to rounding.

M07_STIN7167_03_SE_C07_pp155-178.indd 160 25/10/16 12:19 PM

7.2 RULES FOR PROBABILITY 161

3 If the workers continue their duties in the same manner regardless of the sign, then the chances remain the same. If the sign changes attitudes toward safety, then the chances might change. 4 The answer is no, for two reasons. First, the LLN only applies in the long run. Second, the local economy of this bank may have changed: Borrowers are more or less likely to default than in the past. 5 The histogram shows births are about equally likely during the week, but less likely on the weekend. So we recommend equal staffing Monday to Friday, but with fewer on Saturday and Sunday. 6 Almost 75% of people pick 3. About 20% pick 2 or 4. If the choice were at random, you’d expect 25% to pick each number. That’s not what people seem to do.

7.2 ❘ RULES FOR PROBABILITY Many situations require manipulating probabilities. Consider risk managers, who decide which loans to approve at a bank. Their experience shows that 2% of borrowers default, 15% of borrowers fall behind, and the rest repay the bank on schedule. It is essential for the managers to know these probabilities, but they also need other probabilities that are derived from these. For exam- ple, the bank may exhaust its cash reserves if too many borrowers default. What are the chances that more than 30 of the 100 borrowers who were ap- proved this week do not repay their loans on time?

Probability appears intuitive—it’s a relative frequency—but the manipu- lations that convert one probability into another become confusing without rules. Fortunately, only three intuitive rules are needed. We will start with these rules and then add two extensions that simplify common calculations.

Probability rules describe the possible outcomes as sets. Consider a single borrower at a bank. A loan made to this borrower results in one of three possi- ble outcomes: default, late, or on-time. These are the only possible outcomes and define a sample space. The sample space is the set of all possible outcomes that can happen in a chance situation. The boldface letter S denotes the sample space (some books use the Greek letter V ). The sample space depends on the problem. The status of the next call to the agent at the call center is either easy or hard, so the sample space for a call is S = {easy, hard}. For a loan, the sam- ple space has three possible outcomes, S = {default, late, on@time}. Curly braces { } enclose the list of the items in a set. Notice that the outcomes that make up a sample space don’t have to be equally likely. Outcomes from tossing a fair coin or rolling a die are equally likely. Those in most business applications are not.

The sample space can be very large. The sample space for the status of a single loan at the bank has 3 elements; the sample space for 2 loans has 3 * 3 = 9 outcomes 1not 2 * 32. Each element of this sample space is a pair. The first ele- ment of each pair is the outcome of the first loan, and the second is the outcome of the second loan. Because this sample space is small, we can list the outcomes:

(default, default) (default, late) (default, on-time)

(late, default) (late, late) (late, on-time)

(on-time, default) (on-time, late) (on-time, on-time)

tip

What Do You Think? a. Supervisors at a construction site post a sign that shows the number of days since the last serious accident. Does the chance for an accident increase the longer it has been since the last accident? Explain why or why not.3

b. Over its 40-year history, a bank made thousands of mortgage loans. Of these loans, 1% resulted in default. Does the LLN mean that 1 out of the next 100 mortgages made by the bank will result in default?4

c. On the basis of Figure 7.6, how would you recommend staffing the maternity ward?5

d. Pick a number at random from 1, 2, 3, or 4. Which did you pick? How many others do you think pick the same number as you?6

sample space S The sample space, denoted S, is the set of all possible outcomes.

M07_STIN7167_03_SE_C07_pp155-178.indd 161 25/10/16 12:19 PM

162 CHAPTER 7 Probability

The sample space for 10 calls to the help line has 210 = 1,024 elements, and the sample space for 100 loans has 3100 < 5 * 1047 elements.

Subsets of a sample space are known as events. Following convention, we denote events by single capital letters, such as A, B, or C. For example, when considering two loans, the event

A = {(default, default2, 1late, late2, 1on@time, on@time2} includes the outcomes in which both loans have the same status. We can list the three outcomes in this event, but that will not be feasible when the event includes hundreds of outcomes.

Three Essential Rules

Every event A has a probability, denoted P1A2. All probabilities must obey three rules. To illustrate each rule, we use a Venn diagram. A Venn diagram is a graphical method for depicting the relationship among sets. Since events are sets, Venn diagrams are helpful for seeing the rules of probability. In a Venn diagram, the area of a set represents its probability. Events with larger probabilities have larger area.

The first rule of probability says that something must happen. Because the sample space is the collection of all possible outcomes, it is assigned probabil- ity 1. Our Venn diagrams display the sample space as a shaded rectangle such as the one shown here.

event A portion (subset) of the sample space.

Venn diagram An abstract graphic that shows one or more sets.

S

In the Venn diagram, the area of the rectangle that denotes the sample space— the probability of the sample space—is 1. The something-must-happen rule provides a useful check on a collection of probabilities. When we assign prob- abilities to the outcomes that make up the sample space we must distribute all of the probability. If the probabilities do not add up to 1, we have forgotten something, double counted, or made an error elsewhere in our calculations.

The second rule of probability comes from the definition of a probability as a relative frequency: Every probability lies between 0 and 1.

Rule 1. The probability of an outcome in the sample space is 1, P1S2 = 1.

Rule 2. For any event A, the probability of A is between 0 and 1, 0 … P1A2 … 1.

The Venn diagram makes the second rule appear reasonable. This Venn diagram shows an event A. Since events are subsets contained within the sam- ple space, P1A2 cannot be larger than 1. It’s also clear that the probability of A cannot be negative. Even if you think an event is inconceivable, its probability

S

A

M07_STIN7167_03_SE_C07_pp155-178.indd 162 25/10/16 12:19 PM

7.2 RULES FOR PROBABILITY 163

cannot be less than zero. You might say casually, “The probability that I will loan you $1,000 is -1,” but zero is as low as a probability can go. You can imagine an event with probability zero, but you won’t observe it. Events with probability zero never occur.

The third rule of probability shows how to combine probabilities. The events described by this rule must be disjoint. Disjoint events have no out- comes in common; disjoint events are also said to be mutually exclusive, like the cells in a contingency table. An outcome in one cannot be in the other. In a Venn diagram, disjoint events A and B have nothing in common and hence do not overlap.

The third rule gives the probability of a union of disjoint events. The union of two events A and B is the collection of outcomes in A, in B, or in both. We will write unions using words as

A or B

(Math texts typically use the symbol < in place of or.) Because disjoint events have no outcomes in common, the probability of their union is the sum of the probabilities. This rule is known as the Addition Rule for Disjoint Events.

Rule 3. Addition Rule for Disjoint Events The probability of a union of disjoint events is the sum of the probabili- ties. If A and B are disjoint events, then

P1A or B2 = P1A2 + P1B2

The union (shown in green in the Venn diagram) combines A (which was yellow) with B (which was cyan). The Addition Rule for Disjoint Events states the obvious: The area (probability) of the union of A with B is the sum of the two areas.

caution Sometimes when speaking, we say the word or to mean one or the other, but not both. “I’m going to finish this paper or go to the gym.”

That usage is not correct when describing unions of events. The union (A or B) is the collection of outcomes in A, in B, or in both of them.

The Addition Rule for Disjoint Events extends to more than two events. So long as the events E1, E2, c, Ek are disjoint, the probability of their union is the sum of the probabilities.

P1E1 or E2 or c or Ek2 = P1E12 + P1E22 + g + P1Ek2

This extension is intuitively reasonable. Since the events have nothing in com- mon, the probabilities accumulate without any double counting.

That’s it. Every other rule for manipulating probabilities comes from these three.

Essential Rules of Probability

1. P1S2 = 1. 2. For any event A, 0 … P1A2 … 1. 3. For disjoint events A and B, P1A or B2 = P1A2 + P1B2.

disjoint events Events that have no outcomes in common; mutually exclusive events.

mutually exclusive Events are mutually exclusive if no element of one is in the other.

union A or B. Outcomes in A, in B, or in both.

S

A B

S

A B

M07_STIN7167_03_SE_C07_pp155-178.indd 163 25/10/16 12:19 PM

164 CHAPTER 7 Probability

The Complement and Addition Rules

Two additional rules make solving many problems easier and faster. The first gives you a way to figure out the probability of an event using what does not happen. Suppose that the probability of an investment increasing in value next year is 0.6. What’s the probability that the investment does not increase in value? It’s 0.4 = 1 - 0.6. To express this intuitive notion as a rule, we need to denote an event that does not happen.

The outcomes that are not in the event A form another event called the complement of A, denoted Ac. The Complement Rule says that the probabil- ity of an event is 1 minus the probability that it does not happen.

Rule 4. Complement Rule The prob- ability of an event is one minus the probability of its complement.

P1A2 = 1 - P1Ac2

The complement of A in the Venn diagram is everything in the sample space S that is not in A (colored cyan in the Venn diagram).

The second extra rule generalizes the Addition Rule for Disjoint Events (Rule 3). It shows how to find the probability of unions of events that are not disjoint. In general, events may share outcomes. Suppose the sample space consists of customers who regularly shop in the local mall, and define the events.

A = {customers who use credit cards}

and

B = {customers who are women}

Female customers who use credit cards are in both A and B. Because the events A and B share outcomes, these events are not disjoint. Hence, the prob- ability of the union A or B is not equal to the sum of the probabilities of the two events.

To find the probability of the union of any events, we first identify the over- lap between events. The intersection of two events A and B is the event con- sisting of the outcomes in both A and B. We write the intersection of two events as A and B. (Math texts generally use the symbol x .)

In general, the probability of the union A or B is less than P 1A2 + P 1B2. You can see why by looking at the Venn diagram. Events that are not dis- joint overlap; this overlap is the intersection. Adding the probabilities double counts the intersection. The Venn diagram also suggests an easy fix. Add the probabilities of the two events and then subtract the probability of the inter- section to compensate for double counting. This logic leads to the Addition Rule.

Rule 5. Addition Rule For any two events A and B, the probability that one or the other occurs is the sum of the probabilities minus the probability of their intersection.

P1A or B2 = P1A2 + P1B2 - P1A and B2

complement Ac. Outcomes not in A.

intersection A and B. Out- comes in both A and B.

S

A

Ac

S

A A and B

B

M07_STIN7167_03_SE_C07_pp155-178.indd 164 25/10/16 12:19 PM

7.2 RULES FOR PROBABILITY 165

The Addition Rule for Disjoint Sets is a special case. If A and B are disjoint, then their intersection is empty and P1A and B2 = 0. To see how these rules are derived from the three essential rules, read the section Behind the Math: Making More Rules at the end of the chapter.

An Example

Let’s try these rules with an example. Movies are commonly shown in cinemas that combine several theaters. That way, one cashier and snack bar can serve customers for several theaters. Here is the schedule of the next six movies that are about to start.

Movie Time Genre

M 1

9:00 p.m. Action

M 2

9:00 p.m. Drama

M 3

9:00 p.m. Drama

M 4

9:05 p.m. Horror

M 5

9:05 p.m. Comedy

M 6

9:10 p.m. Drama

Because all six movies start in the next few minutes, let’s assume that the cus- tomers waiting to buy tickets are equally likely to buy a ticket for each of these movies. What’s the probability that the next customer buys a ticket for a movie that starts at 9 o’clock or is a drama?

Let’s practice using the notation for events. Define the events A = {movie starts at 9 p.m.} and B = {movie is a drama}. The probability sought by the question is P 1A or B2. We can draw the Venn diagram like this, with M’s rep- resenting the movies.

(p. 169)

The event A = {M1, M2, M3} and the event B = {M2, M3, M6}. Because tickets for all movies are equally likely to be purchased, P1A2 = 3>6 and P1B2 = 3>6. However, P1A or B2 ? P1A2 + P1B2. The events A and B are not disjoint; mov- ies M2 and M3 belong to both.

We have two ways to find P1A or B2. The first is obvious when the sample space is small and the outcomes are equally likely: Count the outcomes in the union.

A or B = {M1, M2, M3 } or {M2, M3, M6} = {M1, M2, M3, M6}

Because the outcomes are equally likely, P1A or B2 = 4>6 = 2>3. The second way to find P1A or B2 uses the Addition Rule. M2 and M3 form

the intersection, and hence P1A and B2 = 2>6. From the Addition Rule, P1A or B2 = P1A2 + P1B2 - P1A and B2

= 3>6 + 3>6 - 2>6 = 2>3 Both ways give the same answer. If they don’t, then we’ve made a mistake.

S

A M1 M6

M5M4

M2

M3

B

M07_STIN7167_03_SE_C07_pp155-178.indd 165 25/10/16 12:19 PM

166 CHAPTER 7 Probability

7 Use Rules 1, 3, and 4. The probabilities of the possibilities have to sum to 1, so the probability of re- paid on time is 1 - 10.02 + 0.152 = 0.83. 8 These are disjoint events, so the probabilities add. The probability of default or late is 0.02 + 0.15 = 0.17. 9 Use the Complement Rule. The probability of not defaulting is 1 - P1default2 = 1 - 0.02 = 0.98.

independent events Events that do not influence each other; the probabilities of in- dependent events multiply.

P1A and B2 = P1A2 * P1B2.

What Do You Think? Let’s revisit the example of the risk manager at a bank who has just approved a loan. The loan can end in one of three states: default, late, or repaid on time. We’re given that P1default2 = 0.02 and P1late2 = 0.15. a. What is P1repaid on time2?7 b. Is P1default or late2 = 0.17, or is it smaller?8 c. Determine the probability that a loan does not default.9

7.3 ❘ INDEPENDENT EVENTS Many applications of probability concern a large number of similar events. Consider bank loans. Risk managers are concerned about the chances for de- fault among a large number of customers, not just one or two. Similarly, in- surance companies sell thousands of policies. The chances associated with each loan or policy may be quite similar. For instance, the bank believes that P1default2 = 0.02 for any one loan. What can be said about the chances for default among a large collection of loans?

A common approach that simplifies the calculation of probabilities is to treat the individual loans or policies as independent events. Two events are said to be independent events if the occurrence of one has no effect on the probability of the other. The simplest familiar example occurs when you roll two dice when playing a board game like Monopoly™. The value shown on one die doesn’t influence the other. The decisions of two managers are inde- pendent if the choices made by one don’t influence the choices made by the other. The precise definition of independence says that probabilities multi- ply. We summarize this property of independent events in the Multiplication Rule for Independent Events.

Multiplication Rule for Independent Events Two events A and B are independent if the probability that both A and B occur is the product of the probabilities of the two events.

P1A and B2 = P1A2 * P1B2

The Addition Rule applies to unions; the Multiplication Rule for Independent Events applies to intersections.

In the context of the call center, the probability that an arriving call is easy is 0.7. What’s the probability that the next two calls are easy? If the calls are independent, then the probability is 0.7 * 0.7 = 0.49. Independence extends to more than two events. What is the probability that the next five calls are easy? If they are independent, the probabilities again multiply.

P1easy1 and easy2 and easy3 and easy4 and easy52 = 0.75 < 0.17

For loans at the bank, what is the probability that all 100 loans made to customers are repaid on time? The probability that any one loan repays on time is 0.83. If the events are independent, then the probability that all repay

M07_STIN7167_03_SE_C07_pp155-178.indd 166 25/10/16 12:19 PM

7.3 INDEPENDENT EVENTS 167

on time is 0.83100 < 8 * 10-9. Even though each loan is likely to repay on time, the chance that all 100 repay on time is very small.

The assumption of independence simplifies these calculations but may not be realistic. At the call center, calls for assistance that arrive around the same time of day may be related to each other. That could mean that an easy call is likely to be followed by another easy call. For bank loans, the closure of a large factory could signal problems that would affect several borrowers. A default by one may signal problems with others as well.

caution Events that are not independent are said to be dependent events. You cannot simply multiply the probabilities if the events are depen-

dent. For a specific illustration, let’s use the previous example of selling tickets at a movie theater. Are the events A = {9 p.m.} and B = {Drama} independent? We know that P1A2 = 1>2 and P1B2 = 1>2. If these events are independent, then P1A and B2 = 1>4. Is it? No. The intersection contains two movies, so P1A and B2 = 1>3, not 1>4. These events are dependent. Drama movies make up two-thirds of the films offered at 9 p.m. but only one-third of those offered at other times.

Much of statistics concerns deciding when it is reasonable to treat events as independent. The presence of association in a contingency table (Chapter 5) or a scatterplot (Chapter 6) indicates dependence. Data in the example of Web shopping in Chapter 5, for example, revealed that the relative frequency of making a purchase depends on the host site that pulls the shopper to Amazon. We will return to the topic of dependence in the next chapter.

dependent events Events that are not independent, for which

P1A and B2 ? P1A2 * P1B2.

4M ANALYTICS 7.1 MANAGING A PROCESS

MOTIVATION ▶ STATE THE QUESTION A manufacturer has a large order to fulfill. To complete the order on schedule, the assembly line needs to run without a breakdown for the next five days. What is the probabil- ity that a breakdown on the assembly line will interfere with completing the order? ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Past data indicate that the assembly line runs well most of the time. On 95% of days, everything runs without a hitch. That is, the probability is 0.95 that the assembly line runs the full day without a delay. Barring information to the contrary, assume that the performance is independent from one day to the next. ◀

MECHANICS ▶ DO THE ANALYSIS All but the simplest problems require you to combine several rules to find the needed probability. State a rule as you use it so that when you look back at what you’ve done, you’ll be able to check your thinking.

In this example, the event of a breakdown during the next five days can hap- pen in many ways. A breakdown could happen, for example, on the first and second days, or not until the third. Adding up those probabilities is tedious. There is only one way, however, for no breakdown to occur. The Multiplica- tion Rule for Independent Events implies that the probability that the line runs well for five consecutive days is the product

P1OK for 5 days2 = 0.955 < 0.774

tip

M07_STIN7167_03_SE_C07_pp155-178.indd 167 25/10/16 12:19 PM

168 CHAPTER 7 Probability

The Complement Rule shows that the probability that the line has a break- down is

P1breakdown during 5 days2 = 1 - P1OK for 5 days2 = 1 - 0.774 = 0.226 ◀

MESSAGE ▶ SUMMARIZE THE RESULTS The probability that a breakdown disrupts production during the next five days is 0.226. There is about 1 chance in 4 that a breakdown will interfere with the schedule. It would be wise to warn the customer that the delivery may be late.

Clearly state any assumptions that you have made in nontechnical language. This calculation relies on past performance of the assembly line; if things have changed, the probability might be too high or too low. The calculation also relies on independence from day to day. ◀

Boole’s Inequality

Probabilities of unions are hard to find unless the sets are disjoint. One needs to adjust for double counting, requiring the probability of intersections. These difficulties compound for unions of many sets, as in the prior example of breakdowns in five days. If the probabilities are small, however, a simple up- per bound for the probability of the union is often very accurate.

Boole’s Inequality

P1A1 or A2 or cor Ak2 … p1 + p2 + c + pk

Boole’s inequality (sometimes called Bonferroni’s inequality) is named for the English mathematician George Boole (1815–1864), known for creating Boolean logic. If the events A1, A2, c, Ak have probabilities pi, then Boole’s inequality says that the probability of a union is less than or equal to the sum of the probabilities.

Look at the Venn diagram with k = 3 events to convince yourself; the area of the union of these three events is less than the sum of the areas because of the overlap. Boole’s inequality estimates the probability of the union as if the events are disjoint and do not overlap. By ignoring the double counting, it gives a probability that is too large unless the events are disjoint. If the events have the same probability P1Ai2 = p, then Boole’s inequality simplifies to

P1A1 or A2 or c or Ak2 … p + p + c + p = k p Boole’s inequality is most useful when the events have small probabilities.

Let’s apply Boole’s inequality to the previous example of breakdowns on the assembly line. The event A1 means that at least one breakdown occurs on Monday, A2 means that at least one breakdown occurs on Tuesday, and so forth up to A5 for Friday. Each of these events has probability 0.05. In Example 7.1, we assumed that these events were independent, as if breakdowns one day do not affect the next day. With Boole’s inequality, we don’t need that assumption.

P1breakdown during 5 days2 = P1A1 or A2 or A3 or A4 or A52 … 0.05 + 0.05 + 0.05 + 0.05 + 0.05 = 0.25

Boole’s inequality Probability of a union … sum of the prob- abilities of the events.

S

A1 A2

A3

M07_STIN7167_03_SE_C07_pp155-178.indd 168 25/10/16 12:19 PM

BEHIND THE MATH 169

The exact answer if the events are independent is 0.226. Boole’s inequal- ity comes close, without requiring that we assume that the events are independent.

Best Practices

■■ Make sure that your sample space includes all of the possibilities. It may be helpful to list a few members of the sample space, as we did early in the chapter.

■■ Include all of the pieces when describing an event. As events become more complex, it be- comes easy to omit an outcome and get the wrong count.

■■ Check that the probabilities assigned to all of the possible outcomes add up to 1. Once you are sure you have identified the entire sample space, check that the total probability for the possible outcomes in the sample space is 1. If the sum is less than 1, you need another cat- egory. If the sum is more than 1, check that the outcomes are disjoint.

■■ Only add probabilities of disjoint events. As the name says, events must be disjoint to use the Addition Rule for Disjoint Events. If the

events are not disjoint, you’ll double count the intersection.

■■ Be clear about independence when you use prob- abilities. Independence means that the occur- rence of one event does not affect others. That’s natural for tossing coins but may not be rea- sonable for events that describe the economy, for instance. Unemployment, inflation, and the stock market are all linked.

■■ Only multiply probabilities of independent events. Many auto insurance companies, for example, believe that the probability that you are late paying your credit card and you have an accident is not the product of the probability of paying late times the probability of being in an accident. Their data indicate that these events are not independent, and so they do not multiply the probabilities. Rather, they adjust the probabilities (and insurance rates) to reflect credit records.

Pitfalls

■■ Do not assume that events are disjoint. It’s tempting and makes it easy to find the prob- ability of a union of events, but that does not make it right. If you are unsure about whether the events are disjoint, you can use Boole’s in- equality to approximate the probability of the union.

■■ Avoid assigning the same probability to ev- ery outcome. The faces of a coin or die may

be equally likely, but these are special cases. Most outcomes in business, even when you can count the possibilities, are not equally likely.

■■ Do not confuse independent events with disjoint events. Independence means that the occur- rence of one event does not affect the chances for the other. That cannot happen for disjoint events. If one of two disjoint events happens, the other cannot. Disjoint events are dependent.

BEHIND the MATH

Making More Rules

The three essential rules are important because they bring mathematical precision to probability. The only way to add a new rule is to derive it from these three. No matter how intuitive or obvious, you have to be able to derive a rule from these principles.

For example, it seems obvious that the probability of an event A cannot be larger than 1. We’ve even

listed this property as part of Rule 2. Though in- tuitively obvious, the fact that P1A2 … 1 is a conse- quence of the other rules.

To see that the remaining rules imply that P1A2 … 1, write the sample space as the union of two disjoint events, A and its complement Ac, S = A or Ac. Because A and Ac are disjoint, the probability of their union is the sum of their

M07_STIN7167_03_SE_C07_pp155-178.indd 169 25/10/16 12:19 PM

170 CHAPTER 7 Probability

probabilities by the Addition Rule for Disjoint Events. Hence,

P1S2 = P1A or Ac2 = P1A2 + P1Ac2 Now use the other two rules. The first rule tells us that 1 = P1S2 = P1A2 + P1Ac2. Because the sec- ond rule tells us that P1Ac2 Ú 0, it follows that

P1A2 = 1 - P1Ac2 … 1 For a more challenging exercise, consider the general addition rule.

P1A or B2 = P1A2 + P1B2 - P1A and B2 To derive this rule from the three essential rules, build a little machinery first. Define the difference between two sets as

A - B = 5those elements of A that are not in B6

Assume B is a subset of A. In this case, we can write A as a union of two disjoint sets, A = 1A - B2 or B. Hence, the addition rule for disjoint sets (Rule 3) shows that P1A2 = P1A - B2 + P1B2, from which we get P1A - B2 = P1A2 - P1B2.

Now for the Addition Rule itself. Write the union of any two sets A and B as

A or B = A or 1B - 1A and B22 The sets on the right-hand side are disjoint, so

P1A or B2 = P1A2 + P 1B - 1A and B22 = P1A2 + P 1B2 - P1A and B2

The last step works because the intersection is a subset of B.

CHAPTER SUMMARY

The collection of all possible outcomes forms the sam- ple space S. An event is a subset of the sample space. The probability of an event is the long-run relative frequency of the event. The Law of Large Numbers guarantees that the relative frequency of an event in data that lack patterns converges to the probability of the event in the long run. Several rules allow us to manipulate probabilities. If A and B are events, then

1. Something must happen

P1S2 = 1

2. Probabilities lie between 0 and 1

0 … P1A2 … 1

3. Addition Rule P1A or B2 = P1A2 + P1B2 - P1A and B2 = P1A2 + P1B2 if A and B are disjoint

4. Complement Rule

P1A2 = 1 - P1Ac2

5. Multiplication Rule for Independent Events

P1A and B2 = P1A2 * P1B2

Independence is often assumed in order to find the probability of a large collection of similar events. Boole’s inequality puts an upper bound on the probability of a large collection of events with- out requiring the assumption of independence.

■■ Key Terms Addition Rule for Disjoint, Events, 163 Boole’s inequality, 168 complement, 164 dependent events, 167 disjoint events, 163

event, 162 independent events, 166 intersection, 164 Law of Large Numbers, 159 Multiplication Rule for, Independent Events, 166

mutually exclusive, 163 outcome, 158 probability, 158 sample space S, 161 union, 163 Venn diagram, 162

■■ Objectives • Interpret probability as counts of the times that an

event occurs over many repeated trials, so a prob- ability of 0.3 means 3 times out of 10.

• Speak and read the language of probability, in- cluding complements, intersections, and unions of events, and use the rules of probability to find these.

M07_STIN7167_03_SE_C07_pp155-178.indd 170 25/10/16 12:19 PM

EXERCISES 171

• Recognize disjoint events and use addition to find the probability of their union. If the events over- lap, then know how to adjust for double counting.

• Recognize independent events and use multiplica- tion to find the probability of their intersection.

■■ Formulas and Notation Complement of an Event

Ac = 5outcomes not in A6

Union of Events

A or B = 5outcomes in A, in B, or in both6 1=A h B2

Intersection of Events

A and B = 5outcomes in both A and B6 1= A > B2

Boole’s Inequality

P1A1 or A2 or c or Ak2 … p1 + p2 + g + pk

A

Ac

A

A or B

B

A A and B

B

Mix and Match

Find the matching item from the second column.

EXERCISES

1. Independent events (a) P1A and B2 ? P1A2 * P1B2 2. Disjoint events (b) Ac

3. Union (c) S

4. Intersection (d) P1A or B2 + P1A and B2 = P1A2 + P1B2 5. Complement of A (e) P1A or B2 … P1A2 + P1B2 6. Sample space (f ) A and B

7. Addition Rule (g) P1A and B2 = 0 8. Complement Rule (h) A or B

9. Boole’s inequality (i) P1Ac2 = 1 - P1A2 10. Dependent events ( j) P1A and B2 = P1A2 * P1B2

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

Exercises 11–16. A market research assistant watches five customers as they leave a store. He records whether the customer is carrying a store bag that indicates any

M07_STIN7167_03_SE_C07_pp155-178.indd 171 25/10/16 12:19 PM

172 CHAPTER 7 Probability

customer made a purchase. He writes down a yes or a no for each. The sample space consists of all possible sequences of yes or no for these five customers. Define the events.

A = 5first two shoppers both have a bag6 B = 5last two shoppers both have a bag6 C = 5last three shoppers all have a bag6

11. The sample space S for this experiment has 10 elements.

12. The assumption of independence implies that each shopper has the same probability for carrying a bag.

13. P1A2 + P1B2 = P1A or B2. 14. The probability that both events B and C occur is

equal to P(B).

15. The probability that all of these customers made a purchase is P1A and C2.

16. If the probability any customer makes a purchase is p and shoppers behave independently, then P1A and C2 = 5 p.

Exercises 17–22. The Human Resources (HR) group at a firm interviews candidates for new hires. The firm rates each on a 10-point scale, with the rating 10 denoting exceptionally good candidates and 1 denoting those that

the firm rates poor. HR rated 6 candidates on Monday and 6 candidates on Tuesday. The outcomes of these 12 ratings form the sample space.

17. The events A = 53 candidates on Monday rate above 76 and B = 5two candidates on Tuesday rate above 76 are disjoint events.

18. If 8 of the 12 candidates on Monday and Tuesday rate above 6, then the probability of a candidate rating above 6 on Wednesday is 8>12.

19. The HR group has monitored the outcome of these interviews for years. The Law of Large Numbers assures us that HR can use these data to learn the probability of a candidate scoring above 8.

20. The probability that the ratings of the six candidates on Monday are 56, 4, 3, 8, 11, 86 is zero.

21. Define the events A = 56 out of the 12 candidates rate 8 or better6, B = 53 out of the 6 candidates on Monday rate 8 or better6, and C = 53 out of the 6 candidates on Tuesday rate 8 or better6. Indepen- dence implies that P1A2 = P1B2 * P1C2.

22. Define the events A = 5first candidate is rated 8, 9, or 106 and B = 5first candidate is rated 5, 6, or 76. Then P1A or B2 = P1A2 + P1B2.

Think About It

23. Each of the following scatterplots shows a sequence of observations; the x-axis enumerates the sequence as in the example of the arrival of calls to the agent at the help desk in this chapter. For which cases does the Law of Large Numbers apply to probabilities based on these observations?

(a)

80706050403020100 x

345

350

355

360

365

370

375

385

380

340

Y

(b)

80706050403020100 x

5

10

15

20

25

30

35

40

0

Y

(c)

80706050403020100 x

-0.5

0

0.5

1.0

1.5

2.0

2.5

3.0

-1.0

Y

(d)

80706050403020100 x

-3

-2

-1

0

1

2

3

4

-4

Y

M07_STIN7167_03_SE_C07_pp155-178.indd 172 25/10/16 12:19 PM

24. As in Exercise 23, these scatterplots graph a sequence of observations taken under similar conditions. If we could watch these processes longer and longer and accumulate more data, in which cases would the Law of Large Numbers apply?

(a)

10.5-0.5-1 0 x

5

10

15

20

25

30

35

40

0

Y (b)

10.5-0.5-1 0 x

345

350

355

360

365

370

375

385

380

340

Y

(c)

10.5-0.5-1 0 x

-0.5

0

0.5

1.0

1.5

2.0

2.5

3.0

-1.0

Y

(d)

10.5-0.5-1 0 x

-3

-2

-1

0

1

2

3

4

-4

Y

25. A shopper in a convenience store can make a food selection from frozen items, refrigerated packages, fresh foods, or deli items. Let the event A = 5frozen, refrigerated, fresh6 and B = 5fresh, deli6. (a) Find the intersection A and B. (b) Find the union of A and B. (c) Find the event Ac.

26. A credit-rating agency assigns ratings to corporate bonds. The agency rates bonds offered to companies that are most likely to honor their liabilities AAA. The ratings fall as the company becomes more likely to default, dropping from AAA to AA, A, down to BBB, BB, B, CCC, CC, R, and then D (for in default). Let the event W = 5AAA, AA, A, BBB, BB, B6 and V = 5BBB, BB, B, CCC, CC6. (a) Find the intersection W and V. (b) Describe the union W or V. (c) Find the complement 1W or V2c.

27. A brand of men’s pants offered for sale at a clothing store comes in various sizes. The possible waist sizes are

Waist: 524 inches, 26 inches, c, 46 inches6 with inseams (length of the pant leg)

Inseam: 528 inches, 29 inches, c , 40 inches6

Define the event B = 5waist 40 inches or larger6 and T = 5inseam 36 inches or larger6. (a) Describe the choice of a customer that is in the

event (B and T). (b) What would it mean if P1B and T2 = P1B2 * P1T2? (c) Does the choice of a tall, thin customer lie in the

event (B and T) or the event (B or T)?

28. An auto dealer sells several brands of domestic and foreign cars in several price ranges. The brands sold by the dealer are Kia, Nissan, Buick, Acura, and Ferrari. The prices of cars range from under $20,000 to well over $100,000 for a luxury car. Define the event D = 5Kia, Nissan, Buick6 and C = 5price above +60,0006. (a) Does the sale of a Ferrari lie in the union of D

with C? (b) Do you think it’s appropriate to treat the events D

and C as independent? (c) If all of the models of Kia, Nissan, and Buick

vehicles sold cost less than $55,000, then what is the probability of the event D and C?

29. A company seeks to hire engineering graduates who also speak a foreign language. Should you describe the combination of talents as an intersection or a union?

EXERCISES 173

M07_STIN7167_03_SE_C07_pp155-178.indd 173 25/10/16 12:19 PM

174 CHAPTER 7 Probability

30. Customers visit a store with several departments. A randomly chosen customer goes to the sporting goods department. After wandering for a bit, she buys a pair of running shoes. This event is most naturally viewed as a union or an intersection?

31. Which of the following implications comes from the Law of Large Numbers? (a) Independence produces a sequence of observations

without patterns. (b) Probability calculations require a large number

of experiments. (c) Proportions get close to probabilities in the long

run.

32. The combination of which two of the following characteristics would produce the sort of data needed in order for the Law of Large Numbers to guarantee that a proportion converges to a probability?

(a) Independent trials. (b) Constant chance for the event to occur on each trial. (c) The probability of the event must be 1>2.

33. Count the number of cars that pass through an intersection during consecutive five-minute periods. Would these data allow us to use the Law of Large Numbers eventually to learn P(more than 50 cars in five minutes)?

34. An airline would like to know the probability of a piece of luggage weighing more than 40 pounds. To learn this probability, a baggage handler weighs every 20th bag. Do you think these data allow the airline to use the Law of Large Numbers eventually to learn P(luggage weighs more than 40 pounds)?

35. A Web site recorded whether visitors click on a shown ad. The following plot shows the outcomes for a sequence of 100 visitors, with a 1 shown if the visitor clicked on the ad and a 0 otherwise. (a) Does it appear that the Law of Large Numbers is applicable if this sequence continues indefinitely? (b) Can we find the probability of clicking on the shown ad from looking at these 100 observations?

80 90 100706050403020100 Visitor

C lic

k

0

1

36. The manager of a beachside vacation resort tracks the local weather. Each day, she records a 1 if the day is cloudy. She records a 0 if the day is sunny. The following plot shows a sequence from her data. Would these data be well suited for estimating the probability of sunny weather?

80 90 120100 110706050403020100 Day

C lo

ud y

0

1

M07_STIN7167_03_SE_C07_pp155-178.indd 174 25/10/16 12:19 PM

37. A basketball team is down by 2 points with only a few seconds remaining in the game. There’s a 50% chance that the team will be able to make a 2-point shot and tie the game, compared to a 30% chance that it will make a 3-point shot and win. If the game ends in a tie, the game continues to overtime. In overtime, the team has a 50% chance of winning. What should the coach do, go for the 2-point shot or the 3-point shot? Be sure to identify any assumptions you make.

38. An advertising firm is considering two approaches for increasing the amount a current client spends to adver- tise. The first approach will submit two proposals that taken together increase the amount of business. There’s a 75% chance that the first of these will be accepted and a 50% chance that the second will be accepted. Alternatively, the advertising firm can offer a single, unified proposal that bundles these together as one. If it takes this approach, there’s a 40% chance that the cli- ent will approve the increased budget. Which approach should the advertising firm take if it wants to grow the business? Identify any assumptions you’ve made.

39. A market analyst on TV makes predictions such as saying that there is a 25% chance that the market will move up the next week. What do you think is the meaning of such a phrase?

40. A basketball player who has missed his last seven consecutive shots then makes the game-winning shot. When speaking to reporters afterward, the player says he was very confident that last time because he knew he was “due to make a basket.” Comment on his statement.

41. In the weeks following a crash, airlines often report a drop in the number of passengers, probably because people are wary of flying because they just learned of an accident. (a) A travel agent suggests that, since the Law of

Large Numbers makes it highly unlikely to have two plane crashes within a few weeks of each other, flying soon after a crash is the safest time. What do you think?

(b) If the airline industry proudly announces that it has set a new record for the longest period of safe flights, would you be reluctant to fly? Are the airlines due to have a crash?

42. Many construction sites post a sign that proclaims the number of days since the last accident. (a) The count has grown from 20 days to 60 days to

100 days. An accident is coming any day now. Do you agree?

(b) Would you feel safer visiting a work site that proclaimed it had been 100 days since the last accident, or one that showed that the last accident was 14 days ago?

You Do It

43. The Web site for M&M™ candies used to claim that 24% of plain M&M candies are blue, 20% are orange, 16% green, 14% yellow, and 13% each red and brown.

(a) Pick one M&M at random from a package. 1. Describe the sample space. 2. What is the probability that the one you pick

is blue or red? 3. What is the probability that the one you pick

is not green? (b) You pick three M&M’s in a row randomly from

three separate packages. 1. Describe the sample space for the outcomes of

your three choices. 2. What is the probability that every M&M is blue? 3. What is the probability that the third M&M is red? 4. What is the probability that at least one is blue?

44. A manufacturer reassigns employees to differ- ent tasks each month. It is known that 55% of the employees have less than or equal to two years of experience, 32% have between three and five years of experience, and the rest have more than five years of experience. Assume that teams of three employees are formed randomly. Dave works for this manufacturer. (a) Consider one of Dave’s teammates. What is the

probability that this teammate has 1. two or fewer years of experience? 2. more than five years of experience?

(b) What is the probability that, considering Dave’s two teammates, 1. both have more than two years of experience? 2. exactly one of them has more than five years

of experience? 3. at least one has more than five years of

experience?

45. A survey found that 62% of callers in the United States complain about the service they receive from a call center if they suspect that the agent who handled the call is foreign.10 If so, what is the probability that (state your assumptions) (a) the next three consecutive callers complain about

the service provided by a foreign agent? (b) the next two calls produce a complaint, but not

the third? (c) two out of the next three calls produce

a complaint? (d) none of the next 10 calls produces a complaint?

46. An assembly line has problems with intermittent breakdowns. It seems that the equipment fails at some point during 15% of any eight-hour shift. Each day contains three consecutive shifts. What is the probability that the assembly line (state your assumptions) (a) works fine throughout the three shifts on Monday? (b) works fine on Monday but breaks down during

the first shift on Tuesday? (c) breaks down on a given day? (d) breaks down during only one shift during a chosen

day?

47. Because of the difficulty and time to test the separate components, a computer is completely assembled

10 ”Making Bangalore Sound Like Boston,” Business Week, April 10, 2006.

EXERCISES 175

M07_STIN7167_03_SE_C07_pp155-178.indd 175 25/10/16 12:19 PM

176 CHAPTER 7 Probability

before the power is ever applied. Assume that the computer fails to work if any component fails. (a) Suppose that the rate of defects for each compo-

nent is 0.1%. If the computer is assembled out of 100 components, what is the probability that the assembled computer works when switched on, assuming the components are independent?

(b) For the probability that the system works to be 99%, what is the largest allowable defect rate for the components? Assume independence.

(c) Use Boole’s inequality to estimate the probabil- ity that the system described in part (a) works. Why does Boole’s inequality work so well in this example?

48. A chain of fast-food restaurants offers an instant- winner promotion. Drink cups and packages of French fries come with peel-off coupons. According to the promotion, 1 in 20 packages is a winner. (a) A group of friends buys four drinks and three

orders of fries. What’s the probability that some- one in the group wins, assuming the prizes are attached to the cups and fry containers independently?

(b) Assuming independence, how many drinks and fries would the group have to order to have a 50% chance of finding a winner?

(c) Use Boole’s inequality to estimate the probability in part (a). Do you expect this approach to work well in this application?

49. You take a quiz with six multiple-choice questions. You estimated that you have an 80% chance of getting any question right. What are your chances of getting them all right?

50. A friend took a multiple-choice quiz and got all six questions right, but now claims to have guessed on every question. If each question offers four possible answers, do you believe her? Explain why you either believe her or not.

51. Some call centers use technology to route calls to operators who work from home. Recent analysis shows that, compared to the usual staff, home-based opera- tors have more education and stay on the job longer.11

Call Center Home Based

College education 20% 75%

Annual attrition 100% 25%

(a) What is the probability that a home-based opera- tor stays for more than a year and has a college education? Identify any assumptions that you make and offer some justification.

(b) What is the probability that a college-educated operator at a call center stays for two years? Again, note your assumptions.

52. Intel and AMD compete to put their processing chips into personal computers. In 2014, Intel claimed 93%

11 ”Call Centers in the Rec Room,” Business Week Online, January 23, 2006.

of the laptop market and 83% of the market for desk- top computers. In 2010, Intel commanded 86% of the laptop market and 72% of the desktop market. (a) What is the probability that someone who bought

a laptop in 2010 and again in 2014 purchased computers with Intel chips? Identify any assumptions needed for your answer.

(b) Four customers purchased desktop computers in 2014. What is the probability that all of the computers have Intel chips?

(c) Which changed more from 2010 to 2014 the probability of one customer purchasing a desktop computer with an Intel chip or the probability of four customers all purchasing desktop computers with Intel chips?

53. The social networking site Facebook is popular in the United States and around the world. This table shows the number of Facebook users and populations in four countries in 2014. All of these counts are in millions of people.12

Country Facebook Users Population

United States 152 319

India 109 1,250

Indonesia 60 250

Mexico 44 122

(a) If we select a Facebook member from these at random, what is the probability that that person lives in Asia?

(b) If we pick a person at random in any of these countries, in which country is the probability of picking a Facebook member lowest? Highest?

(c) If we are randomly picking a person in each country, what is the probability that all four are not members of Facebook?

54. A study reported in the New England Journal of Medi- cine revealed surprisingly large differences in rates of lung cancer among smokers of different races.13 For each group, the study reported a rate among smokers per 100,000.

Race Male Female

Black 264 161

White 158 134

Japanese American 121 50

Latino 79 47

(a) What is the probability that a black male smoker develops lung cancer?

(b) What is the probability that at least one of four Japanese American women who smoke develops

12 From Wikipedia and the CIA World Factbook. 13 C. A. Haiman, D. O. Stram, L. R. Wilkens, M. C. Pike, L. N. Kolonel, B. E. Henderson, and L. Le Marchand, “Ethnic and Racial Differences in the Smoking-Related Risk of Lung Cancer,” New England Journal of Medicine, 354 (2006), 333–342.

M07_STIN7167_03_SE_C07_pp155-178.indd 176 25/10/16 12:19 PM

cancer? Do you need any assumptions for this calculation?

(c) If the four women were from the same family, would you question any assumptions used in answering the previous item?

55. To boost interest in its big sale, a retailer offers spe- cial scratch-off coupons. When the shopper makes a purchase, the clerk scratches off the covering to reveal the amount of the discount. The discount is 10%, 20%, 30%, 40%, or 50% of the initial amount. Half of the coupons give the shopper 10% off, 1>4 give 20% off, 1>8 give 30% off, 3>32 give 40% off, and 1>32 give 50% off. (a) What are your chances of getting more than 30%

off of your purchase? (b) A clerk was surprised when three shoppers in a

row appeared with coupons that gave them half off. Should he have been suspicious?

(c) Half of the customers at a register purchase a sweater that retails for $50 and another half purchase a suit that retails for $200. What is the probability that a customer saves more than $20 by using one of these coupons? Be clear about any assumptions you need to make.

56. A fast-food chain randomly attaches coupons for prizes to the packages used to serve french fries. Most of the coupons say “Play again,” but a few are winners. Seventy-five percent of the coupons pay nothing, with the rest evenly divided between “Win a free order of fries” and “Win a free sundae.” (a) If each member of a family of three orders fries

with her or his meal, what is the probability that someone in the family is a winner?

(b) What is the probability that one member of the family gets a free order of fries and another gets the sundae? The third wins nothing.

(c) The fries normally cost $1 and the sundae $2. What are the chances of the family winning $5 or more in prizes?

57. In an NBA basketball game on January 22, 2006, Kobe Bryant of the Los Angeles Lakers scored 81 points, a total score second only to a 100-point performance by Wilt Chamberlain in 1962. The data file [kobe] contains the sequence of his attempts, with 0 indicating a miss, 2 for a made regular basket, and 3 for a 3-point basket. (a) Does it seem to you that Kobe had a hot hand in

this game, or do these data resemble a sequence of independent events?

(b) Kobe missed five out of his last six shots. Ear- lier in the third quarter, he made seven in a row. Explain why this difference does not nec- essarily mean that he cooled off at the end of the game.

58. 4M ANALYTICS: Racetrack Odds

Racetracks give the odds for each horse in the race rather than a probability. Here are the racetrack odds for the 2015 Belmont Stakes. American Pharoah won this race and the Triple Crown.

Post Position Horse Odds

1 Mubtaahij 10-1

2 Tale of Verve 15-1

3 Made from Lucky 12-1

4 Frammento 30-1

5 American Pharoah 3-5

6 Frosted 5-1

7 Keen Ice 20-1

8 Materiality 6-1

Racetrack odds indicate the amount won for a bet on the winning horse. For example, these race- track odds mean that a $1 bet on Mubtaahij wins a $10 payout (plus the $1 wager) if Mubtaahij wins the race.

True odds, rather than racetrack odds, are equivalent to probabilities. If you know the true odds, you can get the probabilities and vice versa. The true odds for an event E are the ratio of the probability that this event happens to the probability that it does not happen.

Odds1E2 = P 1E2

1 - P1E2

If you know the odds, you can get the probability from this formula by solving for P(E):

P1E2 = Odds 1E2

1 + Odds1E2

If the shown racetrack odds are true odds against a horse winning the race, then the following calculation illustrated for Mubtaahij shows how to convert them into probabilities:

P1Mubtaahij wins2 = 1 10 + 1

= 1 11

A bet is known as a fair bet if the chance of winning equals the share of the payout. For example, suppose two people each wager a $1 on the toss of a coin, heads or tails. If the winner gets the total pot of $2, then the bet is fair. Now suppose that the two bettors wager on the roll of a die, with the first bettor winning if the number is 1 or 2. For the winner-take-all bet to be fair, if the first bettor puts $2 in the pot, then the second bettor must put in $4.

Motivation

(a) Operating a racetrack is expensive. If these bets are fair, do you think that the track will be able to stay in business?

Method

(b) Suppose that the true odds for Mubtaahij are 11-1. Why would the racetrack prefer to offer gamblers the odds shown in the table (10-1)?

EXERCISES 177

M07_STIN7167_03_SE_C07_pp155-178.indd 177 25/10/16 12:19 PM

178 CHAPTER 7 Probability

Method

(b) In order for the probability calculations to be simplified, how should the inventory items be chosen?

(c) Suppose the corporation is divided into several sales divisions. The directors would like to sim- plify the gathering of sales records by picking one division at random. Would this be a good idea?

Mechanics

For these calculations, assume that 2% of sales receipts have been date shifted and 3% of inventory levels have been overstated.

(d) What is the probability that a random sample of 25 sales receipts will find none that are fraudulent?

(e) What is the probability that a random sample of 25 sales receipts and 25 inventory counts will find at least one incident of fraud?

(f ) You can afford to audit a total of at most 100 sales receipts and inventory items. Under these conditions, how many of each should you sample in order to maximize the chances of finding at least one fraudulent record? Are you satisfied with this result?

Message

(g) Suppose that checking a random sample of 25 sales receipts and 25 inventory counts finds no evidence of fraud. Does this mean that the probability of fraud at this company is zero?

Mechanics

(c) Treat the racetrack odds as true odds and find the implied probability for each horse winning the 2015 Belmont Stakes.

(d) What is the sum of the probabilities found in (c)?

Message

(e) Interpret the results of this analysis. Are racetrack odds true odds?

59. 4M ANALYTICS: Auditing a Business

Businesses have to be on the lookout for fraud. Sales fraud occurs when managers inflate reported sales figures in order to get a bonus. For example, to reach a sales target for last year, a manager could change a date to make it appear as though something that was sold yesterday got sold last year. Inventory fraud happens when an employee overstates the level of inventory to conceal off-book transactions.

An auditor can spot sales fraud by contacting clients and confirming dates of transactions. The auditor can inspect the inventory to see what is actually on hand. For a corpo- ration that conducts thousands of transactions, however, an auditor cannot check every one. At best, the auditor can check a sample of sales receipts and inventoried items.

For this exercise, you’re the auditor. The directors of a large corporation suspect that some sales receipts have been incorrectly dated and some inventory levels have been overstated.

Motivation

(a) Explain why an auditor need only inspect a sub- set of transactions rather than every transaction.

M07_STIN7167_03_SE_C07_pp155-178.indd 178 25/10/16 12:19 PM

179

8.1 FROM TABLES TO PROBABILITIES

8.2 DEPENDENT EVENTS

8.3 ORGANIZING PROBABILITIES

8.4 ORDER IN CONDITIONAL PROBABILITIES

CHAPTER SUMMARY

Conditional Probability

EDUCATION AFFECTS THE INCOME THAT YOU CAN EXPECT TO EARN. Table 8.1 summarizes the income and education of a sample of 200,000 adults in the United States. Each cell gives a percentage of this group, divided into five income brackets and seven levels of education. For instance, the leading cell indicates that 7.8% of these adults earn less than $25,000 and lack a high school diploma. We can treat these percentages as probabilities. If you were to pick one of these people at random, there’s a 0.078 chance that he or she lacks a high school diploma and earned less than $25,000. The shaded margins give the totals for the rows and columns, defining marginal probabilities. Ignoring education, we find that the probability that a randomly chosen adult earns less than $25,000 is 0.4095.

Other percentages convey the relationship between education and income. For example, only 10% of adults are in the top income bracket. Among adults with a doctorate or a professional degree, however, more than 40% have incomes in the top bracket (1.48/3.37 < 0.439). Education and income are dependent events.

8 c h a p t e r

TABLE  8.1 Adult income and education in the United States. (From the Current Population Survey; cells show percentages.)

0.10

1.03

0.96

0.71

2.37

1.19

1.92

9.55

5.25

3.38

5.70

1.98

0.44

0.43

3.09

2.36

1.84

4.03

2.03

0.58 0.39

0.11

0.80

0.99

0.60

3.61

2.13

1.48

1

2

3

4

5

6

7

8

9

10

B C D E F GA

Education $1 to $24,999 $25,000 to $49,999 $50,000 to $74,999 $75,000 to $99,999 $100,000+ Row Total No high school diploma

High School graduate

Some college, no degree

Associate's degree

Bachelor's degree

Master's degree

Professional or Doctorate

Column Total 28.22 14.36 6.75 9.72

Individual Income Bracket in 2014

7.80

14.66

7.41

3.63

5.30

1.67

0.48

40.95

10.36

29.13

16.97

10.16

21.01

9.00

3.37

100

Percentages within rows or columns of the table corresPond to conditional Probabilities, the topic of this chapter. Conditional probabilities reflect how existing circumstances, or conditions, affect the chances for an event. As with income and education, condi- tional probabilities are often quite different from marginal probabilities.

M08_STIN7167_03_SE_C08_pp179-201.indd 179 26/10/16 12:45 PM

8.1 ❘ FROM TABLES TO PROBABILITIES The following contingency table summarizes two categorical variables, Host and Purchase, that were considered in Chapter 5. This contingency table summarizes two variables for 30,000 visits to amazon.com: the host that sent the visitor and whether a purchase was made.

Host

TotalComcast Google Nextag

Purchase No 270 28,000 665 28,935

Yes 30 1,000 35 1,065

Total 300 29,000 700 30,000

TABLE  8.2 Amazon purchase counts.

Rather than think of these counts as summaries of past events, we can use them to define probabilities. We will define three types of probabilities based on the counts in Table 8.2: joint, marginal, and conditional. The differences among these are important and explain why a manager should interpret the 30 purchases from Comcast very differently from the 35 purchases from Nextag.

To convert this contingency table into probabilities, think about the next visitor to Amazon from one of these hosts. If the next visitor behaves like a random choice from the these 30,000 visits, then we can use these counts to define probabilities. It’s a big assumption to pretend that the next person behaves just like those who came before, but a common way to associate counts with probabilities. Table 8.3 divides each count in Table 8.2 by 30,000 and rounds the resulting table of relative frequencies to three decimals while making sure the cells sum to 1. For the rest of this chapter, we’ll treat these relative frequencies as probabilities.

tip

Host

TotalComcast Google Nextag

Purchase No 0.009 0.934 0.022 0.965

Yes 0.001 0.033 0.001 0.035

Total 0.010 0.967 0.023 1

TABLE 8.3 This table converts the counts into relative frequencies that we interpret as probabilities.

The resulting sample space for the status of a single visit has six outcomes, one for each of the six cells in Table 8.3.

{No and Comcast} {No and Google} {No and Nextag}

{Yes and Comcast} {Yes and Google} {Yes and Nextag}

These outcomes are not equally likely. The most common outcome is {No and Google}, which occurs with probability 0.934. The least common outcome, {Yes and Comcast}, has probability (0.001).

Joint Probability

Each of the outcomes defined by the cells in Table 8.3 has two attributes, the host and whether a purchase will be made. Because the probabilities in the cells of the table describe more than one attribute, these probabilities are called joint probabilities. A joint probability gives the chance for an out- come having two or more attributes. In the language of Chapter 7, a joint

joint probability Probability of an outcome with two or more attributes, as found in the cells of a table; probability of an intersection.

180

M08_STIN7167_03_SE_C08_pp179-201.indd 180 18/11/16 3:14 PM

8.1 FROM TABLES TO PROBABILITIES 181

probability is the probability of an intersection of two or more events. For ex- ample, define the event Y as purchasing visits, and define the event C as visits from Comcast. The first cell in the second row of Table 8.3 tells us the joint probability that the next visitor makes a purchase and comes from Comcast is P1Y and C2 = 0.009.

Marginal Probability

If we care only about one attribute, we need marginal probabilities. A marginal probability is the probability of observing an outcome with a single attribute, regardless of its other attributes. Typically, marginal probabilities are linked to rows and columns of a table and displayed in the margins, as in Table 8.3. This common positioning is the origin of the name marginal probability.

As an example, what is the probability that the next visitor will make a pur- chase? This is the probability of the event Y defined in the prior section.

Y = {{Yes and Comcast} or {Yes and Google} or {Yes and Nextag}}

These outcomes have the form {Yes and *}. We write * because we don’t care which host sends the visitor; any of them will do. We care about the first at- tribute of the outcome, regardless of whether a purchase will be made. These outcomes form the second row of Table 8.4 (yellow).

marginal probability Proba- bility that takes account of one attribute of the event; found in margins of a table.

Host

TotalComcast Google Nextag

Purchase No 0.009 0.934 0.022 0.965

Yes 0.001 0.033 0.001 0.035

Total 0.010 0.967 0.023 1

TABLE  8.4 The second row identifies visits that result in a purchase, the marginal event Y.

The probability of a purchase is the sum

P1Y2 = P1{Yes and Comcast} or {Yes and Google} or {Yes and Nextag}2 = 0.001 + 0.033 + 0.001 = 0.035

The probabilities add because the cells define disjoint events. Though it may seem like common sense, we are using the Addition Rule for Disjoint Events defined in Chapter 7.

Analogously, the probability that the next visit comes from Comcast is an- other marginal probability. The event C = {*, Comcast} is represented by the first column of Table 8.5 (blue).

Host

TotalComcast Google Nextag

Purchase No 0.009 0.934 0.022 0.965

Yes 0.001 0.033 0.001 0.035

Total 0.010 0.967 0.023 1

TABLE  8.5 The Comcast column defines the marginal event C.

The marginal probability of a visit from Comcast is the sum of the probabili- ties in this column.

P1C2 = P1{No and Comcast} or {Yes and Comcast}2 = 0.009 + 0.001 = 0.010

These probabilities also add because the cells define disjoint events.

M08_STIN7167_03_SE_C08_pp179-201.indd 181 26/10/16 12:45 PM

182 CHAPTER 8 Conditional Probability

Among visitors from Comcast, the chance of a purchase is 0.001>0.010 = 0.1. This ratio is the conditional probability of a purchase (the event Y) given that the visitor comes from Comcast (the event C). Symbolically, the condi- tional probability of Y given C is

P1Y u C2 = P1Y and C2 P1C2 =

0.001 0.010

= 0.1

The symbol u in P1Y u C2 means “given.” The phrases “given that,” “conditional on,” or “if it is known that” signal conditional probabilities. In general, the condi- tional probability of the event A given that the event B occurs is

P1A u B2 = P1A and B2 P1B2

tip

Conditional Probability

Table 8.3 shows the joint and marginal probabilities, but these do not directly answer an important question: Which host will deliver the best visitors, those who are more likely to make purchases? The answer to this question requires conditional probabilities.

When working with a table of probabilities, we obtain a conditional probability when we restrict the sample space to a particular row or column. To find the proportion of visitors from Comcast who make purchases, we con- dition on the first column of the table. That is, we restrict the sample space to this one column, as in Table 8.6.

This restriction makes the rest of the sample space irrelevant. It’s as though we know that C must occur. Conditioning on C means that we confine our at- tention to outcomes in C, {No and Comcast} and {Yes and Comcast}. Within this new sample space, what’s the probability of {Yes and Comcast}? If we use the joint probability 0.001, we violate Rule 1 of probability: The probabilities of the outcomes in the new sample space sum to 0.010, not 1. The solution is to scale up the probabilities so that they do sum to 1, as in Table 8.7.

Host

TotalComcast Google Nextag

Purchase No 0.009 0.934 0.022 0.965

Yes 0.001 0.033 0.001 0.035

Total 0.010 0.967 0.023 1

TABLE  8.6 Conditioning on the event that the visit comes from Comcast limits the sample space to the first column of the table.

Host

TotalComcast Google Nextag

Purchase No 0.009/0.010 = 0.9 0.934 0.022 0.965

Yes 0.001/0.010 = 0.1 0.033 0.001 0.035

Total 1 0.967 0.023 1

TABLE  8.7 Conditional probabilities must add to 1.

conditional probability The conditional probability of A given B is

P(A u B) = P(A and B)>P(B) Conditional probabilities in a table refer to proportions within a row or column.

M08_STIN7167_03_SE_C08_pp179-201.indd 182 26/10/16 12:45 PM

Suppose that the event G = {*, Google} is known to occur (a visit from Google, the second column of the table). What is P1Y u G2, the probability of a purchase given that the visit comes from Google? By conditioning on G, we move our focus from the first column of the table to the second column, as shown in Table 8.8.

What Do You Think? Conditional probabilities apply to rows as well as columns. a. In words, describe the probability P1C u Y2.1 b. Which row or column defines the sample space for P1C u Y2?2 c. What is P1C u Y2?3

1 Among purchases (given that a purchase occurs), P1C u Y2 is the probability that the visit comes from Comcast. 2 By conditioning on Y, the second row of the table (the Yes row) becomes the new sample space. 3 P1C u Y2 = P1C and Y2>P1Y2 = 0.001>10.001 + 0.033 + 0.0012 < 0.029. About 3% of purchases originate from Comcast.

The conditional probability of a purchase within this column is less than the conditional probability of a purchase in the first column.

P1Y u G2 = P1Y and G2 P1G2 =

0.033 0.967

< 0.034

In Chapter 5, we learned that Host and Purchase are associated. That’s im- portant for advertising because it means that some sites deliver a higher per- centage of purchasing customers. The association between Host and Purchase is evident in the conditional probabilities of a purchase. To three decimals, these are (with T denoting the visits from Nextag)

P1Y u C2 = 0.100, P1Y u G2 = 0.034, and P1Y u T2 = 0.043 Purchases occur at a much higher rate among visits from Comcast than from Google or Nextag. Rather than having a one-size-fits-all probability that ap- plies to visits from all hosts, the probability of a purchase depends on the host.

What Do You Think? A firm revised its advertising to emphasize the energy-conserving features of its new cars. These ads, as well as ads that do not emphasize conservation, were shown to potential customers. This table shows the joint probabilities.

Customer Reaction

Negative Neutral Positive

Theme of Advertisement Conservation 0.05 0.15 0.30

Standard 0.15 0.15 0.20

8.1 FROM TABLES TO PROBABILITIES 183

Host

TotalComcast Google Nextag

Purchase No 0.009 0.934 0.022 0.965

Yes 0.001 0.033 0.001 0.035

Total 0.010 0.967 0.023 1

TABLE  8.8 Conditioning on the event that the visit comes from Google.

M08_STIN7167_03_SE_C08_pp179-201.indd 183 26/10/16 12:45 PM

184 CHAPTER 8 Conditional Probability

4 Customers respond in the same way to both types of ads. 5 P(positive) = 0.3 + 0.2 = 0.5 P(conservation ad) = 0.05 + 0.15 + .3 = 0.5 6 Dependent: P(positive) * P(conservation ad) = 0.25 ? 0.30 = P(positive and conservation ad). Conservation ads are getting a higher rate of positive responses.

a. What would it mean to advertising managers at this firm if the reaction and theme were independent?4

b. Find the marginal probability that a customer has a positive reaction to this firm’s advertisements and the marginal probability that a customer sees the new conservation-themed advertisement.5

c. Are the reaction and theme independent or dependent? Interpret.6

8.2 ❘ DEPENDENT EVENTS Chapter 7 defines two events A and B to be independent if the probability that both occur is the product of the probabilities:

P1A and B2 = P1A2 * P1B2 Independence makes it easy to find the probability of a combination of

events because we can treat the events one at a time, multiplying their prob- abilities. Many events in business, however, are intentionally dependent. For instance, suppose event A identifies customers who see an advertisement for a service and event B identifies customers who purchase the service. If these events are independent, then the conditional probability of a purchase given that the customer has seen the advertisement is

P1B u A2 = P1A and B2 P1A2 =

P1A2P1B2 P1A2 = P1B2

In words, independence implies that seeing the ad has no effect on the chance that a customer will purchase the service. The business that paid for this ad expected dependence: the chance for a purchase should depend on whether the customer saw the ad.

The events Y and C in the Web-hosting example are dependent. The product of the marginal probabilities does not match the probability of the intersection.

P1Y2 * P1C2 = 0.035 * 0.01 = 0.00035 ? P1Y and C2 = 0.001 The probability of the intersection is three times larger than implied by independence. Similarly, other events in Table 8.3 are also dependent. For in- stance, P1Y2 * P1G2 ? P1Y and G2 and P1Y2 * P1T2 ? P1Y and T2.

Another way to recognize dependent events is to compare the conditional probability P1A u B2 to the marginal probability P1A2. If A and B are indepen- dent, then P1A2 = P1A u B2 and P1B2 = P1B u A2. Independence means that the chance for A is the same whether or not B occurs, and the chance for B is the same whether or not A occurs. For the events Y and C, the marginal prob- ability of a purchase is P1Y2 = 0.034, whereas the conditional probability of a purchase by a visitor from Comcast is P1Y u C2 = 0.1. Purchases are more likely among visits from Comcast than in general. Hence, Y and C are dependent.

The Multiplication Rule

We multiply marginal probabilities to find the probability of several independent events. If the events are dependent, we have to use conditional probabilities.

dependent events Events that are not independent, indicated by P(A and B) ? P(A) * P( B) or P(A) ? P(A u B).

M08_STIN7167_03_SE_C08_pp179-201.indd 184 26/10/16 12:45 PM

The definition of the conditional probability P1A u B2 implies a rule for finding the probability of intersections that applies in general.

P1A u B2 = P1A and B2 P1B2 1 P1A and B2 = P1B2 * P1A u B2

We can also write this rule with the events A and B in the other order.

P1B u A2 = P1A and B2 P1A2 1 P1A and B2 = P1A2 * P1B u A2

These expressions form the Multiplication Rule.

Multiplication Rule The joint probability of two events A and B is the product of the marginal probability of one times the conditional probability of the other.

P1A and B2 = P1A2 * P1B u A2 = P1B2 * P1A u B2

The probability that events A and B both occur is the probability of A times the probability of B given that A occurs (or the probability of B times the probability of A given that B occurs).

Because it is so easy, it is common to see a joint probability routinely calculated as the product of marginal probabilities, as if all events are indepen- dent. For example, a key component of credit risk is the probability that the borrower defaults. Consider three loans, with probabilities of default given by

P1Loan 1 defaults2 = p1, P1Loan 2 defaults2 = p2, P1Loan 3 defaults2 = p3 Assigning a probability of default to each loan is only the beginning. What is the probability that all three default? If the outcomes are independent, the joint probability is the product.

P1Loan 1 defaults and Loan 2 defaults and Loan 3 defaults2 = p1 p2 p3 Is this correct, or might the failure of one borrower signal problems for others as well? Suppose, for instance, that these borrowers all work for the same company. In that case, problems at this company affect all three simultane- ously, making the outcomes dependent. The probability that all three default might be much larger than p1 p2 p3. Anytime you see unconditional probabilities multiplied together, stop and ask whether the events are independent.tip

What Do You Think? A company promotes products developed from medical discoveries. It claims that the probability that such a product becomes a bestseller is 0.10. If the launch of the product is preceded by a major publicity event, the probability of a bestseller is 0.25. The probability of a major event is 0.40.

a. Represent the three probabilities given in this question as probabilities of events, such as P(B) = 0.10. You do the other two.7

b. Are bestselling product launches independent of major events?8

c. What is the probability of a bestselling product and a major publicity event?9

7 P(B) = 0.10, P(M) = 0.40, P(B u M) = 0.25. 8 No, dependent. P(B) ? P(B u M). 9 P(B and M) = P(B u M) * P(M) = 0.25 * 0.4 = 0.1.

8.2 DEPENDENT EVENTS 185

M08_STIN7167_03_SE_C08_pp179-201.indd 185 26/10/16 12:45 PM

186 CHAPTER 8 Conditional Probability

Order Matters

The order in which events are listed does not matter in unions or intersec- tions. The union A or B is the same as the union B or A, and the intersection A and B is the same as B and A. For conditional probabilities, however, order matters. The conditional probability P1A u B2 usually differs from P1B u A2.

For example, compare the conditional probabilities P1Y u C 2 and P1C u Y2 in the Web-hosting example. These conditional probabilities are very differ- ent. P1Y u C2 is the probability of a purchase given that the visit comes from Comcast. In contrast, P1C u Y2 is the probability of selecting a visit from Com- cast given that a purchase occurs. Both conditional probabilities involve pur- chases and Comcast, but that’s where the similarity ends. P1Y u C2 restricts the outcomes to the first column of Table 8.3, whereas P1C u Y2 restricts the outcomes to the second row. There’s no reason for the chance of a purchase among visits from Comcast to be the same as the chance for a visit from Com- cast among purchases. They are indeed very different. Visits from Comcast are rare among purchases,

P1C u Y2 = P1Y and C2 P1Y2 =

P1{Yes and Comcast}2 P1{Yes and *}2 =

0.001 0.035

< 0.029

whereas purchases are more common among visits from Comcast, P1Y u C2 = 0.1.

Independence in Venn Diagrams

What does independence look like in a Venn diagram? Think about it, and then draw a Venn diagram with two independent events.

caution A common, but incorrect, choice is to draw a Venn diagram with dis- joint events. Disjoint (mutually exclusive) events are never independent.

The conditional probability of one disjoint event given the other occurs is zero. If A and B are disjoint events, then

P1A u B2 = P1A and B2 P1B2 =

0 P1B2 = 0 ? P1A2

Since P1A2 ? P1A u B2, disjoint events are dependent. How does the Venn diagram of independent events look? Figure 8.1 shows

an example.

A

A and B

B

S

FIGURE  8.1 Venn diagram of independent events.

The total area of the sample space in Figure 8.1 is 1. The event A (yellow) occupies the left quarter of the sample space, so we assign P(A) = 1>4. The event B (cyan) has probability P(B) = 1>2 and occupies the lower half of the sample space. In the Venn diagram, the intersection of the two events is col- ored green. Now suppose we condition on the event A, restricting attention to the left quarter of the sample space. The event B still has half of the probabil- ity, so P(B u A) = 1>2. Since conditioning on A does not affect the probability of B, P(B u A) = P(B), the events A and B are independent.

M08_STIN7167_03_SE_C08_pp179-201.indd 186 26/10/16 12:45 PM

It is easy to confuse disjoint events with independent events because of their importance in the rules for adding and multiplying probabilities.

If A and B are disjoint, P1A or B2 = P1A2 + P1B2. If A and B are independent, P1A and B2 = P1A2 * P1B2.

For disjoint events, probabilities add. For independent events, probabilities multiply. Try not to confuse when to add and when to multiply. Sums occur when you are looking for the probability of one event or the other. Products occur when you are looking for the probability that both events occur.

tip

10 The probability of a customer making a purchase is probably less than the conditional probability of a customer with a coupon making a purchase. Given that the customer has gone to the trouble to bring a coupon, chances are that the customer has a purchase in mind. P1purchase2 is marginal, P1purchase u coupon2 is the conditional probability. 11 Let’s use set notation. Let D denote a drop in stocks and R denote a recession. Then it must be the case that P1D2 7 P1D and R2. The joint probability must be less than either marginal probability. 12 Each component may have little chance of failing. If we calculate P1C1 fails and C2 fails and c2 by multiplying small probabilities, the product gets very small. If failures are dependent, however, the probability may be much larger. For example, components may share a common defect.

What Do You Think? a. Which is larger, the probability that a customer at a department store makes a purchase or the probability that a customer who is known to have a store coupon makes a purchase? Which is a marginal probability, and which is a conditional probability?10

b. Which is larger, the probability of a fall in the stock market or the prob- ability of a fall in the stock market and a recession?11

c. Officials believe that the probability of a major accident at a nuclear plant is so small that we should not expect a failure for hundreds of years. However, three such failures occurred within about 25 years—Fukushima, Chernobyl, and Three Mile Island. How could the estimates be wrong?12

8.3 ❘ ORGANIZING PROBABILITIES In order to find a probability that involves several events, it is useful to orga- nize the information into a tree or a table. These schemes help you see the information and keep track of the various pieces.

Probability Trees

Pictures help us understand data. They’re also an important way to under- stand probabilities. Venn diagrams help visualize basic manipulations, such as unions or complements. Probability trees organize the conditional prob- abilities that describe a sequence of events.

Consider the success of advertising on apps that you have on your phone. Did you pay for the premier version of Spotify or Evernote, or do you put up with those annoying ads that pop up? Your choice affects how app developers make money. The developers earn money from the free version if users click on featured ads. Their income depends on the ads. Developers earn more consistent income if users pay for a premier version that suppresses ads and adds bonus features.

Imagine randomly picking a customer who is using an app on her phone. Suppose that there’s a 15% chance that at this moment she is using Evernote to keep track of her projects. To keep things simple, let’s assume she only has two other apps that she regularly uses, YouTube and Spotify. At this moment, there’s a 35% chance she’s watching YouTube and a 50% chance she’s using Spotify. If an advertiser pushes an ad to these apps, what are the chances that

8.3 ORGANIZING PROBABILITIES 187

M08_STIN7167_03_SE_C08_pp179-201.indd 187 26/10/16 12:45 PM

188 CHAPTER 8 Conditional Probability

she sees the ad? We need some more information to find the answer. Assume that one-third of Evernote’s users pay for the premium version and don’t see ads. Half of Spotify’s users have the premium version, compared to 20% of YouTube’s users. What’s the probability, based on these percentages, that a randomly chosen customer is using Spotify and sees the ad?

To start, the probability of selecting someone using Spotify is one-half. Now we can use the Multiplication Rule,

P1Using Spotify and See ads2 = P1Using Spotify2 * P1See ads u Using Spotify2

= 1 2

* 1 2

= 1 4

That’s quick and easy. For bigger problems, a probability tree organizes the information in a way that avoids formulas. A probability tree (or tree diagram) shows sequences of events as paths that suggest branches of a tree. The number of paths can grow large, so we usually draw the tree sideways, starting from the left and growing vinelike across the page. You might also see trees drawn from the bottom up or top down. Similar decision trees are used to organize a sequence of management choices.

The first branch of the probability tree for this example separates users of each app. That’s in keeping with the information stated in the problem. We are given the percentage using each app. Each of the initial branches takes us to a possible app. The labels on the branches show the probabilities. The branches must include all possible outcomes, so the probabilities that label the initial branches add to 1.

Evernote

YouTube

Spotify

0.15

0.50

0.35

The tree grows when we show whether a customer sees advertisements. For each app, we add branches that identify the alternatives; conditional probabil- ities label the paths. As with the initial branches, the conditional probabilities leaving each node add to 1.

Evernote

YouTube

Spotify

0.15

0.67

0.33

0.80

0.20

0.50

0.50

0.50

0.35

Evernote and See ads

Evernote and No ads

YouTube and See ads

YouTube and No ads

Spotify and See ads

Spotify and No ads

probability tree (tree dia- gram) A graphical depiction of conditional probabilities that shows the probabilities as paths that join a sequence of labeled events.

M08_STIN7167_03_SE_C08_pp179-201.indd 188 26/10/16 12:45 PM

Let’s use this tree to find the probability of picking a viewer using Spotify and sees ads. The gold nodes identify these viewers. To find the probability of this combination, follow the branches and multiply the probabilities along the way.

P1Using Spotify and See ads2 = 0.50 * 0.50 = 0.25 This is exactly the calculation of this probability we got from the Multiplica- tion Rule, only now it is laid out in a tree.

The events at the far right of the probability tree are disjoint. These final nodes, or leaves as they are sometimes called, identify all the possibilities. Hence, the probabilities of the leaves of the tree sum to 1.

0.15 * 0.67 + 0.15 * 0.33 + 0.35 * 0.80 + 0.35 * 0.20 + 0.50 * 0.50 + 0.50 * 0.50 = 1

We can use these final outcomes to find other probabilities. For example, what’s the probability that a randomly chosen viewer sees ads? For this, add the probabilities for each disjoint event on the far right that identifies viewers who see ads. Because the final outcomes are disjoint, we can add their prob- abilities. There are three of these, and the sum of the probabilities is

P1See ads2 = 0.15 * 0.67 + 0.35 * 0.80 + 0.50 * 0.50 < 0.63 The probability that a randomly chosen viewer sees ads is a bit less than two-thirds.

Probability Tables

A probability tree works well when, as in the prior example, the context puts the information in order. We’re given the app that the customer is using, and then the chances that a user of each app sees ads. So long as your questions align with this order, trees are effective. Otherwise, you have to add the proba- bilities for many leaves. Tables, in contrast, are insensitive to sequencing, plus they imbed the probability rules. Rules like P1A2 = 1 - P1Ac2 are automatic in a table, making it easy to check your calculations.

For comparison, let’s fill in a probability table with the information we just presented as a tree. We have three apps: Evernote, YouTube, and Spotify. These choices label the columns of Table 8.9; the rows identify whether a user sees ads. (You could arrange it the other way; this way fits better on our page.)

EverNote Spotify YouTube

Sees ads

No ads

TABLE  8.9 Initial layout for probability table.

Now let’s fill in the table. Cells inside Table 8.9 are meant to show joint prob- abilities. The margins shaded at the right and bottom will show the marginal probabilities. We will start with the given information. The initial description gives the probability that a customer is using each app. These are marginal probabilities and go in the bottom margin of Table 8.10.

EverNote Spotify YouTube

Sees ads

No ads

0.15 0.35 0.50 1

TABLE  8.10 Probability table with marginal probabilities for the columns.

8.3 ORGANIZING PROBABILITIES 189

M08_STIN7167_03_SE_C08_pp179-201.indd 189 26/10/16 12:45 PM

190 CHAPTER 8 Conditional Probability

The 1 in the lower right corner is the sum of the three marginal probabilities. Tables are self-diagnosing: If we leave something out, the probabilities in the table do not sum to 1 and we are warned that we’ve made a mistake.

Now let’s add the information about seeing ads. The description gives con- ditional probabilities of seeing ads among those using each app. Of those using Evernote, the conditional probability of seeing ads is 0.67. That’s 67% of the probability in the first column, or 0.15 * 0.67 = 0.10. Of those using YouTube, 80% of the probability in the second column goes to those who see ads 10.35 * 0.80 = 0.282. For the third column, half see ads. With these cells filled, we get Table 8.11.

tip

EverNote Spotify YouTube

Sees ads 0.10 0.28 0.25

No ads

0.15 0.35 0.50 1

TABLE  8.11 The conditional probabilities determine the first row of joint probabilities.

EverNote YouTube Spotify

Sees ads 0.10 0.28 0.25 0.63

No ads 0.05 0.07 0.25 0.37

0.15 0.35 0.50 1

TABLE  8.12 Completed probability table.

With the first row done, it is straightforward to fill in the rest of the table. The joint probabilities in each column and row must sum to the associated marginal total. For example, P1Evernote and No ads2 = 0.15 - 0.10 = 0.05. Table 8.12 is the finished table, after we have filled in all of the cells and margins.

Once we have filled in the table, we can answer any question about the marginal, conditional, and joint probabilities. For example, the marginal probability P1See ads2 = 0.63. Plus, if we cannot completely fill in the table, then we are missing some information.

8.4 ❘ ORDER IN CONDITIONAL PROBABILITIES If we know a viewer sees ads, then what is the probability that the viewer is using YouTube? That’s an interesting question because the answer describes those who see ads. Among those who watch ads, which app is most common? Marginally, Spotify is the largest draw; it pulls in one-half of the audience. Does it also provide the largest audience for ads?

The tree we built is not organized to answer this question easily. Trees work well if the question we need to answer matches the direction of the tree; this one does not. The tree shows us conditional probabilities such as P1See ads u Spotify2. It’s not so easy to find P1Spotify u See ads). Those who watch ads are scattered among the branches.

This task is easier with the completed probability table. Visually, to find P1Spotify u See ads2, we need to focus on the first row of the table, as in Table 8.13.

M08_STIN7167_03_SE_C08_pp179-201.indd 190 26/10/16 12:45 PM

The conditional probability rescales the joint probabilities in the first row so that the sum of the conditional probabilities is 1. For example,

P1Spotify u See ads2 = P1Spotify and See ads2 P1See ads2

= 0.25 0.63

< 0.397

The conditional probability is the proportion of the probability in the first row (those who see ads) that is in the second column. Similarly,

P1YouTube u See ads2 = P1YouTube and See ads2 P1See ads2

= 0.28 0.63

< 0.444

Even though Spotify draws a larger audience overall, YouTube generates a larger share of those who watch ads.

4M ANALYTICS 8.1 DIAGNOSTIC TESTING

MOTIVATION ▶ STATE THE QUESTION Breast cancer is the most common non- skin malignancy among women in the United States, and it is the second lead- ing cause of death from cancer among women (lung cancer ranks first). A mammogram is a diagnostic procedure designed to quickly detect breast can- cer. Such a test can exhibit two kinds of errors. One error is known as a false positive: The test incorrectly indicates cancer in a healthy woman. The other error (which is more serious in this situation) is a false negative: The test fails to detect cancer.

The probabilities of these errors are established in clinical trials. In a clinical trial, the test is applied to two groups of people, some known to be healthy and others known to have cancer. Among women with breast cancer, a large clinical trial estimated the probability that mammography detects the cancer is 0.87. Among women without breast cancer, the chance for a negative result is 0.97.13

If a mammogram indicates that a 55-year-old Caucasian woman tests posi- tive for breast cancer, then what is the probability that she in fact has breast cancer? ◀

8.4 ORDER IN CONDITIONAL PROBABILITIES 191

EverNote YouTube Spotify

Sees ads 0.10 0.28 0.25 0.63

No ads 0.05 0.07 0.25 0.37

0.15 0.35 0.50 1

TABLE  8.13 Only the first row is relevant for the conditional probability P1App u See ads2.

13 Banks E, Reeves G, Beral V, et al., “Influence of Personal Characteristics of Individual Women on Sensitivity and Specificity of Mammography in the Million Women Study,” British Medical Journal 2004 Aug 28;329(7464):477.

M08_STIN7167_03_SE_C08_pp179-201.indd 191 26/10/16 12:45 PM

192 CHAPTER 8 Conditional Probability

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH This problem requires conditional probabilities. In such problems, be precise in stating the order of conditioning. You do not have to use formal set notation, but you must express the conditioning. The clinical trials of this mammogra- phy exam indicate that

P1Test positive u Cancer2 = 0.87 P1Test negative u No cancer2 = 0.97

The motivating question asks us to find P1Cancer u Test positive2. A small probability table with two rows and two columns is useful to organize the cal- culations. The test outcomes define the columns, and the presence or absence of cancer defines the rows.

It may not be obvious, but we need one more probability: the overall pres- ence of this cancer in the population. In this case, epidemiologists claim that P1Cancer2 = 0.003 in Caucasian women of this age. ◀

MECHANICS ▶ DO THE ANALYSIS The challenge is to fill in the probability table. Then it will be easy to find the needed conditional probability. The first step is to use the probability from the epidemiologists to fill in the marginal probabilities of the rows.

Positive Negative Total

Cancer 0.003

No cancer 0.997

Total 1

The results from the clinical trials allocate these marginal probabilities across the columns. The rule P1A and B2 = P1A2 * P1B u A2 implies that

P1Cancer and Test positive2 = P1Cancer2 * P1Test positive u Cancer2 = 0.003 * 0.87 = 0.00261

This goes in the first cell of the table. Since the probability in the first row sums to 0.003, that leaves 0.00039 for the other cell of the first row. For the second row, the probability of a healthy person taking the test and getting a negative result is

P1Healthy and Test negative2 = P1Healthy2 * P1Test negative u Healthy2 = 0.997 * 0.97 = 0.96709

That leaves 0.997 - 0.96709 = 0.02991 in the final cell. Here’s the finished table.

Positive Negative

Cancer 0.00261 0.00039 0.003

No cancer 0.02991 0.96709 0.997

0.03252 0.96748 1

As a check, the total probabilities for the cells inside the table sum to 1. With the complete table, it’s easy to use the definition P1A u B2 = P1A and B2>P1B2 to find the needed conditional probability.

P1Cancer u Test positive2 = P1Cancer and Test positive2>P1Test positive2 = 0.00261>0.03252 < 0.0803 ◀

M08_STIN7167_03_SE_C08_pp179-201.indd 192 26/10/16 12:45 PM

MESSAGE ▶ SUMMARIZE THE RESULTS The test is hardly definitive. The chance that a woman who tests positive actu- ally has cancer is small, about 8%.

The outcome is less certain than it might seem. With events of low probability, the result of reversing the conditioning can be surprising. That’s the reason people who test positive are sent for follow-up testing. Even though they test positive, they have a small chance of actually having cancer. Disease testing can have unexpected consequences if people interpret testing positive as being positive. ◀

Bayes’ Rule

Instead of using a table, we can express the calculation that reverses a condi- tional probability algebraically. The formula for doing this is known as Bayes’ Rule. The inputs to Bayes’ Rule are the conditional probabilities in the direc- tion opposite to the one that you want.

Bayes’ Rule The conditional probability of A given B can be found from the condi- tional probability of B given A by using the formula

P1A u B2 = P1A and B2 P1B2 =

P1B u A2 * P1A2 P1B u A2 * P1A2 + P1B u Ac2 * P1Ac2

Although the rule shown here has only two events, it extends to more. The principle remains the same, but the formula gets more complicated.

The formula used in the bottom of the fraction is important as well (though we won’t need it). This expression equates the marginal probability P1B2 to a weighted average of conditional probabilities: P1B2 = P1B u A2 * P1A2 + P1B u Ac2 * P1Ac2.

4M ANALYTICS 8.2 FILTERING JUNK MAIL

MOTIVATION ▶ STATE THE QUESTION Junk email consumes system resources and distracts every- one. Is there a way to help workers filter out junk mail?

A rigid set of rules won’t work at most businesses. The type of messages that each individual receives depends upon his or her work at the company. A good filter has to adapt to the type of messages an individual sees. ◀

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Bayes’ Rule (really, conditional probabilities) makes it possible to build an adaptive system that can filter most junk mail. Spam filters such as Spam As- sassin use Bayes’ Rule to learn your preferences rather than a rigid set of rules.

A spam filter based on Bayes’ Rule works by having each user train the sys- tem. As an employee works through his or her email, he or she identifies mes- sages that are junk. The spam filter then looks for patterns in the words that appear. For example, the system might learn conditional probabilities such as

P1Nigerian general u Junk mail2 = 0.20

8.4 ORDER IN CONDITIONAL PROBABILITIES 193

M08_STIN7167_03_SE_C08_pp179-201.indd 193 26/10/16 12:45 PM

194 CHAPTER 8 Conditional Probability

That is, the phrase Nigerian general appears in 20% of the junk mail of an employee. Evidently, this employee is the target of a scam that offers money for helping a “millionaire” in Nigeria move wealth to the United States. Varia- tions on this scam have been going around since email was invented.

For a spam filter to reverse this conditional probability and find P1Junk mail u Nigerian general2, it tracks how often the term Nigerian gen- eral shows up in mail that the employee identifies as not junk. Let’s as- sume this phrase is unusual in the normal work of this employee, say P1Nigerian general u Not junk mail2 = 0.001. As in the prior example, we need a marginal probability. Let’s assume that half of the employee’s email is junk. We’ll again use a table to organize the information and reverse the conditional probability rather than use the algebraic formula for Bayes’ Rule. ◀

MECHANICS ▶ DO THE ANALYSIS Here’s the completed table. The marginal information tells us that the prob- ability in each column is one-half. The given conditional probabilities allocate the probability within the columns. For example,

P1Nigerian general and Junk2 = P1Junk2 * P1Nigerian general u Junk2 = 0.5 * 0.2 = 0.1

Junk Mail Not Junk Mail Total

Nigerian general appears 0.1 0.0005 0.1005

Nigerian general does not appear 0.4 0.4995 0.8995

Total 0.5 0.5 1

From the table, we find that

P1Junk mail u Nigerian general2 = 0.1>0.1005 = 0.995 The spam filter can then classify messages as junk if it finds this phrase, sav- ing this employee the hassle. ◀

MESSAGE ▶ SUMMARIZE THE RESULTS Email messages to this employee with the phrase Nigerian general have a high probability (more than 99%) of being spam. The system can classify these as junk so that the employee can keep up with the real messages. ◀

An added benefit of a Bayes’ filter for spam is the way that it adapts to the hab- its of each user. The rule we just found for removing spam would not be good for everyone. If an employee managed trade with a Nigerian subsidiary, then this spam filter would be a poor setup. Let’s suppose that half of this employee’s mail is junk, too. The difference is that the phrase Nigerian general shows up in a lot of regular messages. If the phrase appears in half of the regular messages, then his spam filter builds the probability table shown in Table 8.14.

Junk Mail Not Junk Mail Total

Nigerian general appears 0.1 0.25 0.35

Nigerian general does not appear 0.4 0.25 0.65

Total 0.5 0.5 1

TABLE  8.14 Alternative probability table for different user.

Now the system learns

P1Junk mail u Nigerian general2 = 0.1>0.35 = 0.286

M08_STIN7167_03_SE_C08_pp179-201.indd 194 26/10/16 12:46 PM

CHAPTER SUMMARY 195

Best Practices

■■ Think conditionally. Many problems that in- volve probability are more easily understood and solved by using conditional probability. Rather than count in haphazard ways, condi- tional probability provides a framework for solving complex problems.

■■ Presume events are dependent and use the Mul- tiplication Rule. When finding the probability of two events simultaneously occurring, use the multiplication rule with a conditional probability

P1A and B2 = P1A2 * P1B u A2 rather than routinely multiplying the marginal probabilities, unless it is known that the events are independent.

■■ Use tables to organize probabilities. There’s no need to memorize Bayes’ Rule if you organize the information in a table. Tables provide built-

in checks that the probabilities that you have found make sense.

■■ Use probability trees for sequences of condi- tional probabilities. If the problem has a long sequence of events, particularly three or more, tables get cumbersome.

■■ Check that you have included all of the events. It is easy to lose track of some of the probabilities when several things happen at once. In a prob- ability tree, make sure that the probabilities assigned to the leaves add to 1. In a probability ta- ble, check that the probabilities assigned to cells add up to the marginal probabilities and that the total probability assigned to the cells sums to 1.

■■ Use Bayes’ Rule to reverse the order of condi- tioning. In general, the probability of A given B does not resemble the probability of B given A or the probability of A. Tables and trees can help you see what’s happening.

Pitfalls

■■ Do not confuse P1A u B2 for P1B u A2. Be sure to condition on the event that you know has oc- curred, not the other way around. The order of the events for a conditional probability matters.

■■ Don’t think that “mutually exclusive” means the same thing as “independent.” It’s just the oppo- site. Mutually exclusive events are disjoint; they have nothing in common. Once you know one has occurred, the other cannot.

■■ Do not confuse counts with probabilities. The difference lies in the interpretation. A contin- gency table describes two columns of observed

data. Probabilities describe the chances for events that have yet to occur. You don’t need data to come up with a collection of joint prob- abilities. You can propose any probabilities you like, so long as they satisfy the rules introduced in Chapter 7 (no negative probabilities, prob- abilities add to 1, and so forth). Unless you con- nect probabilities to data, however, there’s no reason that the probabilities will be useful. You can make up a joint distribution that shows advertising generates large sales, but that’s not going to boost sales in real stores.

CHAPTER SUMMARY

A joint probability gives the chance for an intersec- tion of events, such as those associated with cells in a table. A marginal probability gives the chance of a single event. A conditional probability is the chance for an event to occur once it is known that another event has occurred. The Multiplication Rule for the probability of two events occurring simultaneously

is P1A and B2 = P1A2 * P1B u A2 . This rule does not presume independence. For independent events, P1B u A2 = P1B2 and P1A u B2 = P1A2. Probability trees and probability tables organize calculations in- volving conditional probabilities. Bayes’ Rule shows how to get P1A u B2 from P1B u A2 and P1B u Ac2 with- out organizing probabilities into a table or tree.

It would not likely classify messages with this term as junk and would instead leave them in the inbox to be read.

M08_STIN7167_03_SE_C08_pp179-201.indd 195 26/10/16 12:46 PM

196 CHAPTER 8 Conditional Probability

■■ Key Terms Bayes’ Rule, 193 conditional probability, 182 dependent events, 184

joint probability, 180 marginal probability, 181 Multiplication Rule, 185

probability tree (tree diagram), 188

■■ Objectives • Distinguish joint, marginal, and conditional prob-

abilities and use the rules and notation of prob- ability to convert from one type of probability to another.

• Use conditional probability to describe dependent events, particularly when there are several depen- dent events.

• Organize large collections of probabilities using trees and tables.

• Manipulate the structure of conditional probabili- ties using tables that simplify reversing the order of conditioning.

• Explain the impact of ordering when using condi- tional probability, such as the difference between P(test positive | disease) and P(disease | test positive).

■■ Formulas

Conditional Probability

P1A u B2 = P1A and B2 P1B2

If P1B2 = 0, the definition of P1A u B2 doesn’t make sense. That’s not a problem in practice since events with probability 0 never occur.

Multiplication Rule

P1A and B2 = P1A2 * P1B u A2 = P1B2 * P1A u B2

Bayes’ Rule

P1A u B2 = P 1B u A2 * P1A2

P1B u A2 * P1A2 + P1B u Ac2 * P1Ac2

■■ About the Data The initial table at the beginning of this chapter comes from a summary of the Current Population Survey, available online from the Bureau of the Cen- sus. We converted the counts to percentages (and

made sure that the percentages in Table 8.1 add to 100%). That makes the counts look more like probabilities.

Mix and Match

Match the item from the first column to the best item from the second.

EXERCISES

1. Probability of B given A (a) P1A and B2 = P1A2 * P1B u A2 2. Probability of Bc given A (b) P1A u B2 = P1B u A2 * P1A2>P1B2 3. Identifies independent events (c) P1A2 = P1A u B2 4. Identifies dependent events (d) 1 - P1B u A2 5. Bayes’ Rule (e) P1A2 ? P1A u B2 6. Multiplication Rule (f) P1A and B2>P1A2

M08_STIN7167_03_SE_C08_pp179-201.indd 196 26/10/16 12:46 PM

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

Exercises 7–14. An administrator tracks absences among the staff working in an office. For each employee, define the events.

A = {employee is absent}

K = {employee is sick}

Let A1 and K1 refer to one employee and let A2 and K2 refer to another.

7. The probability of an employee being absent is greater than the probability that the employee is absent given that the employee is sick.

8. The probability that an employee is sick when it is known that the employee is absent is equal to the probability that an employee is absent when it is known that the employee is sick.

9. If the chance for an employee to be absent is greater than the chance for an employee to be sick, then A and K are dependent events.

10. If the chance for an employee to be absent is greater than the chance for an employee to be sick, then P1A u K2 7 P1K u A2.

11. If she knows that P1A2 = 0.20 and P1K2 = 0.15, the administrator can find P1K u A2.

12. If the event A is independent of the event K, then K is independent of A.

13. If A1 is independent of A2, then finding out that Employee 1 is absent increases the chance that Employee 2 is absent.

14. If A1 is independent of A2, P1A1 u A22 = P1A22. Exercises 15–20. The Human Resources division classifies employees of the firm into one of three categories: admin- istrative, clerical, or management. Suppose we choose an employee at random. Define the events.

A = {administrative}

C = {clerical}

M = {management}

Event A occurs, for example, if the randomly chosen employee is an administrator. Event W occurs if the ran- domly chosen employee (from any category) makes more than +120,000 annually. 15. P1A and C2 = P1A2 * P1C2. 16. If event A is independent of event W, then

P1A u W2 = P1W u A2. 17. Independence of W with each of A, C, and M implies

that an equal proportion of employees within these categories makes above +120,000 annually.

18. If 20% of the employees who make more than +120,000 annually are in management and 40% of the employees who make more than +120,000 annu- ally are administrative, then P1M2 6 P1A2.

19. If we pick an employee at random from the clerical employees, then P1W u C2 is the probability that this employee makes more than +120,000 annually.

20. If 75% of employees who work in management make more than +120,000 annually but only 30% of employ- ees in the firm as a whole make more than +120,000 annually, then the events W and M are dependent.

Think About It

Exercises 21–26. Describe the outcomes of these random processes as either independent or dependent. Briefly explain your choice.

21. Recording whether the manufacturer is located in Europe, Asia, or North America for a sequence of cars observed on an interstate highway

22. Recording the type of accident reported to an auto insurance firm in the weeks following a major ice storm in a region where the firm sells insurance

23. Tracking the number of visits to a major Web site day by day

24. Tracking the daily price of a share of stock in Microsoft

25. Recording the amount purchased by individual customers entering a large retail store (with a zero for those who do not make a purchase)

26. Counting the number of gallons of gasoline put into cars arriving at a service station on the turnpike

In the following Venn diagrams, we have drawn the sam- ple space S as a square, and you may assume that the area of the regions is proportional to the probability. Describe the events A and B as either independent or dependent, with a short explanation of your reasoning. The areas of A and B are both one-quarter of the area of S.

27. S

A

B

28. S

A

B

29. (a) Many companies test employees for recreational drug use. What simple test procedure for whether the employee has used drugs has sensitivity equal to 1? The sensitivity of a test procedure is the probability that the test detects the indicated

EXERCISES 197

M08_STIN7167_03_SE_C08_pp179-201.indd 197 26/10/16 12:46 PM

198 CHAPTER 8 Conditional Probability

problem given that the problem is present. (Hint: To obtain this high sensitivity, this simple proce- dure requires neither measurements nor data.)

(b) What is the problem with such a test procedure?

30. A pharmaceutical company has developed a diagnos- tic test for a rare disease. The test has sensitivity 0.99 (the probability of testing positive among people with the disease) and specificity 0.995 (the probability of testing negative among people who do not have the disease). What other probability must the company determine in order to find the probability that a per- son who tests positive is in fact healthy?

31. An insurer is studying the characteristics of those who buy its policies. It discovered that, among young drivers, 45% insure a foreign-made car. Among those who drive foreign-made cars, the insurer also discov- ered that 45% are young. Consider the events.

Y = {randomly chosen driver is young} F = {randomly chosen driver insures foreign@made car}

Does the equality of these percentages mean that Y and F are independent events?

32. Does the insurer described in Exercise 31 cover more drivers who are young or more drivers who insure foreign-made cars?

33. A study looked into the amount of debt accumulated by recent college graduates. The study found that, among those with student loans, 42% said working during college affected their grades?14

(a) Convert this statement into a conditional prob- ability, including a short description of the as- sociated sample space.

(b) What percentage of students working during col- lege have student loans? Can you tell?

34. The study of recent college graduates described in Exercise 33 found that among those who had gradu- ated with debts from student loans, 33% had sold possessions since graduating. Among those who had graduated free of debt, only 17% had sold posses- sions since graduating. (a) Express these two percentages as conditional

probabilities, including a description of the rel- evant sample space.

(b) What do you need to know in order to determine the proportion of recent college graduates who sold possessions after college?

You Do It

Throughout the following exercises, you can form the probability table with whichever variables you want along the rows and columns. The solutions show the tables drawn one way, but you can swap the rows and columns and still get the right answers.

35. An auto manufacturer has studied the choice of options available to customers when ordering a new vehicle. Of the customers who order a car, 25% choose a sunroof, 35% choose a leather interior,

and 10% choose both. (The rest opt for neither.) Of the customers who order a truck, 20% choose an extended cab, 40% choose all-wheel drive, and 15% choose both. You cannot get a sunroof on a truck or an extended cab on a car. For this brand, half of the customers who order a vehicle choose a truck. (a) Organize these probabilities as a probability tree

or probability table. Explain why you picked a tree or a table.

(b) What’s the probability that the customer orders a vehicle with a sunroof?

36. A vendor specializing in outdoor gear offers custom- ers three ways to place orders for home delivery. Cus- tomers can place orders either in the store, online, or via telephone. Of the orders, one-half are made online with one-quarter each made in stores or via tele- phone. Of the online orders, 40% are only for cloth- ing, 30% are only for camping supplies, and 30% are for both. For orders placed in the store, all are for clothing (to order a size that is not in stock). Among orders placed via telephone, 60% are for clothing alone, 20% are for camping supplies alone, and 20% are for both clothing and camping supplies. (a) Organize these probabilities as a probability tree

or probability table. Explain why you picked a tree or a table.

(b) What’s the probability that the customer orders clothing, either by itself or in combination with camping supplies?

37. Seventy percent of customers at the snack counter of a movie theater buy drinks. Among those who buy drinks, 30% also purchase popcorn. What’s the prob- ability that a customer at the counter buys a drink and popcorn? Theaters use this type of calculation to decide which products should be bundled to appeal to customers.

38. Seventy percent of service calls regarding kitchen appliances involve a minor repair. For example, the customer did not read the instructions correctly or failed to connect the appliance correctly. These service calls cost +75. Service calls for major repairs are more expensive. More than one-half of service calls for major repairs cost the customer +200 or more. What’s the probability that a randomly selected service call costs +200 or more?

39. Some electronic devices get better with age: The failure rate is higher when they are new than when they are six months old. Suppose half of the personal music players of a particular brand have a flaw. If the player has the flaw, it fails in the first six months. If it does not have this flaw, then only 10% fail in the first six months. If a player fails after three months, what is the probability that it has this flaw?

40. Some of the managers at a company have an MBA degree. Of managers at the level of director or higher, 60% have an MBA. Among other managers of lower rank, 35% have an MBA. For this company, 15% of managers have a position at the level of director or higher. If you meet an MBA from this firm, what are the chances that this person is a director (or higher)?14 ”Mom? Dad? I’m Home,” Businessweek, May 5, 2006.

M08_STIN7167_03_SE_C08_pp179-201.indd 198 26/10/16 12:46 PM

41. A box of 12 indistinguishable parts is used to assem- ble a finished product. Five parts in the box are defec- tive and will not fit during assembly. A worker picks parts one at a time and attempts to install them. Find the probability of each outcome. (a) The first two parts chosen are both good. (b) At least one of the first three is good. (c) The first four picked are all good. (d) The worker has to pick five parts to find one that

is good.

42. A shopper wanting two medium blouses heads for the sale rack, which is a mess, with sizes jumbled together. Hanging on the rack are 4 medium, 10 large, and 6 extra-large blouses. Find the probability of each event described. (a) The first two blouses she grabs are the wrong

size. (b) The first medium blouse she finds is the third one

she checks. (c) The first four blouses she picks are all extra-large. (d) At least one of the first four blouses she checks is

a medium.

43. After assembling an order for 12 computer systems, the assembler noticed that an electronic component that was to have been installed was left over. The assembler then checked the 12 systems in order to find the system missing the component. Assume that he checks them in a random order. (a) What is the probability that the first system the

assembler checks is the system that is missing the component?

(b) What is the probability that the second system the assembler checks is missing the component, assum- ing that the first system he checked was OK?

(c) Explain why the answers to parts (a) and (b) are different.

44. Among employees at a retailer, there is a 15% chance that an employee will be absent from work for some reason. The floor manager has been talking with two randomly selected employees each day to get sugges- tions to improve sales. (a) What is the probability that the first employee he

chooses to speak with is absent? (b) Describe the probability that the second employee

he chooses to speak with is absent, assuming that the first employee is present. Is it higher, lower, or the same as the probability in part (a)?

(c) Should your answers to parts (a) and (b) match? Explain why or why not.

45. Choice leads for developing new business opportu- nities are randomly assigned to 50 employees who make up the sales team. Half of the sales team is male, and half is female. An employee can receive at most one choice lead per day. On a particular day, five choice leads are assigned. (a) Are the events {first lead given to a male} and

{second lead given to a male} dependent or independent?

(b) If the first four leads all go to men, what is the probability that the fifth lead also goes to a man?

(c) What is the probability that all five leads go to men if you know that at least four of the leads go to men? (This item is harder than most. A proce- dure to handle these sorts of problems is given in Chapter 11, but you can do this exercise from principles in this chapter.)

46. An appliance requires two components, Type A and Type B, supplied by different manufacturers. A robot randomly selects a component from a batch of Type A components and a second component from a batch of Type B components. The appliance works only if both components are within specifications. It is known that 5% of Type A components and 10% of Type B components are out of specifications. (a) What is the probability that the appliance

produced by the robot selecting one component from each batch at random works? State any as- sumptions clearly.

(b) The robot tests the appliance as it is assembled. It discovers that the current appliance does not work because the parts are not compatible. What is the probability that the component that is out of spec is the Type A component?

47. The following table gives the percentages of men and women 20 years and older employed in various occupations in the U.S. workforce in 2010, from the Current Population Survey. Each column sums to 100%. There are about 70 million men and 70 million women in the workforce.

Sex

Men Women

Occupation

Management & Professional 35 42

Service 14 20

Sales & Office 17 32

Mining & Construction 17 1

Production & Transportation 17 5

Total 100% 100%

(a) The number 14 in the second row of the table, after dividing by 100, represents which of these: P(Man and Service), P(Man | Service), P(Man) * P(Service), P(Service | Man).

(b) If you pick a person at random from the sales and office occupations, what is the probability you picked a woman?

(c) If you know someone’s occupation, can you guess whether the person is male or female? Which oc- cupation makes this guess easiest? Hardest?

(d) What is the probability of picking someone at random from the service industry?

48. A company buys components from two suppliers. One produces components that are of higher qual- ity than the other. The high-quality supplier, call it Supplier A, has a defect rate of 2%. The low-quality

EXERCISES 199

M08_STIN7167_03_SE_C08_pp179-201.indd 199 26/10/16 12:46 PM

200 CHAPTER 8 Conditional Probability

supplier, Supplier B, has a defect rate of 10% but offers lower prices. This company buys in equal volume from both suppliers, with half of the orders going to each supplier. (a) What is the probability that a component to be

installed is defective? (b) If a defective component is found, what is the

probability it came from Supplier A?

49. You fly from Philadelphia to San Francisco with a connection in Dallas. The probability that your flight from Philadelphia to Dallas arrives on time is 0.8. If you arrive on time, then the probability that your lug- gage makes the connection to San Francisco is 0.9. If you are delayed, then the chance of your luggage making the connection with you is 0.5. In either case, you make the flight. (a) What is the probability that your luggage arrives

with you in San Francisco? (b) If your luggage is not there to meet you, what is

the probability that you were late in arriving in Dallas?

50. A survey reports that 62% of callers to a help desk complain about the service if they think they spoke to a foreign agent, compared to 31% who complain if they think they spoke to a native agent. Suppose that 40% of all calls to service centers are handled by agents in the United States. If a caller complains, what is the probability she was dealing with a foreign agent?

51. Recent surveys report that although Internet access has grown rapidly, it’s not universal: Only 71% of U.S. households have Internet access at home. Internet access isn’t universal for households that have com- puters. Of the 77% of households that have comput- ers, 8% do not have Internet access.15 Treat these percentages as probabilities. What is the probability that, among households not connected to the Inter- net, the household does not have a computer?

52. Suppose that 10% of the clerical staff in an office smoke cigarettes. Research shows that 60% of smok- ers and 15% of nonsmokers suffer a breathing illness by age 65. (a) Do these percentages indicate that smoking and

this breathing illness are independent? (b) What’s the probability that a randomly selected

65-year-old employee who has this breathing illness smokes?

53. 4M ANALYTICS: Scanner Data

Many supermarkets offer customers discounts in return for using a shopper’s card. These stores also have scan- ner equipment that records the items purchased by the customer. The combination of a shopping card and scan- ner software allows the supermarket to track which items customers regularly purchase.

The data in this table are based on items purchased in 54,055 shopping trips made by families participating in a test of scanners. As part of the study, the families also reported a few things about themselves, such as the number of children and pets. These data are the basis of the following probabilities.

No Dogs 1 Dog 2 Dogs 3 Dogs

More than

3 Dogs

No dog items 0.0487 0.0217 0.0025 0.0002 0

1 to 3 0.1698 0.0734 0.0104 0.0004 0.0002

4 to 6 0.1182 0.0516 0.0093 0.0006 0.0002

7 to 12 0.1160 0.0469 0.0113 0.0012 0.0005

More than 12 0.2103 0.0818 0.0216 0.0021 0.0011

This table shows the number of dogs owned by the person who applied for the shopping card. The table also shows in rows the number of dog food items purchased at the supermarket during each shopping trip in the time period of the study. The table gives the probability for each combination.

Motivation

(a) Should markets anticipate the row and column events described in this table to be independent?

(b) What could it mean that the probability of customers with no dogs buying dog food is larger than zero?

Method

(c) Which conditional probability will be most useful here? Will it be more useful to mar- kets to calculate row probabilities or column probabilities?

(d) The smallest probabilities in the table are in the last column. Does this mean that owners of more than three dogs buy relatively little dog food?

(e) These probabilities come from counts of actual data. What would it mean if the probabilities added up to a value slightly different from 1?

Mechanics

(f) Expand the table by adding row and column marginal probabilities to the table.

(g) Find the conditional probability of buying more than three items among customers reported to own no dogs. Compare this to the conditional probability of buying more than three dog food items among customers reported to own more than three dogs.

(h) If a customer just bought eight cans of dog food, do you think that she or he owns dogs? Give a probability.

15 “Exploring the Digital Domain,” U.S. Department of Commerce, November 2011.

M08_STIN7167_03_SE_C08_pp179-201.indd 200 26/10/16 12:46 PM

Message

(i) What do you conclude about the use of this ver- sion of the scanner data to identify customers likely to purchase dog food?

54. 4M ANALYTICS: Fraud Detection

Credit-card fraud costs business in the United States billions of dollars each year in stolen goods. Compounding the prob- lem, the risk of fraud increases with the rapidly growing online retail market. To reduce fraud, businesses and credit card vendors have devised systems that recognize charac- teristics of fraudulent transactions. These systems are not perfect, however, and sometimes flag honest transactions as fraudulent and sometimes miss fraudulent transactions.

A business has been offered a fraud detection system to protect its online retail site. The system promises very high accuracy. The system catches 99% of fraudulent transac- tions; that is, given a transaction is fraudulent, the system signals a problem 99% of the time. The system flags hon- est transactions as fraudulent only 2% of the time.

Motivation

(a) What would be the possible consequences to the retailer of mistaking honest transactions for fraudulent transactions? Mistaking fraudulent transactions for honest transactions?

Method

(b) The description of this system gives several conditional probabilities, but are these the con- ditional probabilities that are most relevant to owners of the retail site? What other probabilities would be helpful?

Mechanics

(c) Suppose that the prevalence of fraud among transactions at the retailer is 1%. What are the chances that the system incorrectly labels honest transactions as fraudulent?

(d) Suppose that the prevalence of fraud is higher, at 5%. How does the performance of this system change?

Message

(e) Summarize your evaluation of this system for the retailer. Do you think that this system will be adequate for its needs?

EXERCISES 201

M08_STIN7167_03_SE_C08_pp179-201.indd 201 26/10/16 12:46 PM

DAY TRADING IS POPULAR WITH YOUNG PEOPLE AROUND THE WORLD. The Japanese homemaker Yuka Yamamoto got bored watching television. After seeing an ad, she took $2,000 in savings and started trading stocks from home. Within a year, she turned the initial $2,000 into $1 million and became a celebrity in Japan. Her success as a day trader made her into a popular speaker and author.1

Day traders guess when the price of a stock is headed up or down. They’ve invented rules for deciding when to buy and sell stocks, such as the “cup-and-handle” rule shown in the chart below. A big valley in the sequence of prices followed by a small valley signals a change in the trend of stock in Walt Disney, if you believe the rule. The objective of these trading rules is simple: Enable the trader to buy at a low price, then sell at a higher price.

Day traders need to be right more often than wrong. They’d like to have trading rules that always signal when the market is heading up, but that’s asking for too much. Sometimes the price of stock that they bought stays the same or goes down.

This chapTer defines a concise language and noTaTion for describing random processes, such as sTock reTurns, weaTher paTTerns, or manufacTuring. This language is essential for making decisions in the face of uncertainty. The main component of this language is called a random variable. Whether we’re anticipating the price of a stock, scheduling shipments, or forecasting economic trends, random variables use what we know today to anticipate what will happen tomorrow.

9 c h a p t e r Random Variables

9.1 RANDOM VARIABLES

9.2 PROPERTIES OF RANDOM VARIABLES

9.3 PROPERTIES OF EXPECTED VALUES

9.4 COMPARING RANDOM VARIABLES

CHAPTER SUMMARY

1“In Japan, Day-Trading Like It’s 1999,” New York Times, February 19, 2006. 202

M09_STIN7167_03_SE_C09_pp202-223.indd 202 25/10/16 12:20 PM

9.1 ❘ RANDOM VARIABLES Our first random variable expresses the expectations of a day trader who is interested in stock in IBM. She can buy one share of stock in IBM for $100 at the close of the stock market today. The stock market tomorrow determines whether she makes money. To simplify the arithmetic, we’ll restrict what can happen to three possible outcomes. The price of the stock at the close of the market tomorrow will either go up to $105, stay at $100, or fall to $95. If she buys one share, she might make $5, break even, or lose $5. To decide if IBM is a good investment, she needs to assign probabilities to these outcomes. To- gether, these possible values and the corresponding probabilities define a ran- dom variable.

A random variable describes the probabilities for an uncertain future nu- merical outcome of a random process. For a day trader considering stock in IBM, the random variable is the change in the price of IBM stock tomorrow. She can’t be sure today how activity in the market tomorrow will impact the price of IBM stock, but she can assign probabilities to the possible outcomes.

Following convention, the capital letter X denotes the random variable that represents the change in the price of IBM stock. It takes some getting used to, but you have to remember that X does not stand for a number, as in algebra. A random variable represents the correspondence between the collection of pos- sibilities and their probabilities.

Let’s express the opinions of the day trader as a random variable. To keep the calculations manageable, we will limit the random variable to three pos- sible outcomes. Most of the time, the day trader expects the price to stay the same; she assigns this event probability 0.80. She divides the rest of the proba- bility almost equally between going up (with probability 0.11) and going down (with probability 0.09). There’s a slight edge in favor of an increase. Because one of these outcomes has to happen, the probabilities add to 1. Table 9.1 summarizes the outcomes and probabilities.

random variable The uncertain outcome of a random process, often denoted by a capital letter.

discrete random variable A random variable that takes on one of a list of possible values; typically counts.

continuous random variable A random variable that takes on any value in an interval.

TABLE 9.1 Probabilities that define the random variable X.

Stock Price Change

x Probability P(X 5 x)

Increases $5 0.11

Stays same 0 0.80

Decreases -$5 0.09

This table defines the random variable X. It lists the possible outcomes 1+5, 0, and -+52 and assigns a probability to each. Because we can list all of the possible outcomes, X is called a discrete random variable. A continuous random variable can take on any value within an interval; a continuous ran- dom variable shows how probability is spread over an interval rather than assigned to specific values. We will examine continuous random variables in Chapter 12.

caution The notation for random variables is case sensitive. Capital letters denote random variables, and the corresponding lowercase letters

represent possible outcomes, or realizations. When describing the random variable X, we use x to denote a possible outcome.

This convention produces possibly confusing statements, such as P 1X = x2 in the heading of the third column of Table 9.1. It might seem as though we’re saying something as useless as P 11 = 12. That is not what is meant, and you must pay attention to the capitalization. The statement P 1X = x2 is shorthand for the proba- bility of any one of the possible outcomes: P 1X = 52, P 1X = 02, or P 1X = - 52.

203

M09_STIN7167_03_SE_C09_pp202-223.indd 203 25/10/16 12:20 PM

204 CHAPTER 9 Random Variables

Graphs of Random Variables

Tables work well for discrete random variables that have few outcomes, but plots are better if there are more. Table 9.1 would be huge if we allowed the random variable to take on all possible changes in the price of real shares of IBM stock. To prepare for those situations, we need to get comfortable with plots.

How do we plot a random variable? What plot would you use to de- scribe the relative frequencies of past daily changes? If the timeplot of the changes lacks a pattern, we would create a histogram. An analogous graph shows a random variable. There’s a big difference, however, between the histogram of data and the graph of a random variable. A histogram shows relative frequencies of what happened in the past. The graph of a random variable shows probabilities that anticipate relative frequencies expected in the future.

The following graph of a random variable shows its probability distribution (also called its probability density function, probability distribution function, or probability mass function). The probability distribution of a random variable is often abbreviated as

p1x2 = P1X = x2 Whereas a histogram sets the heights of the bars by counting data, the heights of the probability distribution p(x) are probabilities. Figure 9.1 graphs p(x). [The blue vertical lines that connect p(x) to the x-axis are not part of the prob- ability distribution. We include them for discrete random variables to make it easier to identify positions on the x-axis.]

probability distribution A function that assigns probabilities to each possible value of the random variable.

p(x)

x -5

0.2

0.4

0.6

0.8

0 5

Statistical model A descrip- tion of an uncertain outcome using random variables that characterize sources of variation.

Random Variables as Models

A random variable is an example of a statistical model. A statistical model presents a simplified or idealized view of reality. When the day trader uses the random variable X to describe what she thinks will happen to IBM tomorrow, she hopes that her probabilities match reality.

Data affect the choice of a probability distribution for a random variable. Experiences in the past affect what we expect in the future. If a stock has al- ways gone up or down by $5 and never stayed the same, this random variable wouldn’t be a good description of what is likely to happen. This model sug- gests that it is common to see no change in the value of the stock. If the day trader’s model for the probabilities is right, you’d expect to find 80% of the fu- ture changes at zero, 11% at $5, and 9% at - $5. The Law of Large Numbers (Chapter 7) shows that the relative frequencies of outcomes in the future will eventually match her probability distribution in the long run.

Recall the definition of probability in Chapter 7: A probability is a long-run rela- tive frequency. When we say that X is a random variable with these probabilities,

FIGURE 9.1 Plot of the probability distribution of X.

M09_STIN7167_03_SE_C09_pp202-223.indd 204 25/10/16 12:20 PM

9.2 PROPERTIES OF RANDOM VARIABLES 205

we claim to know the chances for what will happen tomorrow and the days that follow. We cannot say which outcome will happen tomorrow any more than we can say whether a coin will land on heads or tails, but we can describe what will happen for lots of tomorrows. In this sense, the probability distribution of a ran- dom variable describes a histogram of possibilities rather than a histogram of data.

2 Since the probability distribution sums to 1, P1Y = 42 = 1 - 10.5 + 0.25 + 0.06252 = 0.1875=3>16. 3 P1Y 7 12 = 1 - P1Y = 12 = 1>2. 4 The graph should show the points 11, 1>22, 12, 1>42, 13, 1>162 , and 14, 3>162 , possibly connected by vertical lines to the x-axis.

parameter A characteristic of a random variable, such as its mean or standard deviation. Parameters are typically denoted by Greek letters.

9.2 ❘ PROPERTIES OF RANDOM VARIABLES The mean x and standard deviation s are two important summary statistics of data. Both of these statistics are connected to the histogram: x lies at the center of the histogram, and s measures its variability. A random variable has analo- gous properties that go by the same names, mean and standard deviation. The mean and standard deviation of a random variable are linked to its graph in the same way that x and s are linked to the histogram.

This similarity can cause confusion. Do not confuse the characteristics of a random variable, called parameters, with statistics computed from data. To distinguish parameters from statistics, we will typically use Greek symbols to denote the parameters of a random variable.

Mean of a Random Variable

The probability distribution of a random variable determines its mean. Even though the mean of the random variable is not an average of data, it has a simi- lar definition. To see the connection, imagine that we have data whose relative frequencies happen to match the probabilities for X. We have n = 100 observed changes in the price of IBM stock. Of these, the price fell by $5 on 9 days, stayed the same 80 times, and increased by $5 on 11 days. The histogram of these 100 values matches the probability distribution of X in Figure 9.1.

What is the mean of these imagined data? Because of the repetition, this is an easy calculation. 9 80 11

x = 1-52 + c + 1-52 + 0 + c + 0 + 5 + c + 5

100

= -5192 + 01802 + 51112

100 = -510.092 + 010.802 + 510.112 = +0.10

The average of these 100 changes is $0.10.

tip

(+++)+++* (+)+* (+)+*

What Do You Think? Customers who buy tires at an auto service center purchase one tire, two tires, three tires, or a full set of four tires. The probability of buying one tire is 1/2, the probability of buying two is 1/4, and the probability of buying three is 1/16. The random variable Y denotes the number of tires purchased by a customer who has just entered the service center.

a. What is P1Y = 42?2 b. What is P1Y 7 1)?3 c. Graph the probability distribution of Y.4

M09_STIN7167_03_SE_C09_pp202-223.indd 205 25/10/16 12:20 PM

206 CHAPTER 9 Random Variables

The mean M of a random variable is defined in just this way, with rela- tive frequencies replaced by probabilities. The mean of the random variable X is a weighted average of the possible outcomes, with probabilities p(x) for the weights. The most common symbol for the mean of a random variable is the Greek letter m (mu, pronounced “mew”). The mean of the random variable X is

m = -5 p1-52 + 0 p102 + 5 p152 = -510.092 + 010.802 + 510.112 = +0.10

The mean m has the same units as the possible values of the random variable. The mean m tells us that the day trader expects on average to make 10 cents on every share of IBM. In general, the mean of a discrete random variable X that has k possible values x1, x2, c, xk is

m = x1 p1x12 + x2 p1x22 + g+ xk p1xk2

The sum includes all of the possible outcomes of the random variable. (When dealing with several random variables, a subscript distinguishes each mean, as in mX = +0.10.2

The data m reveals the same thing about the random variable X that x tells us about. To balance the histogram of data, we locate the fulcrum at x. To bal- ance the probability distribution of a random variable, we locate the fulcrum at m. Figure 9.2 shows the probability distribution of the random variable X with a fulcrum added at m = 0.1, just to the right of zero.

mean M of a random variable The weighted sum of possible values, with the prob- abilities as weights.

p(x)

x -5

0.8

0.6

0.4

0.2

0.0 5

FIGURE 9.2 The mean balances the probability distribution.

The average gain seems small until you think about what happens over time. Each share costs $100, so a gain of $0.10 amounts to an increase of 0.10/100 = 0.1% daily. If you had a savings account that paid 0.1% daily, that would pile up more than 44% interest annually!

Expected Value

The mean of a random variable is a special case of a more general concept. A weighted average of outcomes that uses probabilities as weights is known as an expected value. Because the mean of a random variable is a weighted av- erage of its possible values, the mean of X is also known as the expected value of X. It is written in this notation as

E1X2 = m = x1 p1x12 + x2 p1x22 + g+ xk p1xk2

expected value A weighted average that uses probabilities to weight the possible outcomes.

M09_STIN7167_03_SE_C09_pp202-223.indd 206 25/10/16 12:20 PM

caution The name expected value can be confusing. For the stock, the expected value of X is not one of the possible outcomes. The price of the stock

either stays the same or changes by five dollars. It never changes by 10 cents. The ex- pected value of a random variable need not match any of the possible outcomes.

Expected values are relevant in situations other than the mean of a random variable. The expected value is relevant any time we’re dealing with random outcomes. In each case, remember that the expected value is the weighted sum of possibilities, with probabilities as the weights. The results can some- times be surprising, as in the following example.

A retailer uses free prizes to increase the number of shoppers who visit its stores. Four prize-winning tickets are hidden in randomly chosen items. The total prize of $30,000 is split among the winners. Because some tickets may not be found by the end of the contest, the amount won depends on the number of winning tickets that are claimed. Table 9.2 defines the random variable W that denotes the number of winners in a future contest. The mean, or expected value, of W is 3.

m = E1W2 = 110.052 + 210.202 + 310.452 + 410.302 = 3

Figure 9.3 graphs the probability distribution of W with a triangle locating m.

TABLE 9.2 Probabilities of the number of winners in the future contest.

Number of Winners P(W 5 w)

1 0.05

2 0.20

3 0.45

4 0.30

p(w)

1 2 3 4 w0.0

0.1

0.2

0.3

0.4

TABLE 9.3 Calculations for the expected amount won.

FIGURE 9.3 Graph of the probability distribution of W.

The amount won by those who find a ticket depends on the number who claim winning prizes. A single winner gets all $30,000. If two win, they split the money and each gets $15,000. In general, the amount won by each is $30,000 divided by the number of winners. Let’s find the expected value of the amount won, E(30,000>W). The expected value of W is 3, but the expected amount won is not $10,000.

We start by organizing the calculations. The first column in Table 9.3 lists the possible values w = 1, 2, 3, or 4 of the random variable W. The second column shows the amount won if w customers claim prizes. The third col- umn lists the probabilities of these outcomes, and the last column uses these

Number of Winners w

Amount Won 30,000,w

Probability P(W 5 w)

Amount : P(W 5 w)

1 30,000 0.05 1,500

2 15,000 0.20 3,000

3 10,000 0.45 4,500

4 7,500 0.30 2,250

Sum 11,250

9.2 PROPERTIES OF RANDOM VARIABLES 207

M09_STIN7167_03_SE_C09_pp202-223.indd 207 25/10/16 12:20 PM

208 CHAPTER 9 Random Variables

probabilities to weight the outcomes. The sum of the values in the fourth column is the expected value of the amount won.

If you find a winning ticket in your package, you win—on average— E1+30,000>W2 = +11,250. The expected value of the amount won is probably larger than you anticipated. We expect E1W2 = 3 winners to share the prize, but the expected value E130,000>W2 is larger than 30,000>E1W2 = 30,000>3 = +10,000.

5 Your graph should have four points: p102 = 0.55, p112 = 0.25, p152 = 0.15, and p1202 = 0.05. 7 m = 0(0.55) + 1(0.25) + 5(0.15) + 20(0.05) = +2. This is a so-called fair game. On average, custom- ers win as much as they spend.

6 Skewness suggests that the mean is larger than $1 or $2, but probably not larger than $5.

What Do You Think? People who play the carnival game Lucky Ducks pick a plastic duck from among those floating in a small pool. Each duck has a value written on the bottom that determines the prize. Most say “Sorry, try again!” Otherwise, 25% pay $1, 15% pay $5, and 5% pay $20. Let the random variable Y denote the winnings from picking a duck at random.

a. Graph the probability distribution of Y.5

b. Estimate the mean of Y from the graph of its probability distribution.6

c. Find the expected value of Y. What’s your interpretation of the mean if the carnival charges $2 to play this game?7

Variance and Standard Deviation

Day traders, and anyone else who has to make decisions, need to appreciate the variation in what might happen. The mean change for IBM stock is posi- tive, E(X) = $0.10 The fact that the mean is positive does not imply that stock in IBM goes up in value every day. On average, the value increases, but an investor could lose money on any given day. The same goes for players of the Lucky Ducks carnival game. The expected payoff ($2) matches the cost of the game, but the only way to break even is to play the game several times.

Parameters that describe the variation of a random variable resemble statistics that describe variation in data. The parameters even use the same names: variance and standard deviation. The variance of data s2 is the average squared deviation of the observed values from the mean. The corresponding parameter of a random variable replaces averages by expected values and the letter s by the Greek character s (“sigma”).

The variance of a random variable X is the expected value of the squared deviation from m, E(X - m)2. We often write the variance of X as Var(X) and denote it by s2. Because the variance of X is an expected value, it is a weighted sum of possible values, with the possible values weighted by probabilities. Written out, the variance of X is

Var1X2 = s2 = E1X - m22 = 1x1 - m22p1x12 + 1x2 - m22p1x22 + g+ 1xk - m22p1xk2

As an example, we’ll use the day trader’s random variable. The calculations resemble those used to find s2 from data. With data, we subtract x from each value, square these deviations, add them up, and divide the sum by n - 1. Similar steps produce the variance. First, find the deviation of each outcome from m, then square the deviations, as in Table 9.4.

variance The variance of a random variable X is the expected value of the squared deviation from its mean m:

s2 = E1X - m22

M09_STIN7167_03_SE_C09_pp202-223.indd 208 25/10/16 12:21 PM

Next, multiply the squared deviations by the corresponding probabilities. Finally, add the weighted, squared deviations. The variance of the day trader’s random variable is

Var1X2 = s2 = 1-5 - 0.102210.092 + 10 - 0.102210.802 + 15 - 0.102210.112 = 4.99

Variances have units. The variance is the expected value of the squared devia- tion from m. Hence, the measurement units of s2 are the square of the units of the random variable. Squaring makes the variance hard to interpret.

We remedy this problem for random variables, as we did for data, by taking the square root of the variance. The standard deviation of a random variable is the square root of its variance.

s = SD1X2 =1Var1X2 = 14.99 < +2.23

The expected daily change in this model for the price of IBM stock is m = +0.10, with standard deviation s = +2.23. Both m and s have the same units as the random variable X. If we want to distinguish these parameters from those of a different random variable, we use subscripts and write them as mX and sX.

4M ANALYTICS 9.1 COMPUTER SHIPMENTS AND QUALITY

MOTIVATION ▶ STATE THE QUESTION CheapO Computers promptly shipped two serv- ers to its biggest client. The company profits $5,000 on each one of these big systems. Execu- tives are horrified, though, to learn that some- one restocked 4 refurbished computers along with 11 new systems in the warehouse. The guys in shipping randomly selected the systems that were delivered from the 15 computers in stock.

If the client gets two new servers, CheapO earns $10,000 profit. If the client gets a refurbished computer, it’s coming back. CheapO loses the $5,000 profit on this system and must pay a $500 penalty as well. That leaves $4,500 profit. If both servers that were shipped are refur- bished, however, the client will return both and cancel the order. CheapO will be out any profit and be left with $1,000 in penalties. What are the expected value and the standard deviation of CheapO’s profits? ◀

TABLE 9.4 Calculating the variance of a random variable

1

2

3

4

5

6

B C D EA

p(x)

-0.1 0.01

0.09 2.3409

4.9 24.01

26.01

0.80 0.0080

0.11 2.6411

4.99

Change in Price

x

Deviation Squared Deviation Probability Weighted

-$5

0

$5

Sum

x - m = x - 0.1

-5.1

(x - m)2 (x - m)2 p(x)

standard deviation The square root of the vari- ance, whether from data or probabilities.

9.2 PROPERTIES OF RANDOM VARIABLES 209

M09_STIN7167_03_SE_C09_pp202-223.indd 209 25/10/16 12:21 PM

210 CHAPTER 9 Random Variables

METHOD ▶ DESCRIBE THE DATA AND SELECT AN APPROACH Use random variables to identify and manipulate uncertain quantities. In this ex- ample, the random variable X is the amount of profit earned on this order. The probability distribution of X comes directly from the statement of the problem. We’ll arrange the information as a tree. (You could use a table as well.) ◀

MECHANICS ▶ DO THE ANALYSIS We will approach this problem sequentially using conditional probabilities. Conditional probabilities are needed because the probability that the second system chosen for this order is refurbished depends on whether the first is refurbished. If X denotes the profit earned on the shipment, then this tree lists its values and probabilities.

4/15

11/15

10/14

11/14

3/14

4/14

44/210

12/210

44/210

110/210

Refurbished

Refurbished

Refurbished

New

New

New

Outcome x P 1X 5 x2 Both refurbished -+1,000 4>15 * 3>14 = 12>210 One refurbished $4,500 4>15 * 11>14 + 11>15 * 4>14 = 88>210 New/New $10,000 11>15 * 10>14 = 110>210

The expected value of X is a weighted sum of the possible profits, with the probabilities defining the weights.

E1X2 = -1,000 112>2102 + 4,500 188>2102 + 10,000 1110>2102 < +7,067 The variance is expected squared deviation of the profit from the mean.

Var1X2 = 1-1,000 - 7,06722112>2102 + 14,500 - 7,06722188>2102 + 110,000 - 7,067221110>2102 < 10,986,032 +2

The measurement units of the variance are unusual, so we wrote them in this odd style of “squared dollars” as a reminder. Taking the square root gives the standard deviation, which is measured in dollars and is easier to interpret.

SD1X2 = 1Var1X2 = 110,986,032 < +3,315 ◀ MESSAGE ▶ SUMMARIZE THE RESULTS

This is a very profitable deal on average, unless both systems that were shipped are refurbished units. The expected value of the sale is $7,067. The large SD (about $3,300) is a reminder that the profits are wiped out if both systems are refurbished. ◀

M09_STIN7167_03_SE_C09_pp202-223.indd 210 25/10/16 12:21 PM

9.3 PROPERTIES OF EXPECTED VALUES 211

9.3 ❘ PROPERTIES OF EXPECTED VALUES We often add a constant to a random variable or multiply a random variable by a constant. For example, suppose the computer shipper in 4M Analytics 9.1 has to pay fee of $1,000 to an agent who found this customer. In this case the net profits are not X dollars, but X - $1,000. Similarly, we often multiply random variables by constants. The car shop that sells tires makes a sale of Y tires to a customer. If the shop charges $5 to mount each tire, then the mount- ing charge for a sale of Y tires is $5 * Y.

What are the mean and standard deviation of these quantities? We don’t need to start over by defining a new random variable. An easier approach uses the properties of the random variables that we already have. If you think back to the properties of histograms, you’ll know right away what happens to a ran- dom variable when we shift its values or change its scale. Let’s start with the effect of adding (or subtracting) a constant.

Adding or Subtracting a Constant

Adding or subtracting a constant from a random variable shifts every possible value of the random variable, changing the expected value by the constant. Figure 9.4 shows the effect of subtracting the $1,000 fee from the profits of the computer vendor. The fee shifts each possible value down by 1,000. Hence, the mean shifts down by $1,000 as well. In general, if c is any constant, then

E(X { c) = E(X ) { c

Capital letters denote random variables, whereas lowercase letters like a, b, or c identify constants. When you see a capital letter, think random. When you see a small letter, think of numbers such as 2 or 3.14.

tip

p(x)

-2000 2000 4000 6000 8000 10,000 x

0.1

0.2

0.3

0.4

0.5

FIGURE 9.4 Subtracting a constant from a random variable shifts every outcome by the same amount without changing the probabilities.

Hence, the expected net profit to the computer shipper after paying the fixed fee of $1,000 is

E1X - 1,0002 = E1X2 - +1,000 = +7,067 - +1,000 = +6,067 What about the standard deviation? How does shifting the outcomes

affect the standard deviation? Look back at the effect of shifting the dis- tribution in Figure 9.4. Each outcome moves, but the differences among them remain the same. With data, adding or subtracting a constant doesn’t

M09_STIN7167_03_SE_C09_pp202-223.indd 211 25/10/16 12:21 PM

212 CHAPTER 9 Random Variables

change the variance or standard deviation. The same holds for random variables. Adding or subtracting a constant has no effect on the variance or standard deviation of a random variable.

Var1X { c2 = Var1X2 SD1X { c2 = SD1X2

Multiplying by a Constant

Multiplying the value of a random variable by a constant changes both the expected value and the variance. Recall from Chapter 4 that multiplying or dividing data by a constant change the mean and the standard deviation by that factor. Variance, being the square of standard deviation, changes by the square of the constant. The same is true for random variables.

Multiplying a random variable by a constant c multiplies the mean and standard deviation by c. The absolute value is needed in the expression for the SD because the constant c could be negative, but the SD cannot be negative.

E1c X2 = c E1X2 SD1cX2 = 0c 0 SD1X2

Because it has squared units, multiplying by a constant changes the variance by the square of c.

Var1c X2 = c 2 Var1X2

For example, Figure 9.5 shows the effect on the distribution of the tire shop of multiplying by 5. Rather than run from 1 to 4, the values now extend from 5 to 20. The probabilities are the same, but the gaps between adjacent values have grown by the factor 5. The car shop (page 215) sells on average

E1Y2 = 110.52 + 210.252 + 310.06252 + 410.18752 = 1.9375 tires A similar calculation shows that SD1Y2 < 1.144 tires. Since it charges $5 to install each tire, the expected charge for this service is E15Y2 = 5E1Y2 = 511.93752 < 9.69, with standard deviation SD15Y2 = 5SD1Y2 < 511.1442 = +5.72.

Let’s do another example, returning to the day trader. Suppose rather than buying one share of stock, she buys two. By investing $200 and buying two shares of the same stock, the day trader doubles m and s. With c = 2, the expected change in the value of her investment is twice what we found previously:

E12X2 = 2E1X2 = 2 * 0.10 = +0.20 with standard deviation

SD12X2 = 2 SD1X2 = +4.46 She expects to earn twice as much on average, but she also doubles the stan- dard deviation. The variance grows by a factor of 4.

Var12X2 = 22Var1X2 = 4 * 4.99 = 19.96 +2 For another example, let the random variable S denote the number of win-

dows shipped daily by a manufacturer of building materials. Some days the plant produces more and on other days less. The mean of S is 2,500 windows,

M09_STIN7167_03_SE_C09_pp202-223.indd 212 25/10/16 12:21 PM

with standard deviation 500 windows. Each window has eight panes of glass. What is the expected number of panes of glass shipped each day and its stan- dard deviation?

Notice that we don’t need the probability distribution of the random vari- able S. All we need are its mean and standard deviation. The expected number of panes is E18 S2 = 8E1S2 = 812,5002 = 20,000 panes, with standard de- viation SD18 S2 = 8SD1S2 = 815002 = 4,000 panes.

The following equations summarize the rules for finding expected values and variances when constants are mixed with random variables. Lowercase letters denote constants.

Rules for Expected Values If a and b are constants and X is a random variable, then

E1a + b X2 = a + b E1X2 SD1a + b X2 = 0b 0 SD1X2 Var1a + b X2 = b2 Var1X2

Expected values of sums are sums of expected values. The expected value of a constant is that constant and, when they appear in a product, constants factor out (with squares for the variance).

1 2 3 4 y0.0

0.1

0.2

0.3

0.4

0.5

5 10 15 20 0.0

0.1

0.2

0.3

0.4

0.5

Times 5 p(5y)

p(y)

5y

FIGURE 9.5 Multiplying a random variable by a constant scales the gaps between the outcomes by the constant, without changing the probabilities.

9.3 PROPERTIES OF EXPECTED VALUES 213

M09_STIN7167_03_SE_C09_pp202-223.indd 213 25/10/16 12:21 PM

214 CHAPTER 9 Random Variables

What Do You Think? a. Suppose that the day trader has to pay a $0.10 transaction fee each time that she buys a share of stock. What is the expected net change in the value of her investment?8

b. Suppose the day trader prefers to track the value of her investments in cents ($1 = 100 cents) rather than dollars. What are the mean and variance of 100X?9

9.4 ❘ COMPARING RANDOM VARIABLES Expressions that combine random variables with constants often arise when comparing random variables. Comparison requires putting the random vari- ables on a common scale. Consider the following example.

An international business based in the United States is considering launch- ing a product in two locations. One location is in Europe, and the other is in India. Regional management in each location offers a random variable to express what it believes will happen if the product is launched. These random variables are expressed in the local currency. The division in Europe believes that the launch will produce net profits of 2.5 million euros in the first year, with standard deviation 0.75 million euros. The division in India expects that the launch will generate net profits of 125 million rupees, with standard devia- tion 50 million rupees.

We cannot directly compare these random variables because they are mea- sured on different scales, one in euros and the other in rupees. To compare them, as well as to make them more familiar to managers in the United States, let’s convert both into dollars. Assume the exchange rates are 1 dollar = 0.90 euro and 1 dollar = 60 rupees. Let X denote the random variable that models the launch in Europe, and let Y denote the random variable for India. To convert X into dollars, we have to divide by 0.90. Hence, the mean and variance of a launch in Europe are E1X>0.902 = 2.5>0.90 = +2.78 million and SD1X>0.902 = 0.75>0.90 = +0.83 million. By comparison, the mean and SD of the proposed launch in India are E1Y>602 = 125>60 = +2.08 million and SD1Y>602 = 50>60 = +0.83 million. If both random variables accurately reflect the potential gains, then the launch in Europe seems the better choice. This option offers a larger mean with the same SD (higher expected sales with matching uncertainty).

Sharpe Ratio

Comparisons of some random variables require adjustments even if they have the same units. In the prior example, the launch in Europe wins be- cause it offers larger expected sales along with the same uncertainty. Often, however, the comparison is less one-sided. When comparing the perfor- mance of two stocks, for example, the stock with higher average return usually has a higher standard deviation. For instance, during the 15 years 1990–2014, stock in Apple on average grew 2.45% monthly, with standard deviation 13.3%. During this same period, stock in McDonald’s grew 1.14% with SD = 6.2%. Stock in McDonald’s grew at a slower average rate, but the growth was steadier.

We can use these characteristics to define two random variables as models for what we expect to happen in future months, as shown in Table 9.5.

8 Zero: E1X - 0.12 = E1X2 - 0.1 = 0.1 - 0.1 = 0. Trading costs are spoilers. 9 E1100X2 = 100E1X2 = 10.; Var1100X2 = 1002 Var1X2 = 1002 * 4.99 = 49,900.2.

M09_STIN7167_03_SE_C09_pp202-223.indd 214 25/10/16 12:21 PM

BEST PRACTICES 215

Which random variable represents the better investment going forward? The ran- dom variables have the same units (monthly percentage changes), but the com- parison is difficult because the one with the higher average also has the higher SD.

Finance offers many ways to judge the relative size of rewards in compari- son to risks. One of the most popular is the Sharpe ratio. The higher the Sharpe ratio, the better the investment. The Sharpe ratio is a fraction derived from percentage changes in the value of a stock. The numerator of the Sharpe ratio is the mean return minus what you would earn on an investment that presents no risk. The denominator of the Sharpe ratio is the standard devia- tion of the return on the investment. Suppose X is a random variable that measures the performance of an investment, and assume that the mean of X is m and its SD is s. The Sharpe ratio of X is

S1X2 = m - rf s

The symbol r f stands for the return on a risk-free investment (similar to the

rate of interest on a savings account). Don’t confuse the Sharpe ratio S1X2 with the standard deviation SD1X2.

Let’s use the Sharpe ratio to compare investing in Apple and McDonald’s. We’ll set the risk-free rate of interest to 0.1% per month (about 1% annual interest). Subscripts on m and s identify the random variables. The Sharpe ratio for Apple is

S1A2 = mA - rf sA

= 2.45 - 0.10

13.3 < 0.177

The Sharpe ratio for McDonald’s is

S1M2 = mM - rf sM

= 1.14 - 0.10

6.2 < 0.168

An investor who judges investments with the Sharpe ratio prefers investing in Apple to investing in McDonald’s because S1A2 7 S1M2. Apple offers a higher average rate of return above the risk-free rate relative to its standard deviation.

TABLE 9.5 Random variables based on monthly returns on Apple and McDonald’s.

Company Random Variable Mean SD

Apple A 2.45% 13.3%

McDonald’s M 1.14% 6.2%

Sharpe ratio Ratio of an investment’s net expected gain to its standard deviation.

Best Practices

■■ Use random variables to represent uncertain out- comes. Associating a random variable with a random process forces you to become specific about what you think might happen. You have to specify the possibilities and the chances of each.

■■ Graph the random variable. Plots provide impor- tant summaries of data, and the same applies to random variables. The graph of the probability distribution of a random variable is analogous to the histogram of data. We can guess the mean and the standard deviation from the probability distribution. If you graph the probability distri- bution, you are less likely to make a mistake cal- culating its mean or its standard deviation.

■■ Recognize that random variables represent mod- els. The stock market is more complicated than flipping coins. We’ll never know every prob- ability. Random variables are useful, but they should not be confused with reality. Question probabilities as you would data. Just because you have written down a random variable does not mean that the phenomenon actually be- haves that way.

■■ Keep track of the units of the random variables. As with data, the units of random variables are important. It does not make sense to compare data that have different units, and the same applies to random variables.

M09_STIN7167_03_SE_C09_pp202-223.indd 215 18/11/16 3:33 PM

216 CHAPTER 9 Random Variables

Pitfalls

■ Do not confuse x with m or s with s. Greek let- ters highlight properties of random variables, not properties of data.

■ Do not mistake a histogram for a probability dis- tribution. They are similar, but recognize that the probability distribution of the random vari- able describes a model for what will happen, but it’s only a model.

■ Do not mix up X with x. Uppercase letters de- note random variables, and the corresponding lowercase letters indicate a generic possible value of the random variable.

■ Do not forget to square constants in variances. Remember that Var1cX2 = c2Var1X2. Vari- ances have squared units.

CHAPTER SUMMARY

A random variable represents the uncertain out- come of a random process. The probability distri- bution of a random variable X identifies the possible outcomes and assigns a probability to each. The probability distribution is analogous to the histo- gram of numerical data. An expected value is a weighted sum of the possible values of an expression that includes a random variable. The expected value

of a random variable itself is the mean m = E1X2 of the random variable. The variance of a random variable s2 = Var1X2 is the expected value of the squared deviation from the mean. The standard deviation is the square root of the variance. The Sharpe ratio of an investment is the ratio of its net expected gain to its standard deviation.

■ Formulas

■ Key Terms expected value, 206 mean m of a random variable, 206 parameter, 205 probability distribution, 204 random variable, 203

continuous, 203 discrete, 203 Sharpe ratio, 215 standard deviation, 209 statistical model, 204

symbol, m (mean), 206 s (std. dev.), 209 s2 (variance), 208 variance, 208

The letters a, b, and c stand for constants in the fol- lowing formulas, and subscripted lowercase symbols (such as x

1 ) represent possible values of the random

variable X.

Mean or Expected Value of a Discrete Random Variable X

Assuming that the random variable can take on any of the k possible values x1, x2, c, xk, then its expected value is

m = E1X2 = x1 p1x12 + x2 p1x22 + g+ xk p1xk2 = a

k

i = 1 xi p1xi2

Variance of a Discrete Random Variable

Assuming that the random variable can take on any of the k possible values x1, x2, c, xk, then its variance is

s2 = Var1X2 = E1X - m22 = 1x1 - m22p1x12 + 1x2 - m22p1x22 + g

+ 1xk - m22p1xk2 = a

k

i = 1 1xi - m22p1xi2

■ Objectives • Use the language of probability to represent

uncertain outcomes as random variables. • Relate the information in a histogram to the prob-

ability distribution of a random variable, appreci- ating their similarities and differences.

• Find the mean and variance of random variables from their probability distributions.

• Manipulate the mean, variance, and standard deviation of random variables when transformed by adding or multiplying by a constant.

M09_STIN7167_03_SE_C09_pp202-223.indd 216 25/10/16 12:21 PM

EXERCISES 217

A short-cut formula simplifies the calculations (avoid- ing the need to subtract m before squaring) and is use- ful if you have to do these calculations by hand:

s2 = E1X22 - m2 = x21 p1x12 + x22 p1x22 + g+ x2k p1xk2 - m2

Standard deviation of a random variable X is the square root of its variance.

s = 2Var1X2 = 2s2 Adding a Constant to a Random Variable

E1X { c2 = E1X2 { c SD1X { c2 = SD1X2, Var1X { c2 = Var1X2

Multiplying a Random Variable by a Constant

E1cX2 = cE1X2 SD1cX2 = 0c 0SD1X2, Var1cX2 = c2Var1X2

Adding a Constant and Multiplying by a Constant

E1a + bX2 = a + bE1X2 Var1a + bX2 = b2Var1X2

Sharpe Ratio of the Random Variable X with Mean M and Variance S2

S1X2 = m - rf s

The symbol r f is the risk-free rate of interest, the rate

of interest paid on savings that are not at risk of loss or default.

■ About the Data This chapter models changes in the price of stock in IBM using a random variable with three pos- sible outcomes, -$5, 0, and $5. Actual changes in the value of IBM stock take on many more pos- sibilities, but they have a lot in common with this simple model. In particular, this random variable shares the same mean and standard deviation as percentage changes in the actual stock. Over a duration of 10 years (2,519 trading days in 1994 through 2003), IBM stock changed by 0.103% daily on average, with standard deviation 2.25%. Those statistics are very similar to the parameters m = 0.1 and s = 2.23 of this random variable.

This random variable was a good match, but much has happened since 2003. Static models based on past data can fall out of sync with a changing economy. In this case, the more recent performance of IBM stock shows the effects of the credit crises and recession. From 2004 through 2015 (3,021 days of market trading), IBM stock increased on average daily by less than one-third as much as in the prior years, 0.029%, with a smaller standard deviation s = 1.33%. A model that matches the data during one period may not be such a good match during other periods.

Mix and Match

Match the descriptive phrases to the symbols on the right. In these exercises, X denotes a random variable.

EXERCISES

1. Expected value of X (a) E1X - m2 2. Variance of X (b) 10X

3. Standard deviation of X (c) 1X - 0.042>s 4. Shorthand notation for P(X = x) (d) X + 10

5. Has 10 times the standard deviation of X (e) E(X - m)2

6. Is always equal to zero (f) 1Var1X2 7. Increases the mean of X by 10 (g) m

8. Has standard deviation 1 (h) p(x)

M09_STIN7167_03_SE_C09_pp202-223.indd 217 25/10/16 12:21 PM

218 CHAPTER 9 Random Variables

True/False

Mark each statement True or False. If you believe that a statement is false, briefly say why you think it is false.

Exercises 9–14. A cable television provider is planning to change the way that it bills customers. The new billing would depend on the number of computers and televi- sions that customers connect to its network. The provider has asked managers to participate in the planning by de- scribing a random variable X that represents the number of connections used by a current customer and a second random variable Y that represents the number of connec- tions used by a customer after the change in billing. All customers must have at least one connection.

9. The function defined as p112 = 1>2, p122 = 1>4, p132 = 1>8, p142 = 1>8, and p1x2 = 0 otherwise is a probability distribution that could be used to model X.

10. The function defined as p112 = 1>2, p122 = 1>4, p132 = 1>2, p142 = -1>4, and p1x2 = 0 otherwise is a probability distribution that could be used to model X.

11. If the new style of billing increases the average number of connections used by customers, then E1X2 7 E1Y2.

12. If a customer can have from 1 to 10 possible connec- tions in the current setup, then 1 … E1X) … 10.

13. The mean E1Y2 must be an integer because this random variable can only be an integer.

14. If the cable network counts the number of connec- tions used by 100 customers, then the average count of these customers is the same as E1X2.

Exercises 15–20. An insurance company uses a random variable X to model the cost of an accident incurred by a female driver who is 20 to 30 years old. A second random variable Y models the cost of an accident by a male driver in the same range of ages. Both random variables are measured in dollars.

15. If the mean of Y is $3,200, then the variance of Y must be larger than $3,200.

16. If E1X2 = +2,300, then P1X … 2,3002 = 1>2. 17. The units of both the mean and standard deviation of

Y are dollars.

18. If an insurance policy limits the coverage of an accident to $500,000, then both E1X2 and E1Y2 must be less than $500,000.

19. If costs associated with accidents rise next year by 5%, then the mean of the random variable X should also increase by 5%.

20. If costs associated with accidents rise next year by 5%, then the standard deviation of the random vari- able Y should also increase by 5%.

Think About It

The following plots show the probability functions of four random variables: X, Y, Z, and W. Use these pictures to answer Exercises 21–30. Estimate the needed probabili- ties from the information shown in the plots.

21. What is the probability that X = -2? 22. What is the probability that Y Ú 2.5? 23. What is the probability that Z … -3 or Z = 3? 24. Which is larger, P1W 7 02 or P1Y 7 02? 25. Is the E(Z) positive, negative, or close to zero?

26. Which random variable has the largest mean?

27. Which random variable has the largest SD? (You do not need to compute the SD from the shown prob- abilities to answer this question.)

28. Pick the SD of W: 0.15, 1.5, or 15. (You do not need to compute the SD from the shown probabilities to select from these three choices.)

29. The median of the random variable Y is a value m such that P1Y … m2 = 1>2. What is the median of Y?

X

p(x)

x -3 -2 -1

0.1

0.0

0.3

0.2

0.4

0 1

Y

p(x)

x -3 -2 -1

0.05

0.00

0.15

0.10

0.20

0.25

0 1 2 3

Z

p(x)

x -3 -2 -1

0.05

0.00

0.15

0.10

0.20

0.25

0.30

0 1 2 3

W

p(x)

x -3 -2 -1

0.05

0.00

0.15

0.10

0.20

0.25

0.30

0 1 2 3

M09_STIN7167_03_SE_C09_pp202-223.indd 218 25/10/16 12:21 PM

30. The lower quartile of a random variable X is the smallest q so that P1X … q2 Ú 0.25. (This is analo- gous to the lower edge of the box in the boxplot.) What’s the lower quartile of X?

31. A game involving chance is said to be a fair game if the expected amount won or lost is zero. Consider the following arcade game. A player pays $1 and chooses a number from 1 to 10. A spinning wheel then ran- domly selects a number from 1 to 10. If the numbers match, the player wins $5. Otherwise the player loses the $1 entry fee. (a) Define a random variable W that is the amount

won by the player and draw its probability distri- bution. (Capital letters other than X can be used for random variables.) Use negative values for losses, and positive values for winnings.

(b) Find the mean of W. Is this a fair game?

32. In a common type of lottery, a customer buys a ticket with a three-digit number from 000 to 999. A machine (such as one with bouncing balls num- bered 0 to 9) then selects a number in this range at random. Each ticket bought by a customer costs $1, whether the customer wins or loses. Customers with winning tickets are paid $500 for each winning ticket. (a) Sketch the probability distribution of the ran-

dom variable X that denotes the net amount won by a customer. (Notice that each customer pays $1 regardless of whether he or she wins or loses.)

(b) Is this a fair game? (See Exercise 31 for the definition of a fair game.) Does the state want a fair game?

(c) Interpret the expected value of X for a person who plays the lottery.

33. A fair game requires that a player’s probability of win- ning must be equal to the player’s share of the pot of money awarded to the winner. All money is put into a pot at the start of the game, and the winner claims the entire amount. (Compare this definition to that in Exercise 31.) (a) The player and the host each put $20 into the

pot. The player rolls a die and wins the pot if the die produces an even number. Is this game fair to the player?

(b) A player at a casino puts $1 into the pot and names a card from a standard deck (e.g., ace of diamonds). The casino puts $99 into the pot. If a randomly selected card matches the choice of the player, the player wins the pot. Is this game fair to the player?

34. Two companies are competing to define a new stan- dard for digital recording. Only one standard will be adopted commercially. Company A invests $10 million to promote its standard, and Company B invests $20 million to promote its. If the outcome is to be viewed as a fair game (in the sense defined in Exercise 33), what is the chance that the market accepts the pro- posal of Company A?

You Do It

Bold names shown with the question identify the data table for the problem.

35. Given that the random variable X has mean m = 120 and SD s = 15, find the mean and SD of each of these random variables that are defined from X: (a) X>3 (b) 2X - 100 (c) X + 2 (d) X - X

36. Given that the random variable Y has mean m = 25 and SD s = 10, find the mean and SD of each of these random variables that are defined from Y: (a) 2Y + 20 (b) 3Y (c) Y>2 + 0.5 (d) 6 - Y

37. An investor buys the stock of two companies, investing $10,000 in each. The stock of each com- pany either goes up by 80% after a month (rising to $18,000) with probability 1/2 or drops by 60% (fall- ing to $4,000) with probability 1/2. Assume that the changes in each are independent. Let the random variable X denote the value of the amount invested after one month. (a) Find the probability distribution of X. (b) Find the mean value of X. (c) Does the mean value represent the experience of

the typical investor?

38. Imagine that the investor in Exercise 37 invests $10,000 for one month in a company whose stock either goes up by 80% after a month with prob- ability 1/2 or drops 60% with probability 1/2. After one month, the investor sells this stock and uses the proceeds to buy stock in a second company. Let the random variable Y denote the value of the $10,000 investment after two months.10

(a) Find the probability distribution of Y. (b) Find the mean value of Y. (c) Does the mean value represent the experience of

the typical investor?

39. A construction firm bids on a contract. It anticipates a profit of $50,000 if it gets the contract for the full proj- ect, and it estimates its profit to be $20,000 on a shared project. The firm estimates there’s a 20% chance it will get the full contract and a 75% chance it will get the shared contract; otherwise, it gets nothing. (a) Define a random variable to model the outcome

of the bid for this firm. (b) What is the expected profit earned on these

contracts? Report units with your answer. (c) What is the standard deviation of the profits?

40. A law firm takes cases on a contingent fee basis. If the case goes to trial, the firm expects to earn $25,000 as part of the settlement if it wins and nothing if it does not. The firm wins one-third of the cases that go to

10 See A Mathematician Plays the Stock Market by J. A. Paulos for similar problems.

EXERCISES 219

M09_STIN7167_03_SE_C09_pp202-223.indd 219 25/10/16 12:21 PM

220 CHAPTER 9 Random Variables

trial. If the case does not go to trial, the firm earns nothing. Half of the cases do not go to trial. Be sure you add the appropriate units. (a) Define a random variable to model the earning of

taking a case of this type. (b) What is the expected value of such a case to the

firm? (c) What is the standard deviation of the earnings?

41. The manager of an office has estimated how many reams (packs of 500 sheets) are used each day and has assigned these probabilities.

X 5 # of reams of paper used

0 1 2 3 4 5

P 1X 5 x2 0.05 0.25 0.35 0.15 0.15 0.05

(a) How many reams should the manager expect to use each day?

(b) What is the standard deviation of the number of reams?

(c) If the storeroom has 20 reams at the start of the day, how many should the manager expect to find at the end of the day?

(d) Find the standard deviation of the number of reams found at the end of the day.

(e) Find the mean and standard deviation of the number of pages of paper used each day (assum- ing that each ream is completely used).

42. The maintenance staff of a large office building regu- larly replaces fluorescent ceiling lights that have gone out. During a visit to a typical floor, the staff may have to replace several lights. The manager of this staff has given the following probabilities to the number of lights (identified by the random variable Y) that need to be replaced on a floor:

Y 0 1 2 3 4

P(Y 5 y) 0.2 0.15 0.2 0.3 0.15

(a) How many lights should the manager expect to replace on a floor?

(b) What is the standard deviation of the number of lights on a floor that are replaced?

(c) If a crew takes six lights to a floor, how many should it expect to have left after replacing those that are out?

(d) If a crew takes six lights to a floor, find the stan- dard deviation of the number of lights that remain after replacing those that are out on a floor.

(e) If it takes 10 minutes to replace each light, how long should the manager expect the crew to take when replacing the lights on a floor?

43. Companies based in the United States that operate in foreign nations must convert currencies in order to pay obligations denoted in those currencies. Consider a company that has a contract that requires it to pay 1,000,000 Venezuelan bolivars for an oil shipment in six months. Because of economic uncertainties, managers at the company are unsure of the future exchange rate. Local contacts indicate that the

exchange rate in six months could be 2.15 bolivars per dollar (with probability 0.6) or might rise dramat- ically to 5 bolivars per dollar (with probability 0.4). (a) If managers accept these estimates, what is the

expected value today, in dollars, of this contract? (b) The expected value of the exchange rate is

2.1510.62 + 510.42 = 3.29 bolivars/dollar. If we divide 1,000,000 by 3.29, do we get the answer found in part (a)? Explain why or why not.

44. During the 1960s and into the 1970s, the Mexican government pegged the value of the Mexican peso to the U.S. dollar at 12 pesos per dollar. Because interest rates in Mexico were higher than those in the United States, many investors (including banks) bought bonds in Mexico to earn higher returns than were available in the United States. The benefits of the higher interest rates, however, masked the possibility that the peso would be allowed to float and lose sub- stantial value compared to the dollar. Suppose you are an investor and believe that the probability of the exchange rate for the next year remains at 12 pesos per dollar is 0.9, but that the rate could soar to 24 per dollar with probability 0.1. (a) Consider two investments: Deposit $1,000 today

in a U.S. savings account that pays 8% annual interest (rates were high in the 1970s), or deposit $1,000 in a Mexican account that pays 16% inter- est. The latter requires converting the dollars into pesos at the current rate of 12 pesos/dollar, and then after a year converting the pesos back into dollars at whatever rate then applies. Which choice has the higher expected value in one year? (In fact, the peso fell in value in 1976 by nearly 50%, catching some investors by surprise.)

(b) Now suppose you are a Mexican with 12,000 pesos to invest. You can convert these pesos to dollars, collect 8% interest, and then convert them back at the end of the year, or you can get 16% from your local Mexican investment. Com- pare the expected value in pesos of each of these investments. Which looks better?

(c) Explain the difference in strategies that obtain the higher expected value.

45. A manufacturer of inexpensive printers offers a model that retails for $150. Each sale of one of these models earns the manufacturer $60 in profits. The manufacturer is considering offering a $30 mail-in rebate. Assume that a randomly selected customer has probability p of purchasing this model of printer. (a) Assume that the availability of a rebate increases

the probability of purchase from p to p*. How much does the chance of purchase need to increase in order for the expected net profits to increase? (Assume that the manufacturer has plenty of printers to sell and that the cost to the manufacturer for the rebate is the stated $30.)

(b) Not all rebates that are offered are used, with the chance of the rebate being used hovering around 40%. If this is so, how much does the chance of purchase need to increase in order for

M09_STIN7167_03_SE_C09_pp202-223.indd 220 25/10/16 12:21 PM

the expected net profits to increase? Identify any assumptions you make.

46. Suppose that you’ve just bought a $4,000 flat-screen TV. Should you also buy the $50 surge protector that guarantees to protect your TV from electric surges caused by lightning? (a) Let p denote the probability that your home is

hit by lightning during the time that you own this TV, say five years. In order for the purchase of the surge protector to have positive long-term value, what must the chance of being hit by lightning be?

(b) There are about 100 million households in the United States, and fewer than 10,000 get hit by lightning each year. Do you think the surge pro- tector is a good deal on average? Be sure to note any assumptions you make.

47. An insurance salesman visits up to three clients each day, hoping to sell a new policy. He stops for the day once he makes a sale. Each client independently decides whether to buy a policy; 10% of clients pur- chase the policy. (a) Create a probability model for the number of

clients the salesman visits each day. (b) Find the expected number of clients. (c) If the salesman spends about 212 hours with each

client, then how many hours should he expect to be busy each day?

(d) If the salesman earns $3,000 per policy sold, how much can he expect to make per day?

48. An office machine costs $10,000 to replace unless a mysterious, hard-to-find problem can be found and fixed. Repair calls from any service technician cost $500 each, and you’re willing to spend up to $2,000 to get this machine fixed. You estimate that a repair technician has a 25% chance of fixing it. (a) Create a probability model for the number of

visits needed to fix the machine or exhaust your budget of $2,000. (That is, there can be at most four visits.)

(b) Find the expected number of service technicians you’ll call in.

(c) Find the expected amount spent on this machine. (You must spend $10,000 for a new one if you do not find a vendor that repairs yours.)

49. The ATM at a local convenience store allows custom- ers to make withdrawals of $10, $20, $50, or $100. Let X denote a random variable that indicates the amount withdrawn by a customer. The probability distribution of X is

p1102 = 0.2 p1202 = 0.5 p1502 = 0.2

p11002 = 0.1

(a) Draw the probability distribution of X. (b) What is the probability that a customer with-

draws more than $20?

(c) What is the expected amount of money with- drawn by a customer?

(d) The expected value is not a possible value of the amount withdrawn. Interpret the expected value for a manager.

(e) Find the variance and standard deviation of X.

50. A company orders components from Japan for its game player. The prices for the items that it orders are in Japanese yen. When the products are deliv- ered, it must convert dollars into yen to pay the Japanese producer. When its next order is delivered (and must be paid for), it believes that the exchange rate of dollars to yen will take on the following val- ues with the shown probabilities.

Conversion Rate (yen per dollar) Probability

100 0.1

110 0.2

120 0.4

130 0.2

140 0.1

(a) Draw the probability distribution of a random variable X that denotes the exchange rate on the day of the delivery. Describe the shape of this distribution.

(b) If the current exchange rate is 120 ¥/$, what is the probability that the exchange rate will be higher on the delivery date?

(c) What is the expected value of the exchange rate on the date of the delivery?

(d) If the company exchanges $100,000 on the deliv- ery date, how many yen can it expect to receive?

(e) What is the expected value in dollars of ¥10,000 on the delivery date?

51. NBA MVP Star basketball player Derrick Rose of the Chicago Bulls was named Most Valuable Player for the 2010–2011 NBA season.11 This table lists the number of baskets made of several types along with the number of attempts.

Type of Shot Made Attempts

Free throw (1 point) 476 555

Field goal (2 points) 711 1,597

3-point goal 128 385

Use the implied percentages for each type of shot as probabilities to answer the following questions:

(a) Which opportunity has the highest expected value for Rose: two free-throw attempts, one field goal attempt, or one attempted 3-point shot?

(b) Which has larger variation in outcome for Rose: attempt a 2-point basket or attempt a 3-point basket?

11 From http://www.usatoday.com/sports/basketball.

EXERCISES 221

M09_STIN7167_03_SE_C09_pp202-223.indd 221 25/10/16 12:21 PM

222 CHAPTER 9 Random Variables

(c) If his team is losing by 1 point near the end of a game, which of the following offers the bigger chance for a win (i.e., scoring 2 points): Rose attempts a field goal or Rose attempts two free throws? Are any assumptions needed for this comparison?

52. Akers In 13 professional football seasons from 1999 to 2011, kicker David Akers recorded the following results for field goal attempts of various distances.

Distance Made Attempts

0–19 7 7

20–29 111 112

30–39 109 123

40–49 89 128

50+ 22 41

Use the implied percentages as probabilities to answer the following questions:

(a) A made field goal scores 3 points. What is the expected value (in points scored) if Akers at- tempts a 45-yard kick?

(b) During a game, Akers attempts kicks of 27, 38, and 51 yards. What is the expected contribution of these kicks to the score in the game? What assumptions do you need?

(c) His team can allow Akers to attempt a 37-yard kick or, they can also go for a first down. If they go for a first down and fail, they score nothing. If they make the first down, half of the time Akers will get a chance to kick from less than 30 yards and half of the time they score a touchdown (worth 7 points). They have a 35% chance to make the first down. What should they do?

53. Daily Stocks These data are daily percentage changes of several stocks in 2010 and 2011. This question focuses on stock in Apple. (a) Describe the shape and any key features of

the histogram of percentage changes in Apple stock.

(b) Find the mean and standard deviation of the percentage changes in Apple stock. Use the prop- erties of these data to define a random variable that is the percentage change in Apple stock in a future day.

(c) If the interest cost for a daily investment in Apple is 2 basis points (0.02% per day), find the Sharpe ratio for a daily investment of $1,000 in this stock.

(d) Find the Sharpe ratio for a daily investment of $10,000 in this stock. Does it differ from your answer to (c)?

(e) How does the Sharpe ratio for Apple compare to that for owning McDonald’s during this period?

(f) Contrast the results obtained in (e) using these daily returns to the comparison of Sharpe ratios using monthly data described in the chapter. Do these comparisons agree?

54. This exercise considers daily percentage changes in stock of Disney during 2010 and 2011. Use the same data as in Exercise 53. (a) Describe the shape and any key features of the

histogram. (b) Find the mean and standard deviation of the per-

centage changes. Use the properties of these data to define a random variable that is the percentage change in Disney stock in a future day.

(c) If the interest cost for a daily investment in Disney is 2 basis points (i.e., 0.02% per day), find the Sharpe ratio for a daily investment of $1,000 in this stock.

(d) Does the Sharpe ratio indicate whether the inves- tor is better off with more money or less money invested in Disney? That is, can an investor use the Sharpe ratio to decide whether to invest $1,000, $5,000, or $10,000 in Disney?

(e) How does the Sharpe ratio for Disney compare to that for Microsoft during these two years

(f) If an investor compares the Sharpe ratio for Disney to that for Microsoft to decide which investment is better, then what assump- tions about future returns are being made?

55. 4M ANALYTICS: Project Management

A construction company manages two large building projects. Both require about 50 employees with compa- rable skills. At a meeting, the site managers from the two projects got together to estimate the labor needs of their projects during the coming winter. At the end of the meet- ing, each manager provided a table that showed how they expected weather to impact their labor needs.

Site 1:

Winter Conditions

Mild Typical Cold Severe

Number of labor employees